Language Selection

English French German Italian Portuguese Spanish

Gnu Planet

Syndicate content
Planet GNU - https://planet.gnu.org/
Updated: 4 hours 40 min ago

health @ Savannah: GNU Health Control Center 3.6.5 supports Weblate

Thursday 2nd of July 2020 09:50:28 PM

Dear community

As you may know, GNU Health HMIS has migrated its translation server to Weblate.
(see news https://savannah.gnu.org/forum/forum.php?forum_id=9762)

Today, we have released the GH Control Center 3.6.5, which has support to Weblate for the language installation. The syntax is the same, and you won't notice any difference.

To update to the latest GH Control Center, run:

$ cdutil
$ ./gnuhealth-control update

That will fetch and install the latest version, and you're ready to go :)

Happy and Healthy hacking!
Luis

gnucobol @ Savannah: GnuCOBOL 3.1-rc-1 on alpha.gnu.org

Thursday 2nd of July 2020 07:13:19 AM

While this version is a release-candidate (with an expected full release within 3 months) it is the most stable and complete free COBOL compiler ever available.

Source kits can be found at https://alpha.gnu.org/gnu/gnucobol, the first pre-built binaries are already available and the OS package managers are invited to update their packages.

Compared to the last stable release of 2.2 we have such a huge change list that it is too much to note here for the release candidate; but it will be done with the final release and even there the NEWS entry which you can check now at https://sourceforge.net/p/open-cobol/code/HEAD/tree/tags/gnucobol-3.1-rc1/NEWS is very compacted.

The RC1 got extensive tests over the lasts months and is downward compatible to GnuCOBOL 2.2 - you are invited to upgrade GnuCOBOL now or start your own tests with the release candidate to be able to update when the final release arrives.

Parabola GNU/Linux-libre: ath9k wifi devices may not work with linux-libre 5.7.6

Thursday 2nd of July 2020 01:49:09 AM

if you have a USB wifi device which uses the ath9k or ath9k_htc kernel module, you should postpone upgrading to any of the 5.7.6 kernels; or the device may not work when you next reboot - PCI devices do not seem to be affected by this bug

watch this bug report for further details

linux-libre and linux-libre-headers 5.7.6 have been pulled from the repos and replaced with 5.7.2; but other kernels remain at 5.7.6 - if you have already upgraded to one of the 5.7.6 kernels, and your wifi does not work, you will need to revert to the previous kernel:

# pacman -Syuu linux-libre

or boot a parabola LiveISO, mount your / partition, and install it with pacstrap:

# mount /dev/sdXN /mnt # pacstrap /mnt linux-libre

GNU Guix: Securing updates

Wednesday 1st of July 2020 05:40:00 PM

Software deployment tools like Guix are in a key position when it comes to securing the “software supply chain”—taking source code fresh from repositories and providing users with ready-to-use binaries. We have been paying attention to several aspects of this problem in Guix: authentication of pre-built binaries, reproducible builds, bootstrapping, and security updates.

A couple of weeks ago, we addressed the elephant in the room: authentication of Guix code itself by guix pull, the tool that updates Guix and its package collection. This article looks at what we set out to address, how we achieved it, and how it compares to existing work in this area.

What updates should be protected against

The problem of securing distro updates is often viewed through the lens of binary distributions such as Debian, where the main asset to be protected are binaries themselves. The functional deployment model that Guix and Nix implement is very different: conceptually, Guix is a source distribution, like Gentoo if you will.

Pre-built binaries are of course available and very useful, but they’re optional; we call them substitutes because they’re just that: substitutes for local builds. When you do choose to accept substitutes, they must be signed by one of the keys you authorized (this has been the case since version 0.6 in 2014).

Guix consists of source code for the tools as well as all the package definitions—the distro. When users run guix pull, what happens behind the scene is equivalent to git clone or git pull. There are many ways this can go wrong. An attacker can trick the user into pulling code from an alternate repository that contains malicious code or definitions for backdoored packages. This is made more difficult by the fact that code is fetched over HTTPS from Savannah by default. If Savannah is compromised (as happened in 2010), an attacker can push code to the Guix repository, which everyone would pull. The change might even go unnoticed and remain in the repository forever. An attacker with access to Savannah can also reset the main branch to an earlier revision, leading users to install outdated software with known vulnerabilities—a downgrade attack. These are the kind of attacks we want to protect against.

Authenticating Git checkouts

If we take a step back, the problem we’re trying to solve is not specific to Guix and to software deployment tools: it’s about authenticating Git checkouts. By that, we mean that when guix pull obtains code from Git, it should be able to tell that all the commits it fetched were pushed by authorized developers of the project. We’re really looking at individual commits, not tags, because users can choose to pull arbitrary points in the commit history of Guix and third-party channels.

Checkout authentication requires cryptographically signed commits. By signing a commit, a Guix developer asserts that they are the one who made the commit; they may be its author, or they may be the person who applied somebody else’s changes after review. It also requires a notion of authorization: we don’t simply want commits to have a valid signature, we want them to be signed by an authorized key. The set of authorized keys changes over time as people join and leave the project.

To implement that, we came up with the following mechanism and rule:

  1. The repository contains a .guix-authorizations file that lists the OpenPGP key fingerprints of authorized committers.
  2. A commit is considered authentic if and only if it is signed by one of the keys listed in the .guix-authorizations file of each of its parents. This is the authorization invariant.

(Remember that Git commits form a directed acyclic graph (DAG) where each commit can have zero or more parents; merge commits have two parent commits, for instance. Do not miss Git for Computer Scientists for a pedagogical overview!)

Let’s take an example to illustrate. In the figure below, each box is a commit, and each arrow is a parent relationship:

This figure shows two lines of development: the orange line may be the main development branch, while the purple line may correspond to a feature branch that was eventually merged in commit F. F is a merge commit, so it has two parents: D and E.

Labels next to boxes show who’s in .guix-authorizations: for commit A, only Alice is an authorized committer, and for all the other commits, both Bob and Alice are authorized committers. For each commit, we see that the authorization invariant holds; for example:

  • commit B was made by Alice, who was the only authorized committer in its parent, commit A;
  • commit C was made by Bob, who was among the authorized committers as of commit B;
  • commit F was made by Alice, who was among the authorized committers of both parents, commits D and E.

The authorization invariant has the nice property that it’s simple to state, and it’s simple to check and enforce. This is what guix pull implements. If your current Guix, as returned by guix describe, is at commit A and you want to pull to commit F, guix pull traverses all these commits and checks the authorization invariant.

Once a commit has been authenticated, all the commits in its transitive closure are known to be already authenticated. guix pull keeps a local cache of the commits it has previously authenticated, which allows it to traverse only new commits. For instance, if you’re at commit F and later update to a descendant of F, authentication starts at F.

Since .guix-authorizations is a regular file under version control, granting or revoking commit authorization does not require special support. In the example above, commit B is an authorized commit by Alice that adds Bob’s key to .guix-authorizations. Revocation is similar: any authorized committer can remove entries from .guix-authorizations. Key rotation can be handled similarly: a committer can remove their former key and add their new key in a single commit, signed by the former key.

The authorization invariant satisfies our needs for Guix. It has one downside: it prevents pull-request-style workflows. Indeed, merging the branch of a contributor not listed in .guix-authorizations would break the authorization invariant. It’s a good tradeoff for Guix because our workflow relies on patches carved into stone tablets (patch tracker), but it’s not suitable for every project out there.

Bootstrapping

The attentive reader may have noticed that something’s missing from the explanation above: what do we do about commit A in the example above? In other words, which commit do we pick as the first one where we can start verifying the authorization invariant?

We solve this bootstrapping issue by defining channel introductions. Previously, one would identify a channel simply by its URL. Now, when introducing a channel to users, one needs to provide an additional piece of information: the first commit where the authorization invariant holds, and the fingerprint of the OpenPGP key used to sign that commit (it’s not strictly necessary but provides an additional check). Consider this commit graph:

On this figure, B is the introduction commit. Its ancestors, such as A are considered authentic. To authenticate, C, D, E, and F, we check the authorization invariant.

As always when it comes to establishing trust, distributing channel introductions is very sensitive. The introduction of the official guix channel is built into Guix. Users obtain it when they install Guix the first time; hopefully they verify the signature on the Guix tarball or ISO image, as noted in the installation instructions, which reduces chances of getting the “wrong” Guix, but it is still very much trust-on-first-use (TOFU).

For signed third-party channels, users have to provide the channel’s introduction in their channels.scm file, like so:

(channel (name 'my-channel) (url "https://example.org/my-channel.git") (introduction (make-channel-introduction "6f0d8cc0d88abb59c324b2990bfee2876016bb86" (openpgp-fingerprint "CABB A931 C0FF EEC6 900D 0CFB 090B 1199 3D9A EBB5"))))

The guix describe command now prints the introduction if there’s one. That way, one can share their channel configuration, including introductions, without having to be an expert.

Channel introductions also solve another problem: forks. Respecting the authorization invariant “forever” would effectively prevent “unauthorized” forks—forks made by someone who’s not in .guix-authorizations. Someone publishing a fork simply needs to emit a new introduction for their fork, pointing to a different starting commit.

Last, channel introductions give a point of reference: if an attacker manipulates branch heads on Savannah to have them point to unrelated commits (such as commits on an orphan branch that do not share any history with the “official” branches), authentication will necessarily fail as it stumbles upon the first unauthorized commit made by the attacker. In the figure above, the red branch with commits G and H cannot be authenticated because it starts from A, which lacks .guix-authorizations and thus fails the authorization invariant.

That’s all for authentication! I’m glad you read this far. At this point you can take a break or continue with the next section on how guix pull prevents downgrade attacks.

Downgrade attacks

An important threat for software deployment tools is downgrade or roll-back attacks. The attack consists in tricking users into installing older, known-vulnerable software packages, which in turn may offer new ways to break into their system. This is not strictly related to the authentication issue we’ve been discussing, except that it’s another important issue in this area that we took the opportunity to address.

Guix saves provenance info for itself: guix describe prints that information, essentially the Git commits of the channels used during git pull:

$ guix describe Generation 149 Jun 17 2020 20:00:14 (current) guix 8b1f7c0 repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: 8b1f7c03d239ca703b56f2a6e5f228c79bc1857e

Thus, guix pull, once it has retrieved the latest commit of the selected branch, can verify that it is doing a fast-forward update in Git parlance—just like git pull does, but compared to the previously-deployed Guix. A fast-forward update is when the new commit is a descendant of the current commit. Going back to the figure above, going from commit A to commit F is a fast-forward update, but going from F to A or from D to E is not.

Not doing a fast-forward update would mean that the user is deploying an older version of the Guix currently used, or deploying an unrelated version from another branch. In both cases, the user is at risk of ending up installing older, vulnerable software.

By default guix pull now errors out on non-fast-forward updates, thereby protecting from roll-backs. Users who understand the risks can override that by passing --allow-downgrades.

Authentication and roll-back prevention allow users to safely refer to mirrors of the Git repository. If git.savannah.gnu.org is down, one can still update by fetching from a mirror, for instance with:

guix pull --url=https://github.com/guix-mirror/guix

If the repository at this URL is behind what the user already deployed, or if it’s not a genuine mirror, guix pull will abort. In other cases, it will proceed.

Unfortunately, there is no way to answer the general question “is X the latest commit of branch B ?”. Rollback detection prevents just that, rollbacks, but there’s no mechanism in place to tell whether a given mirror is stale. To mitigate that, channel authors can specify, in the repository, the channel’s primary URL. This piece of information lives in the .guix-channel file, in the repository, so it’s authenticated. guix pull uses it to print a warning when the user pulls from a mirror:

$ guix pull --url=https://github.com/guix-mirror/guix Updating channel 'guix' from Git repository at 'https://github.com/guix-mirror/guix'... Authenticating channel 'guix', commits 9edb3f6 to 3e51f9e (44 new commits)... guix pull: warning: pulled channel 'guix' from a mirror of https://git.savannah.gnu.org/git/guix.git, which might be stale Building from this channel: guix https://github.com/guix-mirror/guix 3e51f9e …

So far we talked about mechanics in a rather abstract way. That might satisfy the graph theorist or the Git geek in you, but if you’re up for a quick tour of the implementation, the next section is for you!

A long process

We’re kinda celebrating these days, but the initial bug report was opened… in 2016. One of the reasons was that we were hoping the general problem was solved already and we’d “just” have to adapt what others had done. As for the actual design: you would think it can be implemented in ten lines of shell script invoking gpgv and git. Perhaps that’s a possibility, but the resulting performance would be problematic—keep in mind that users may routinely have to authenticate hundreds of commits. So we took a long road, but the end result is worth it. Let’s recap.

Back in April 2016, committers started signing commits, with a server-side hook prohibiting unsigned commits. In July 2016, we had proof-of-concept libgit2 bindings with the primitives needed to verify signatures on commits, passing them to gpgv; later Guile-Git was born, providing good coverage of the libgit2 interface. Then there was a two-year hiatus during which no code was produced in that area.

Everything went faster starting from December 2019. Progress was incremental and may have been hard to follow, even for die-hard Guix hackers, so here are the major milestones:

Whether you’re a channel author or a user, the feature is now fully documented in the manual, and we’d love to get your feedback!

SHA-1

We can’t really discuss Git commit signing without mentioning SHA-1. The venerable crytographic hash function is approaching end of life, as evidenced by recent breakthroughs. Signing a Git commit boils down to signing a SHA-1 hash, because all objects in the Git store are identified by their SHA-1 hash.

Git now relies on a collision attack detection library to mitigate practical attacks. Furthermore, the Git project is planning a hash function transition to address the problem.

Some projects such as Bitcoin Core choose to not rely on SHA-1 at all. Instead, for the commits they sign, they include in the commit log the SHA512 hash of the tree, which the verification scripts check.

Computing a tree hash for each commit in Guix would probably be prohibitively costly. For now, for lack of a better solution, we rely on Git’s collision attack detection and look forward to a hash function transition.

As for SHA-1 in an OpenPGP context: our authentication code rejects SHA-1 OpenPGP signatures, as recommended.

Related work

A lot of work has gone into securing the software supply chain, often in the context of binary distros, sometimes in a more general context; more recent work also looks into Git authentication and related issues. This section attempts to summarize how Guix relates to similar work that we’re aware of in these two areas. More detailed discussions can be found in the issue tracker.

The Update Framework (TUF) is a reference for secure update systems, with a well-structured spec and a number of implementations. TUF is a great source of inspiration to think about this problem space. Many of its goals are shared by Guix. Not all the attacks it aims to protect against (Section 1.5.2 of the spec) are addressed by what’s presented in this post: indefinite freeze attacks, where updates never become available, are not addressed per se (though easily observable), and slow retrieval attacks aren’t addressed either. The notion of role is also something currently missing from the Guix authentication model, where any authorized committer can touch any files, though the model and .guix-authorizations format leave room for such an extension.

However, both in its goals and system descriptions, TUF is biased towards systems that distribute binaries as plain files with associated meta-data. That creates a fundamental impedance mismatch. As an example, attacks such as fast-forward attacks or mix-and-match attacks don’t apply in the context of Guix; likewise, the repository depicted in Section 3 of the spec has little in common with a Git repository.

Developers of OPAM, the OCaml package manager, adapted TUF for use with their Git-based package repository, later updated to write Conex, a separate tool to authenticate OPAM repositories. OPAM is interesting because like Guix it’s a source distro and its package repository is a Git repository containing “build recipe”. To date, it appears that opam update itself does not authenticate repositories though; it’s up to users or developer to run Conex.

Another very insightful piece of work is the 2016 paper On omitting commits and committing omissions. The paper focuses on the impact of malicious modifications to Git repository meta-data. An attacker with access to the repository can modify, for instance, branch references, to cause a rollback attack or a “teleport” attack, causing users to pull an older commit or an unrelated commit. As written above, guix pull would detect such attacks. However, guix pull would fail to detect cases where metadata modification does not yield a rollback or teleport, yet gives users a different view than the intended one—for instance, a user is directed to an authentic but different branch rather than the intended one. The “secure push” operation and the associated reference state log (RSL) the authors propose would be an improvement.

Wrap-up and outlook

Guix now has a mechanism that allows it to authenticate updates. If you’ve run guix pull recently, perhaps you’ve noticed additional output and a progress bar as new commits are being authenticated. Apart from that, the switch has been completely transparent. The authentication mechanism is built around the commit graph of Git; in fact, it’s a mechanism to authenticate Git checkouts and in that sense it is not tied to Guix and its application domain. It is available not only for the main guix channel, but also for third-party channels.

To bootstrap trust, we added the notion of channel introductions. These are now visible in the user interface, in particular in the output of guix describe and in the configuration file of guix pull and guix time-machine. While channel configuration remains a few lines of code that users typically paste, this extra bit of configuration might be intimidating. It certainly gives an incentive to provide a command-line interface to manage the user’s list of channels: guix channel add, etc.

The solution here is built around the assumption that Guix is fundamentally a source-based distribution, and is thus completely orthogonal to the public key infrastructure (PKI) Guix uses for the signature of substitutes. Yet, the substitute PKI could probably benefit from the fact that we now have a secure update mechanism for the Guix source code: since guix pull can securely retrieve a new substitute signing key, perhaps it could somehow handle substitute signing key revocation and delegation automatically? Related to that, channels could perhaps advertise a substitute URL and its signing key, possibly allowing users to register those when they first pull from the channel. All this requires more thought, but it looks like there are new opportunities here.

Until then, if you’re a user or a channel author, we’d love to hear from you! We’ve already gotten feedback that these new mechanisms broke someone’s workflow; hopefully it didn’t break yours, but either way your input is important in improving the system. If you’re into security and think this design is terrible or awesome, please do provide feedback.

It’s a long and article describing a long ride on a path we discovered as we went, and it felt like an important milestone to share!

Acknowledgments

Thanks to everyone who provided feedback, ideas, or carried out code review during this long process, notably (in no particular order): Christopher Lemmer Webber, Leo Famulari, David Thompson, Mike Gerwitz, Ricardo Wurmus, Werner Koch, Justus Winter, Vagrant Cascadian, Maxim Cournoyer, Simon Tournier, John Soo, and Jakub Kądziołka. Thanks also to janneke, Ricardo, Marius, and Simon for reviewing an earlier draft of this post.

About GNU Guix

GNU Guix is a transactional package manager and an advanced distribution of the GNU system that respects user freedom. Guix can be used on top of any system running the kernel Linux, or it can be used as a standalone operating system distribution for i686, x86_64, ARMv7, and AArch64 machines.

In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. When used as a standalone GNU/Linux distribution, Guix offers a declarative, stateless approach to operating system configuration management. Guix is highly customizable and hackable through Guile programming interfaces and extensions to the Scheme language.

Christopher Allan Webber: Some updates: CapTP in progress, Datashards, chiptune experiments, etc

Tuesday 30th of June 2020 11:54:40 PM

(Originally written as a post for Patreon donors.)

Hello... just figured I'd give a fairly brief update. Since I wrote my last post I've been working hard towards the distributed programming stuff in Goblins.

In general, this involves implementing a protocol called CapTP, which is fairly obscure... the idea is generally to apply the same "object capability security" concept that Goblins already follows but on a networked protocol level. Probably the most prominent other implementation of CapTP right now is being done by the Agoric folks, captp.js. I've been in communication with them... could we achieve interoperability between our implementations? It could be cool, but it's too early to tell. Anyway it's one of those technical areas that's so obscure that I decided to document my progress on the cap-talk mailing list, but that's becoming the length of a small novel... so I guess, beware before you try to read that whole thing. I'm far enough along where the main things work, but not quite everything (CapTP supports such wild things as distributed garbage collection...!!!!)

Anyway, in general I don't think that people get too excited by hearing "backend progress is happening"; I believe that implementing CapTP is even more important than standardizing ActivityPub was in the long run of my life work, but I also am well aware that in general people (including myself!) understand best by seeing an interesting demonstration. So, I do plan another networked demo, akin to the time-travel Terminal Phase demo, but I'm not sure just how fancy it will be (yet). I think I'll have more things to show on that front in 1-2 months.

(Speaking of Goblins and games, I'm putting together a little library called Game Goblin to make making games on top of Goblins a bit easier; it isn't quite ready yet but thought I'd mention it. It's currently going through some "user testing".)

More work is happening on the Datashards front; Serge Wroclawski (project leader for Datashards; I guess you could say I'm "technical engineer") and I have started assembling more documentation and have put together some proto-standards documents. (Warning: WIP WIP WIP!!!) We are exploring with a standards group whether or not Datashards would be a good fit there, but it's too early to talk about that since the standards group is still figuring it out themselves. Anyway, it's taken up a good chunk of time so I figured it was worth mentioning.

So, more to come, and hopefully demos not too far ahead.

But let's end on a fun note. In-between all that (and various things at home, of course), I have taken a bit of what might resemble "downtime" and I'm learning how to make ~chiptunes / "tracker music" with Milkytracker, which is just a lovely piece of software. (I've also been learning more about sound theory and have been figuring out how to compose some of my own samples/"instruments" from code.) Let me be clear, I'm not very good at it, but it's fun to learn a new thing. Here's a dollhouse piano thing (XM file), the start of a haunted video game level (XM file), a sound experiment representing someone interacting with a computer (XM file), and the mandatory demonstration that I've figured out how to do C64-like phase modulation and arpeggios (XM file). Is any of that stuff... "good"? Not really, all pretty amateurish, but maybe in a few months of off-hour experiments it won't be... so maybe some of my future demos / games won't be quite as quiet! ;)

Hope everyone's doing ok out there...

GNU Taler news: Exchange independent security audit report published

Tuesday 30th of June 2020 10:00:00 PM
2020-07: Exchange external security audit completed

We received a grant from NLnet foundation to pay for an external security audit of the GNU Taler exchange cryptography, code and documentation. CodeBlau now concluded their audit. You can find the final report here. We have compiled a preliminary response detailing what changes we have already made and which changes we are still planning to make in the future. We thank CodeBlau for their work, and NLnet and the European Commission's Horizion 2020 NGI initiative for funding this work.

GNUnet News: GNS Specification Milestone 3/4

Monday 29th of June 2020 10:00:00 PM

GNS Technical Specification Milestone 3/4

We are happy to announce the completion of the third milestone for the GNS Specification. The third milestone consists of documenting the GNS zone revocation process. As part of this, we have reworked the proof-of-work algorithms in GNUnet also used for GNS revocations.
The (protocol breaking) changes will be released as part of GNUnet 0.13.0. The specification document LSD001 can be found at:

In preparation for the fourth and last milestone, we have started the IETF process to find a working group and expect to present our work initially at IETF 108.

This work is generously funded by NLnet as part of their Search and discovery fund.

FSF Blogs: June GNU Spotlight with Mike Gerwitz: Twelve new releases!

Monday 29th of June 2020 03:16:08 PM

For announcements of most new GNU releases, subscribe to the info-gnu mailing list: https://lists.gnu.org/mailman/listinfo/info-gnu.

To download: nearly all GNU software is available from https://ftp.gnu.org/gnu/, or preferably one of its mirrors from https://www.gnu.org/prep/ftp.html. You can use the URL https://ftpmirror.gnu.org/ to be automatically redirected to a (hopefully) nearby and up-to-date mirror.

This month, we welcome Jacob Bachmeyer as co-maintainer of DejaGnu.

A number of GNU packages, as well as the GNU operating system as a whole, are looking for maintainers and other assistance: please see https://www.gnu.org/server/takeaction.html#unmaint if you'd like to help. The general page on how to help GNU is at https://www.gnu.org/help/help.html.

If you have a working or partly working program that you'd like to offer to the GNU project as a GNU package, see https://www.gnu.org/help/evaluation.html.

GNU Guile: GNU Guile 3.0.4 released

Wednesday 24th of June 2020 03:00:00 PM

We are pleased but also embarrassed to announce GNU Guile 3.0.4. This release fixes the SONAME of libguile-3.0.so, which was wrongfully bumped in 3.0.3. Distributions should use 3.0.4.

Apologies for the inconvenience!

parallel @ Savannah: GNU Parallel 20200622 ('Floyd') released

Monday 22nd of June 2020 10:05:25 PM

GNU Parallel 20200622 ('Floyd') has been released. It is available for download at: http://ftpmirror.gnu.org/parallel/

Quote of the month:

  Who needs spark when GNU Parallel exists
    -- MatthijsB @MatthijsBrs@twitter

New in this release:

  • No new functionality
  • Bug fixes and man page updates.

News about GNU Parallel:

  • Bug fixes and man page updates.

Get the book: GNU Parallel 2018 http://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html

GNU Parallel - For people who live life in the parallel lane.

About GNU Parallel

GNU Parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU Parallel can then split the input and pipe it into commands in parallel.

If you use xargs and tee today you will find GNU Parallel very easy to use as GNU Parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU Parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU Parallel can even replace nested loops.

GNU Parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU Parallel as input for other programs.

For example you can run this to convert all jpeg files into png and gif files and have a progress bar:

  parallel --bar convert {1} {1.}.{2} ::: *.jpg ::: png gif

Or you can generate big, medium, and small thumbnails of all jpeg files in sub dirs:

  find . -name '*.jpg' |
    parallel convert -geometry {2} {1} {1//}/thumb{2}_{1/} :::: - ::: 50 100 200

You can find more about GNU Parallel at: http://www.gnu.org/s/parallel/

You can install GNU Parallel in just 10 seconds with:

    $ (wget -O - pi.dk/3 || lynx -source pi.dk/3 || curl pi.dk/3/ || \
       fetch -o - http://pi.dk/3 ) > install.sh
    $ sha1sum install.sh | grep 3374ec53bacb199b245af2dda86df6c9
    12345678 3374ec53 bacb199b 245af2dd a86df6c9
    $ md5sum install.sh | grep 029a9ac06e8b5bc6052eac57b2c3c9ca
    029a9ac0 6e8b5bc6 052eac57 b2c3c9ca
    $ sha512sum install.sh | grep f517006d9897747bed8a4694b1acba1b
    40f53af6 9e20dae5 713ba06c f517006d 9897747b ed8a4694 b1acba1b 1464beb4
    60055629 3f2356f3 3e9c4e3c 76e3f3af a9db4b32 bd33322b 975696fc e6b23cfb
    $ bash install.sh

Watch the intro video on http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Walk through the tutorial (man parallel_tutorial). Your command line will love you for it.

When using programs that use GNU Parallel to process data for publication please cite:

O. Tange (2018): GNU Parallel 2018, March 2018, https://doi.org/10.5281/zenodo.1146014.

If you like GNU Parallel:

  • Give a demo at your local user group/team/colleagues
  • Post the intro videos on Reddit/Diaspora*/forums/blogs/ Identi.ca/Google+/Twitter/Facebook/Linkedin/mailing lists
  • Get the merchandise https://gnuparallel.threadless.com/designs/gnu-parallel
  • Request or write a review for your favourite blog or magazine
  • Request or build a package for your favourite distribution (if it is not already there)
  • Invite me for your next conference

If you use programs that use GNU Parallel for research:

  • Please cite GNU Parallel in you publications (use --citation)

If GNU Parallel saves you money:

About GNU SQL

GNU sql aims to give a simple, unified interface for accessing databases through all the different databases' command line clients. So far the focus has been on giving a common way to specify login information (protocol, username, password, hostname, and port number), size (database and table size), and running queries.

The database is addressed using a DBURL. If commands are left out you will get that database's interactive shell.

When using GNU SQL for a publication please cite:

O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.

About GNU Niceload

GNU niceload slows down a program when the computer load average (or other system activity) is above a certain limit. When the limit is reached the program will be suspended for some time. If the limit is a soft limit the program will be allowed to run for short amounts of time before being suspended again. If the limit is a hard limit the program will only be allowed to run when the system is below the limit.

FSF Blogs: Software patents hold back progress -- even in healthcare

Monday 22nd of June 2020 08:30:00 PM

On Friday May 8th, the USPTO announced the COVID-19 Prioritized Examination Pilot Program. Doctored up to look like a helpful response to a global pandemic, it's actually the exact opposite. Under the program, the USPTO will waive some fees associated with accelerated application review for patents on works that require US Food and Drug Administration (FDA) approval. They'll also work to try and get these applications granted within six months. These changes will make it easier and faster for people to gain patents on any technology related to the pandemic, including patents on software. It's not in our scope to determine the impact of other kinds of patents, but we know specifically that they are terrible for software, and at a time where software is critical to saving lives, expediting software patent applications will only cause harm.

To be clear, this program does not speed up FDA approval, or help to get lifesaving technology to the people who need it most. It doesn't create supply chains or help fund the development of medical technologies and software. All it does is make it easier for someone to "own" that technology, to make it quicker and cheaper to restrict others from implementing and sharing tools that people need to survive. It rushes the patent application process so that someone could be able to sue others trying to save seriously ill patients around the world before the global pandemic is over.

While the crisis was unfurling, the GNOME Foundation was still expending resources fighting off a patent suit started in 2019. On May 20, 2020, the GNOME Foundation succeeding in securing a release and covenant not to sue from the patent aggressor for all software released under a free license. This was a major win for software freedom that took months and months to realize. But the threat remains, and the only reason someone would need to get their patent granted sooner is because they want to start their lawsuits sooner, to disrupt the flow of medical technology in order to siphon off profits from those seeking to alleviate the worst pandemic in a century.

Taken together with a recently released report from the USPTO patting themselves on the back for ignoring the US Supreme Court in order to increase the number of software patents, it's clear that the USPTO views its mandate as requiring them to crank out as many patents as they possibly can. In Alice v. CLS Bank, the US Supreme Court limited the patent eligibility of software implemented on a general purpose computer. As the USPTO report stated, this increased the number of patent applications receiving initial rejections. So last year the USPTO released "guidance" that weakened those limitations, and now the number of software patents speeding through its pipeline has increased once again.

Even in normal times, increasing software patent restriction is harmful to everyone. Software patents threaten all developers, putting them in the line of fire for ruinous lawsuits simply for creating and sharing their own code. Until the day when we can completely end all software patents, we should be reducing the harm done, not coming up with programs and guidance to accelerate the damage. But in the midst of a global crisis where hundreds of thousands have already died, where supply chains for medical technology are stretched beyond limits, creating a cheaper and easier fast track for causing further patent disruption is downright criminal.

While the USPTO is pretending to help with the response to COVID-19, it is actually throwing a supercharged wrench into the gears of medical supply distribution, so we the people have to come up with our own response that actually helps. Software patents cut people off from one another, but if we're going to respond adequately and humanely to this crisis, we need to all work together. That is why we are asking you to join our COVID-19 Response Team on the Free Software Directory. The Free Software Directory is a volunteer-run catalog of over 16,000 free software packages. It helps users to find software that they can modify and share with their friends. The ethics of free software are more important than ever, and we need to help people gain access to software that respects their rights. On the COVID-19 Response Team, we're directing our efforts towards ensuring that everyone has access to medical software, 3D printing designs of medical technology, and other tools that will be useful in the fight against this global pandemic. You can jump in and start working right away, or join us every Friday from 12 pm to 3pm EDT in the #fsf irc channel on Freenode.

Software patents are another kind of disease, and the USPTO has decided to become a super-spreader of that infection, which will in turn worsen the physical infections. Join together with us to fight back. We hope to see you every Friday, but here are some other ways that you can help:

health @ Savannah: GNU Health starts the migration to Weblate

Monday 22nd of June 2020 11:32:33 AM

Dear all

Our friends from Weblate (www.weblate.org) have provided hosting for GNU Health!

The import of the strings for the official languages is already there (https://hosted.weblate.org/projects/gnu-health/).

During the coming days, we'll organize the teams, adapt the scripts, links to mercurial, etc..

The server translate.gnusolidario.org is now deprecated and will no longer be active by July 1st.

Let me take the opportunity to thank the great Pootle community, that has been the translation system of GNU Health for many years! Your contribution has been immense.

We are looking forward to this new translation experience with Weblate.

All the best
Luis

mit-scheme @ Savannah: Test release 11.0.90 is available

Monday 22nd of June 2020 05:28:06 AM

Please try it out and report any bugs.

This will become the 11.1 release.

GNU Guile: GNU Guile 3.0.3 released

Sunday 21st of June 2020 09:20:00 PM

We are pleased to announce GNU Guile 3.0.3, the third bug-fix release of the new 3.0 stable series! This release represents 170 commits by 17 people since version 3.0.2.

The highlight of this release is the addition of a new baseline compiler, used at optimizations levels -O1 and -O0. The baseline compiler is designed to generate code fast, for applications where compilation speed matters more than execution time of the generated code. It is around ten times faster than the optimizing continuation-passing style (CPS) compiler.

This version also includes a new pipeline procedure to create shell-like process pipelines, improvements to the bitvector interface, and bug fixes for JIT compilation on ARMv7 machines.

See the release announcement for details and the download page to give it a go!

www-zh-cn @ Savannah: GNU CTT posts an openning position.

Sunday 21st of June 2020 01:39:55 AM

Dear volunteers:
Dear visitors:
Dear GNU lovers:

We are GNU Chinese Translators Team (GNU CTT). Our goal is to translate the http://www.gnu.org webpages into Chinese. According to
https://www.gnu.org/help/help.html

每个翻译团队需要多个母语为目标语言的成员(英语熟练),还需要至少一个母语为英语的成员(目标语言熟练)。

GNU CTT 目前缺少母语是英语的成员。如果您想提供帮助,请发送邮件至 <a href="mailto:web-translators@gnu.org">&lt;web-translators@gnu.org&gt;</a>,或订阅我们的邮件列表<a href="mailto:www-zh-cn-translators@gnu.org">&lt;GNU CTT translators&gt;</a>。

Thank you.
GNU CTT

www @ Savannah: GNU is Looking for Native English Speakers for Proofreading and Translations

Saturday 20th of June 2020 06:47:54 PM

We at the GNU Project love languages. Here are two main things we do in this area and how you can join to help.

  • To bring our message of software freedom to the widest possible audience, we translate http://www.gnu.org into as many languages as we can. Our volunteer translation teams do a wonderful job and more help is always welcome. Apart from native speakers of the target language, these teams also need at least one member that is a native speaker of English and fairly fluent in the target language. Find more about the GNU translation teams and contact them directly or write to <web-translators@gnu.org> if you would like to help.
  • English is the default language of the GNU Project. The vast majority of our web pages, manuals and documentation are originally written in English. Occasionally, we like to have the material proofread by native English speakers with a good command of the written language to ensure quality and readability. To help with this, please subscribe to our low-traffic  GNU documentation proofreaders list.

Other ways to  help GNU

health @ Savannah: Security: GNU Health HMIS Control Center 3.6.4 is out!

Saturday 20th of June 2020 04:25:49 PM

Dear all
The GH control center (gnuhealth-control) version 3.6.4 is out, fixing some minor security issues found by the openSUSE security team[1]

You just need to to the following:

1) Login as GNU Health
2) cdutil
3) ./gnuhealth-control update

That is all. The new gnuhealth-control center is now installed. To verify, please check it with

./gnuhealth-control version

If the update reports that, in addition to the gnuhealth-control, there are other components that need to be upgraded, please re-run gnuhealth-control and update the instance, as explained in the wikibook[2]

Please remember to submit any possible vulnerabilities to security@gnuhealth.org

Have a great weekend!
1.- https://bugzilla.opensuse.org/show_bug.cgi?id=1167128#c13
2.- https://en.wikibooks.org/wiki/GNU_Health/Control_Center

GNU Guix: Guix Further Reduces Bootstrap Seed to 25%

Monday 15th of June 2020 12:00:00 PM

We are delighted to announce that the second reduction by 50% of the Guix bootstrap binaries has now been officially released!

The initial set of binaries from which packages are built now weighs in at approximately 60~MiB, a quarter of what it used to be.

In a previous blog post we elaborate on why this reduction and bootstrappability in general is so important. One reason is to eliminate---or greatly reduce the attack surface of---a “trusting trust” attack. Last summer at the Breaking Bitcoin conference, Carl Dong gave a fun and remarkably gentle introduction and at FOSDEM2020 I also gave a short talk about this. If you choose to believe that building from source is the proper way to do computing, then it follows that the “trusting trust” attack is only a symptom of an incomplete or missing bootstrap story.

Further Reduced Binary Seed bootstrap

Last year, the first reduction removed the GCC, glibc and Binutils binary seeds. The new Further Reduced Binary Seed bootstrap, merged in Guix master last month, removes the “static-binaries tarball” containing GNU Awk, Bash, Bzip2, the GNU Core Utilities, Grep, Gzip, GNU Make, Patch, sed, Tar, and Xz. It replaces them by Gash and Gash Core Utils. Gash is a minimalist POSIX shell written in Guile Scheme, while Gash Core Utils is a Scheme implementation for most of the tools found in GNU Coreutils, as well as the most essential bits of Awk, grep and sed.

After three new GNU Mes releases with numerous Mes C Library updates and fixes, a major update of Gash and the first official Gash Utils release, and the delicate balancing of 17 new bootstrap source packages and versions, the bottom of the package graph now looks like this (woohoo!):

gcc-mesboot (4.9.4) ^ | (...) ^ | binutils-mesboot (2.14), glibc-mesboot (2.2.5), gcc-core-mesboot (2.95.3) ^ | bash-mesboot (2.05), bzip2-mesboot, gawk-mesboot (3.0.0) diffutils-mesboot (2.7), patch-mesboot (2.5.9), sed-mesboot (1.18) ^ | gnu-make-mesboot (3.82) ^ | gzip-mesboot (1.2.4) ^ | tcc-boot ^ | mes-boot ^ | gash-boot, gash-utils-boot ^ | * bootstrap-mescc-tools, bootstrap-mes (~12 MiB) bootstrap-guile (~48 MiB)

full graph

We are excited that the Nlnet Foundation has sponsored this work!

However, we aren't done yet; far from it.

Lost Paths

The idea of reproducible builds and bootstrappable software is not very new. Much of that was implemented for the GNU tools in the early 1990s. Working to recreate it in present time shows us much of that practice was forgotten.

Readers who are familiar with the GNU toolchain may have noticed the version numbers of the *-mesboot source packages in this great new bootstrap: They are ancient! That's a problem.

Typically, newer versions of the tool chain fix all kinds of bugs, make the software easier to build and add support for new CPU architectures, which is great. However---more often than not--- simultaneously new features are introduced or dependencies are added that are not necessary for bootstrapping and may increase the bootstrap hurdle. Sometimes, newer tools are more strict or old configure scripts do not recognise newer tool versions.

A trivial example is GNU sed. In the current bootstrap we are using version 1.18, which was released in 1993. Until recently the latest version of sed we could hope to bootstrap was sed-4.2.2 (2012). Newer releases ship as xz-compressed tarballs only, and xz is notoriously difficult to bootstrap (it needs a fairly recent GCC and try building that without sed).

Luckily, the sed maintainers (Jim Meyering) were happy to correct this mistake and starting from release sed-4.8 (2020) also gzip-compressed tarballs will be shipped. Similar for the GNU Core Utils: Releases made between 2011 and 2019 will probably be useless for bootstrapping. Confronted with this information, also the coreutils maintainers (Pádraig Brady) were happy to release coreutils-8.32 also in gzip compression from now on.

Even these simple cases show that solving bootstrap problems can only be done together: For GNU it really is a project-wide responsibility that needs to be addressed.

Most bootstrap problems or loops are not so easy to solve and sometimes there are no obvious answers, for example:

and while these examples make for a delightful puzzle from a bootstrappability perspective, we would love to see the maintainers of GNU softwares to consider bootstrappability and start taking more responsibility for the bootstrap story of their packages.

Towards a Universal, Full Source Bootstrap

Our next target will be a third reduction by ~50%; the Full-Source bootstrap will replace the MesCC-Tools and GNU Mes binaries by Stage0 and M2-Planet.

The Stage0 project by Jeremiah Orians starts everything from ~512 bytes; virtually nothing. Have a look at this incredible project if you haven’t already done so.

We are most grateful and excited that the Nlnet Foundation has again decided to sponsor this work!

While the reduced bootstrap currently only applies to the i686-linux and x86_64-linux architectures, we are thrilled that ARM will be joining soon. The Trusted ARM bootstrapping work is progressing nicely, and GNU Mes is now passing its entire mescc test suite on native ARMv7, and passing nigh its entire gcc test suite on native ARMv7. Work is underway to compile tcc using that GNU Mes. Adding this second architecture is a very important one towards the creation of a universal bootstrap!

Upcoming releases of Gash and Gash-Utils will allow us to clean up the bottom of the package graph and remove many of the “vintage” packages. In particular, the next version of Gash-Utils will be sophisticated enough to build everything up to gcc-mesboot using only old versions of GNU Make and Gzip. This is largely thanks to improvements to the implementation of Awk, which now includes nearly all of the standard features.

Looking even further into the future, we will likely have to remove the “vintage” GCC-2.95.3 that was such a helpful stepping stone and reach straight for GCC-4.6.4. Interesting times ahead!

About Bootstrappable Builds and GNU Mes

Software is bootstrappable when it does not depend on a binary seed that cannot be built from source. Software that is not bootstrappable---even if it is free software---is a serious security risk for a variety of reasons. The Bootstrappable Builds project aims to reduce the number and size of binary seeds to a bare minimum.

GNU Mes is closely related to the Bootstrappable Builds project. Mes aims to create an entirely source-based bootstrapping path for the Guix System and other interested GNU/Linux distributions. The goal is to start from a minimal, easily inspectable binary (which should be readable as source) and bootstrap into something close to R6RS Scheme.

Currently, Mes consists of a mutual self-hosting scheme interpreter and C compiler. It also implements a C library. Mes, the scheme interpreter, is written in about 5,000 lines of code of simple C. MesCC, the C compiler, is written in scheme. Together, Mes and MesCC can compile a lightly patched TinyCC that is self-hosting. Using this TinyCC and the Mes C library, it is possible to bootstrap the entire Guix System for i686-linux and x86_64-linux.

About GNU Guix

GNU Guix is a transactional package manager and an advanced distribution of the GNU system that respects user freedom. Guix can be used on top of any system running the kernel Linux, or it can be used as a standalone operating system distribution for i686, x86_64, ARMv7, and AArch64 machines.

In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. When used as a standalone GNU/Linux distribution, Guix offers a declarative, stateless approach to operating system configuration management. Guix is highly customizable and hackable through Guile programming interfaces and extensions to the Scheme language.

www @ Savannah: Successful Resistance against Nonfree Software in Schools

Sunday 7th of June 2020 03:31:05 AM

The GNU Education Team presents examples of teachers, students, parents, free software advocates and the community at large who are taking action to stop the use of nonfree programs in schools and are succeeding. Stay tuned for new cases.

https://www.gnu.org/education/successful-resistance-against-nonfree-software.html

More in Tux Machines

AMD Ryzen 5 3600XT / Ryzen 7 3800XT / Ryzen 9 3900XT Linux Performance In 130+ Benchmarks

After the AMD Ryzen 3000XT series was announced last month, these new higher-clocked Zen 2 desktop processors are shipping today. Here are 130+ benchmarks on each of the Ryzen 5 3600XT, Ryzen 7 3800XT, and Ryzen 9 3900XT parts compared to various Intel and AMD CPUs. Tests under Ubuntu Linux and also complemented by performance-per-Watt / power and performance-per-dollar data points. These "XT" processors were announced in mid-June as still being Zen 2 based like the rest of the Ryzen 3000 desktop line-up but with slight increases to the base and boost clock frequencies to ratchet up the competition on Intel's new Comet Lake processors. Read more

The Current State of Open-Source Testing Tools

Tricentis and three survey project collaborators recently published findings that provide new insights on global trends in open-source testing. The study focused on the open-source tool testing industry, not how or if companies and software developers test their code. This is an issue that confronts both open-source and commercial or proprietary software. The results revealed that a lack of technical skills is the major roadblock to open-source tool adoption, with 30 percent of organizations surveyed saying they lacked the skills to adopt open-source tools, according to Kevin Dunne, senior vice president for strategic initiatives at Tricentis. Read more

Why I stick with xterm

I use xterm. That's right, xterm. It may seem like an old school choice, and I do use GNOME 3 now as well, but after many years of trying some and ignoring others, then going back to old standbys, I find I don't need (or like) newer stuff like GNOME Terminal. My philosophy: Start simple, improve over time, and aim for productivity. Read more

GNOME 3.37.3 Released With More Features, Code Improvements

GNOME 3.37.3 is out today as the newest development snapshot working towards the September release of GNOME 3.38. GNOME 3.37.3 is another routine development snapshot inching closer to GNOME 3.38. Among the changes with the 3.37.3 milestone include: - The GNOME Web Browser (Epiphany) now supports muting individual tabs, a run-in-background option for web apps, a --search command line option, a dark mode for the view source mode, and a wide range of other fixes/improvements. - The latest GTK4 toolkit development code has added more APIs, a Tracker3-based search engine implementation under the GtkFileChooser, dropping App Menu support from GtkApplication, improving X11 sync when the NVIDIA binary driver is used, various OpenGL renderer improvements, and other changes. Read more