Python: Security and NumPy 1.20 Release

• Python Package Index nukes 3,653 malicious libraries uploaded soon after security shortcoming highlighted

  The Python Package Index, also known as PyPI, has removed 3,653 malicious packages uploaded days after a security weakness in the use of private and public registries was highlighted. Python developers use PyPI to add software libraries written by other developers in their own projects. Other programming languages implement similar package management systems, all of which demand some level of trust. Developers are often advised to review any code they import from an external library though that advice isn't always followed. Package management systems like npm, PyPI, and RubyGems have all had to remove subverted packages in recent years. Malware authors have found that if they can get their code included in popular libraries or applications, they get free distribution and trust they haven't earned. Last month, security researcher Alex Birsan demonstrated how easy it is to take advantage of these systems through a form of typosquatting that exploited the interplay between public and private package registries.

• A pair of Python vulnerabilities [LWN.net]

  Two separate vulnerabilities led to the fast-tracked release of Python 3.9.2 and 3.8.8 on February 19, though source-only releases of 3.7.10 and 3.6.13 came a few days earlier. The vulnerabilities may be problematic for some Python users and workloads; one could potentially lead to remote code execution. The other is, arguably, not exactly a flaw in the Python standard library—it simply also follows an older standard—but it can lead to web cache poisoning attacks. [...] [Update: As pointed out in an email from Moritz Muehlenhoff, Python 2.7 actually is affected by this bug. He notes that python2 on Debian 10 ("Buster") is affected and has been updated. Also, Fedora has a fix in progress for its python2.7 package.]

• NumPy 1.20 has been released

  NumPy is a Python library that adds an array data type to the language, along with providing operators appropriate to working on arrays and matrices. By wrapping fast Fortran and C numerical routines, NumPy allows Python programmers to write performant code in what is normally a relatively slow language. NumPy 1.20.0 was announced on January 30, in what its developers describe as the largest release in the history of the project. That makes for a good opportunity to show a little bit about what NumPy is, how to use it, and to describe what's new in the release. [...] NumPy adds a new data type to Python: the multidimensional ndarray. This a container, like a Python list, but with some crucial differences. A NumPy array is usually homogeneous; while the elements of a list can be of various types, an ndarray will, typically, only contain a single, simple type, such as integers, strings, or floats. However, these arrays can instead contain arbitrary Python objects (i.e. descendants of object). This means that the elements will, for simple data types, all occupy the same amount of space in memory. The elements of an ndarray are laid out contiguously in memory, whereas there is no such guarantee for a list. In this way, they are similar to Fortran arrays. These properties of NumPy arrays are essential for efficiency because the location of each element can be directly calculated. Beyond just adding efficient arrays, NumPy also overloads arithmetic operators to act element-wise on the arrays. This allows the Python programmer to express computations concisely, operating on arrays as units, in many cases avoiding the need to use loops. This does not turn Python into a full-blown array language such as APL, but adds to it a syntax similar to that incorporated into Fortran 90 for array operations.

4 Best Free and Open Source Graphical MPD Clients

MPD is a powerful server-side application for playing music. In a home environment, you can connect an MPD server to a Hi-Fi system, and control the server using a notebook or smartphone. You can, of course, play audio files on remote clients. MPD can be started system-wide or on a per-user basis. MPD runs in the background playing music from its playlist. Client programs communicate with MPD to manipulate playback, the playlist, and the database. The client–server model provides advantages over all-inclusive music players. Clients can communicate with the server remotely over an intranet or over the Internet. The server can be a headless computer located anywhere on a network. There’s graphical clients, console clients and web-based clients. To provide an insight into the quality of software that is available, we have compiled a list of 4 best graphical MPD clients. Hopefully, there will be something of interest here for anyone who wants to listen to their music collection via MPD. Here’s our recommendations. They are all free and open source goodness.

LWN on Kernel: 5.12 Merge, Lockless Algorithms, and opy_file_range()

• 5.12 Merge window, part 1 [LWN.net]

  The beginning of the 5.12 merge window was delayed as the result of severe weather in the US Pacific Northwest. Once Linus Torvalds got going, though, he wasted little time; as of this writing, just over 8,600 non-merge changesets have been pulled into the mainline repository for the 5.12 release — over a period of about two days. As one might imagine, that work contains a long list of significant changes.

• An introduction to lockless algorithms [LWN.net]

  Low-level knowledge of the memory model is universally recognized as advanced material that can scare even the most seasoned kernel hackers; our editor wrote (in the July article) that "it takes a special kind of mind to really understand the memory model". It's been said that the Linux kernel memory model (and in particular Documentation/memory-barriers.txt) can be used to frighten small children, and the same is probably true of just the words "acquire" and "release". At the same time, mechanisms like RCU and seqlocks are in such widespread use in the kernel that almost every developer will sooner or later encounter fundamentally lockless programming interfaces. For this reason, it is a good idea to equip yourself with at least a basic understanding of lockless primitives. Throughout this series I will describe what acquire and release semantics are really about, and present five relatively simple patterns that alone can cover most uses of the primitives.

• How useful should copy_file_range() be? [LWN.net]

  Its job is to copy len bytes of data from the file represented by fd_in to fd_out, observing the requested offsets at both ends. The flags argument must be zero. This call first appeared in the 4.5 release. Over time it turned out to have a number of unpleasant bugs, leading to a long series of fixes and some significant grumbling along the way. In 2019 Amir Goldstein fixed more issues and, in the process, removed a significant limitation: until then, copy_file_range() refused to copy between files that were not located on the same filesystem. After this patch was merged (for 5.3), it could copy between any two files, falling back on splice() for the cross-filesystem case. It appeared that copy_file_range() was finally settling into a solid and useful system call. Indeed, it seemed useful enough that the Go developers decided to use it for the io.Copy() function in their standard library. Then they ran into a problem: copy_file_range() will, when given a kernel-generated file as input, copy zero bytes of data and claim success. These files, which include files in /proc, tracefs, and a large range of other virtual filesystems, generally indicate a length of zero when queried with a system call like stat(). copy_file_range(), seeing that zero length, concludes that there is no data to copy and the job is already done; it then returns success. But there is actually data to be read from this kind of file, it just doesn't show in the advertised length of the file; the real length often cannot be known before the file is actually read. Before 5.3, the prohibition on cross-filesystem copies would have caused most such attempts to return an error code; afterward, they fail but appear to work. The kernel is happy, but some users can be surprisingly stubborn about actually wanting to copy the data they asked to be copied; they were rather less happy.

Banana Pi BPI-M2 Pro is a compact Amlogic S905X3 SBC

Banana Pi has already designed an Amlogic S905X3 SBC with Banana Pi BPI-M5 that closely follows Raspberry Pi 3 Model B form factor, but they’ve now unveiled a more compact model with Banana Pi BPI-M2 Pro that follow the design of the company’ earlier BPI-MP2+ SBC powered by the good old Allwinner H3 processor. BPI-M2 Pro comes with 2GB RAM, 16GB eMMC storage, HDMI video output, Gigabit Ethernet, Wifi & Bluetooth connectivity, as well as two USB 3.0 ports.