Language Selection

English French German Italian Portuguese Spanish

Programming Leftovers

Filed under
Development
  • Quicksort in Python

    Quicksort is a popular sorting algorithm and is often used, right alongside Merge Sort. It's a good example of an efficient sorting algorithm, with an average complexity of O(n logn). Part of its popularity also derives from the ease of implementation.

    We will use simple integers in the first part of this article, but we'll give an example of how to change this algorithm to sort objects of a custom class.

    Quicksort is a representative of three types of sorting algorithms: divide and conquer, in-place, and unstable.

  • GCC 10.0 Status Report (2019-11-18), Stage 3 in effect now
  • GCC 10 Feature Development Is Over - Now The Focus Turns To Bug Fixing

    GCC 10 has moved to its next stage of development that shifts away from feature work to instead general bug fixing with hopes of shipping the GNU Compiler Collection 10 release in the months ahead.

    GCC 10 release manager Richard Biener of SUSE announced this Monday morning that "stage three" development phase.

  • ttdo 0.0.4: Extension

    A first update release to the still very new (and still very small) ttdo package arrived on CRAN today. Introduced about two months ago in September, the ttdo package extends the most excellent (and very minimal / zero depends) unit testing package tinytest by Mark van der Loo with the very clever and well-done diffobj package by Brodie Gaslam.

  • Adventures optimizing a bytecode based scripting language
  • Keeping a simple markdown work-log, via emacs

    For the past few years I've been keeping a work-log of everything I do. I don't often share these, though it is sometimes interesting to be able to paste into a chat-channel "Oh on the 17th March I changed that .."

  • Implement C++ coroutines.
    This patch series is an initial implementation of a coroutine feature,
    expected to be standardised in C++20.
    
    Standardisation status (and potential impact on this implementation):
    ----------------------
    
    The facility was accepted into the working draft for C++20 by WG21 in
    February 2019.  During two following WG21 meetings, design and national
    body comments have been reviewed, with no significant change resulting.
    
    Mature implementations (several years) of this exist in MSVC, clang and
    EDG with some experience using the clang one in production - so that the
    underlying principles are thought to be sound.
    
    At this stage, the remaining potential for change comes from two areas of
    national body comments that were not resolved during the last WG21 meeting:
    (a) handling of the situation where aligned allocation is available.
    (b) handling of the situation where a user wants coroutines, but does not
        want exceptions (e.g. a GPU).
    
    It is not expected that the resolution to either of these will produce any
    major change.
    
    The current GCC implementation is against n4835 [1].
    
    ABI
    ---
    
    The various compiler developers have discussed a minimal ABI to allow one
    implementation to call coroutines compiled by another; this amounts to:
    
    1. The layout of a public portion of the coroutine frame.
    2. A number of compiler builtins that the standard library might use.
    
    The eventual home for the ABI is not decided yet, I will put a draft onto
    the wiki this week.
    
    The ABI has currently no target-specific content (a given psABI might elect
    to mandate alignment, but the common ABI does not do this).
    
    There is not need to add any new mangling, since the components of this are
    regular functions with manipulation of the coroutine via a type-erased handle.
    
    Standard Library impact
    -----------------------
    
    The current implementations require addition of only a single header to
    the standard library (no change to the runtime).  This header is part of
    the patch series.
    
    GCC Implementation outline
    --------------------------
    
    The standard's design for coroutines does not decorate the definition of
    a coroutine in any way, so that a function is only known to be a coroutine
    when one of the keywords (co_await, co_yield, co_return) is encountered.
    
    This means that we cannot special-case such functions from the outset, but
    must process them differently when they are finalised - which we do from
    "finish_function ()".
    
    At a high level, this design of coroutine produces four pieces from the
    original user's function:
    
      1. A coroutine state frame (taking the logical place of the activation
         record for a regular function).  One item stored in that state is the
         index of the current suspend point.
      2. A "ramp" function
         This is what the user calls to construct the coroutine frame and start
         the coroutine execution.  This will return some object representing the
         coroutine's eventual return value (or means to continue it when it it
         suspended).
      3. A "resume" function.
         This is what gets called when a the coroutine is resumed when suspended.
      4. A "destroy" function.
         This is what gets called when the coroutine state should be destroyed
         and its memory returned.
    
    The standard's coroutines involve cooperation of the user's authored function
    with a provided "promise" class, which includes mandatory methods for
    handling the state transitions and providing output values.  Most realistic
    coroutines will also have one or more 'awaiter' classes that implement the
    user's actions for each suspend point.  As we parse (or during template
    expansion) the types of the promise and awaiter classes become known, and can
    then be verified against the signatures expected by the standard.
    
    Once the function is parsed (and templates expanded) we are able to make the
    transformation into the four pieces noted above.
    
    The implementation here takes the approach of a series of AST transforms.
    The state machine suspend points are encoded in three internal functions
    (one of which represents an exit from scope without cleanups).  These three 
    IFNs are lowered early in the middle end, such that the majority of GCC's
    optimisers can be run on the resulting output.
    
    As a design choice, we have carried out the outlining of the user's function
    in the front end, and taken advantage of the existing middle end's abilities
    to inline and DCE where that is profitable.
    
    Since the state machine is actually common to both resumer and destroyer
    functions, we make only a single function "actor" that contains both the
    resume and destroy paths.  The destroy function is represented by a small
    stub that sets a value to signal the use of the destroy path and calls the
    actor.  The idea is that optimisation of the state machine need only be done
    once - and then the resume and destroy paths can be identified allowing the
    middle end's inline and DCE machinery to optimise as profitable as noted above.
    
    The middle end components for this implementation are:
     1. Lower the coroutine builtins that allow the standard library header to
        interact with the coroutine frame (these fairly simple logical or
        numerical substitution of values given a coroutine frame pointer).
     2. Lower the IFN that represents the exit from state without cleanup.
        Essentially, this becomes a gimple goto.
     3. Lower the IFNs that represent the state machine paths for the resume and
        destroy cases.
     4. A very late pass that is able to re-size the coroutine frame when there
        are unused entries and therefore choose the minimum allocation for it.
    
    There are no back-end implications to this current design.
    
    GCC Implementation Status
    -------------------------
    
    The current implementation should be considered somewhat experimental and is
    guarded by a "-fcoroutines" flag.  I have set out to minimise impact on the
    compiler (such that with the switch off, coroutines should be a NOP).
    
    The branch has been feature-complete for a few weeks and published on Compiler
    Explorer since late September.  I have been keeping a copy of the branch on
    my github page, and some bug reports have been filed there (and dealt with).
    
    The only common resource taken is a single bit in the function decl to flag
    that this function is determined to be a coroutine.
    
    Patch Series
    ------------
    
    The patch series is against r278049 (Mon 11th Nov).
    
    There are 6 pieces to try an localise the reviewer interest areas.  However
    it would not make sense to commit except as possibly two (main and testsuite).
    I have not tested that the compiler would even build part-way through this
    series.
    
    1) Common code and base definitions.
    
    This is the background content, defining the gating flag, keywords etc.
    
    2) Builtins and internal functions.
    
    Definitions of the builtins used by the standard library header and the
    internal functions used to implement the state machine.
    
    3)  Front end parsing and AST transforms.
    
    This is the largest part of the code, and has essentially two phases
     1. parse (and template expansion)
     2. analysis and transformation, which does the code generation for the
        state machine.
    
    4) Middle end expanders and transforms
    
     As per the description above.
    
    5) Standard library header.
    
    This is mostly mandated by the standard, although (of course) the decision
    to implement the interaction with the coroutine frame by inline builtin
    calls is pertinent.
    
    There is no runtime addition for this (the builtins are expanded directly).
    
    6) Testsuite.
    
    There are two chunks of tests.
     1. those that check for correct error handling
     2. those that check for the correct lowering of the state machine
     
    Since the second set are checking code-gen, they are run as 'torture' tests
    with the default options list.
    
    ======
    
    I will put this patch series onto a git branch for those that would prefer
    to view it in that form.
    
    thanks
    Iain
    
  • Initial Patches Wire In C++20 Coroutines For The GCC Compiler

    The GNU Compiler Collection continues picking up new features aligned for the upcoming C++20 standard. The latest are patches pending on the mailing list for implementing coroutines in C++.

    C++20 is expected to have coroutines per the pending technical specification. Coroutines allow a function to have its execution stopped/suspended and then to be resumed later.

  • Faster Winter 1: Vectors
  • Faster Winter 2: SPECIALIZE
  • Attention! Attention! Tutorial Proposal Deadline Approaching
  • Python Software Foundation Fellow Members for Q3 2019

    Congratulations! Thank you for your continued contributions. We have added you to our Fellow roster online.

    The above members have contributed to the Python ecosystem by teaching Python, maintaining popular libraries/tools such as cryptography and pytest, helping document on packaging.python.org, organizing Python events, starting Python communities in their home countries, and overall being great mentors in our community. Each of them continues to help make Python more accessible around the world. To learn more about the new Fellow members, check out their links above.

  • Why Sponsor PyCon 2020?

    Sponsors help keep PyCon affordable and accessible to the widest possible audience. Sponsors are what make this conference possible. From low ticket prices to financial aid, to video recording, the organizations who step forward to support PyCon, in turn, support the entire Python community. They make it possible for so many to attend, for so many to be presenters, and for the people at home to watch along.

  • Pandas GroupBy: Your Guide to Grouping Data in Python

    Whether you’ve just started working with Pandas and want to master one of its core facilities, or you’re looking to fill in some gaps in your understanding about .groupby(), this tutorial will help you to break down and visualize a Pandas GroupBy operation from start to finish.

    This tutorial is meant to complement the official documentation, where you’ll see self-contained, bite-sized examples. Here, however, you’ll focus on three more involved walk-throughs that use real-world datasets.

  • Solving CAPTCHA with Web automation

    CAPTCHA is no longer an alien term to the users. An acronym for Completely Automated Public Turing test to tell Computers and Humans Apart. CAPTCHA is basically a computer program built to distinguish between the human and machine to prevent any type of spam or data extraction from websites. The entire concept of CAPTCHA is based on the assumption that only a human would pass this test and bot or automated scripts would fail.

More in Tux Machines

Fedora Deciding Whether CD/DVD Installation Issues Should Still Hold Up Releases

Fedora will continue producing ISO images of their distribution that can be installed to a DVD (or CD in the case of some lightweight spins) or more commonly these days copied to USB flash drives, but they are debating whether any CD/DVD optical media issues should still be considered blocker bugs in 2020 and beyond. Fedora optical media and any issues pertaining to that would be considered non-blocking for Fedora releases. This reflects the fact a majority of Linux users these days are copying their Linux distributions to USB flash drives and installing from there rather than still burning CDs/DVDs. Particularly with many computers these days lacking CD/DVD drives, not having to worry about optical install issues as blocker bugs would free up resources to deal with more pressing bugs around release time. Read more

today's leftovers

  • AMDVLK 2019.Q4.4 Released With Navi 14 Fixes, DoW 3 Perf Optimization

    As anticipated, AMD has now formally released a new version of their AMDVLK open-source Vulkan driver following this week's Radeon Software Adrenalin 2020 Windows driver release. The changes end up being what I was alluding to yesterday with VK_EXT_pipeline_creation_feedback support, subgroup cluster support, a performance optimization for the Dawn of War 3 game, CTS failure fixes for Navi 14, and other fixes.

  • Dominique Leuenberger: openSUSE Tumbleweed – Review of the week 2019/50

    Another week has passed – and we’re almost at the end of the year. During the last week we have released 4 snapshots for Tumbleweed (1206, 1207, 1210 and 1211) containing those noteworthy changes: gpg 2.2.18 libvirt 5.10.0 linux-glibc-devel 5.4 Mozilla Thunderbird 68.3.0 bluez 5.52 libxml 2.9.10 createrepo_c 0.15.4: beware: it is very strict and blocks any snapshot if there is a package with non-UTF8 chars or ASCII < 32 (except 9, 10 and 13) in a changelog. Double check your .changes files before submitting. GNOME 3.34.2 KDE Plasma 5.17.4

  • Why you need to know about Seeed hardware devices

    The microcontroller craze doesn't seem to be dying down—and that's a good thing because these products consistently succeed where the mobile market consistently fails: Users get open software and hardware, a portable form factor, and a wide choice of vendors and products that are built to last. Among the best of the open hardware and software vendors is Seeed, the self-proclaimed "IoT Hardware Enabler." I recently started seeing the Seeed logo on projects, so I contacted the company to learn about the interesting things they're doing. In response, they generously sent me one of their latest products: the Seeeduino Nano, a compact board that the company says is fully compatible with the Arduino Nano but at half the price and a quarter the size, along with a sample sensor to get me started. I spent a few days with it, and I'm already working on a project to improve my home garden and thinking of several others for home automation. Far from just another Arduino-like product, the Seeeduino Nano solves several problems new makers face when they get a microcontroller and want to use it.

  • Marco Zehe: A quick introduction to using Gutenberg

    Late in November, I published a personal opinion on the state of Gutenberg accessibility. Today, I’d like to give an introduction to Gutenberg from a screen reader user perspective. Gutenberg, the WordPress block editor, is the new way to create content and build sites in WordPress. It is a rich web application that uses many modern techniques such as dynamic updates, toolbars, side bars and other items to completely update the posting experience. It can also be quite daunting at first. Let us try to shed a little light on some of the mysteries around it.

  • Pitfalls for OMEMO Implementations – Part 1: Inactive Devices

    Smack’s OMEMO implementation received a security audit a while ago (huge thanks to the Guardian Project for providing the funding!). Radically Open Security, a non-profit pentesting group from the Netherlands focused on free software and ethical hacking went through the code in great detail to check its correctness and to search for any vulnerabilities. In the end they made some findings, although I wouldn’t consider them catastrophically bad (full disclosure – its my code, so I might be biased :D). In this post I want to go over two of the finding and discuss, what went wrong and how the issue was fixed.

  • Support FSF's copyleft and licensing work

    We launched our annual fundraiser with the goal of welcoming 600 new associate members before December 31st. New members are critical to the cause, and by becoming a member you will stand in solidarity with others who care about computer user freedom. As is the case with any social movement, the numbers matter, and it is a very powerful gesture to make for only $10 a month ($5 if you are a student). Please support the work that gives hope for a future with software freedom: make a donation or – better yet -- join us and become a member today. The Free Software Foundation is a global leader for copyleft, and the licensing team plays a vital role in disseminating useful knowledge about free software while working to protect it. We accomplish this in part by answering licensing questions from the public and by providing resources like our list of free software licenses. We also increase access to software freedom by managing the Respects Your Freedom certification program, and cataloging free software through our endorsed distributions program and the Free Software Directory. To protect free software, we handle license compliance for the GNU Project, resulting in a stronger community and more respect for the power of copyleft. We are proud to accomplish this as just two staff working with our executive director, board, and legal counsel. These resources combined make a potent force for software freedom, and your support will ensure our work continues with the aim to do an even better job in 2020. Let us share a bit about the work we did in 2019 and elaborate on why it is so vital that this work continues.

  • OpenJS Foundation Welcomes Electron As Its New Incubating Project [Ed: OpenJS is run by people from Microsoft]

    Initially developed by GitHub in 2013, today the framework is maintained by a number of developers and organization

  • Twitter Is Funding Effort To Create A 'Decentralized Standard?'For Social Media

    The project is called Bluesky and eventually, it should enable Twitter to "access and contribute to a much larger corpus of public conversation," pushing it to be far more innovative than in the past.

Programming: GAction, Research on Developers, JavaBeans and Python

  • Sébastien Wilmet: Providing GActions in a library

    GAction represents an action that the user can do in an application, it’s usually present in a menu item or a button. It’s not just a function to launch, it’s a little more involved than that. Overall, providing GActions in a library can be done quite naturally, once the library provides a framework for the application. TeplApplication and TeplApplicationWindow both provide GActions in their public API. They are namespaced with the "tepl-" prefix, to avoid conflicts with other libraries or the application; so the full name of the GActions are "app.tepl-something" or "win.tepl-something". And all the GActions are documented in the class description. Note that TeplApplication and TeplApplicationWindow are not subclasses of GtkApplication and GtkApplicationWindow, because several libraries might want to extend those GTK classes and an application needs to be able to use all those extensions at the same time. A nice solution that doesn’t require to hold a new object in the application: use this design pattern that I’ve already described on my blog.

  • Research: Developers are trusted by the business but the alignment is not felt evenly across different generations

    Welcome to the first in a series of in-depth articles looking at the developer’s role in the modern organisation. In this first post: a new generation has arrived. As organisations shift to becoming technology-focused, developers’ roles have evolved so that they are now playing a crucial role in decision making across their businesses. However, all this newfound alignment isn’t so keenly felt across the whole developer workforce…

  • Jakarta EE: Creating an Enterprise JavaBeans timer

    Enterprise JavaBeans (EJB) has many interesting and useful features, some of which I will be highlighting in this and upcoming articles. In this article, I’ll show you how to create an EJB timer programmatically and with annotation. Let’s go! The EJB timer feature allows us to schedule tasks to be executed according a calendar configuration. It is very useful because we can execute scheduled tasks using the power of Jakarta context. When we run tasks based on a timer, we need to answer some questions about concurrency, which node the task was scheduled on (in case of an application in a cluster), what is the action if the task does not execute, and others. When we use the EJB timer we can delegate many of these concerns to Jakarta context and care more about business logic. It is interesting, isn’t it?

  • Python Dictionary Comprehension

    In this tutorial, we will learn about Python dictionary comprehension and how to use it with the help of examples.

Mozilla and Curl Leftovers

  • Ending QA community events, for now

    QMO events have been around for several years now, with many loyal Mozilla contributors engaged in various types of manual testing activities– some centered around verification of bug fixes, others on trying out exciting new features or significant changes made to the browser’s core ones. The feedback we received through them, during the Nightly and Beta phases, helped us ship polished products with each iteration, and it’s something that we’re very grateful for. We also feel that we could do more with the Testday and Bugday events. Their format has remained unchanged since we introduced them and the lack of a fresh new take on these events is now more noticeable than ever, as the overall interest in them has been dialing down for the past couple of years. We think it’s time to take a step back, review things and think about new ways to engage the community going forward.

  • Tips to improve your Ring camera security

    We cannot stress this enough. Weak and reused passwords are a serious vulnerability to your personal security and privacy. The software that the Nulled crew is using to tap into Ring feeds can be used to take over other things like, say, a Disney+ account. Or your bank account.

  • The Mozilla Blog: Petitioning for rehearing in Mozilla v. FCC

    Today, Mozilla continues the fight to preserve net neutrality protection as a fundamental digital right. Alongside other petitioners in our FCC challenge, Mozilla, Etsy, INCOMPAS, Vimeo and the Ad Hoc Telecom Users Committee filed a petition for rehearing and rehearing en banc in response to the D.C. Circuit decision upholding the FCC’s 2018 Order, which repealed safeguards for net neutrality. Our petition asks the original panel of judges or alternatively the full complement of D.C. Circuit judges to reconsider the decision both because it conflicts with D.C. Circuit or Supreme Court precedent and because it involves questions of exceptional importance.

  • Daniel Stenberg: Reporting documentation bugs in curl got easier

    After I watched a talk by Marcus Olsson about docs as code (at foss-sthlm on December 12 2019), I got inspired to provide links on the curl web site to make it easier for users to report bugs on documentation. Starting today, there are two new links on the top right side of all libcurl API function call documentation pages. File a bug about this page – takes the user directly to a new issue in the github issue tracker with the title filled in with the name of the function call, and the label preset to ‘documentation’. All there’s left is for the user to actually provide a description of the problem and pressing submit (and yeah, a github account is also required).