Language Selection

English French German Italian Portuguese Spanish

Development

gitgeist: a git-based social network proof of concept

Filed under
Development
Web

Are you tired of not owning the data or the platform you use for social postings? I know I am.

It's hard to say when I "first" used a social network. I've been on email for about 30 years and one of the early ad-hoc forms of social networks were chain emails. Over the years I was asked to join all sorts of "social" things such as IRC, ICQ, Skype, MSN Messenger, etc. and eventually things like Orkut, MySpace, Facebook, etc. I'll readily admit that I'm not the type of person that happily jumps onto every new social bandwagon that appears on the Internet. I often prefer preserving the quietness of my own thoughts. That, though, hasn't stopped me from finding some meaningfulness participating in Twitter, Facebook, LinkedIn and more recently Google+. Twitter was in fact the first social network that I truly embraced. And it would've remained my primary social network had they not killed their own community by culling the swell of independently-developed Twitter clients that existed. That and their increased control of their API effectively made me look for something else. Right around that time Google+ was being introduced and many in the open source community started participating in that, in some ways to find a fresh place where techies can aggregate away from the noise and sometimes over-the-top nature of Facebook. Eventually I took to that too and started using G+ as my primary social network. That is, until Google recently decided to pull the plug on G+.

While Google+ might not have represented a success for Google, it had become a good place for sharing information among the technically-inclined. As such, I found it quite useful for learning and hearing about new things in my field. Soon-to-be-former users of G+ have gone in all sorts of directions. Some have adopted a "c'mon guys, get over it, Facebook is the spot" attitude, others have adopted things like Mastodon, others have fallen back to their existing IDs on Twitter, and yet others, like me, are still looking.

Read more

Programming Leftovers

Filed under
Development
  • Packaging PyQt5 apps with fbs

    fbs is a cross-platform PyQt5 packaging system which supports building desktop applications for Windows, Mac and Linux (Ubuntu, Fedora and Arch). Built on top of PyInstaller it wraps some of the rough edges and defines a standard project structure which allows the build process to be entirely automated. The included resource API is particularly useful, simplifying the handling of external data files, images or third-party libraries — a common pain point when bundling apps.

  • Infrastructure monitoring: Defense against surprise downtime

    There are a number of tools available that can build a viable and strong monitoring system. The only decision to make is which to use; your answer lies in what you want to achieve with monitoring as well as various financial and business factors you must consider.

    While some monitoring tools are proprietary, many open source tools, either unmanaged or community-managed software, will do the job even better than the closed source options.

    In this article, I will focus on open source tools and how to use them to create a strong monitoring architecture.

  • GSlice considerations and possible improvements

    The paper Mesh: Compacting Memory Management for C/C++ Applications is about moving memory allocations for compaction, even though the memory pointers are exposed. The idea is to merge allocation blocks from different pages that are not overlapping at page offsets, and then letting multiple virtual page pointers point to the same physical page. Some have asked about the applicability to the GSlice allocator.

  • plprofiler – Getting a Handy Tool for Profiling Your PL/pgSQL Code
  • Reading and Writing Files in Python (Guide)
  • Today is a Good Day to Learn Python

5 of the Best Linux Distros for Developers and Programmers

Filed under
Development
GNU
Linux

One of the reasons Linux is great is because of how flexible it is. For example, it can run on everything from servers to your old laptop to a Raspberry Pi. For this reason, it’s also a fantastic platform for developers.

Whether you’re a seasoned developer or just using Linux to learn to program, you still have to choose a distribution. You could just choose Ubuntu and run with it, but there are plenty of “other options available to you.”

Read more

Programming: OpenJDK, Python, PyGame and Pandas

Filed under
Development
  • OpenJDK

    OpenJDK is a free, open-source version of the Java Development Kit for the Java Platform, Standard Edition (Java SE). OpenJDK, which stands for Open Java Development Kit, originated from an effort initiated by Sun Microsystems in 2006 and is now sponsored and led by Oracle. The project is licensed under the GNU General Public License (GNU GPL) version 2 with a linking exception. Without the linking exception, components that linked to the Java class library would be subject to the terms of the GPL license.

    Since the release of Java SE version 7, OpenJDK has been the official reference implementation. A few notable components that fall under the OpenJDK project include the Java class library, the Java compiler, the Java Runtime Environment (JRE) and the Java virtual machine (JVM). Unlike other JDK release projects, which focused on releasing one feature at a time before terminating, OpenJDK is a long-term, ongoing project. OpenJDK follows a strict, time-based model that is split into development branches and will release new features every six months.

  • Pandas Tutorial in Python

    According to the Pandas homepage: pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.

    One of the coolest things about Pandas is that it makes reading data from common data formats like CSV, SQL etc. very easy which makes it equally usable in production grade applications or just some demo applications.

  • New Course: Learn Data Cleaning with Python and Pandas
  • Adjust the boy sprite animation

    Hello, and welcome back, we are almost done coding the player animation mechanism after we have finished the player boundary detection mechanism in the last article but before we can go to the next stage we need to tidy up the player animation mechanism first by introducing the standstill image of the boy when the boy is not moving and that image will either face left or right or up or down based on the direction of the boy at the time he stops moving. In order to achieve this we only need to edit two files.

    The first file we need to edit is the main file where we will include the keyup event so we can set the x different or y different to zero when the boy who is moving in either x or y-direction suddenly stop moving.

Events: Qt World Summit 2018, NetSurf Developer, LibreOffice Asia Conference

Filed under
Development
LibO
OSS
  • Networking in Berlin: Qt World Summit 2018

    At our little booth we showcased Plasma running on a variety of devices, ranging from a Nexus 5X running Plasma Mobile through two ARM laptops to the powerful KDE Slimbook. Plasma was praised for its performance and reliability and since the focus of the event was mostly on embedded systems, we could easily demonstrate with our selection of devices that Plasma and the KDE Frameworks are a viable option for an endeavor in this area, too.

    It was very interesting to see the diverse set of people presenting their products and roaming the stalls, to see where Qt is in use today without you even realizing. We were approached by several companies evaluating using KDE Frameworks in their products and also tried to lay a foundation for an eventual partnership. And then there was Daimler who just parked an A-Class in the hallway, whose MBUX infotainment system is also powered by Qt.

  • Vincent Sanders: A very productive weekend

    I just hosted a NetSurf Developer weekend which is an opportunity for us to meet up and make use of all the benefits of working together. We find the ability to plan work and discuss solutions without loosing the nuances of body language generally results in better outcomes for the project.

    [...]

    We rounded the Saturday off by going out for a very pleasant meal with some mutual friends. Sunday started by adding a bunch of additional topics to consider and we made good progress addressing these.

    We performed a bug triage and managed to close several issues and commit to fixing a few more. We even managed to create a statement of work of things we would like to get done before the next meetup.

    My main achievement on the Sunday was to add WEBP image support. This uses the Google libwebp library to do all the heavy lifting and adding a new image content handler to NetSurf is pretty straightforward.

  • First LibreOffice Asia Conference to Take Place May 25-26, 2019 in Tokyo, Japan

    The Document Foundation published today more information on when and where the first ever LibreOffice Asia Conference event will take place this year.

    LibreOffice Asia Conference 2019 will be the project's first conference event to take place in a country in the Asia region where the free and open source software movement is rapidly growing. The Document Foundation decided it's time to put together a conference in Asia after the massive success of the LibreOffice Conference Indonesia 2018 event.

    "It’s a real leap of faith," says Franklin Weng, an Asian member in the Board of Directors of The Document Foundation. "Asia is a rapidly growing area in adoptions of ODF and LibreOffice, but our ecosystem for LibreOffice and FOSS has not been good enough yet. In this conference, we’re not only trying to make the FOSS ecosystem in Asia more healthy but also to encourage Asian community members to show their potential.”

Programming: RenPyWeb, OpenCL 2.2-10, x86 vs. ARM for Web Crawling in Python

Filed under
Development
  • RenPyWeb - Ren'Py in your HTML5 web browser

    I like the Ren'Py project, a popular game engine aimed at Visual Novels - that can also be used as a portable Python environment.

    One limitation was that it required downloading games, while nowadays people are used to Flash- or HTML5- based games that play in-browser without having to (de)install.

    Can this fixed? While maintaining compatibility with Ren'Py's several DSLs? And without rewriting everything in JavaScript?
    Can Emscripten help? While this is a Python/Cython project?
    After lots of experimenting, and full-stack patching/contributing, it turns out the answer is yes!

  • OpenCL 2.2-10 Released With Fixes

    While "OpenCL-Next" will hopefully be on track for releasing later this year as the next big update to OpenCL, OpenCL 2.2-10 was released today by The Khronos Group as the latest maintenance update to the nearly two year old OpenCL 2.2 specification.

    OpenCL-Next can't come soon enough to hopefully bolster OpenCL GPU programming adoption and OpenCL 2.2 showing its age with the provisional specification for it approaching three years old. With today's OpenCL 2.2-10 update there are various fixes to community reported problems. Also, the KHR OpenCL extensions have been folded into the extensions specification.

  • SPEED TEST: x86 vs. ARM for Web Crawling in Python

    Can you imagine if your job was to trawl competitor websites and jot prices down by hand, again and again and again? You’d burn your whole office down by lunchtime.

    So, little wonder web crawlers are huge these days. They can keep track of customer sentiment and trending topics, monitor job openings, real estate transactions, UFC results, all sorts of stuff.

    For those of a certain bent, this is fascinating stuff. Which is how I found myself playing around with Scrapy, an open source web crawling framework written in Python.

  • The hard part in becoming a command line wizard
  • How to Parse Hidden HTML With Selenium Headless Mode and Deploy it to Heroku
  • PyCoder’s Weekly: Issue #356 (Feb. 19, 2019)
  • PyCon 2019 Tutorial Schedule! [Ed: OK, but it is already compromised. It took a bribe from Microsoft (the top sponsor) and posted Azure ads in its site in exchange. Appalling trend.]

Programming: CRuby, C++, R, Qt, LSP/Python, and C

Filed under
Development
  • Register Transfer Language for CRuby

    For the last two years, I have been trying to improve CRuby performance. I have been working simultaneously on two major fronts: introducing register transfer language (RTL) for the CRuby virtual machine (VM) and just-in-time (JIT) compilation. For background on the goal of having Ruby 3 be 3 times faster than version 2 (3X3), see my previous article, “Towards the Ruby 3×3 Performance Goal“.

    The JIT project (MJIT) is advancing successfully. The JIT approach and engine I proposed and implemented has been adopted by the CRuby community. Takashi Kokubun hardened the code and adapted it to the current CRuby stack machine and recently MJIT became an experimental feature of the CRuby 2.6 release.

    Introducing a Register Transfer Language (RTL) to the CRuby VM turned out to be an even harder task than introducing the initial JIT compiler. The required changes to the VM are far more invasive than the ones needed for the JIT compiler.

  • Template meta-programming: Some testing and debugging tricks

    There are only a few things more fun in this world than doing template meta-programming (TMP) and reading all those long poems that the compiler writes out when we make even the smallest mistake.

    While we don’t usually welcome these messages, there are ways to make them useful.

    One of the main causes of errors in TMP code are unexpected types – types that the compiler is deducing instead of the types that we expect it to deduce.

    This results in error messages occurring in seemingly random places in our code.

  • Open Science, Open Source and R

    Psychology changed forever on the August 27, 2015. For the previous four years, the 270 psychologists of the Open Science Collaboration had been quietly re-running 100 published psychology experiments. Now, finally, they were ready to share their findings. The results were shocking. Less than half of the re-run experiments had worked.

    When someone tries to re-run an experiment, and it doesn't work, we call this a failure to replicate. Scientists had known about failures to replicate for a while, but it was only quite recently that the extent of the problem became apparent. Now, an almost existential crisis loomed. That crisis even gained a name: the Replication Crisis. Soon, people started asking the same questions about other areas of science. Often, they got similar answers. Only half of results in economics replicated. In pre-clinical cancer studies, it was worse; only 11% replicated.

  • Qt Design Studio 1.1 released

    We are happy to announce the release of Qt Design Studio 1.1 !

    Qt Design Studio is a UI design and development tool that enables designers and developers to rapidly prototype and develop complex UIs. Both designers and developers use Qt Design Studio and this makes collaboration between the two a lot simpler and more streamlined. To get an impression, you should watch this video.

    Since the Qt Design Studio 1.0 release last year we worked hard on bug fixes and new features.

  • Sublime Text and Language Server Protocol

    LSP - protocol for interactions between IDE and language server. The latter provides such means like autocompletion, goto implementation and etc. When IDE needs to show autocomplete choices on, for example, python language - it sends a request to the special server. And it responds with the necessary data. The cool part here is that it is an initiative of a big company - Microsoft.

  • C Programming Tutorial Part 5 - Character variables

XFS Copy-On-Write Support Being Improved, Always CoW Option

Filed under
Development
Linux

One of the recent XFS innovations under work and maturing with time has been Copy on Write (CoW) support for this mature Linux file-systems. The XFS CoW support continues to be improved upon and an "always CoW" option is being prepared to always force this behavior.

Christoph Hellwig has been among the XFS developers working on the copy-on-write improvements as well as adding an always_cow sysfs option. The latest XFS CoW patches were sent out on Monday for review.

Read more

Also: ACPI 6.3 Support Coming With The Linux 5.1 Kernel

HowTos and Development Leftovers

Filed under
Development
HowTos
Syndicate content

More in Tux Machines

Games: Surviving Mars and OpenMW

Kernel and Security: BPF, Mesa, Embedded World, Kernel Address Sanitizer and More

  • Concurrency management in BPF
    In the beginning, programs run on the in-kernel BPF virtual machine had no persistent internal state and no data that was shared with any other part of the system. The arrival of eBPF and, in particular, its maps functionality, has changed that situation, though, since a map can be shared between two or more BPF programs as well as with processes running in user space. That sharing naturally leads to concurrency problems, so the BPF developers have found themselves needing to add primitives to manage concurrency (the "exchange and add" or XADD instruction, for example). The next step is the addition of a spinlock mechanism to protect data structures, which has also led to some wider discussions on what the BPF memory model should look like. A BPF map can be thought of as a sort of array or hash-table data structure. The actual data stored in a map can be of an arbitrary type, including structures. If a complex structure is read from a map while it is being modified, the result may be internally inconsistent, with surprising (and probably unwelcome) results. In an attempt to prevent such problems, Alexei Starovoitov introduced BPF spinlocks in mid-January; after a number of quick review cycles, version 7 of the patch set was applied on February 1. If all goes well, this feature will be included in the 5.1 kernel.
  • Intel Ready To Add Their Experimental "Iris" Gallium3D Driver To Mesa
    For just over the past year Intel open-source driver developers have been developing a new Gallium3D-based OpenGL driver for Linux systems as the eventual replacement to their long-standing "i965 classic" Mesa driver. The Intel developers are now confident enough in the state of this new driver dubbed Iris that they are looking to merge the driver into mainline Mesa proper.  The Iris Gallium3D driver has now matured enough that Kenneth Graunke, the Intel OTC developer who originally started Iris in late 2017, is looking to merge the driver into the mainline code-base of Mesa. The driver isn't yet complete but it's already in good enough shape that he's looking for it to be merged albeit marked experimental.
  • Hallo Nürnberg!
    Collabora is headed to Nuremberg, Germany next week to take part in the 2019 edition of Embedded World, "the leading international fair for embedded systems". Following a successful first attendance in 2018, we are very much looking forward to our second visit! If you are planning on attending, please come say hello in Hall 4, booth 4-280! This year, we will be showcasing a state-of-the-art infrastructure for end-to-end, embedded software production. From the birth of a software platform, to reproducible continuous builds, to automated testing on hardware, get a firsthand look at our platform building expertise and see how we use continuous integration to increase productivity and quality control in embedded Linux.
  • KASAN Spots Another Kernel Vulnerability From Early Linux 2.6 Through 4.20
    The Kernel Address Sanitizer (KASAN) that detects dynamic memory errors within the Linux kernel code has just picked up another win with uncovering a use-after-free vulnerability that's been around since the early Linux 2.6 kernels. KASAN (along with the other sanitizers) have already proven quite valuable in spotting various coding mistakes hopefully before they are exploited in the real-world. The Kernel Address Sanitizer picked up another feather in its hat with being responsible for the CVE-2019-8912 discovery.
  • io_uring, SCM_RIGHTS, and reference-count cycles
    The io_uring mechanism that was described here in January has been through a number of revisions since then; those changes have generally been fixing implementation issues rather than changing the user-space API. In particular, this patch set seems to have received more than the usual amount of security-related review, which can only be a good thing. Security concerns became a bit of an obstacle for io_uring, though, when virtual filesystem (VFS) maintainer Al Viro threatened to veto the merging of the whole thing. It turns out that there were some reference-counting issues that required his unique experience to straighten out. The VFS layer is a complicated beast; it must manage the complexities of the filesystem namespace in a way that provides the highest possible performance while maintaining security and correctness. Achieving that requires making use of almost all of the locking and concurrency-management mechanisms that the kernel offers, plus a couple more implemented internally. It is fair to say that the number of kernel developers who thoroughly understand how it works is extremely small; indeed, sometimes it seems like Viro is the only one with the full picture. In keeping with time-honored kernel tradition, little of this complexity is documented, so when Viro gets a moment to write down how some of it works, it's worth paying attention. In a long "brain dump", Viro described how file reference counts are managed, how reference-count cycles can come about, and what the kernel does to break them. For those with the time to beat their brains against it for a while, Viro's explanation (along with a few corrections) is well worth reading. For the rest of us, a lighter version follows.

Blacklisting insecure filesystems in openSUSE

The Linux kernel supports a wide variety of filesystem types, many of which have not seen significant use — or maintenance — in many years. Developers in the openSUSE project have concluded that many of these filesystem types are, at this point, more useful to attackers than to openSUSE users and are proposing to blacklist many of them by default. Such changes can be controversial, but it's probably still fair to say that few people expected the massive discussion that resulted, covering everything from the number of OS/2 users to how openSUSE fits into the distribution marketplace. On January 30, Martin Wilck started the discussion with a proposal to add a blacklist preventing the automatic loading of a set of kernel modules implementing (mostly) old filesystems. These include filesystems like JFS, Minix, cramfs, AFFS, and F2FS. For most of these, the logic is that the filesystems are essentially unused and the modules implementing them have seen little maintenance in recent decades. But those modules can still be automatically loaded if a user inserts a removable drive containing one of those filesystem types. There are a number of fuzz-testing efforts underway in the kernel community, but it seems relatively unlikely that any of them are targeting, say, FreeVxFS filesystem images. So it is not unreasonable to suspect that there just might be exploitable bugs in those modules. Preventing modules for ancient, unmaintained filesystems from automatically loading may thus protect some users against flash-drive attacks. If there were to be a fight over a proposal like this, one would ordinarily expect it to be concerned with the specific list of unwelcome modules. But there was relatively little of that. One possible exception is F2FS, the presence of which raised some eyebrows since it is under active development, having received 44 changes in the 5.0 development cycle, for example. Interestingly, it turns out that openSUSE stopped shipping F2FS in September. While the filesystem is being actively developed, it seems that, with rare exceptions, nobody is actively backporting fixes, and the filesystem also lacks a mechanism to prevent an old F2FS implementation from being confused by a filesystem created by a newer version. Rather than deal with these issues, openSUSE decided to just drop the filesystem altogether. As it happens, the blacklist proposal looks likely to allow F2FS to return to the distribution since it can be blacklisted by default. Read more

gitgeist: a git-based social network proof of concept

Are you tired of not owning the data or the platform you use for social postings? I know I am. It's hard to say when I "first" used a social network. I've been on email for about 30 years and one of the early ad-hoc forms of social networks were chain emails. Over the years I was asked to join all sorts of "social" things such as IRC, ICQ, Skype, MSN Messenger, etc. and eventually things like Orkut, MySpace, Facebook, etc. I'll readily admit that I'm not the type of person that happily jumps onto every new social bandwagon that appears on the Internet. I often prefer preserving the quietness of my own thoughts. That, though, hasn't stopped me from finding some meaningfulness participating in Twitter, Facebook, LinkedIn and more recently Google+. Twitter was in fact the first social network that I truly embraced. And it would've remained my primary social network had they not killed their own community by culling the swell of independently-developed Twitter clients that existed. That and their increased control of their API effectively made me look for something else. Right around that time Google+ was being introduced and many in the open source community started participating in that, in some ways to find a fresh place where techies can aggregate away from the noise and sometimes over-the-top nature of Facebook. Eventually I took to that too and started using G+ as my primary social network. That is, until Google recently decided to pull the plug on G+. While Google+ might not have represented a success for Google, it had become a good place for sharing information among the technically-inclined. As such, I found it quite useful for learning and hearing about new things in my field. Soon-to-be-former users of G+ have gone in all sorts of directions. Some have adopted a "c'mon guys, get over it, Facebook is the spot" attitude, others have adopted things like Mastodon, others have fallen back to their existing IDs on Twitter, and yet others, like me, are still looking. Read more