Language Selection

English French German Italian Portuguese Spanish

Debian

Syndicate content
Planet Debian - https://planet.debian.org/
Updated: 8 hours 53 min ago

Andrew Cater: Debian Stretch release 9.13 - testing of images now complete 202007181950 or so

Saturday 18th of July 2020 08:00:16 PM
And we're getting a lot closer. Last edits to the wiki for the testing page. Tests marked as passed. There are a bunch of architectures where the media images are still to be built and the source media will still need to be written as well.

It's been a really good (and fairly long day). I'm very glad indeed that I'm not staying up until the bitter end. It's always fun - I'm also very glad that, as it panned out, we didn't end up also doing the next Buster point release this weekend - it would have been a massive workload.

So Stretch passes to LTS, Jessie has passed to ELTS - and we go onwards and upwards to the next point release - and then, in due course, to Bullseye when it's ready (probably the first quarter of next year?).

As ever, thanks due to all involved - the folks behind the scenes who maintain cdimage.debian.org, to Sledge, RattusRattus, Isy and schweer. Done for about a fortnight or so until the next Buster release.

I will try to blog on other things, honest :)

Andrew Cater: Debian Stretch 9.13 release - blog post 2 - testing of basic .iso images ongoing as at 202007181655

Saturday 18th of July 2020 06:10:29 PM
The last of the tests for the basic .iso images are finishing. Live image testing is starting. Lots from RattusRattus, Isy, Sledge and schweer besides myself. New this time round is an experiment in videoconference linking which has made this whole experience much more human in lockdown - and means I'm missing Cambridge.

Two questions that have come up as we've been going through. There are images made for Intel-based mac that were first made for Jessie or so: there are also images for S390X. These can't be tested because none of the testers have the hardware. Nobody has yet recorded issues with them but it could be that nobody has ever used them recently or has reported problems

If nobody is bothered with these: they are prime candidates for removal prior to Bullseye.

Dirk Eddelbuettel: tint 0.1.3: Fixes for html mode, new demo

Saturday 18th of July 2020 04:06:00 PM

A new version 0.1.3 of the tint package arrived at CRAN today. It corrects some features for html output, notably margin notes and references. It also contains a new example for inline references.

The full list of changes is below.

Changes in tint version 0.1.3 (2020-07-18)
  • A new minimal demo was added showing inline references (Dirk addressing #42).

  • Code for margin notes and reference in html mode was updated with thanks to tufte (Dirk in #43 and #44 addressing #40).

  • The README.md was updated with a new 'See Also' section and a new badge.

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the tint page.

For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Dirk Eddelbuettel: drat 0.1.8: Minor test fix

Saturday 18th of July 2020 03:52:00 PM

A new version of drat arrived on CRAN today. This is a follow-up release to 0.1.7 from a week ago. It contains a quick follow-up by Felix Ernst to correct on of the tests which misbehaved under the old release of R still being tested at CRAN.

drat stands for drat R Archive Template, and helps with easy-to-create and easy-to-use repositories for R packages. Since its inception in early 2015 it has found reasonably widespread adoption among R users because repositories with marked releases is the better way to distribute code.

As your mother told you: Friends don’t let friends install random git commit snapshots. Rolled-up releases it is. drat is easy to use, documented by five vignettes and just works.

The NEWS file summarises the release as follows:

Changes in drat version 0.1.8 (2020-07-18)
  • The archive pruning test code was corrected for r-oldrel (Felix Ernst in #105 fixing #104).

Courtesy of CRANberries, there is a comparison to the previous release. More detailed information is on the drat page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Ritesh Raj Sarraf: Laptop Mode Tools 1.74

Saturday 18th of July 2020 03:45:04 PM
Laptop Mode Tools 1.74

Laptop Mode Tools version 1.74 has been released. This release includes important bug fixes, some defaults settings updated to current driver support in Linux and support for devices with nouveau based nVIDIA cards.

A filtered list of changes is mentioned below. For the full log, please refer to the git repository

1.74 - Sat Jul 18 19:10:40 IST 2020 * With 4.15+ kernels, Linux Intel SATA has a better link power saving policy, med_power_with_dipm, which should be the recommended one to use * Disable defaults for syslog logging * Initialize LM_VERBOSE with default to disabled * Merge pull request #157 from rickysarraf/nouveau * Add power saving module for nouveau cards * Disable ethernet module by default * Add board-specific folder and documentation * Add execute bit on module radeon-dpm * Drop unlock because there is no lock acquired Resources What is Laptop Mode Tools Description: Tools for Power Savings based on battery/AC status Laptop mode is a Linux kernel feature that allows your laptop to save considerable power, by allowing the hard drive to spin down for longer periods of time. This package contains the userland scripts that are needed to enable laptop mode. . It includes support for automatically enabling laptop mode when the computer is working on batteries. It also supports various other power management features, such as starting and stopping daemons depending on power mode, automatically hibernating if battery levels are too low, and adjusting terminal blanking and X11 screen blanking . laptop-mode-tools uses the Linux kernel's Laptop Mode feature and thus is also used on Desktops and Servers to conserve power

Andrew Cater: Debian "Stretch" 9.13 release preparations ongoing

Saturday 18th of July 2020 12:36:49 PM
Just checking in. Debian "Jessie" == oldoldstable == Debian 8 was the previous Debian Long Term Support release. Debian LTS seeks to provide support for Debian releases for five years. LTS support for Jessie ended on 30th June 2020.

A limited subset of Jessie will now move to ELTS - Extended Long Term Support and another two years projected support.

Neither LTS nor ELTS are supported  any longer by the main Debian folks: instead, they are supported on a commercial basis by a group of Debian volunteers and companies, coordinated by a company led by Raphael Hertzog.

Debian releases are fully supported by the Debian project for two years after the release of the next version.  Today is the final release of Stretch by Debian to incorporate security fixes and so on up to the handover to LTS.
if you are currently running Stretch: you do not need the new CD images. Apt / apt-get update will supply you the updates up until today. Hereafter, Stretch will be supported only as LTS - see LTS


David Bremner: git-annex and ikiwiki, not as hard as I expected

Saturday 18th of July 2020 12:09:00 PM
Background

So apparently there's this pandemic thing, which means I'm teaching "Alternate Delivery" courses now. These are just like online courses, except possibly more synchronous, definitely less polished, and the tuition money doesn't go to the College of Extended Learning. I figure I'll need to manage share videos, and our learning management system, in the immortal words of Marie Kondo, does not bring me joy. This has caused me to revisit the problem of sharing large files in an ikiwiki based site (like the one you are reading).

My goto solution for large file management is git-annex. The last time I looked at this (a decade ago or so?), I was blocked by git-annex using symlinks and ikiwiki ignoring them for security related reasons. Since then two things changed which made things relatively easy.

  1. I started using the rsync_command ikiwiki option to deploy my site.

  2. git-annex went through several design iterations for allowing non-symlink access to large files.

TL;DR

In my ikiwiki config

# attempt to hardlink source files? (optimisation for large files) hardlink => 1,

In my ikiwiki git repo

$ git annex init $ git annex add foo.jpg $ git commit -m&aposadd big photo&apos $ git annex adjust --unlock # look ikiwiki, no symlinks $ ikiwiki --setup ~/.config/ikiwiki/client # rebuild my local copy, for review $ ikiwiki --setup /home/bremner/.config/ikiwiki/rsync.setup --refresh # deploy

You can see the result at photo

Dima Kogan: Converting images while extracting a tarball

Saturday 18th of July 2020 05:31:00 AM

Last week at the lab I received a data dump: a gzip-compressed tarball with lots of images in it. The images are all uncompressed .pgm, with the whole tarball weighing in at ~ 1TB. I tried to extract it, and after chugging all day, it ran out of disk space. Added more disk, tried again: out of space again. Just getting a listing of the archive contents (tar tvfz) took something like 8 hours.

Clearly this is unreasonable. I made an executive decision to use .jpg files instead: I'd take the small image quality hit for the massive gains in storage efficiency. But the tarball has .pgm and just extracting the thing is challenging. So I'm now extracting the archive, and converting all the .pgm images to .jpg as soon as they hit disk. How? Glad you asked!

I'm running two parallel terminal sessions (I'm using screen, but you can do whatever you like).

Session 1 < archive.tar.gz unpigz -p20 | tar xv

Here I'm just extracting the archive to disk normally. Using unpigz instead of plain, old tar to get parallelization.

Session 2 inotifywait -r PATH -e close_write -m | mawk -Winteractive '/pgm$/ { print $1$3 }' | parallel -v -j10 'convert {} -quality 96 {.}.jpg && rm {}'

This is the secret sauce. I'm using inotifywait to tell me when any file is closed for writing in a subdirectory of PATH. Then I mawk it to only tell me when .pgm files are done being written, then I convert them to .jpg, and delete the .pgm when that's done. I'm using GNU Parallel to parallelize the image conversion. Otherwise the image conversion doesn't keep up.

This is going to take at least all day, but I'm reasonably confident that it will actually finish successfully, and I can then actually do stuff with the data.

Abhijith PA: Workstation setup

Saturday 18th of July 2020 04:37:00 AM

Hello,

Recently I’ve seen lot of people sharing about their home office setup. I thought why don’t I do something similar. Not to beat FOMO, but in future when I revisit this blog, it will be lovely to understand that I had some cool stuffs.

There are people who went deep down in the ocean to lay cables for me to have a remote job and I am thankful to them.

Being remote my home is my office. On my work table I have a Samsung R439 laptop. I’ve blogged about it earlier. New addition is that it have another 4GB RAM, a total of 6GB and 120GB SSD. I run Debian testing on it. Laptop is placed on a stand. Dell MS116 as external mouse always connected to it. I also use an external keyboard from fingers. The keys are very stiff so I don’t recommend this to anyone. The only reason I took this keyboard that it is in my budget and have a backlit, which I needed most.

I have a Micromax MM215FH76 21 inch monitor as my secondary display which stacked up on couple of old books to adjust the height with laptop stand. Everything is ok with this monitor except that it don’t have a HDMI point and stand is very weak. I use i3wm and this small script help me to manage my monitor arrangement.

# samsung r439 xrandr --output LVDS1 --primary --mode 1366x768 --pos 1920x312 --rotate normal --output DP1 --off --output HDMI1 --off --output VGA1 --mode 1920x1080 --pos 0x0 --rotate normal --output VIRTUAL1 --off # thinkpad t430s #xrandr --output LVDS1 --primary --mode 1600x900 --pos 1920x180 --rotate normal --output DP1 --off --output DP2 --off --output DP3 --off --output HDMI1 --off --output HDMI2 --off --output HDMI3 --off --output VGA1 --mode 1920x1080 --pos 0x0 --rotate normal --output VIRTUAL1 --off i3-msg workspace 2, move workspace to left i3-msg workspace 4, move workspace to left i3-msg workspace 6, move workspace to left

I also have another Viewsonic monitor 19 inch, it started to show some lines and unpleasant colors. Thus moved back to shelf.

I have an orange pi zero plus 2 running Armbian which serve as my emby media server. I don’t own any webcam or quality headset at the moment. I have a boat, and Mi, headphones. My laptop inbuilt webcam is horrible, so for my video conferencing need I use jitsi app on my mobile device.

Abhijith PA: Workstation setup

Saturday 18th of July 2020 04:37:00 AM

Hello,

Recently I’ve seen lot of people sharing about their home office setup. I thought why don’t I do something similar. Not to beat FOMO, but in future when I revisit this blog, it will be lovely to understand that I had some cool stuffs.

There are people who went deep down in the ocean to lay cables for me to have a remote job and I am thankful to them.

Being remote my home is my office. On my work table I have a Samsung R439 laptop. I’ve blogged about it earlier. New addition is that it have another 4GB RAM, a total of 6GB and 120GB SSD. I run Debian testing on it. Laptop is placed on a stand. Dell MS116 as external mouse always connected to it. I also use an external keyboard from fingers. The keys are very stiff so I don’t recommend this to anyone. The only reason I took this keyboard that it is in my budget and have a backlit, which I needed most.

I have a Micromax MM215FH76 21 inch monitor as my secondary display which stacked up on couple of old books to adjust the height with laptop stand. Everything is ok with this monitor except that it don’t have a HDMI point and stand is very weak. I use i3wm and this small script help me to manage my monitor arrangement.

# samsung r439 xrandr --output LVDS1 --primary --mode 1366x768 --pos 1920x312 --rotate normal --output DP1 --off --output HDMI1 --off --output VGA1 --mode 1920x1080 --pos 0x0 --rotate normal --output VIRTUAL1 --off # thinkpad t430s #xrandr --output LVDS1 --primary --mode 1600x900 --pos 1920x180 --rotate normal --output DP1 --off --output DP2 --off --output DP3 --off --output HDMI1 --off --output HDMI2 --off --output HDMI3 --off --output VGA1 --mode 1920x1080 --pos 0x0 --rotate normal --output VIRTUAL1 --off i3-msg workspace 2, move workspace to left i3-msg workspace 4, move workspace to left i3-msg workspace 6, move workspace to left

I also have another Viewsonic monitor 19 inch, it started to show some lines and unpleasant colors. Thus moved back to shelf.

I have an orange pi zero plus 2 running Armbian which serve as my emby media server. I don’t own any webcam or quality headset at the moment. I have a boat, and Mi, headphones. My laptop inbuilt webcam is horrible, so for my video conferencing need I use jitsi app on my mobile device.

Reproducible Builds (diffoscope): diffoscope 152 released

Saturday 18th of July 2020 12:00:00 AM

The diffoscope maintainers are pleased to announce the release of diffoscope version 152. This version includes the following changes:

[ Chris Lamb ] * Bug fixes: - Don't require zipnote(1) to determine differences in a .zip file as we can use libarchive directly. * Reporting improvements: - Don't emit "javap not found in path" if it is available in the path but it did not result in any actual difference. - Fix "... not available in path" messages when looking for Java decompilers; we were using the Python class name (eg. "<class 'diffoscope.comparators.java.Javap'>") over the actual command we looked for (eg. "javap"). * Code improvements: - Replace some simple usages of str.format with f-strings. - Tidy inline imports in diffoscope.logging. - In the RData comparator, always explicitly return a None value in the failure cases as we return a non-None value in the "success" one. [ Jean-Romain Garnier ] * Improve output of side-by-side diffs, detecting added lines better. (MR: reproducible-builds/diffoscope!64) * Allow passing file with list of arguments to ArgumentParser (eg. "diffoscope @args.txt"). (MR: reproducible-builds/diffoscope!62)

You find out more by visiting the project homepage.

Dirk Eddelbuettel: RcppArmadillo 0.9.900.2.0

Friday 17th of July 2020 04:37:00 PM

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 757 other packages on CRAN.

Conrad just released a new minor upstream version 9.900.2 of Armadillo which we packaged and tested as usual first as a ‘release candidate’ build and then as the release. As usual, logs from reverse-depends runs are in the rcpp-logs repo.

All changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.900.2.0 (2020-07-17)
  • Upgraded to Armadillo release 9.900.2 (Nocturnal Misbehaviour)

    • In sort(), fixes for inconsistencies between checks applied to matrix and vector expressions

    • In sort(), remove unnecessary copying when applied in-place to vectors function when applied in-place to vectors

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Sven Hoexter: Debian and exFAT

Friday 17th of July 2020 03:58:03 PM

As some might have noticed we now have Linux 5.7 in Debian/unstable and subsequently the in kernel exFAT implementation created by Samsung available. Thus we now have two exFAT implementations, the exfat fuse driver and the Linux based one. Since some comments and mails I received showed minor confusions, especially around the available tooling, it might help to clarify a bit what is required when.

Using the Samsung Linux Kernel Implementation

Probably the most common use case is that you just want to use the in kernel implementation. Easy, install a Linux 5.7 kernel package for your architecture and either remove the exfat-fuse package or make sure you've version 1.3.0-2 or later installed. Then you can just run mount /dev/sdX /mnt and everything should be fine.

Your result will look something like this:

$ mount|grep sdb /dev/sdb on /mnt type exfat (rw,relatime,fmask=0022,dmask=0022,iocharset=utf8,errors=remount-ro

In the past this basic mount invocation utilized the mount.exfat helper, which was just a symlink to the helper which is shipped as /sbin/mount.exfat-fuse. The link was dropped from the package in 1.3.0-2. If you're running a not so standard setup, and would like to keep an older version of exfat-fuse installed you must invoke mount -i to prevent mount from loading any helper to mount the filesystem. See also man 8 mount.

For those who care, mstone@ and myself had a brief discussion about this issue in #963752, which quickly brought me to the conclusion that it's in the best interest of a majority of users to just drop the symlink from the package.

Sticking to the Fuse Driver

If you would like to stick to the fuse driver you can of course just do it. I plan to continue to maintain all packages for the moment. Just keep the exfat-fuse package installed and use the mount.exfat-fuse helper directly. E.g.

$ sudo mount.exfat-fuse /dev/sdb /mnt FUSE exfat 1.3.0 $ mount|grep sdb /dev/sdb on /mnt type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096)

In case this is something you would like to make permanent, I would recommend that you create yourself a mount.exfat symlink pointing at the mount.exfat-fuse helper.

mkfs and fsck - exfat-utils vs exfatprogs

Beside of the filesystem access itself we now also have two implementations of tooling, which support filesystem creation, fscking and label adjustment. The older one is exfat-utils by the creator of the fuse driver, which is also part of Debian since the fuse driver was packaged in 2012. New in Debian is the exfatprogs package written by the Samsung engineers. And here the situation starts to get a bit more complicated.

Both packages ship a mkfs.exfat and fsck.exfat tool, so we can not co-install them. In the end both packages declare a conflict with each other at the moment. As outlined in this thread I do not plan to overcomplicate the situation by using the alternatives system. I do feel strongly that this would just create more confusion without a real benefit. Since the tools do not have matching cli options, that could also cause additional issues and confusion.

I plan to keep that as is, at least for the bullseye release. Afterwards it's possible, depending on how the usage evolves, to drop the mkfs.exfat and fsck.exfat from exfat-utils, they are in fact again only symlinks. pain point might be tools interfacing with the differing implementations. Currently I see only three reverse depedencies, so that should be manageable to consolidate if required.

Last but not least it might be relevant to mention that the exfat-utils package also contains a dumpexfat tool which could be helpful if you're more into forensics, or looking into other lower level analysis of an exFAT filsystem. Thus there is a bit of an interest to have those tools co-installed in some - I would say - niche cases.

buster-backports

Well if you use buster with a backports kernel you're a bit on your own. In case you want to keep the fuse driver installed, but would still like to mount, e.g. for testing, with the kernel exFAT driver, you must use mount -i. I do not plan any uploads to buster-backports. If you need a mkfs.exfat on buster, I would recommend to just use the one from exfat-utils for now. It has been good enough for the past years, should not get sour before the bullseye release, which ships exfatprogs for you.

Kudos

My sincere kudos go to:

  • Andrew Nayenko who wrote the exFAT fuse implemetation which was very helpful to many people for the past years. He's a great upstream to work with.
  • Namjae Jeon and Hyunchul Lee who maintain the Linux exFAT driver and exfatprogs. They are also very responsive upstreams and easy to work with.
  • Last but not least our ftp-master who reviewed the exfatprogs package way faster than what I had anticipated looking at the current NEW backlog.

Dima Kogan: Visualizing a broken internet connection

Thursday 16th of July 2020 11:47:00 PM

We've all seen our ISPs crap out periodically. Things feel slow, we then go ping some server, and see lost packets or slow response times. When bitching to the ISP it's useful to have evidence so you can clearly explain exactly how they are fucking up. This happened to me again last week, so I wrote a quick oneliner to do the visualization. Here it is:

( echo '# i dt'; ping 8.8.8.8 | perl -ne 'BEGIN { $|=1} next if /PING/; s/.*seq=([0-9]+).*time=([0-9]+).*/\1 \2/ && print') | tee /tmp/ping.vnl | vnl-filter --stream -p i,dt,di='diff(i)' | vnl-filter --stream --has di | feedgnuplot --domain --stream --with 'linespoints pt 7 palette' --tuplesizeall 3 --ymin 0 --set 'cbrange [1:]' --xlabel "ping index" --ylabel "Response time (ms)" --title "ping 8.8.8.8 response times. Consecutive indices shown as colors"

You run that, and you get a realtime-updating plot of ping times and missed packets. The one-liner also logs the data to a file, so the saved data can be re-visualized. Showing what happens on my currently-working internet connection:

< /tmp/ping.vnl vnl-filter -p i,dt,di='diff(i)' | vnl-filter --has di | feedgnuplot --domain --with 'linespoints pt 7 palette' --tuplesizeall 3 --ymin 0 --set 'cbrange [1:]' --xlabel "ping index" --ylabel "Response time (ms)" --title "ping 8.8.8.8 response times. Consecutive indices shown as colors" --hardcopy /tmp/isp-good.svg

On a broken network it looks like this (I edited the log to show the kind of broken-ness I was seeing earlier):

The x axis is the ping index. The y axis is the response time. Misses responses are shows as gaps in the points and also as colors, for legibility.

As usual, this uses the vnlog and feedgnuplot tools, so

sudo apt install feedgnuplot vnlog

if you want to try this

Louis-Philippe Véronneau: DebConf Videoteam Sprint Report -- DebConf20@Home

Thursday 16th of July 2020 09:45:19 PM

DebConf20 starts in about 5 weeks, and as always, the DebConf Videoteam is working hard to make sure it'll be a success. As such, we held a sprint from July 9th to 13th to work on our new infrastructure.

A remote sprint certainly ain't as fun as an in-person one, but we nonetheless managed to enjoy ourselves. Many thanks to those who participated, namely:

  • Carl Karsten (CarlFK)
  • Ivo De Decker (ivodd)
  • Kyle Robbertze (paddatrapper)
  • Louis-Philippe Véronneau (pollo)
  • Nattie Mayer-Hutchings (nattie)
  • Nicolas Dandrimont (olasd)
  • Stefano Rivera (tumbleweed)
  • Wouter Verhelst (wouter)

We also wish to extend our thanks to Thomas Goirand and Infomaniak for providing us with virtual machines to experiment on and host the video infrastructure for DebConf20.

Advice for presenters

For DebConf20, we strongly encourage presenters to record their talks in advance and send us the resulting video. We understand this is more work, but we think it'll make for a more agreeable conference for everyone. Video conferencing is still pretty wonky and there is nothing worse than a talk ruined by a flaky internet connection or hardware failures.

As such, if you are giving a talk at DebConf this year, we are asking you to read and follow our guide on how to record your presentation.

Fear not: we are not getting rid of the Q&A period at the end of talks. Attendees will ask their questions — either on IRC or on a collaborative pad — and the Talkmeister will relay them to the speaker once the pre-recorded video has finished playing.

New infrastructure, who dis?

Organising a virtual DebConf implies migrating from our battle-tested on-premise workflow to a completely new remote one.

One of the major changes this means for us is the addition of Jitsi Meet to our infrastructure. We normally have 3 different video sources in a room: two cameras and a slides grabber. With the new online workflow, directors will be able to play pre-recorded videos as a source, will get a feed from a Jitsi room and will see the audience questions as a third source.

This might seem simple at first, but is in fact a very major change to our workflow and required a lot of work to implement.

== On-premise == || == Online == || Camera 1 || Jitsi | || | v ---> Frontend || v ---> Frontend | || | Slides -> Voctomix -> Backend -+--> Frontend || Questions -> Voctomix -> Backend -+--> Frontend | || | ^ ---> Frontend || ^ ---> Frontend | || | Camera 2 || Pre-recorded video

In our tests, playing back pre-recorded videos to voctomix worked well, but was sometimes unreliable due to inconsistent encoding settings. Presenters will thus upload their pre-recorded talks to SReview so we can make sure there aren't any obvious errors. Videos will then be re-encoded to ensure a consistent encoding and to normalise audio levels.

This process will also let us stitch the Q&As at the end of the pre-recorded videos more easily prior to publication.

Reducing the stream latency

One of the pitfalls of the streaming infrastructure we have been using since 2016 is high video latency. In a worst case scenario, remote attendees could get up to 45 seconds of latency, making participation in events like BoFs arduous.

In preparation for DebConf20, we added a new way to stream our talks: RTMP. Attendees will thus have the option of using either an HLS stream with higher latency or an RTMP stream with lower latency.

Here is a comparative table that can help you decide between the two protocols:

HLS RTMP Pros
  • Can be watched from a browser
  • Auto-selects a stream encoding
  • Single URL to remember
  • Lower latency (~5s)
Cons
  • Higher latency (up to 45s)
  • Requires a dedicated video player (VLC, mpv)
  • Specific URLs for each encoding setting
Live mixing from home with VoctoWeb

Since DebConf16, we have been using voctomix, a live video mixer developed by the CCC VOC. voctomix is conveniently divided in two: voctocore is the backend server while voctogui is a GTK+ UI frontend directors can use to live-mix.

Although voctogui can connect to a remote server, it was primarily designed to run either on the same machine as voctocore or on the same LAN. Trying to use voctogui from a machine at home to connect to a voctocore running in a datacenter proved unreliable, especially for high-latency and low bandwidth connections.

Inspired by the setup FOSDEM uses, we instead decided to go with a web frontend for voctocore. We initially used FOSDEM's code as a proof of concept, but quickly reimplemented it in Python, a language we are more familiar with as a team.

Compared to the FOSDEM PHP implementation, voctoweb implements A / B source selection (akin to voctogui) as well as audio control, two very useful features. In the following screen captures, you can see the old PHP UI on the left and the new shiny Python one on the right.

Voctoweb is still under development and is likely to change quite a bit until DebConf20. Still, the current version seems to works well enough to be used in production if you ever need to.

Python GeoIP redirector

We run multiple geographically-distributed streaming frontend servers to minimize the load on our streaming backend and to reduce overall latency. Although users can connect to the frontends directly, we typically point them to live.debconf.org and redirect connections to the nearest server.

Sadly, 6 months ago MaxMind decided to change the licence on their GeoLite2 database and left us scrambling. To fix this annoying issue, Stefano Rivera wrote a Python program that uses the new database and reworked our ansible frontend server role. Since the new database cannot be redistributed freely, you'll have to get a (free) license key from MaxMind if you to use this role.

Ansible & CI improvements

Infrastructure as code is a living process and needs constant care to fix bugs, follow changes in DSL and to implement new features. All that to say a large part of the sprint was spent making our ansible roles and continuous integration setup more reliable, less buggy and more featureful.

All in all, we merged 26 separate ansible-related merge request during the sprint! As always, if you are good with ansible and wish to help, we accept merge requests on our ansible repository :)

Enrico Zini: Build Qt5 cross-builder with raspbian sysroot: compiling with the sysroot

Thursday 16th of July 2020 10:00:00 AM

This is part of a series of posts on compiling a custom version of Qt5 in order to develop for both amd64 and a Raspberry Pi.

Now that I have a sysroot, I try to use it to build Qt5 with QtWebEngine.

Nothing seems to work straightforwardly with Qt5's build system, and hit an endless series of significant blockers to try and work around.

Problem in wayland code

QtWayland's source currently does not compile:

../../../hardwareintegration/client/brcm-egl/qwaylandbrcmeglwindow.cpp: In constructor ‘QtWaylandClient::QWaylandBrcmEglWindow::QWaylandBrcmEglWindow(QWindow*)’: ../../../hardwareintegration/client/brcm-egl/qwaylandbrcmeglwindow.cpp:131:67: error: no matching function for call to ‘QtWaylandClient::QWaylandWindow::QWaylandWindow(QWindow*&)’ , m_eventQueue(wl_display_create_queue(mDisplay->wl_display())) ^ In file included from ../../../../include/QtWaylandClient/5.15.0/QtWaylandClient/private/qwaylandwindow_p.h:1, from ../../../hardwareintegration/client/brcm-egl/qwaylandbrcmeglwindow.h:43, from ../../../hardwareintegration/client/brcm-egl/qwaylandbrcmeglwindow.cpp:40: ../../../../include/QtWaylandClient/5.15.0/QtWaylandClient/private/../../../../../src/client/qwaylandwindow_p.h:97:5: note: candidate: ‘QtWaylandClient::QWaylandWindow::QWaylandWindow(QWindow*, QtWayland Client::QWaylandDisplay*)’ QWaylandWindow(QWindow *window, QWaylandDisplay *display); ^~~~~~~~~~~~~~ ../../../../include/QtWaylandClient/5.15.0/QtWaylandClient/private/../../../../../src/client/qwaylandwindow_p.h:97:5: note: candidate expects 2 arguments, 1 provided make[5]: Leaving directory '/home/build/armhf/qt-everywhere-src-5.15.0/qttools/src/qdoc'

I am not trying to debug here. I understand that Wayland support is not a requirement, and I'm adding -skip wayland to Qt5's configure options.

Next round.

nss not found

Qt5 embeds Chrome's sources. Chrome's sources require libnss3-dev to be available for both host and target architectures. Although I now have it installed both on the build system and in the sysroot, the pkg-config wrapper that Qt5 hooks into its Chrome's sources, failes to find it:

Command: /usr/bin/python2 /home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/3rdparty/chromium/build/config/linux/pkg-config.py -s /home/build/sysroot/ -a arm -p /usr/bin/arm-linux-gnueabihf-pkg-config --system_libdir lib nss -v -lssl3 Returned 1. stderr: Package nss was not found in the pkg-config search path. Perhaps you should add the directory containing `nss.pc' to the PKG_CONFIG_PATH environment variable No package 'nss' found Traceback (most recent call last): File "/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/3rdparty/chromium/build/config/linux/pkg-config.py", line 248, in <module> sys.exit(main()) File "/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/3rdparty/chromium/build/config/linux/pkg-config.py", line 143, in main prefix = GetPkgConfigPrefixToStrip(options, args) File "/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/3rdparty/chromium/build/config/linux/pkg-config.py", line 82, in GetPkgConfigPrefixToStrip "--variable=prefix"] + args, env=os.environ).decode('utf-8') File "/usr/lib/python2.7/subprocess.py", line 223, in check_output raise CalledProcessError(retcode, cmd, output=output) subprocess.CalledProcessError: Command '['/usr/bin/arm-linux-gnueabihf-pkg-config', '--variable=prefix', 'nss']' returned non-zero exit status 1 See //build/config/linux/nss/BUILD.gn:15:3: whence it was called. pkg_config("system_nss_no_ssl_config") { ^--------------------------------------- See //crypto/BUILD.gn:218:25: which caused the file to be included. public_configs += [ "//build/config/linux/nss:system_nss_no_ssl_config" ] ^-------------------------------------------------- Project ERROR: GN run error!

It's trying to look into $SYSROOT/usr/lib/pkgconfig, while it should be $SYSROOT//usr/lib/arm-linux-gnueabihf/pkgconfig.

I worked around this this patch to qtwebengine/src/3rdparty/chromium/build/config/linux/pkg-config.py:

--- pkg-config.py.orig 2020-07-16 11:46:21.005373002 +0200 +++ pkg-config.py 2020-07-16 11:46:02.605296967 +0200 @@ -61,6 +61,7 @@ libdir = sysroot + '/usr/' + options.system_libdir + '/pkgconfig' libdir += ':' + sysroot + '/usr/share/pkgconfig' + libdir += ':' + sysroot + '/usr/lib/arm-linux-gnueabihf/pkgconfig' os.environ['PKG_CONFIG_LIBDIR'] = libdir return libdir

Next round.

g++ 8.3.0 Internal Compiler Error

Qt5's sources embed Chrome's sources that embed the skia library sources.

One of the skia library sources, when cross-compiled to ARM with -O1 or -O2 with g++ 8.3.0, produces an Internal Compiler Error:

/usr/bin/arm-linux-gnueabihf-g++ -MMD -MF obj/skia/skcms/skcms.o.d -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DOFFICIAL_BUILD -DTOOLKIT_QT -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNO_UNWIND_TABLES -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DCR_SYSROOT_HASH=76e6068f9f6954e2ab1ff98ce5fa236d3d85bcbd -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -I../../3rdparty/chromium/third_party/skia/include/third_party/skcms -Igen -I../../3rdparty/chromium -w -std=c11 -mfp16-format=ieee -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -fno-unwind-tables -fno-asynchronous-unwind-tables -fPIC -pipe -pthread -march=armv7-a -mfloat-abi=hard -mtune=generic-armv7-a -mfpu=vfpv3-d16 -mthumb -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wno-psabi -Wno-unused-local-typedefs -Wno-maybe-uninitialized -Wno-deprecated-declarations -fno-delete-null-pointer-checks -Wno-comments -Wno-packed-not-aligned -Wno-dangling-else -Wno-missing-field-initializers -Wno-unused-parameter -O2 -fno-ident -fdata-sections -ffunction-sections -fno-omit-frame-pointer -g0 -fvisibility=hidden -std=gnu++14 -Wno-narrowing -Wno-class-memaccess -Wno-attributes -Wno-class-memaccess -Wno-subobject-linkage -Wno-invalid-offsetof -Wno-return-type -Wno-deprecated-copy -fno-exceptions -fno-rtti --sysroot=../../../../../../sysroot/ -fvisibility-inlines-hidden -c ../../3rdparty/chromium/third_party/skia/third_party/skcms/skcms.cc -o obj/skia/skcms/skcms.o during RTL pass: expand In file included from ../../3rdparty/chromium/third_party/skia/third_party/skcms/skcms.cc:2053: ../../3rdparty/chromium/third_party/skia/third_party/skcms/src/Transform_inl.h: In function ‘void baseline::exec_ops(const Op*, const void**, const char*, char*, int)’: ../../3rdparty/chromium/third_party/skia/third_party/skcms/src/Transform_inl.h:766:13: internal compiler error: in convert_move, at expr.c:218 static void exec_ops(const Op* ops, const void** args, ^~~~~~~~ Please submit a full bug report, with preprocessed source if appropriate. See <file:///usr/share/doc/gcc-8/README.Bugs> for instructions.

I reported the bug at https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96206

Since this source compiles with -O0, I attempted to fix this by editing qtwebkit/src/3rdparty/chromium/build/config/compiler/BUILD.gn and replacing instances of -O1 and -O2 with -O0.

Spoiler: wrong attempt. We'll see it in the next round.

Impossible constraint in asm

Qt5's sources embed Chrome's sources that embed the ffmpeg library sources. Even if ffmpeg's development libraries are present both in the host and in the target system, the build system insists in compiling and using the bundled version.

Unfortunately, using -O0 breaks the build of ffmpeg:

/usr/bin/arm-linux-gnueabihf-gcc -MMD -MF obj/third_party/ffmpeg/ffmpeg_internal/opus.o.d -DHAVE_AV_CONFIG_H -D_POSIX_C_SOURCE=200112 -D_XOPEN_SOURCE=600 -DPIC -DFFMPEG_CONFIGURATION=NULL -DCHROMIUM_NO_LOGGING -D_ISOC99_SOURCE -D_LARGEFILE_SOURCE -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DOFFICIAL_BUILD -DTOOLKIT_QT -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNO_UNWIND_TABLES -DCR_SYSROOT_HASH=76e6068f9f6954e2ab1ff98ce5fa236d3d85bcbd -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DOPUS_FIXED_POINT -I../../3rdparty/chromium/third_party/ffmpeg/chromium/config/Chromium/linux/arm -I../../3rdparty/chromium/third_party/ffmpeg -I../../3rdparty/chromium/third_party/ffmpeg/compat/atomics/gcc -Igen -I../../3rdparty/chromium -I../../3rdparty/chromium/third_party/opus/src/include -fPIC -Wno-deprecated-declarations -fomit-frame-pointer -w -std=c99 -pthread -fno-math-errno -fno-signed-zeros -fno-tree-vectorize -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -fno-unwind-tables -fno-asynchronous-unwind-tables -fPIC -pipe -pthread -march=armv7-a -mfloat-abi=hard -mtune=generic-armv7-a -mfpu=vfpv3-d16 -mthumb -g0 -fvisibility=hidden -Wno-psabi -Wno-unused-local-typedefs -Wno-maybe-uninitialized -Wno-deprecated-declarations -fno-delete-null-pointer-checks -Wno-comments -Wno-packed-not-aligned -Wno-dangling-else -Wno-missing-field-initializers -Wno-unused-parameter -O0 -fno-ident -fdata-sections -ffunction-sections -std=gnu11 --sysroot=../../../../../../sysroot/ -c ../../3rdparty/chromium/third_party/ffmpeg/libavcodec/opus.c -o obj/third_party/ffmpeg/ffmpeg_internal/opus.o In file included from ../../3rdparty/chromium/third_party/ffmpeg/libavutil/intmath.h:30, from ../../3rdparty/chromium/third_party/ffmpeg/libavutil/common.h:106, from ../../3rdparty/chromium/third_party/ffmpeg/libavutil/avutil.h:296, from ../../3rdparty/chromium/third_party/ffmpeg/libavutil/audio_fifo.h:30, from ../../3rdparty/chromium/third_party/ffmpeg/libavcodec/opus.h:28, from ../../3rdparty/chromium/third_party/ffmpeg/libavcodec/opus_celt.h:29, from ../../3rdparty/chromium/third_party/ffmpeg/libavcodec/opus.c:32: ../../3rdparty/chromium/third_party/ffmpeg/libavcodec/opus.c: In function ‘ff_celt_quant_bands’: ../../3rdparty/chromium/third_party/ffmpeg/libavutil/arm/intmath.h:77:5: error: impossible constraint in ‘asm’ __asm__ ("usat %0, %2, %1" : "=r"(x) : "r"(a), "i"(p)); ^~~~~~~

The same source compiles with using -O2 instead of -O0.

I worked around this by undoing the previous change, and limiting -O0 to just the source that causes the Internal Compiler Error.

I edited qtwebengine/src/3rdparty/chromium/third_party/skia/third_party/skcms/skcms.cc to prepend:

#pragma GCC push_options #pragma GCC optimize ("O0")

and append:

#pragma GCC pop_options

Next round.

Missing build-deps for i386 code

Qt5's sources embed Chrome's sources that embed the V8 library sources.

For some reason, torque, that is part of V8, wants to build some of its sources into 32 bit code with -m32, and I did not have i386 cross-compilation libraries installed:

/usr/bin/g++ -MMD -MF v8_snapshot/obj/v8/torque_base/csa-generator.o.d -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DOFFICIAL_BUILD -DTOOLKIT_QT -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNO_UNWIND_TABLES -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=64 -DENABLE_MINOR_MC -DV8_INTL_SUPPORT -DV8_CONCURRENT_MARKING -DV8_ENABLE_LAZY_SOURCE_POSITIONS -DV8_EMBEDDED_BUILTINS -DV8_SHARED_RO_HEAP -DV8_WIN64_UNWINDING_INFO -DV8_ENABLE_REGEXP_INTERPRETER_THREADED_DISPATCH -DV8_31BIT_SMIS_ON_64BIT_ARCH -DV8_DEPRECATION_WARNINGS -DV8_TARGET_ARCH_ARM -DCAN_USE_ARMV7_INSTRUCTIONS -DCAN_USE_VFP3_INSTRUCTIONS -DUSE_EABI_HARDFLOAT=1 -DV8_HAVE_TARGET_OS -DV8_TARGET_OS_LINUX -DDISABLE_UNTRUSTED_CODE_MITIGATIONS -DV8_31BIT_SMIS_ON_64BIT_ARCH -DV8_DEPRECATION_WARNINGS -Iv8_snapshot/gen -I../../3rdparty/chromium -I../../3rdparty/chromium/v8 -Iv8_snapshot/gen/v8 -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -fno-unwind-tables -fno-asynchronous-unwind-tables -fPIC -pipe -pthread -m32 -msse2 -mfpmath=sse -mmmx -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wno-unused-local-typedefs -Wno-maybe-uninitialized -Wno-deprecated-declarations -fno-delete-null-pointer-checks -Wno-comments -Wno-packed-not-aligned -Wno-dangling-else -Wno-missing-field-initializers -Wno-unused-parameter -fno-omit-frame-pointer -g0 -fvisibility=hidden -Wno-strict-overflow -Wno-return-type -O3 -fno-ident -fdata-sections -ffunction-sections -std=gnu++14 -Wno-narrowing -Wno-class-memaccess -Wno-attributes -Wno-class-memaccess -Wno-subobject-linkage -Wno-invalid-offsetof -Wno-return-type -Wno-deprecated-copy -fvisibility-inlines-hidden -fexceptions -frtti -c ../../3rdparty/chromium/v8/src/torque/csa-generator.cc -o v8_snapshot/obj/v8/torque_base/csa-generator.o In file included from ../../3rdparty/chromium/v8/src/torque/csa-generator.h:8, from ../../3rdparty/chromium/v8/src/torque/csa-generator.cc:5: /usr/include/c++/8/iostream:38:10: fatal error: bits/c++config.h: No such file or directory #include <bits/c++config.h> ^~~~~~~~~~~~~~~~~~ compilation terminated.

New build dependencies needed:

apt install lib32stdc++-8-dev apt install libc6-dev-i386 dpkg --add-architecture i386 apt install linux-libc-dev:i386

Next round.

OpenGL build issues

Next bump are OpenGL related compiler issues:

/usr/bin/arm-linux-gnueabihf-g++ -MMD -MF obj/QtWebEngineCore/gl_ozone_glx_qt.o.d -DCHROMIUM_VERSION=\"80.0.3987.163\" -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DOFFICIAL_BUILD -DTOOLKIT_QT -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNO_UNWIND_TABLES -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DCR_SYSROOT_HASH=76e6068f9f6954e2ab1ff98ce5fa236d3d85bcbd -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DQT_NO_LINKED_LIST -DQT_NO_KEYWORDS -DQT_USE_QSTRINGBUILDER -DQ_FORWARD_DECLARE_OBJC_CLASS=QT_FORWARD_DECLARE_CLASS -DQTWEBENGINECORE_VERSION_STR=\"5.15.0\" -DQTWEBENGINEPROCESS_NAME=\"QtWebEngineProcess\" -DBUILDING_CHROMIUM -DQTWEBENGINE_EMBEDDED_SWITCHES -DQT_NO_EXCEPTIONS -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -DQT_NO_DEBUG -DQT_QUICK_LIB -DQT_GUI_LIB -DQT_QMLMODELS_LIB -DQT_WEBCHANNEL_LIB -DQT_QML_LIB -DQT_NETWORK_LIB -DQT_POSITIONING_LIB -DQT_CORE_LIB -DQT_WEBENGINECOREHEADERS_LIB -DVK_NO_PROTOTYPES -DGL_GLEXT_PROTOTYPES -DUSE_GLX -DUSE_EGL -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DHAVE_PTHREAD -DU_USING_ICU_NAMESPACE=0 -DU_ENABLE_DYLOAD=0 -DUSE_CHROMIUM_ICU=1 -DU_STATIC_IMPLEMENTATION -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_FILE -DUCHAR_TYPE=uint16_t -DWEBRTC_NON_STATIC_TRACE_EVENT_HANDLERS=0 -DWEBRTC_CHROMIUM_BUILD -DWEBRTC_POSIX -DWEBRTC_LINUX -DABSL_ALLOCATOR_NOTHROW=1 -DWEBRTC_USE_BUILTIN_ISAC_FIX=1 -DWEBRTC_USE_BUILTIN_ISAC_FLOAT=0 -DHAVE_SCTP -DNO_MAIN_THREAD_WRAPPING -DSK_HAS_PNG_LIBRARY -DSK_HAS_WEBP_LIBRARY -DSK_USER_CONFIG_HEADER=\"../../skia/config/SkUserConfig.h\" -DSK_GL -DSK_HAS_JPEG_LIBRARY -DSK_USE_LIBGIFCODEC -DSK_VULKAN_HEADER=\"../../skia/config/SkVulkanConfig.h\" -DSK_VULKAN=1 -DSK_SUPPORT_GPU=1 -DSK_GPU_WORKAROUNDS_HEADER=\"gpu/config/gpu_driver_bug_workaround_autogen.h\" -DVK_NO_PROTOTYPES -DLEVELDB_PLATFORM_CHROMIUM=1 -DLEVELDB_PLATFORM_CHROMIUM=1 -DV8_31BIT_SMIS_ON_64BIT_ARCH -DV8_DEPRECATION_WARNINGS -I../../3rdparty/chromium/skia/config -I../../3rdparty/chromium/third_party -I../../3rdparty/chromium/third_party/boringssl/src/include -I../../3rdparty/chromium/third_party/skia/include/core -Igen -I../../3rdparty/chromium -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/core -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/core/api -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQuick/5.15.0 -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQuick/5.15.0/QtQuick -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include/QtGui/5.15.0 -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include/QtGui/5.15.0/QtGui -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQuick -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include/QtGui -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQmlModels/5.15.0 -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQmlModels/5.15.0/QtQmlModels -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQml/5.15.0 -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQml/5.15.0/QtQml -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include/QtCore/5.15.0 -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include/QtCore/5.15.0/QtCore -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQmlModels -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebchannel/include -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebchannel/include/QtWebChannel -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQml -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include/QtNetwork -I/home/build/armhf/qt-everywhere-src-5.15.0/qtlocation/include -I/home/build/armhf/qt-everywhere-src-5.15.0/qtlocation/include/QtPositioning -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include/QtCore -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/include -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/include/QtWebEngineCore -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/include/QtWebEngineCore/5.15.0 -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/include/QtWebEngineCore/5.15.0/QtWebEngineCore -I.moc -I/home/build/sysroot/opt/vc/include -I/home/build/sysroot/opt/vc/include/interface/vcos/pthreads -I/home/build/sysroot/opt/vc/include/interface/vmcs_host/linux -Igen/.moc -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/mkspecs/devices/linux-rasp-pi2-g++ -Igen -Igen -I../../3rdparty/chromium/third_party/libyuv/include -Igen -I../../3rdparty/chromium/third_party/jsoncpp/source/include -I../../3rdparty/chromium/third_party/jsoncpp/generated -Igen -Igen -I../../3rdparty/chromium/third_party/khronos -I../../3rdparty/chromium/gpu -I../../3rdparty/chromium/third_party/vulkan/include -I../../3rdparty/chromium/third_party/perfetto/include -Igen/third_party/perfetto/build_config -Igen -Igen -Igen/third_party/dawn/src/include -I../../3rdparty/chromium/third_party/dawn/src/include -Igen -I../../3rdparty/chromium/third_party/boringssl/src/include -I../../3rdparty/chromium/third_party/protobuf/src -Igen/protoc_out -I../../3rdparty/chromium/third_party/protobuf/src -I../../3rdparty/chromium/third_party/ced/src -I../../3rdparty/chromium/third_party/icu/source/common -I../../3rdparty/chromium/third_party/icu/source/i18n -I../../3rdparty/chromium/third_party/webrtc_overrides -I../../3rdparty/chromium/third_party/webrtc -Igen/third_party/webrtc -I../../3rdparty/chromium/third_party/abseil-cpp -I../../3rdparty/chromium/third_party/skia -I../../3rdparty/chromium/third_party/libgifcodec -I../../3rdparty/chromium/third_party/vulkan/include -I../../3rdparty/chromium/third_party/skia/third_party/vulkanmemoryallocator -I../../3rdparty/chromium/third_party/vulkan/include -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -I../../3rdparty/chromium/third_party/crashpad/crashpad -I../../3rdparty/chromium/third_party/crashpad/crashpad/compat/non_mac -I../../3rdparty/chromium/third_party/crashpad/crashpad/compat/linux -I../../3rdparty/chromium/third_party/crashpad/crashpad/compat/non_win -I../../3rdparty/chromium/third_party/libwebm/source -I../../3rdparty/chromium/third_party/leveldatabase -I../../3rdparty/chromium/third_party/leveldatabase/src -I../../3rdparty/chromium/third_party/leveldatabase/src/include -I../../3rdparty/chromium/v8/include -Igen/v8/include -I../../3rdparty/chromium/third_party/mesa_headers -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -fno-unwind-tables -fno-asynchronous-unwind-tables -fPIC -pipe -pthread -march=armv7-a -mfloat-abi=hard -mtune=generic-armv7-a -mfpu=vfpv3-d16 -mthumb -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wno-psabi -Wno-unused-local-typedefs -Wno-maybe-uninitialized -Wno-deprecated-declarations -fno-delete-null-pointer-checks -Wno-comments -Wno-packed-not-aligned -Wno-dangling-else -Wno-missing-field-initializers -Wno-unused-parameter -O2 -fno-ident -fdata-sections -ffunction-sections -fno-omit-frame-pointer -g0 -fvisibility=hidden -g -O2 -fdebug-prefix-map=/home/build/armhf/qt-everywhere-src-5.15.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -O2 -fno-exceptions -Wall -Wextra -D_REENTRANT -I/home/build/sysroot/usr/include/nss -I/home/build/sysroot/usr/include/nspr -std=gnu++14 -Wno-narrowing -Wno-class-memaccess -Wno-attributes -Wno-class-memaccess -Wno-subobject-linkage -Wno-invalid-offsetof -Wno-return-type -Wno-deprecated-copy -fno-exceptions -fno-rtti --sysroot=../../../../../../sysroot/ -fvisibility-inlines-hidden -g -O2 -fdebug-prefix-map=/home/build/armhf/qt-everywhere-src-5.15.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -O2 -std=gnu++1y -fno-exceptions -Wall -Wextra -D_REENTRANT -Wno-unused-parameter -Wno-unused-variable -Wno-deprecated-declarations -c /home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/core/ozone/gl_ozone_glx_qt.cpp -o obj/QtWebEngineCore/gl_ozone_glx_qt.o In file included from ../../3rdparty/chromium/ui/gl/gl_bindings.h:497, from ../../3rdparty/chromium/ui/gl/gl_gl_api_implementation.h:12, from /home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/core/ozone/gl_ozone_glx_qt.cpp:49: ../../3rdparty/chromium/ui/gl/gl_bindings_autogen_egl.h:227:5: error: ‘EGLSetBlobFuncANDROID’ has not been declared EGLSetBlobFuncANDROID set, ^~~~~~~~~~~~~~~~~~~~~ ../../3rdparty/chromium/ui/gl/gl_bindings_autogen_egl.h:228:5: error: ‘EGLGetBlobFuncANDROID’ has not been declared EGLGetBlobFuncANDROID get); ^~~~~~~~~~~~~~~~~~~~~ ../../3rdparty/chromium/ui/gl/gl_bindings_autogen_egl.h:571:46: error: ‘EGLSetBlobFuncANDROID’ has not been declared EGLSetBlobFuncANDROID set, ^~~~~~~~~~~~~~~~~~~~~ ../../3rdparty/chromium/ui/gl/gl_bindings_autogen_egl.h:572:46: error: ‘EGLGetBlobFuncANDROID’ has not been declared EGLGetBlobFuncANDROID get) = 0; ^~~~~~~~~~~~~~~~~~~~~ cc1plus: warning: unrecognized command line option ‘-Wno-deprecated-copy’ /usr/bin/arm-linux-gnueabihf-g++ -MMD -MF obj/QtWebEngineCore/display_gl_output_surface.o.d -DCHROMIUM_VERSION=\"80.0.3987.163\" -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DOFFICIAL_BUILD -DTOOLKIT_QT -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNO_UNWIND_TABLES -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DCR_SYSROOT_HASH=76e6068f9f6954e2ab1ff98ce5fa236d3d85bcbd -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DQT_NO_LINKED_LIST -DQT_NO_KEYWORDS -DQT_USE_QSTRINGBUILDER -DQ_FORWARD_DECLARE_OBJC_CLASS=QT_FORWARD_DECLARE_CLASS -DQTWEBENGINECORE_VERSION_STR=\"5.15.0\" -DQTWEBENGINEPROCESS_NAME=\"QtWebEngineProcess\" -DBUILDING_CHROMIUM -DQTWEBENGINE_EMBEDDED_SWITCHES -DQT_NO_EXCEPTIONS -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -DQT_NO_DEBUG -DQT_QUICK_LIB -DQT_GUI_LIB -DQT_QMLMODELS_LIB -DQT_WEBCHANNEL_LIB -DQT_QML_LIB -DQT_NETWORK_LIB -DQT_POSITIONING_LIB -DQT_CORE_LIB -DQT_WEBENGINECOREHEADERS_LIB -DVK_NO_PROTOTYPES -DGL_GLEXT_PROTOTYPES -DUSE_GLX -DUSE_EGL -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DHAVE_PTHREAD -DU_USING_ICU_NAMESPACE=0 -DU_ENABLE_DYLOAD=0 -DUSE_CHROMIUM_ICU=1 -DU_STATIC_IMPLEMENTATION -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_FILE -DUCHAR_TYPE=uint16_t -DWEBRTC_NON_STATIC_TRACE_EVENT_HANDLERS=0 -DWEBRTC_CHROMIUM_BUILD -DWEBRTC_POSIX -DWEBRTC_LINUX -DABSL_ALLOCATOR_NOTHROW=1 -DWEBRTC_USE_BUILTIN_ISAC_FIX=1 -DWEBRTC_USE_BUILTIN_ISAC_FLOAT=0 -DHAVE_SCTP -DNO_MAIN_THREAD_WRAPPING -DSK_HAS_PNG_LIBRARY -DSK_HAS_WEBP_LIBRARY -DSK_USER_CONFIG_HEADER=\"../../skia/config/SkUserConfig.h\" -DSK_GL -DSK_HAS_JPEG_LIBRARY -DSK_USE_LIBGIFCODEC -DSK_VULKAN_HEADER=\"../../skia/config/SkVulkanConfig.h\" -DSK_VULKAN=1 -DSK_SUPPORT_GPU=1 -DSK_GPU_WORKAROUNDS_HEADER=\"gpu/config/gpu_driver_bug_workaround_autogen.h\" -DVK_NO_PROTOTYPES -DLEVELDB_PLATFORM_CHROMIUM=1 -DLEVELDB_PLATFORM_CHROMIUM=1 -DV8_31BIT_SMIS_ON_64BIT_ARCH -DV8_DEPRECATION_WARNINGS -I../../3rdparty/chromium/skia/config -I../../3rdparty/chromium/third_party -I../../3rdparty/chromium/third_party/boringssl/src/include -I../../3rdparty/chromium/third_party/skia/include/core -Igen -I../../3rdparty/chromium -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/core -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/core/api -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQuick/5.15.0 -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQuick/5.15.0/QtQuick -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include/QtGui/5.15.0 -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include/QtGui/5.15.0/QtGui -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQuick -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include/QtGui -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQmlModels/5.15.0 -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQmlModels/5.15.0/QtQmlModels -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQml/5.15.0 -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQml/5.15.0/QtQml -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include/QtCore/5.15.0 -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include/QtCore/5.15.0/QtCore -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQmlModels -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebchannel/include -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebchannel/include/QtWebChannel -I/home/build/armhf/qt-everywhere-src-5.15.0/qtdeclarative/include/QtQml -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include/QtNetwork -I/home/build/armhf/qt-everywhere-src-5.15.0/qtlocation/include -I/home/build/armhf/qt-everywhere-src-5.15.0/qtlocation/include/QtPositioning -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/include/QtCore -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/include -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/include/QtWebEngineCore -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/include/QtWebEngineCore/5.15.0 -I/home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/include/QtWebEngineCore/5.15.0/QtWebEngineCore -I.moc -I/home/build/sysroot/opt/vc/include -I/home/build/sysroot/opt/vc/include/interface/vcos/pthreads -I/home/build/sysroot/opt/vc/include/interface/vmcs_host/linux -Igen/.moc -I/home/build/armhf/qt-everywhere-src-5.15.0/qtbase/mkspecs/devices/linux-rasp-pi2-g++ -Igen -Igen -I../../3rdparty/chromium/third_party/libyuv/include -Igen -I../../3rdparty/chromium/third_party/jsoncpp/source/include -I../../3rdparty/chromium/third_party/jsoncpp/generated -Igen -Igen -I../../3rdparty/chromium/third_party/khronos -I../../3rdparty/chromium/gpu -I../../3rdparty/chromium/third_party/vulkan/include -I../../3rdparty/chromium/third_party/perfetto/include -Igen/third_party/perfetto/build_config -Igen -Igen -Igen/third_party/dawn/src/include -I../../3rdparty/chromium/third_party/dawn/src/include -Igen -I../../3rdparty/chromium/third_party/boringssl/src/include -I../../3rdparty/chromium/third_party/protobuf/src -Igen/protoc_out -I../../3rdparty/chromium/third_party/protobuf/src -I../../3rdparty/chromium/third_party/ced/src -I../../3rdparty/chromium/third_party/icu/source/common -I../../3rdparty/chromium/third_party/icu/source/i18n -I../../3rdparty/chromium/third_party/webrtc_overrides -I../../3rdparty/chromium/third_party/webrtc -Igen/third_party/webrtc -I../../3rdparty/chromium/third_party/abseil-cpp -I../../3rdparty/chromium/third_party/skia -I../../3rdparty/chromium/third_party/libgifcodec -I../../3rdparty/chromium/third_party/vulkan/include -I../../3rdparty/chromium/third_party/skia/third_party/vulkanmemoryallocator -I../../3rdparty/chromium/third_party/vulkan/include -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -Igen/third_party/perfetto -I../../3rdparty/chromium/third_party/crashpad/crashpad -I../../3rdparty/chromium/third_party/crashpad/crashpad/compat/non_mac -I../../3rdparty/chromium/third_party/crashpad/crashpad/compat/linux -I../../3rdparty/chromium/third_party/crashpad/crashpad/compat/non_win -I../../3rdparty/chromium/third_party/libwebm/source -I../../3rdparty/chromium/third_party/leveldatabase -I../../3rdparty/chromium/third_party/leveldatabase/src -I../../3rdparty/chromium/third_party/leveldatabase/src/include -I../../3rdparty/chromium/v8/include -Igen/v8/include -I../../3rdparty/chromium/third_party/mesa_headers -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -fno-unwind-tables -fno-asynchronous-unwind-tables -fPIC -pipe -pthread -march=armv7-a -mfloat-abi=hard -mtune=generic-armv7-a -mfpu=vfpv3-d16 -mthumb -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wno-psabi -Wno-unused-local-typedefs -Wno-maybe-uninitialized -Wno-deprecated-declarations -fno-delete-null-pointer-checks -Wno-comments -Wno-packed-not-aligned -Wno-dangling-else -Wno-missing-field-initializers -Wno-unused-parameter -O2 -fno-ident -fdata-sections -ffunction-sections -fno-omit-frame-pointer -g0 -fvisibility=hidden -g -O2 -fdebug-prefix-map=/home/build/armhf/qt-everywhere-src-5.15.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -O2 -fno-exceptions -Wall -Wextra -D_REENTRANT -I/home/build/sysroot/usr/include/nss -I/home/build/sysroot/usr/include/nspr -std=gnu++14 -Wno-narrowing -Wno-class-memaccess -Wno-attributes -Wno-class-memaccess -Wno-subobject-linkage -Wno-invalid-offsetof -Wno-return-type -Wno-deprecated-copy -fno-exceptions -fno-rtti --sysroot=../../../../../../sysroot/ -fvisibility-inlines-hidden -g -O2 -fdebug-prefix-map=/home/build/armhf/qt-everywhere-src-5.15.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -O2 -std=gnu++1y -fno-exceptions -Wall -Wextra -D_REENTRANT -Wno-unused-parameter -Wno-unused-variable -Wno-deprecated-declarations -c /home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/core/compositor/display_gl_output_surface.cpp -o obj/QtWebEngineCore/display_gl_output_surface.o In file included from ../../3rdparty/chromium/gpu/command_buffer/client/gles2_interface.h:8, from ../../3rdparty/chromium/gpu/command_buffer/client/client_transfer_cache.h:15, from ../../3rdparty/chromium/gpu/command_buffer/client/gles2_implementation.h:28, from /home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/core/compositor/display_gl_output_surface.cpp:47: /home/build/sysroot/opt/vc/include/GLES2/gl2.h:78: warning: "GL_FALSE" redefined #define GL_FALSE (GLboolean)0 In file included from ../../3rdparty/chromium/gpu/command_buffer/client/client_context_state.h:10, from ../../3rdparty/chromium/gpu/command_buffer/client/gles2_implementation.h:27, from /home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/core/compositor/display_gl_output_surface.cpp:47: ../../3rdparty/chromium/third_party/khronos/GLES3/gl3.h:85: note: this is the location of the previous definition #define GL_FALSE 0 In file included from ../../3rdparty/chromium/gpu/command_buffer/client/gles2_interface.h:8, from ../../3rdparty/chromium/gpu/command_buffer/client/client_transfer_cache.h:15, from ../../3rdparty/chromium/gpu/command_buffer/client/gles2_implementation.h:28, from /home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/core/compositor/display_gl_output_surface.cpp:47: /home/build/sysroot/opt/vc/include/GLES2/gl2.h:79: warning: "GL_TRUE" redefined #define GL_TRUE (GLboolean)1 In file included from ../../3rdparty/chromium/gpu/command_buffer/client/client_context_state.h:10, from ../../3rdparty/chromium/gpu/command_buffer/client/gles2_implementation.h:27, from /home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/core/compositor/display_gl_output_surface.cpp:47: ../../3rdparty/chromium/third_party/khronos/GLES3/gl3.h:86: note: this is the location of the previous definition #define GL_TRUE 1 In file included from ../../3rdparty/chromium/gpu/command_buffer/client/gles2_interface.h:8, from ../../3rdparty/chromium/gpu/command_buffer/client/client_transfer_cache.h:15, from ../../3rdparty/chromium/gpu/command_buffer/client/gles2_implementation.h:28, from /home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/core/compositor/display_gl_output_surface.cpp:47: /home/build/sysroot/opt/vc/include/GLES2/gl2.h:600:37: error: conflicting declaration of C function ‘void glShaderSource(GLuint, GLsizei, const GLchar**, const GLint*)’ GL_APICALL void GL_APIENTRY glShaderSource (GLuint shader, GLsizei count, const GLchar** string, const GLint* length); ^~~~~~~~~~~~~~ In file included from ../../3rdparty/chromium/gpu/command_buffer/client/client_context_state.h:10, from ../../3rdparty/chromium/gpu/command_buffer/client/gles2_implementation.h:27, from /home/build/armhf/qt-everywhere-src-5.15.0/qtwebengine/src/core/compositor/display_gl_output_surface.cpp:47: ../../3rdparty/chromium/third_party/khronos/GLES3/gl3.h:624:29: note: previous declaration ‘void glShaderSource(GLuint, GLsizei, const GLchar* const*, const GLint*)’ GL_APICALL void GL_APIENTRY glShaderSource (GLuint shader, GLsizei count, const GLchar *const*string, const GLint *length); ^~~~~~~~~~~~~~ cc1plus: warning: unrecognized command line option ‘-Wno-deprecated-copy’

I'm out of the allocated hour budget, and I'll stop here for now.

Building Qt5 has been providing some of the most nightmarish work time in my entire professional life. If my daily job became being required to deal with this kind of insanity, I would strongly invest in a change of career.

Update

Andreas Gruber wrote:

Long story short, a fast solution for the issue with EGLSetBlobFuncANDROID is to remove libraspberrypi-dev from your sysroot and do a full rebuild. There will be some changes to the configure results, so please review them - if they are relevant for you - before proceeding with your work.

And thanks to Andreas, the story can continue...

Enrico Zini: Build Qt5 cross-builder with raspbian sysroot: building the sysroot

Thursday 16th of July 2020 08:00:00 AM

This is part of a series of posts on compiling a custom version of Qt5 in order to develop for both amd64 and a Raspberry Pi.

As an attempt to get webview to compile, I'm reattempting to build a Qt5 cross-compiling environment using a raspbian sysroot, instead of having dependencies for both arm and amd64 installed in the build system.

Using dependencies installed in a straightforward way in the build system has failed because of issues like #963136, where some of the build dependencies are needed for both architectures, but the corresponding Debian -dev packages are not yet coinstallable.

This is something that causes many people much pain.

Start from a clean sysroot

Looking for a Raspbian image, I found out that it has been renamed to "Raspberry Pi OS". I realised that software names are like underwear: as soon as they become well used, they need to be changed.

I downloaded RaspbianRaspberry Pi OS Lite from https://www.raspberrypi.org/downloads/raspberry-pi-os/ to start with something minimal. It came out as something like 1.5G uncompressed, which wasn't as minimal as I would have hoped, but that'll be what I'll have to work with.

Adding build dependencies

I have acquired significant experience manipulating RaspbianRaspberry Pi OS images from working with Himblick.

This time I'm working with the disk image directly, instead of an SD card, since I will be needing it as a sysroot during the build, and I won't need to actually boot it on real hardware.

The trick is to work with kpartx to make the partitions in the image available as loopback block devices.

I have extracted a lot of relevant code from Himblick into a Python library I called Transilience

The result is this provisioning script, which is able to take a RaspbianRaspberry Pi OS image, enlarge it, and install Debian packages into it.

I find this script pretty cool, also in the way it embeds quite a bit of experience gathered on the field. I can also be integrated in a fully automated setup and provisioning system.

The next step will be to use the result as a sysroot to build Qt.

Russell Coker: Windows 10 on Debian under KVM

Thursday 16th of July 2020 04:07:47 AM

Here are some things that you need to do to get Windows 10 running on a Debian host under KVM.

UEFI Booting

UEFI is big and complex, but most of what it does isn’t needed at all. If all you want to do is boot from an image of a disk with a GPT partition table then you just install the package ovmf and add something like the following to your KVM start script:

UEFI"-drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd -drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_VARS.fd"

Note that some of the documentation on this doesn’t have the OVMF_VARS.fd file set to readonly. Allowing writes to that file means that the VM boot process (and maybe later) can change EFI variables that affect later boots and other VMs if they all share the same file. For a basic boot you don’t need to change variables so you want it read-only. Also having it read-only is necessary if you want to run KVM as non-root.

As an experiment I tried booting without the OVMF_VARS.fd file, it didn’t boot and then even after configuring it to use the OVMF_VARS.fd file again Windows gave a boot error about the “boot configuration data file” that required booting from recovery media. Apparently configuration mistakes with EFI can mess up the Windows installation, so be careful and backup the Windows installation regularly!

Linux can boot from EFI but you generally don’t want to unless the boot device is larger than 2TB. It’s relatively easy to convert a Linux installation on a GPT disk to a virtual image on a DOS partition table disk or on block devices without partition tables and that gives a faster boot. If the same person runs the host hardware and the VMs then the best choice for Linux is to have no partition tables just one filesystem per block device (which makes resizing much easier) and have the kernel passed as a parameter to kvm. So booting a VM from EFI is probably only useful for booting Windows VMs and for Linux boot loader development and testing.

As an aside, the Debian Wiki page about Secure Boot on a VM [4] was useful for this. It’s unfortunate that it and so much of the documentation about UEFI is about secure boot which isn’t so useful if you just want to boot a system without regard to the secure boot features.

Emulated IDE Disks

Debian kernels (and probably kernels from many other distributions) are compiled with the paravirtualised storage device drivers. Windows by default doesn’t support such devices so you need to emulate an IDE/SATA disk so you can boot Windows and install the paravirtualised storage driver. The following configuration snippet has a commented line for paravirtualised IO (which is fast) and an uncommented line for a virtual IDE/SATA disk that will allow an unmodified Windows 10 installation to boot.

#DRIVE="-drive format=raw,file=/home/kvm/windows10,if=virtio" DRIVE="-drive id=disk,format=raw,file=/home/kvm/windows10,if=none -device ahci,id=ahci -device ide-drive,drive=disk,bus=ahci.0" Spice Video

Spice is an alternative to VNC, Here is the main web site for Spice [1]. Spice has many features that could be really useful for some people, like audio, sharing USB devices from the client, and streaming video support. I don’t have a need for those features right now but it’s handy to have options. My main reason for choosing Spice over VNC is that the mouse cursor in the ssvnc doesn’t follow the actual mouse and can be difficult or impossible to click on items near edges of the screen.

The following configuration will make the QEMU code listen with SSL on port 1234 on all IPv4 addresses. Note that this exposes the Spice password to anyone who can run ps on the KVM server, I’ve filed Debian bug #965061 requesting the option of a password file to address this. Also note that the “qxl” virtual video hardware is VGA compatible and can be expected to work with OS images that haven’t been modified for virtualisation, but that they work better with special video drivers.

KEYDIR=/etc/letsencrypt/live/kvm.example.com-0001 -spice password=xxxxxxxx,x509-cacert-file=$KEYDIR/chain.pem,x509-key-file=$KEYDIR/privkey.pem,x509-cert-file=$KEYDIR/cert.pem,tls-port=1234,tls-channel=main -vga qxl

To connect to the Spice server I installed the spice-client-gtk package in Debian and ran the following command:

spicy -h kvm.example.com -s 1234 -w xxxxxxxx

Note that this exposes the Spice password to anyone who can run ps on the system used as a client for Spice, I’ve filed Debian bug #965060 requesting the option of a password file to address this.

This configuration with an unmodified Windows 10 image only supported 800*600 resolution VGA display.

Networking

To setup bridged networking as non-root you need to do something like the following as root:

chgrp kvm /usr/lib/qemu/qemu-bridge-helper setcap cap_net_admin+ep /usr/lib/qemu/qemu-bridge-helper mkdir -p /etc/qemu echo "allow all" > /etc/qemu/bridge.conf chgrp kvm /etc/qemu/bridge.conf chmod 640 /etc/qemu/bridge.conf

Windows 10 supports the emulated Intel E1000 network card. Configuration like the following configures networking on a bridge named br0 with an emulated E1000 card. MAC addresses that have a 1 in the second least significant bit of the first octet are “locally administered” (like IPv4 addresses starting with “10.”), see the Wikipedia page about MAC Address for details.

The following is an example of network configuration where $ID is an ID number for the virtual machine. So far I haven’t come close to 256 VMs on one network so I’ve only needed one octet.

NET="-device e1000,netdev=net0,mac=02:00:00:00:01:$ID -netdev tap,id=net0,helper=/usr/lib/qemu/qemu-bridge-helper,br=br0" Final KVM Settings KEYDIR=/etc/letsencrypt/live/kvm.example.com-0001 SPICE="-spice password=xxxxxxxx,x509-cacert-file=$KEYDIR/chain.pem,x509-key-file=$KEYDIR/privkey.pem,x509-cert-file=$KEYDIR/cert.pem,tls-port=1234,tls-channel=main -vga qxl" UEFI="-drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd -drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_VARS.fd" DRIVE="-drive format=raw,file=/home/kvm/windows10,if=virtio" NET="-device e1000,netdev=net0,mac=02:00:00:00:01:$ID -netdev tap,id=net0,helper=/usr/lib/qemu/qemu-bridge-helper,br=br0" kvm -m 4000 -smp 2 $SPICE $UEFI $DRIVE $NET Windows Settings

The Spice Download page has a link for “spice-guest-tools” that has the QNX video driver among other things [2]. This seems to be needed for resolutions greater than 800*600.

The Virt-Manager Download page has a link for “virt-viewer” which is the Spice client for Windows systems [3], they have MSI files for both i386 and AMD64 Windows.

It’s probably a good idea to set display and system to sleep after never (I haven’t tested what happens if you don’t do that, but there’s no benefit in sleeping). Before uploading an image I disabled the pagefile and set the partition to the minimum size so I had less data to upload.

Problems

Here are some things I haven’t solved yet.

The aSpice Android client for the Spice protocol fails to connect with the QEMU code at the server giving the following message on stderr: “error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca:../ssl/record/rec_layer_s3.c:1544:SSL alert number 48“.

Spice is supposed to support dynamic changes to screen resolution on the VM to match the window size at the client, this doesn’t work for me, not even with the Red Hat QNX drivers installed.

The Windows Spice client doesn’t seem to support TLS, I guess running some sort of proxy for TLS would work but I haven’t tried that yet.

Related posts:

  1. Debian PPC64EL Emulation In my post on Debian S390X Emulation [1] I mentioned...
  2. Debian S390X Emulation I decided to setup some virtual machines for different architectures....
  3. installing Xen domU on Debian Etch I have just been installing a Xen domU on Debian...

Evgeni Golov: Scanning with a Brother MFC-L2720DW on Linux without any binary blobs

Wednesday 15th of July 2020 07:08:05 PM

Back in 2015, I've got a Brother MFC-L2720DW for the casual "I need to print those two pages" and "I need to scan these receipts" at home (and home-office ;)). It's a rather cheap (I paid less than 200€ in 2015) monochrome laser printer, scanner and fax with a (well, two, wired and wireless) network interface. In those five years I've never used the fax or WiFi functions, but printed a scanned a few pages.

Brother offers Linux drivers, but those are binary blobs which I never really liked to run.

The printer part works just fine with a "Generic PCL 6/PCL XL" driver in CUPS or even "driverless" via AirPrint on Linux. You can also feed it plain PostScript, but I found it rather slow compared to PCL. On recent Androids it works using the built in printer service or Mopria Printer Service for older ones - I used to joke "why would you need a printer on your phone?!", but found it quite useful after a few tries.

However, for the scanner part I had to use Brother's brscan4 driver on Linux and their iPrint&Scan app on Android - Mopria Scan wouldn't support it.

Until, last Friday, I've seen a NEW package being uploaded to Debian: sane-airscan. And yes, monitoring the Debian NEW queue via Twitter is totally legit!

sane-airscan is an implementation of Apple's AirScan (eSCL) and Microsoft's WSD/WS-Scan protocols for SANE. I've never heard of those before - only about AirPrint, but thankfully this does not mean nobody has reverse-engineered them and created something that works beautifully on Linux. As of today there are no packages in the official Fedora repositories and the Debian ones are still in NEW, however the upstream documentation refers to an openSUSE OBS repository that works like a charm in the meantime (on Fedora 32).

The only drawback I've seen so far: the scanner only works in "Color" mode and there is no way to scan in "Grayscale", making scanning a tad slower. This has been reported upstream and might or might not be fixable, as it seems the device does not announce any mode besides "Color".

Interestingly, SANE has an eSCL backend on its own since 1.0.29, but it's disabled in Fedora in favor of sane-airscan even though the later isn't available in Fedora yet. However, it might not even need separate packaging, as SANE upstream is planning to integrate it into sane-backends directly.

Utkarsh Gupta: GSoC Phase 2

Wednesday 15th of July 2020 05:00:00 PM

Hello,

In early May, I got selected as a Google Summer of Code student for Debian to work on a project which is to write a linter (an extension to RuboCop).
This tool is mostly to help the Debian Ruby team. And that is the best part, I love working in/for/with the Ruby team!
(I’ve been an active part of the team for 19 months now :))

More details about the project can be found here, on the wiki.
And also, I have got the best mentors I could’ve possibly asked for: Antonio Terceiro and David Rodríguez

More in Tux Machines

Linux Devices and Open Hardware

  • Mini-PC and SBC build on Whiskey Lake

    Supermicro’s 3.5-inch “X11SWN-H-WOHS” SBC and “SYS-E100-9W-H” mini-PC based it feature an 8th Gen UE-series CPU, HDMI and DP, 4x USB 3.1 Gen2, 2x GbE, and 3x M.2. Supermicro has launched a fanless, 8th Gen Whiskey Lake SBC and mini-PC. The SYS-E100-9W-H mini-PC (or SuperServer E100-9W-H), which was reported on by Fanless Tech, is certified only to run Windows 10, but the 3.5-inch X11SWN-H-WOHS SBC supports Ubuntu. Applications include industrial automation, retail, smart medical expert systems, kiosks, interactive info systems, and digital signage.

  • Exor nanoSOM nS02 System-on-Module Features the 800MHz version of STM32MP1 Processor

    Exor provides a Linux RT board support package (BSP) or Android BSP for the module which also fully supports the company’s X Platform including Exor Embedded Open HMI software, Corvina Cloud IIoT platform, and IEC61131 CODESYS or Exor xPLC runtime.

  • Onyx Boox Poke2 Color eReader Launched for $299

    Manga and comics fans, rejoice! After years of getting black & white eReaders, the first commercial color eReaders are coming to market starting with Onyx Boox Poke2 Color eReader sold for $299 (but sadly sold out at the time of writing). The eReader comes with a 6-inch, 1448 x 1072 E-Ink display that supports up to 4096 colors, and runs Android 9.0 on an octa-core processor coupled with 2GB RAM and 32GB storage.

  • xDrill Smart Power Drill Supports Intelligent Speed/Torque, Laser Measuring, Digital Leveling (Crowdfunding)

    Many home appliances now have smart functions, and in my cases, I fail to see the added value, and I’m not sure why I’d want/need a connected refrigerator with a touchscreen display. So when I first saw somebody make a “smart” power drill with a small touchscreen display I laughed. But after having a closer look, Robbox xDrill smart power drill could actually be a very useful device saving you time and helping work better.

  • Raspberry Pi calls out your custom workout routine
  • Odyssey Blue: A powerful x86 and Arduino machine that supports Windows 10 and Linux

    It has been a few months since we reported on the Odyssey, a single-board computer (SBC) designed by Seeedstudio. Unlike many SBCs, the Odyssey, or ODYSSEY-X86J4105800 to give it its full name, supported the x86 instruction set. While the Odyssey can run Windows 10, it is also compatible with the Arduino ecosystem. Now, Seeedstudio has expanded on the design of the Odyssey with the Odyssey Blue.

  • Bring two analog meters out of retirement to display temperature and humidity

    Tom of Build Comics created a unique analog weather station that shows temperature and humidity on a pair of recycled gauges. An Arduino Nano reads the levels using a DHT22 sensor and outputs them in the proper format for each display. Both units have a new printed paper backing to indicate conditions, along with a trimmer pot for calibration. To set the build off nicely, the Nano and other electronics are housed inside a beautiful custom wooden box, to which the antique meters are also affixed.

Programming Leftovers

  • Engineer Your Own Electronics With PCB Design Software

    A lot of self-styled geeks out there tend to like to customize their own programs, devices, and electronics. And for the true purists, that can mean building from the ground up (you know, like Superman actor Henry Cavill building a gaming PC to the delight of the entire internet). Building electronics from the ground up can mean a lot of different things: acquiring parts, sometimes from strange sources; a bit of elbow grease on the mechanical side of things; and today, even taking advantage of the 3D printing revolution that’s finally enabling people to manufacture customized objects in their home. Beyond all of these things though, engineering your own devices can also mean designing the underlying electronics — beginning with printed circuit boards, also known as PCBs. [...] On the other hand, there are also plenty of just-for-fun options to consider. For example, consider our past buyer’s guide to the best Linux laptop, in which we noted that you can always further customize your hardware. With knowledge of PCB design, that ability to customize even a great computer or computer setup is further enhanced. You might, for instance, learn how to craft PCBs and devices amounting to your own mouse, gaming keyboard, or homemade speakers — all of which can make your hardware more uniquely your own. All in all, PCB design is a very handy skill to have in 2020. It’s not typically necessary, in that there’s usually a device or some light customization that can give you whatever you want or need out of your electronics. But for “geeks” and tech enthusiasts, knowledge of PCB design adds another layer to the potential to customize hardware.

  • Programming pioneer Fran Allen dies aged 88 after a career of immense contributions to compilers

    Frances Allen, one of the leading computer scientists of her generation and a pioneer of women in tech, died last Tuesday, her 88th birthday. Allen is best known for her work on compiler organisation and optimisation algorithms. Together with renowned computer scientist John Cocke, she published a series of landmark papers in the late '60s and '70s that helped to lay the groundwork for modern programming. In recognition of her efforts, in 2006 Allen became the first woman to be awarded the AM Turing Award, often called the Nobel Prize of computing.

  • Excellent Free Tutorials to Learn ECMAScript

    ECMAScript is an object‑oriented programming language for performing computations and manipulating computational objects within a host environment. The language was originally designed as a scripting language, but is now often used as a general purpose programming language. ECMAScript is best known as the language embedded in web browsers but has also been widely adopted for server and embedded applications.

  • Alexander Larsson: Compatibility in a sandboxed world

    Compatibility has always been a complex problems in the Linux world. With the advent of containers/sandboxing it has become even more complicated. Containers help solve compatibility problems, but there are still remaining issues. Especially on the Linux desktop where things are highly interconnected. In fact, containers even create some problems that we didn’t use to have. Today I’ll take a look at the issues in more details and give some ideas on how to best think of compatibility in this post-container world, focusing on desktop use with technologies like flatpak and snap. [...] Another type of compatibility is that of communication protocols. Two programs that talk to each other using a networking API (which could be on two different machines, or locally on the same machine) need to use a protocol to understand each other. Changes to this protocol need to be carefully considered to ensure they are compatible. In the remote case this is pretty obvious, as it is very hard to control what software two different machines use. However, even for local communication between processes care has to be taken. For example, a local service could be using a protocol that has several implementations and they all need to stay compatible. Sometimes local services are split into a service and a library and the compatibility guarantees are defined by the library rather than the service. Then we can achieve some level of compatibility by ensuring the library and the service are updated in lock-step. For example a distribution could ship them in the same package.

  • GXml-0.20 Released

    GXml is an Object Oriented implementation of DOM version 4, using GObject classes and written in Vala. Has a fast and robust serialization implementation from GObject to XML and back, with a high degree of control. After serialization, provides a set of collections where you can get access to child nodes, using lists or hash tables. New 0.20 release is the first step toward 1.0. It provides cleaner API and removes old unmaintained implementations. GXml is the base of other projects depending on DOM4, like GSVG an engine to read SVG documents based on its specificacion 1.0. GXml uses a method to set properties and fill declared containers for child nodes, accessing GObject internals directly, making it fast. A libxml-2.0 engine is used to read sequentially each node, but is prepared to implement new ones in the future.

  • Let Mom Help You With Object-Oriented Programming

    Mom is a shortcut for creating Moo classes (and roles). It allows you to define a Moo class with the brevity of Class::Tiny. (In fact, Mom is even briefer.) A simple example: Mom allows you to use Moo features beyond simply declaring Class::Tiny-like attributes though. You can choose whether attributes are read-only, read-write, or read-write-private, whether they're required or optional, specify type constraints, defaults, etc.

  • Perl Weekly Challenge 73: Min Sliding Window and Smallest Neighbor

    These are some answers to the Week 73 of the Perl Weekly Challenge organized by Mohammad S. Anwar. Spoiler Alert: This weekly challenge deadline is due in a few days from now (on Aug. 16, 2020). This blog post offers some solutions to this challenge, please don’t read on if you intend to complete the challenge on your own.

  • [rakulang] 2020.32 Survey, Please

    The TPF Marketing Committee wants to learn more about how you perceive “The Perl Foundation” itself, and asks you to fill in this survey (/r/rakulang, /r/perl comments). Thank you!

Hardware With Linux Support: NUVIA and AMD Wraith Prism

  • Performance Delivered a New Way

    The server CPU has evolved at an incredible pace over the last two decades. Gone are the days of discrete CPUs, northbridges, southbridges, memory controllers, other external I/O and security chips. In today’s modern data center, the SoC (System On A Chip) does it all. It is the central point of coordination for virtually all workloads and the main hub where all the fixed-function accelerators connect, such as AI accelerators, GPUs, network interface controllers, storage devices, etc.

  • NUVIA Published New Details On Their Phoenix CPU, Talks Up Big Performance/Perf-Per-Watt

    Since leaving stealth last year and hiring some prominent Linux/open-source veterans to complement their ARM processor design experts, we have been quite eager to hear more about this latest start-up aiming to deliver compelling ARM server products. Today they shared some early details on their initial "Phoenix" processor that is coming within their "Orion" SoC. The first-generation Phoenix CPU is said to have a "complete overhaul" of the CPU pipeline and is a custom core based on the ARM architecture. They believe that Phoenix+Orion will be able to take on Intel/AMD x86_64 CPUs not only in raw performance but also in performance-per-Watt.

  • Take control of your AMD Wraith Prism RGB on Linux with Wraith Master

    Where the official vendor doesn't bother with supporting Linux properly, once again the community steps in to provide. If you want to tweak your AMD Wraith Prism lighting on Linux, check out Wraith Master. It's a similar project to CM-RGB that we previously highlighted. With the Wraith Master project, they provide a "feature-complete" UI and command-line app for controlling the fancy LED system on AMD's Wraith Prism cooler with eventual plans to support more.

The Massive Privacy Loopholes in School Laptops

It’s back to school time and with so many school districts participating in distance learning, many if not most are relying on computers and technology more than ever before. Wealthier school districts are providing their students with laptops or tablets, but not all schools can afford to provide each student with a computer which means that this summer parents are scrambling to find a device for their child to use for school. Geoffery Fowler wrote a guide in the Washington Post recently to aid parents in sourcing a computer or tablet for school. Given how rough kids can be with their things, many people are unlikely to give their child an expensive, premium laptop. The guide mostly focuses on incredibly low-cost, almost-disposable computers, so you won’t find a computer in the list that has what I consider a critical feature for privacy in the age of video conferencing: hardware kill switches. Often a guide like this would center on Chromebooks as Google has invested a lot of resources to get low-cost Chromebooks into schools yet I found Mr. Fowler’s guide particularly interesting because of his opinion on Chromebooks in education... Read more Also: Enabling Dark Mode on a Chromebook (Do not try this at home)