Language Selection

English French German Italian Portuguese Spanish

The Bash Fingertips: Making Your Own 'Information Centre'

Filed under
Howtos

Information Centre

FORGET bloated Web browsers. Forget so-called 'social' media (I call it social control media). They're not efficient, they eat up a lot of memory and CPU cycles, and the interfaces are not consistent (across sites). They're sufficiently distracting and they have ads. They erode privacy. They don't scale well; neither for an aging system (my laptop turns 10 in a few months) nor for users. GUIs are good in particular scenarios, but when the same things are repeated over and over again one might as well set up scripts, automating things and tailoring one's own interfaces, which is easy to achieve (relatively fast and simple) in the command line. It's also more accessible, e.g. over SSH. The pertinent tools are already out there (available for download/installation from repositories), they just need to be put together and programming skills aren't required, just batching in a bash file.

Some years ago I 'developed' a little script (I've been scripting since I was about 12). I called it getswap-sorted.sh and it just ran another script that helped me see what applications use the swap (and how much of it). For the sake of speed I like to restart applications that heavily use swap (i.e. depend on magnetic disk operations). I don't have much RAM. I never had more than 2 GB. getswap-sorted.sh just called out ./getswap.sh | sort -n -k 5 and getswap.sh comes from Erik Ljungstrom. Here it is:

#!/bin/bash
# Get current swap usage for all running processes
# Erik Ljungstrom 27/05/2011
SUM=0
OVERALL=0
for DIR in `find /proc/ -maxdepth 1 -type d | egrep "^/proc/[0-9]"` ; do
PID=`echo $DIR | cut -d / -f 3`
PROGNAME=`ps -p $PID -o comm --no-headers`
for SWAP in `grep Swap $DIR/smaps 2>/dev/null| awk '{ print $2 }'`
do
let SUM=$SUM+$SWAP
done
echo "PID=$PID - Swap used: $SUM - ($PROGNAME )"
let OVERALL=$OVERALL+$SUM
SUM=0

done

The output of getswap-sorted.sh would be something like this:


PID=1559 - Swap used: 16472 - (x-terminal-emul )
PID=21980 - Swap used: 16648 - (kwalletd5 )
PID=25548 - Swap used: 16704 - (konversation )
PID=631 - Swap used: 19336 - (kded5 )
PID=23817 - Swap used: 50048 - (pidgin )
PID=23923 - Swap used: 180312 - (thunderbird )


This helps me see which application/process number uses swap and to what degree. It's sorted by the amount of swap taken and the PID helps when I just want to kill a process from the command line (some are small and obsolete anyway).

My script, however, grew bigger over time. I added more things to it, eventually binding it to a special (fifth) mouse key, using xbindkeys -- an immensely valuable and powerful program I've used since around 2004. Extra mouse buttons always seemed worthless (anything more than three), but that's just because there was no program I needed to open or action I needed to invoke often enough. Over time I found that keeping a new terminal one click away (fourth button) and another special terminal also a click away improved my workflow/productivity. I just needed to invest some time in tailoring it. I ended up opening, temporarily, a terminal window with important information displayed, such as weather, disk space (I'm always near the limits), swap usage (I have only 2GB of RAM), uptime, real-time football scores etc. Change of wallpapers was lumped in too, for good measure...

For football tables/scores use one of the following 1) livescore-cli 2) soccer-cli and 3) football-cli.

Sadly, the above CLI football scores' tools got 'stolen' by Microsoft and need to isolate themselves GitHub, in due cource/time. I use the first of the three as it suits my needs best and does not require an API key.

The output looks like this:

 ... Fetching information from www.livescore.com ... 
Displaying Table for Barclay's Premier League
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
                                Barclay's Premier League TABLE
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 LP     Team Name               GP      W       D       L       GF      GA      GD      Pts
--------------------------------------------------------------------------------------------------
 1      Liverpool               24      19      4       1       55      14      41      61
 2      Tottenham Hotspur       25      19      0       6       51      24      27      57
 3      Manchester City         24      18      2       4       63      19      44      56
 4      Chelsea                 25      15      5       5       45      23      22      50
 5      Arsenal                 24      14      5       5       50      33      17      47
 6      Manchester United       24      13      6       5       48      35      13      45
 7      Wolverhampton Wanderers 25      11      5       9       33      32      1       38
 8      Watford                 25      9       7       9       33      34      -1      34
 9      Everton                 25      9       6       10      36      36      0       33
 10     AFC Bournemouth         25      10      3       12      37      44      -7      33
 11     Leicester City          24      9       5       10      30      30      0       32
 12     West Ham United         24      9       4       11      30      37      -7      31
 13     Brighton & Hove Albion  25      7       6       12      27      36      -9      27
 14     Crystal Palace          25      7       5       13      26      33      -7      26
 15     Newcastle United        25      6       6       13      21      33      -12     24
 16     Southampton             25      5       9       11      27      42      -15     24
 17     Burnley                 25      6       6       13      26      46      -20     24
 18     Cardiff City            25      6       4       15      22      46      -24     22
 19     Fulham                  25      4       5       16      25      55      -30     17
 20     Huddersfield Town       25      2       5       18      13      46      -33     11
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 LP = League Position   GP = Games Played       W = Wins        D = Draws       L = Lose 
 GF = Goals For         GA = Goal Against       GD = Goal Differences
--------------------------------------------------------------------------------------------------
 Champions League       Champions League qualification  Europa League
 Europa League qualification    Relegation
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Real-time scores (when matches are on):

 ... Fetching information from www.livescore.com ... 
Displaying Scores for Barclay's Premier League
----------------------------------------------------------------------------
                 Barclay's Premier League SCORES 
----------------------------------------------------------------------------
 January 29  FT     Arsenal                  2 - 1  Cardiff City           
 January 29  FT     Fulham                   4 - 2  Brighton & Hove Albion 
 January 29  FT     Huddersfield Town        0 - 1  Everton                
 January 29  FT     Wolverhampton Wanderers  3 - 0  West Ham United        
 January 29  FT     Manchester United        2 - 2  Burnley                
 January 29  FT     Newcastle United         2 - 1  Manchester City        
 January 30  FT     AFC Bournemouth          4 - 0  Chelsea                
 January 30  FT     Southampton              1 - 1  Crystal Palace         
 January 30  FT     Liverpool                1 - 1  Leicester City         
 January 30  FT     Tottenham Hotspur        2 - 1  Watford                
 February 2  FT     Tottenham Hotspur        1 - 0  Newcastle United       
 February 2  FT     Brighton & Hove Albion   0 - 0  Watford                
 February 2  FT     Burnley                  1 - 1  Southampton            
 February 2  FT     Chelsea                  5 - 0  Huddersfield Town      
 February 2  FT     Crystal Palace           2 - 0  Fulham                 
 February 2  FT     Everton                  1 - 3  Wolverhampton Wanderers
 February 2  FT     Cardiff City             2 - 0  AFC Bournemouth        
 February 3  15:05  Leicester City           ? - ?  Manchester United      
 February 3  17:30  Manchester City          ? - ?  Arsenal                
 February 4  21:00  West Ham United          ? - ?  Liverpool              
----------------------------------------------------------------------------
---------------------------------------

Now putting it all together:


feh --bg-fill --randomize /media/roy/c3fd5b6e-794f-4f24-b3e7-b4ead3722f11/home/roy/Main/Graphics/Wallpapers/Single\ Head/natgeo/* &

livescore -t bpl 

./getswap.sh | sort -n -k 5
 curl -4 http://wttr.in/Manchester
 swapon --summary | grep sda2
 df | grep sda1
uptime

sleep 10

livescore -s bpl 

sleep 40

The first line is feh choosing a wallpaper at random from a collection of award-winning National Geographic photographs. The options and the underlying parameters are self-explanatory.

The football league's table is then shown.

Next, after about 10 seconds of processing, a list of processes will show up based on swap usage (as described above)

The weather at home (Manchester) will then be shown, with colour. Right now I get:

Weather report: Manchester

     \   /     Sunny
      .-.      -5--2 °C       
   ― (   ) ―   ↑ 9 km/h       
      `-’      10 km          
     /   \     0.0 mm         
                                                       ┌─────────────┐                                                       
┌──────────────────────────────┬───────────────────────┤  Sun 03 Feb ├───────────────────────┬──────────────────────────────┐
│            Morning           │             Noon      └──────┬──────┘     Evening           │             Night            │
├──────────────────────────────┼──────────────────────────────┼──────────────────────────────┼──────────────────────────────┤
│    \  /       Partly cloudy  │      .-.      Light drizzle  │  _`/"".-.     Light rain sho…│               Mist           │
│  _ /"".-.     -4-0 °C        │     (   ).    -2-3 °C        │   ,\_(   ).   1-3 °C         │  _ - _ - _ -  0-3 °C         │
│    \_(   ).   ↑ 12-20 km/h   │    (___(__)   ↑ 17-26 km/h   │    /(___(__)  ↗ 7-14 km/h    │   _ - _ - _   ↑ 9-17 km/h    │
│    /(___(__)  20 km          │     ‘ ‘ ‘ ‘   20 km          │      ‘ ‘ ‘ ‘  16 km          │  _ - _ - _ -  13 km          │
│               0.0 mm | 0%    │    ‘ ‘ ‘ ‘    0.4 mm | 83%   │     ‘ ‘ ‘ ‘   0.4 mm | 65%   │               0.0 mm | 0%    │
└──────────────────────────────┴──────────────────────────────┴──────────────────────────────┴──────────────────────────────┘
                                                       ┌─────────────┐                                                       
┌──────────────────────────────┬───────────────────────┤  Mon 04 Feb ├───────────────────────┬──────────────────────────────┐
│            Morning           │             Noon      └──────┬──────┘     Evening           │             Night            │
├──────────────────────────────┼──────────────────────────────┼──────────────────────────────┼──────────────────────────────┤
│      .-.      Light drizzle  │  _`/"".-.     Patchy rain po…│               Cloudy         │               Cloudy         │
│     (   ).    2-6 °C         │   ,\_(   ).   3-7 °C         │      .--.     1-4 °C         │      .--.     -2 °C          │
│    (___(__)   → 16-26 km/h   │    /(___(__)  → 20-27 km/h   │   .-(    ).   → 13-23 km/h   │   .-(    ).   ↗ 9-16 km/h    │
│     ‘ ‘ ‘ ‘   14 km          │      ‘ ‘ ‘ ‘  18 km          │  (___.__)__)  20 km          │  (___.__)__)  20 km          │
│    ‘ ‘ ‘ ‘    0.3 mm | 88%   │     ‘ ‘ ‘ ‘   0.3 mm | 88%   │               0.0 mm | 0%    │               0.0 mm | 0%    │
└──────────────────────────────┴──────────────────────────────┴──────────────────────────────┴──────────────────────────────┘
                                                       ┌─────────────┐                                                       
┌──────────────────────────────┬───────────────────────┤  Tue 05 Feb ├───────────────────────┬──────────────────────────────┐
│            Morning           │             Noon      └──────┬──────┘     Evening           │             Night            │
├──────────────────────────────┼──────────────────────────────┼──────────────────────────────┼──────────────────────────────┤
│    \  /       Partly cloudy  │               Overcast       │               Overcast       │      .-.      Light drizzle  │
│  _ /"".-.     -1-3 °C        │      .--.     2-6 °C         │      .--.     6 °C           │     (   ).    1 °C           │
│    \_(   ).   ↖ 19-31 km/h   │   .-(    ).   ↑ 23-33 km/h   │   .-(    ).   ↑ 24-40 km/h   │    (___(__)   ↑ 24-40 km/h   │
│    /(___(__)  20 km          │  (___.__)__)  19 km          │  (___.__)__)  8 km           │     ‘ ‘ ‘ ‘   9 km           │
│               0.0 mm | 0%    │               0.0 mm | 0%    │               0.0 mm | 0%    │    ‘ ‘ ‘ ‘    0.3 mm | 0%    │
└──────────────────────────────┴──────────────────────────────┴──────────────────────────────┴──────────────────────────────┘


After this I am shown general memory usage and disk usage (for a particular partition) along with uptime thusly:

/dev/sda2                               partition       2097148 381128  -1
/dev/sda1        84035088   77299588   2443660  97% /
 08:03:28 up 116 days, 12:36,  7 users,  load average: 1.70, 1.40, 1.26

It will close on its own after I see what needs seeing, owing to the sleep command. It saves me the clicking (required to then close the window); it just fades away or 'expires', so to speak (until the next time the mouse button gets pressed).

More in Tux Machines

Games: Surviving Mars and OpenMW

Kernel and Security: BPF, Mesa, Embedded World, Kernel Address Sanitizer and More

  • Concurrency management in BPF
    In the beginning, programs run on the in-kernel BPF virtual machine had no persistent internal state and no data that was shared with any other part of the system. The arrival of eBPF and, in particular, its maps functionality, has changed that situation, though, since a map can be shared between two or more BPF programs as well as with processes running in user space. That sharing naturally leads to concurrency problems, so the BPF developers have found themselves needing to add primitives to manage concurrency (the "exchange and add" or XADD instruction, for example). The next step is the addition of a spinlock mechanism to protect data structures, which has also led to some wider discussions on what the BPF memory model should look like. A BPF map can be thought of as a sort of array or hash-table data structure. The actual data stored in a map can be of an arbitrary type, including structures. If a complex structure is read from a map while it is being modified, the result may be internally inconsistent, with surprising (and probably unwelcome) results. In an attempt to prevent such problems, Alexei Starovoitov introduced BPF spinlocks in mid-January; after a number of quick review cycles, version 7 of the patch set was applied on February 1. If all goes well, this feature will be included in the 5.1 kernel.
  • Intel Ready To Add Their Experimental "Iris" Gallium3D Driver To Mesa
    For just over the past year Intel open-source driver developers have been developing a new Gallium3D-based OpenGL driver for Linux systems as the eventual replacement to their long-standing "i965 classic" Mesa driver. The Intel developers are now confident enough in the state of this new driver dubbed Iris that they are looking to merge the driver into mainline Mesa proper.  The Iris Gallium3D driver has now matured enough that Kenneth Graunke, the Intel OTC developer who originally started Iris in late 2017, is looking to merge the driver into the mainline code-base of Mesa. The driver isn't yet complete but it's already in good enough shape that he's looking for it to be merged albeit marked experimental.
  • Hallo Nürnberg!
    Collabora is headed to Nuremberg, Germany next week to take part in the 2019 edition of Embedded World, "the leading international fair for embedded systems". Following a successful first attendance in 2018, we are very much looking forward to our second visit! If you are planning on attending, please come say hello in Hall 4, booth 4-280! This year, we will be showcasing a state-of-the-art infrastructure for end-to-end, embedded software production. From the birth of a software platform, to reproducible continuous builds, to automated testing on hardware, get a firsthand look at our platform building expertise and see how we use continuous integration to increase productivity and quality control in embedded Linux.
  • KASAN Spots Another Kernel Vulnerability From Early Linux 2.6 Through 4.20
    The Kernel Address Sanitizer (KASAN) that detects dynamic memory errors within the Linux kernel code has just picked up another win with uncovering a use-after-free vulnerability that's been around since the early Linux 2.6 kernels. KASAN (along with the other sanitizers) have already proven quite valuable in spotting various coding mistakes hopefully before they are exploited in the real-world. The Kernel Address Sanitizer picked up another feather in its hat with being responsible for the CVE-2019-8912 discovery.
  • io_uring, SCM_RIGHTS, and reference-count cycles
    The io_uring mechanism that was described here in January has been through a number of revisions since then; those changes have generally been fixing implementation issues rather than changing the user-space API. In particular, this patch set seems to have received more than the usual amount of security-related review, which can only be a good thing. Security concerns became a bit of an obstacle for io_uring, though, when virtual filesystem (VFS) maintainer Al Viro threatened to veto the merging of the whole thing. It turns out that there were some reference-counting issues that required his unique experience to straighten out. The VFS layer is a complicated beast; it must manage the complexities of the filesystem namespace in a way that provides the highest possible performance while maintaining security and correctness. Achieving that requires making use of almost all of the locking and concurrency-management mechanisms that the kernel offers, plus a couple more implemented internally. It is fair to say that the number of kernel developers who thoroughly understand how it works is extremely small; indeed, sometimes it seems like Viro is the only one with the full picture. In keeping with time-honored kernel tradition, little of this complexity is documented, so when Viro gets a moment to write down how some of it works, it's worth paying attention. In a long "brain dump", Viro described how file reference counts are managed, how reference-count cycles can come about, and what the kernel does to break them. For those with the time to beat their brains against it for a while, Viro's explanation (along with a few corrections) is well worth reading. For the rest of us, a lighter version follows.

Blacklisting insecure filesystems in openSUSE

The Linux kernel supports a wide variety of filesystem types, many of which have not seen significant use — or maintenance — in many years. Developers in the openSUSE project have concluded that many of these filesystem types are, at this point, more useful to attackers than to openSUSE users and are proposing to blacklist many of them by default. Such changes can be controversial, but it's probably still fair to say that few people expected the massive discussion that resulted, covering everything from the number of OS/2 users to how openSUSE fits into the distribution marketplace. On January 30, Martin Wilck started the discussion with a proposal to add a blacklist preventing the automatic loading of a set of kernel modules implementing (mostly) old filesystems. These include filesystems like JFS, Minix, cramfs, AFFS, and F2FS. For most of these, the logic is that the filesystems are essentially unused and the modules implementing them have seen little maintenance in recent decades. But those modules can still be automatically loaded if a user inserts a removable drive containing one of those filesystem types. There are a number of fuzz-testing efforts underway in the kernel community, but it seems relatively unlikely that any of them are targeting, say, FreeVxFS filesystem images. So it is not unreasonable to suspect that there just might be exploitable bugs in those modules. Preventing modules for ancient, unmaintained filesystems from automatically loading may thus protect some users against flash-drive attacks. If there were to be a fight over a proposal like this, one would ordinarily expect it to be concerned with the specific list of unwelcome modules. But there was relatively little of that. One possible exception is F2FS, the presence of which raised some eyebrows since it is under active development, having received 44 changes in the 5.0 development cycle, for example. Interestingly, it turns out that openSUSE stopped shipping F2FS in September. While the filesystem is being actively developed, it seems that, with rare exceptions, nobody is actively backporting fixes, and the filesystem also lacks a mechanism to prevent an old F2FS implementation from being confused by a filesystem created by a newer version. Rather than deal with these issues, openSUSE decided to just drop the filesystem altogether. As it happens, the blacklist proposal looks likely to allow F2FS to return to the distribution since it can be blacklisted by default. Read more

gitgeist: a git-based social network proof of concept

Are you tired of not owning the data or the platform you use for social postings? I know I am. It's hard to say when I "first" used a social network. I've been on email for about 30 years and one of the early ad-hoc forms of social networks were chain emails. Over the years I was asked to join all sorts of "social" things such as IRC, ICQ, Skype, MSN Messenger, etc. and eventually things like Orkut, MySpace, Facebook, etc. I'll readily admit that I'm not the type of person that happily jumps onto every new social bandwagon that appears on the Internet. I often prefer preserving the quietness of my own thoughts. That, though, hasn't stopped me from finding some meaningfulness participating in Twitter, Facebook, LinkedIn and more recently Google+. Twitter was in fact the first social network that I truly embraced. And it would've remained my primary social network had they not killed their own community by culling the swell of independently-developed Twitter clients that existed. That and their increased control of their API effectively made me look for something else. Right around that time Google+ was being introduced and many in the open source community started participating in that, in some ways to find a fresh place where techies can aggregate away from the noise and sometimes over-the-top nature of Facebook. Eventually I took to that too and started using G+ as my primary social network. That is, until Google recently decided to pull the plug on G+. While Google+ might not have represented a success for Google, it had become a good place for sharing information among the technically-inclined. As such, I found it quite useful for learning and hearing about new things in my field. Soon-to-be-former users of G+ have gone in all sorts of directions. Some have adopted a "c'mon guys, get over it, Facebook is the spot" attitude, others have adopted things like Mastodon, others have fallen back to their existing IDs on Twitter, and yet others, like me, are still looking. Read more