Language Selection

English French German Italian Portuguese Spanish

Custom scripting gives users a safe-du

Filed under
HowTos

My company has a Linux cluster with a terabyte of attached storage. Over time we noticed the head node was becoming more overloaded. Inspection of the system showed that users were starting dozens of copies of the du utility to determine disk space usage. This was a natural thing for them to do, because they had a need to know how much disk space was available. A lack of disk space would cause their software builds and tests to fail. The problem was that it takes five to seven hours for a du of the entire shared filesystem. Thus, when the filesystem was nearly full (as it of course usually was), the number of du processes would increase almost exponentially.

To address this problem, we first set up automated nightly disk space reports, so that users could check the status without running du. This still did not solve the problem, as the amount of used space could fluctuate dramatically over the course of 24 hours. Users still wanted and needed to run their own du processes throughout the workday.

While adding more disk space would have solved the problem, we are using a large disk array that is already filled to maximum capacity. In general, users tend to fill up all available disk space anyway, no matter how much you give them.

We then developed a policy: users could run du on any directory they owned. In addition, user du processes would be allowed to run for a maximum of one hour of wall time. Users in the wheel group would be exempt from these restrictions.

I was given the task of developing a tool to implement this policy. Some sort of wrapper around the existing du seemed like an obvious choice: the script could validate the input, abort if an invalid path was given, and terminate the du process if it ran too long.

I wrote a basic bash script in perhaps an hour's time. Then I thought about how to run it, and that is where I ran into trouble. I had thought that I would make the script set user id (setuid) or set group id (setgid) root, i.e. when run by any user it would actually run in the root group. Then, I could change the permissions on the real du so that only root could run it. The result would be that normal users could only access the real du through the wrapper script.

Of course that would make a pretty boring article, and in reality it didn't turn out to be that simple:

Full Story.

More in Tux Machines

Red Hat News

Linux Kernel Space: eBPF and More (LWN Paywall Expired)

  • A thorough introduction to eBPF
    In his linux.conf.au 2017 talk [YouTube] on the eBPF in-kernel virtual machine, Brendan Gregg proclaimed that "super powers have finally come to Linux". Getting eBPF to that point has been a long road of evolution and design. While eBPF was originally used for network packet filtering, it turns out that running user-space code inside a sanity-checking virtual machine is a powerful tool for kernel developers and production engineers. Over time, new eBPF users have appeared to take advantage of its performance and convenience. This article explains how eBPF evolved how it works, and how it is used in the kernel.
  • Restricting automatic kernel-module loading
    The kernel's module mechanism allows the building of a kernel with a wide range of hardware and software support without requiring that all of that code actually be loaded into any given running system. The availability of all of those modules in a typical distributor kernel means that a lot of features are available — but also, potentially, a lot of exploitable bugs. There have been numerous cases where the kernel's automatic module loader has been used to bring buggy code into a running system. An attempt to reduce the kernel's exposure to buggy modules shows how difficult some kinds of hardening work can be.
  • Container IDs for the audit subsystem
    Linux containers are something of an amorphous beast, at least with respect to the kernel. There are lots of facilities that the kernel provides (namespaces, control groups, seccomp, and so on) that can be composed by user-space tools into containers of various shapes and colors; the kernel is blissfully unaware of how user space views that composition. But there is interest in having the kernel be more aware of containers and for it to be able to distinguish what user space considers to be a single container. One particular use case for the kernel managing container identifiers is the audit subsystem, which needs unforgeable IDs for containers that can be associated with audit trails. Back in early October, Richard Guy Briggs posted the second version of his RFC for kernel container IDs that can be used by the audit subsystem. The first version was posted in mid-September, but is not the only proposal out there. David Howells proposed turning containers into full-fledged kernel objects back in May, but seemingly ran aground on objections that the proposal "muddies the waters and makes things more brittle", in the words of namespaces maintainer Eric W. Biederman.

today's howtos

Graphics: Texture Compression, Enlightenment Foundation Libraries (EFL), and AMD FreeSync

  • Unity Continues Crunching More Out Of Crunch Texture Compression
    Unity is one of the big public users of the open-source Crunch DXT texture compression library. While it's no longer maintained by Rich Geldreich / Binomial, Unity has continued advancing this open-source code to further improve the compression ratio and speed. For months Unity has been talking about their promising findings with Crunch. But this is the project that Rich Geldreich, the former Valve developer, previously expressed regret having open-sourced all of it. While he is on to working on better and more advanced technologies at his Binomial startup, Unity is working to squeeze more out of this open-source library.
  • Improving EFL Graphics With Wayland Application Redraws
    Under X, application redraws are tricky to do without tearing because content can be updated at any chosen time with no clear feedback as to when the compositor will read it. EFL uses some clever tricks to this end (check out the state of the art X redraw timing for yourself), but it’s difficult to get right in all cases. For a lot of people this just works, or they’re not sensitive to the issue when it doesn’t.
  • Improved Wayland Application Redraws Coming To Enlightenment's EFL
    Samsung's Open-Source Group has been working on making their Wayland support in the Enlightenment Foundation Libraries (EFL) even better. The latest Wayland work on the Enlightenment/EFL front has been improving the application redraw process. The EFL toolkit with the upcoming v1.21 release will now be hooking into Wayland's frame callbacks to better dealing with drawing, only drawing when necessary, and doing so without the possibility of tearing.
  • AMD FreeSync For Tear-Free Linux Gaming - Current State In 2017
    If you are thinking of gifting yourself (or someone else) a FreeSync-compatible monitor this holiday season, here's a look at how the AMD FreeSync support is working right now, the driver bits you need to be aware of, and how it's all playing out for those wanting to use this tear-free capability for Linux gaming.