Language Selection

English French German Italian Portuguese Spanish

Supercomputing Articles

Filed under
Server
  • Exascale meets hyperscale: How high-performance computing is transitioning to cloud-like environments

    Twice a year the high-performance computing (HPC) community anxiously awaits the announcement of the latest edition of the Top500 list, cataloging the most powerful computers on the planet. The excitement of a supercomputer breaking the coveted exascale barrier and moving into the top position typically overshadows the question of which country will hold the record. As it turned out, the top 10 systems on the November 2019 Top500 list are unchanged from the previous revision with Summit and Sierra still holding #1 and #2 positions, respectively. Despite the natural uncertainty around the composition of the Top500 list, there is little doubt about software technologies that are helping to reshape the HPC landscape. Starting at the International Supercomputing conference earlier this year, one of the technologies leading this charge is containerization, lending further credence to how traditional enterprise technologies are influencing the next generation of supercomputing applications.

    Containers are borne out of Linux, the operating system underpinning Top500 systems. Because of that, the adoption of container technologies has gained momentum and many supercomputing sites already have some portion of their workflows containerized. As more supercomputers are being used to run artificial intelligence (AI) and machine learning (ML) applications to solve complex problems in science-- including disciplines like astrophysics, materials science, systems biology, weather modeling and cancer research, the focus of the research is transitioning from using purely computational methods to AI-accelerated approaches. This often requires the repackaging of applications and restaging the data for easier consumption, where containerized deployments are becoming more and more important.

  • Exploring AMD’s Ambitious ROCm Initiative

    Three years ago, AMD released the innovative ROCm hardware-accelerated, parallel-computing environment [1] [2]. Since then, the company has continued to refine its bold vision for an open source, multiplatform, high-performance computing (HPC) environment. Over the past three years, ROCm developers have contributed many new features and components to the ROCm open software platform.

    ROCm is a universal platform for GPU-accelerated computing. A modular design lets any hardware vendor build drivers that support the ROCm stack [3]. ROCm also integrates multiple programming languages and makes it easy to add support for other languages. ROCm even provides tools for porting vendor-specific CUDA code into a vendor-neutral ROCm format, which makes the massive body of source code written for CUDA available to AMD hardware and other hardware environments.

  • High-Performance Python – GPUs

    When GPUs became available, C code via CUDA, a parallel computing platform and programming model developed by Nvidia for GPUs, was the logical language of choice. Since then, Python has become the tool of choice for machine learning, deep learning, and, to some degree, scientific code in general.

    Not long after the release of CUDA, the Python world quickly created tools for use with GPUs. As with new technologies, a plethora of tools emerged to integrate Python with GPUs. For some time, the tools and libraries were adequate, but soon they started to show their age. The biggest problem was incompatibility.

    If you used a tool to write code for the GPU, no other tools could read or use the data on the GPU. After making computations on the GPU with one tool, the data had to be copied back to the CPU. Then a second tool had to copy the data from the CPU to the GPU before commencing its computations. The data movement between the CPU and the GPU really affected overall performance. However, these tools and libraries allowed people to write functions that worked with Python.

    In this article, I discuss the Python GPU tools that are being actively developed and, more importantly, likely to interoperate. Some tools don’t need to know CUDA for GPU code, and other tools do need to know CUDA for custom Python kernels.

  • Porting CUDA to HIP

    You’ve invested money and time in writing GPU-optimized software with CUDA, and you’re wondering if your efforts will have a life beyond the narrow, proprietary hardware environment supported by the CUDA language.

    Welcome to the world of HIP, the HPC-ready universal language at the core of AMD’s all-open ROCm platform [1]. You can use HIP to write code once and compile it for either the Nvidia or AMD hardware environment. HIP is the native format for AMD’s ROCm platform, and you can compile it seamlessly using the open source HIP/​Clang compiler. Just add CUDA header files, and you can also build the program with CUDA and the NVCC compiler stack (Figure 1).

  • OpenMP – Coding Habits and GPUs

    When first using a new programming tool or programming language, it’s always good to develop some good general habits. Everyone who codes with OpenMP directives develops their own habits – some good and some perhaps not so good. As this three-part OpenMP series finishes, I highlight best practices from the previous articles that can lead to good habits.

    Enamored with new things, especially those that drive performance and scalability, I can’t resist throwing a couple more new directives and clauses into the mix. After covering these new directives and clauses, I will briefly discuss OpenMP and GPUs. This pairing is fairly recent, and compilers are still catching up to the newer OpenMP standards, but it is important for you to understand that you can run OpenMP code on targeted offload devices (e.g., GPUs).

  • News and views on the GPU revolution in HPC and Big Data:

    Exploring AMD's Ambitious ROCm Initiative
    Porting CUDA to HIP
    Python with GPUs
    OpenMP – Coding Habits and GPUs

AMD and NVIDIA at SC19

  • AMD Announces Radeon Open Compute ROCm 3.0

    AMD just sent out their press release for SuperComputing 19 week in Denver. It turns out being released for SC19 is the latest major iteration of Radeon Open Compute, ROCm 3.0.

    AMD's press release mentions ROCm 3.0 being released though as of writing it has yet to appear via the ROCm repositories on GitHub. Once the actual drop happens, I'll certainly be writing about it and digging deeper into the other changes in full.

  • NVIDIA Releasing Reference Design For Stuffing Their GPUs Into Arm Servers

    NVIDIA CEO Jensen Huang announced from SC19 today in Denver that they are releasing a "reference design" of hardware and software to help in deployments of their graphics processors within Arm-based servers focused on HPC and AI.

    This isn't too surprising considering NVIDIA's past forays into ARM-based servers for HPC/AI and it was just a few months ago NVIDIA said they would be supporting CUDA on ARM Linux for HPC servers. NVIDIA has already been supporting their software for ARM-based SoCs for years as well considering their Tegra platform and Linux 4 Tegra (L4T).

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

More in Tux Machines

Android Document Scanning and Developer-Focused TV Box

  • The 15 Best Document Scanner Apps for Android Devices in 2020

    It doesn’t matter whether you are an Office job holder, a businessman, or a student; you will face a situation where scanning some papers or documents seem to be essential. But finding a scanner is tough in many places nowadays. You can deal with such a problem if you have installed any document scanner apps on your Android device. In PlayStore, some scanner apps can turn your mobile phone into a tiny scanner. So, just by installing a useful document scanner App, you can scan notes and documents anytime, anywhere.

  • Google ADT-3 is a Developer-Focused TV Box for Android TV on Android 10

    Back in 2014, Google killed Google TV and announced Android TV, and as a result, introduced ADT-1, the first developer kit specifically designed for Android TV.

Improving the security model of the LVFS

There are lots of layers of security in the LVFS and fwupd design, including restricted account modes, 2FA, and server side AppStream namespaces. The most powerful one is the so-called vendor-id that the vendors cannot assign themselves, and is assigned by me when creating the vendor account on the LVFS. The way this works is that all firmware from the vendor is tagged with a vendor-id string like USB:0x056A which in this case matches the USB consortium vendor assigned ID. Client side, the vendor-id from the signed metadata is checked against the physical device and the firmware is updated only if the ID matches. This ensures that malicious or careless users on the LVFS can never ship firmware updates for other vendors hardware. About 90% of the vendors on the LVFS are locked down with this mechanism. Some vendors have to have IDs that they don’t actually own, a good example here is for a DFU device like the 8bitdo controllers. In runtime mode they use the USB-assigned 8bitdo VID, but in bootloader mode they use a generic VID which is assigned to the chip supplier as they are using the reference bootloader. This is obviously fine, and both vendor IDs are assigned to 8bitdo on the LVFS for this reason. Another example is where Lenovo is responsible for updating Lenovo-specific NVMe firmware, but where the NVMe vendor isn’t always Lenovo’s PCI ID. Read more

Programming: Vim, Qt Shader and Python

  • Vim Text Editor for Beginners Part 1 - Introduction

    In my newly refreshed Vim series, you'll learn all the things you'll need to know in order to use this text editor in your daily workflow. In this first video, we'll get Vim installed take an initial look.

  • Vim Text Editor for Beginners Part 2 - Combining Files

    In my newly refreshed Vim series, you'll learn all the things you'll need to know in order to use this text editor in your daily workflow.

  • Qt Shader Tools Looks To Become Official Qt6 Module

    The currently-experimental Qt Shader Tools allows for graphics/compute shader conditioning and used by the in-development Qt graphics abstraction layer for supporting Vulkan / Metal / Direct3D / OpenGL APIs. Qt Shader Tools offers various shader features in preparing them for consumption by different graphics APIs. Qt Shader Tools is currently used ahead of time for QtGUI with Qt 5.14+. But for Qt 6.0, Qt Shader Tools is going through the appropriate steps for becoming a formal Qt 6 module for compiling and translating shaders between interfaces.

  • Python Positional-only parameters

    I have downloaded Python 3.8 and start to play around with those latest python functions. In this article, we will look at the Positional-only parameter syntax which is a function parameter syntax / to indicate that some function parameters must be specified positionally and cannot be used as keyword arguments which means after the / syntax we may specify a value for each parameter within that function.

  • For Loop in Python Explained With Practical Examples

    If you are just getting started to learn Python, you must be in search of something to explore for loop in Python. Of course, our list of free python resources should help you learn about it quickly. In either case, we shall help you learn more about the ‘for‘ loop in python using a couple of important examples.

Games: Pygame, The Long Dark, DXVK and Shovel Knight

  • Enable your Python game player to run forward and backward

    In previous entries in this series about creating video games in Python 3 using the Pygame module, you designed your level-design layout, but some portion of your level probably extended past your viewable screen. The ubiquitous solution to that problem in platformer games is, as the term "side-scroller" suggests, scrolling. The key to scrolling is to make the platforms around the player sprite move when the player sprite gets close to the edge of the screen. This provides the illusion that the screen is a "camera" panning across the game world. This scrolling trick requires two dead zones at either edge of the screen, at which point your avatar stands still while the world scrolls by.

  • Survival Mode in The Long Dark just got a lot bigger with the ERRANT PILGRIM update

    As promised, Hinterland Studio have released a huge update to the Survival Mode side of The Long Dark named ERRANT PILGRIM. It brings in a whole new region to explore, Bleak Inlet. Once a home to a thriving industrial Cannery, seismic activity cut-off Bleak Inlet from the rest of the Great Bear mainland. Exploring is not for the faint of heart, being Timberwolf territory but the treasures contained in the industrial complex may just be enough to warrant the journey.

  • DXVK Reportedly Going Into "Maintenance Mode" Due To State Of Code-Base

    While DXVK tends to be much-loved by Linux gamers for allowing more Direct3D 10/11 Windows games to run nicely on Linux with Wine or Proton (Steam Play) thanks to its fairly complete translation of D3D10/D3D11 API calls to Vulkan, it looks like Philip Rebohle is at least contemplating shifting it just into maintenance-mode. The DXVK lead developer recently commented that DXVK is "entering maintenance mode" and he doesn't want to make any significant changes or additions to the code.

  • Shovel Knight: King of Cards and Shovel Knight Showdown are out, completing the series

    Starting off with a successful Kickstarter crowdfunding campaign back in 2013 and growing into a massive multi-part 8-bit inspired world, Shovel Knight: Treasure Trove now finally finished. Note: Keys provided by GOG.com to us. Originally having a goal of $75,000 and a Linux/macOS stretch goal at $130,000 it proved to be popular ending on $311,491. It's taken six years for Yacht Club Games to get here starting with Shovel of Hope, followed by Plague of Shadows in 2015, Specter of Torment in 2017, and now King of Cards and Shovel Knight Showdown in 2019.