GPUDirect is a DMA (Direct Memory Access) technology that enables the NVIDIA graphics card on a device to access the storage unit to facilitate data exchange. This technology is also applicable to network-based systems where data exchange can occur without involving the CPU or the system memory. NVIDIA GPUDirect technology promises to increase performance and reduce load times of the device while simultaneously reducing the strain on the system’s CPU.
How Does GPU Direct Storage Work?
In a traditional computer, whenever any data needs to be processed by the graphics card, it first goes from the storage to the CPU which copies the data into the RAM. Then, the RAM again transfers the data back to the CPU which finally transfers the data to the GPU. This makes dataflow a complicated process that requires the CPU to handle and manage all the data flow to the GPU. GPUDirect reimagines how the data in a system should be handled. Its method of accessing the data directly from the storage significantly reduces processing times. To obtain that, it takes the advantage of the graphics card’s highly specialized building blocks and execution engine. Using this technology, the data does not have to compete with other processes for memory bandwidth, reducing the wait times that the data needs to go through. By utilizing NVIDIA Magnum I/O software stack and DALI (Data Loading Library), Deep Learning and AI based tasks can be processed very efficiently.
Benefits of Using GPUDirect Storage
If implemented, NVIDIA claims GPUDirect will enhance and optimize system performance while providing a high level of parallel computing power. This also means that the technology can be used to process vast amounts of data like artificial intelligence and data warehousing. The following are some of the benefits of GPUDirect:
Reduces CPU and System Memory Utilization: GPUDirect storage technology helps reduce the strain on the CPU and the system memory by reducing the number of I/O (Input/Output) operations. Reduces Load Times and Increases Hardware Data Decompression Rate: If you play games or perform any specialized task that requires the CPU to process large amounts of data, GPUDirect accelerates the process and helps offload the instructions that will be sent to the CPU. Facilitates Deep Learning and AI: AI and deep learning are complex and require multi-level processing. By utilizing GPUDirect, you can process large chunks of data relatively quickly when compared to processing the data through traditional means.Bypass CPU Bottleneck Issue: By reducing the processing load and overheads on the CPU, GPUDirect increases the performance of the system and makes it focus more on logical operations rather than handling data transfer to the GPU. Improves Gaming Performance in Consoles and Supported Hardware: By utilizing the parallel processing power of the GPU, data and assets of the games you play will be loaded much faster. Overall improvement in graphics quality, draw distance, environmental asset, and particle effects can be seen by adopting GPUDirect technology and implementing it in games.
Limitations of GPUDirect Storage
Despite all of its benefits, NVIDIA GPUDirect, being a relatively new technology comes with its own caveats. Issues like limited compatibility and a complicated setup process are some of the factors that can affect the widespread adoption of this technology. Some drawbacks of this technology are:
Limited I/O Compatibility: Being a new technology launched in 2019, NVIDIA GPUDirect I/O acceleration technology has limited compatibility with other systems. Requires Additional Setup to Operate: NVIDIA GPUDirect Storage does not come enabled by default. Users will have to manually set up and install necessary drivers and software for this to operate on their devices. It also requires installation of Magnum I/O software stack to enable file systems, operate and process large data, and AI related operations.Limited Hardware and Software Support: GPUDirect being a new technology, has limited software that can truly take advantage of this feature. This also means that GPUDirect has limited support for legacy systems. The feature requires CUDA parallel computing platform and at least 8.x series of graphics cards to operate on.
What Does it Mean for Gaming?
Average users and household computing machines cannot really take advantage of this feature while doing tasks like playing games. The games and software need to be optimized to take advantage of this feature making GPUDirect a niche for an average user. However, the technology is being adopted at a steady rate by developers and hardware manufacturers. Microsoft Xbox’s new Velocity architecture utilizes similar technology to significantly reduce load times and increase the performance of its series X console. Supported games are also expected to release in the near future.
What Should We Expect in The Future?
Mostly associated with AI, machine learning, and data processing, GPUDirect has usage mostly in the industrial sector of computing. The technology is mostly used by software developers and large data processing centers that rely on the GPU’s parallel operating capability to reduce data processing times. However, the technology is spreading very rapidly and other manufacturers have developed their own alternatives to NVIDIA GPUDirect. Namely AMD’s OpenCL and Windows’s Direct Storage. All in all, NVIDIA GPUDirect Storage technology aims to improve loading and processing times in applications and software by optimizing dataflow in the system.
