1. Introduction
We have been quite regularly sharing updates, results, and, of course, loads of code since our Open Radio Access Network (O-RAN)-native CU/DU solution was announced back in March 2023. Once again, we’d like to lift the hood and show you the bits and bolts of another key piece of our 5G SA gNodeB physical (PHY) layer implementation.
The architecture of our customizable O-RAN-native solution has been designed with scalability, performance, disaggregation, and clean interfacing in mind. Such a design facilitates the use of virtualized RAN (vRAN) functions and different PHY layer implementation approaches, ensuring that the stack runs efficiently on a wide array of computing solutions. In this regard, srsRAN Enterprise 5G (the commercial offering of our O-RAN-native stack) provides built-in support for hardware-accelerated PHY layer processing. The srsRAN solution includes a complete software PHY layer, however adding hardware acceleration for certain elements can be attractive when performance is the principal requirement (e.g., cloud infrastructure).
From a software architecture point of view, support for hardware accelerators has been integrated into the stack based on three basic concepts: resource acquisition is initialization (RAII), factories, and hardware abstraction. RAII is quite an abstract name given to a very helpful C++ technique to bind resources to a scope and make sure they are released when no longer used. By combining the latter with factories, our stack can use different implementations of key functions all conforming to a common generic interface (e.g., optimize PHY layer DSP for a given processor architecture). Last, but not least, those PHY implementations can conveniently offload compute-intensive DSP functions to a hardware accelerator through a specialized hardware abstraction layer (HAL). Hence, from the user’s point of view, a 5G SA RAN stack fully optimized for a given computation solution can be transparently built from a single codebase.
Being an O-RAN-native solution, the hardware accelerator integration is built around the Accelerator Abstraction Layer (AAL) interface specifications [1], as defined by the O-RAN Alliance, of which SRS is an early member. Naturally, hardware acceleration in 5G takes place in the PHY layer, with the forward error correction (FEC) functions taking the lead in terms of computational complexity. Therefore, the focus of this article will be on the support for look-aside hardware acceleration of the FEC in our gNB solution. That is, as part of the PHY layer execution flow, the host CPU will offload the processing of the LDPC encoding, decoding, and rate-matching functions to a specialized hardware accelerator and will receive the results after processing has been completed. While the FEC processing is carried out by the accelerator hardware, the host CPU is free to carry out other processing tasks in parallel. Moreover, the accelerator’s deeply pipelined and massively parallel architecture can complete the offloaded tasks in a considerably shorter time, which boosts the stack performance.
2. Software architecture
Two key aspects of our 5G SA software solution are flexibility and scalability. Consequently, our software architecture aims to efficiently provide support for a wide range of hardware acceleration solutions, including both in-house (e.g., custom FPGA design) and third-party.
Our modular factory-based RAII design greatly facilitates the integration of different hardware abstraction libraries and, thus, transparently interfacing all kinds of hardware accelerator technologies. Given our expertise in FPGA PHY layer acceleration, it was natural for us to introduce support for the vRAN Accelerator ACC100 Adapter, a PCIe-based accelerator card from Intel. The ACC100 embeds an eASIC device which is an intermediary technology between FPGAs and ASICs and provides low-power 4G and 5G FEC acceleration.
The integration of PCIe-based PHY-layer accelerators to our stack makes use of the wireless baseband device (BBDEV) library [2], which is part of the Data Plane Development Kit (DPDK) [3]. Besides being the de facto standard for such acceleration technologies, BBDEV is an open-source C-based implementation of the AAL [4] contributed by Intel, which is also a member of O-RAN Alliance.
srsRAN Enterprise 5G introduces SRS’ own HAL as a dynamic library, which is perfectly aligned with the AAL. Through the HAL, hardware-accelerated implementations of the PUSCH and PDSCH FEC processing functions are enabled as an optional flavor supported by the corresponding factory, similar to other technology-optimized realizations (e.g., AVX-512). Given our modular hierarchical and disaggregated architecture design, those software-only and accelerated implementations can be easily swapped transparently. From a user point of view, this means that the user can implement a gNB with support for hardware-accelerated FEC functions at compile time, while conveniently specifying the details of the underlying hardware accelerator (e.g., ACC100) via the configuration file.
Figure 1 – Overview of SRS HAL software architecture.
Figure 1 provides a summarized view of the software architecture of SRS HAL, focusing on the integration of the look-aside hardware acceleration of LDPC channel coding functions in the CU/DU stack. SRS HAL defines abstract hardware accelerator interfaces for the different supported FEC functions, enabling the gNodeB to transparently offload them to different accelerator technologies (e.g., BBDEV- or FPGA-based). In more detail, the HAL defines generic interfaces with virtual enqueue and dequeue methods are defined (i.e., offload a task and retrieve the results). The specialized implementations need to conform to the defined interface while making use of drivers and libraries defined specifically for the targeted accelerator device. In the ACC100 case, the specialized FEC-function implementations make use of the BBDEV AAL library, which is built on top of DPDK. The latter provides a wider functionality beyond baseband acceleration (i.e., fast packet processing) and, thus, it is an interesting asset to have available in other parts of our stack (e.g., if configured to do so, our open fronthaul implementation can make use of DPDK too). For this reason, the software architecture of our stack enables access to DPDK functions from different parts of our stack efficiently and cleanly.
3. Performance
Besides providing a richer deployment offering, the integration of look-aside hardware accelerators, such as the ACC100, into our stack both boosts the performance and reduces power consumption (e.g., lowering the CPU workload by implementing the complex FEC processing in a fully optimized eASIC device).
Figures 2 and 3 focus on the increased performance by comparing the throughput of the ACC100-accelerated and an avx512-optimized implementation, both running in a SuperMicro IoT A+ server with an AMD EPYC 8534PN CPU (64 core, 3 GHz) processor and 384 GB of DDR5 memory (4800 MHz). In more detail, Figure 2 compares the throughput (Mbps) in terms of the number of threads observed with the PDSCH processor benchmark application [5] of our stack, for a 100 MHz channel bandwidth and a configuration maximizing the attainable rate (256 QAM, scs30_100MHz_256qam_max profile). Each throughput measure represents 100 repetitions of 100 processor jobs per thread, where each thread executes a single PDSCH processor instance. As it can be observed, with the performance boost of the ACC100, the throughput scales up to roughly 38 Gbps when 22 threads are used (that’s a gain of around 22 Gbps with respect to the non-accelerated version).
Figure 2 – Comparative of PDSCH processor benchmark results between ACC100 and avx512 implementations for a 1000 MHz channel bandwidth (256 QAM).
Similarly, Figure 3 shows the counterpart PUSCH processor benchmark application [6] results, for a two-layer four-antenna 100 MHz channel bandwidth configuration (scs30_100MHz_256qam_rv0_4port_nlayer profile). In this case, the observed performance gain is above 4 Gbps when 24 threads are used.
Figure 3 – Comparative of PUSCH processor benchmark results between ACC100 and avx512 implementations for a two-layer four-antenna 100 MHz channel bandwidth configuration (256 QAM).
Finally, the processor loads and memory usage observed by both the ACC100-accelerated gNB implementation and the avx512-optimized one are shown in Figures 4 and 5. In this case, the gNB configuration was modified in each test to implement an increasing number of 4×4 MIMO cells with a 100 MHz bandwidth configuration. Both test mode and ru emulation features have been used when exercising the gNB to ease reproducibility and simplify the required laboratory setup.
Figure 4 – Observed CPU load when executing multiple cells in a single server with the AMD EPYC 8534PN processor.
Figure 5 – Observed memory usage when executing multiple cells in a single server with the AMD EPYC 8534PN processor.
Focusing on the 10-cell test, without hardware acceleration, the combined CPU load scales up to 3070% (slightly more than 30 full core utilization). When using the ACC100, the combined CPU load was reduced to 1720%, representing more than 44% CPU savings (around 13 full cores). On the other hand, very similar memory consumption values were observed for both implementations.
4. Conclusions
The modular hierarchical and disaggregated architecture design of SRS’ CU/DU facilitates the deployment of various PHY layer implementations. The srsRAN 5G Enterprise solution enables the transparent integration of look-aside hardware accelerators, such as the ACC100, into the SRS stack. By offloading the FEC functions to the ACC100, CPU savings above 50% can be obtained with a negligible increase in memory usage.
5. References
[1] O-RAN Acceleration Abstraction Layer General Aspects and Principles, Work Group 6, O-RAN Alliance, 2022. Available on-line at https://www.o-ran.org/specifications.
[2] Wireless Baseband Device Library, https://doc.dpdk.org/guides/prog_guide/bbdev.html
[3] https://github.com/DPDK/dpdk
