Ushering In A New Wave Of Data Acceleration
Here’s how dedicated processing can be the new wave of acceleration.
As big data grows exponentially, our ability to process complex workloads is waning. Jonathan Friedmann, Co-founder & CEO of Speedata, shares some of the most common workloads run by CPUs and the hardware needed to accelerate them.
2.5 quintillion bytes of data are generated daily, and estimates suggest that big data will continue to grow by 23% annually. This trend has permeated nearly every corner of the economy – businesses from airlines, banks, and insurance companies to governmental institutions, hospitals, and telecommunication companies have adopted big data analytics to improve business intelligence, promote growth and streamline efficiency.
As big data only grows, the tools used to analyze all that data must be scaled. However, computer chips currently used to handle big or complex workloads are not up to the task, requiring so many of them that the costs outweigh the benefits and hamper computing efficiency.
Therefore, for all its advantages, the explosion of data creates multiple challenges for the high-tech industry. The key to overcoming this challenge is bolstering processing power from every angle.
To do this, a wave of specialized domain-specific accelerators has been developed to offload workloads from the CPU, the traditional workhorse of computer chips. Such “alternative” accelerators are designed for specific tasks, trading off the flexibility and the general-purpose abilities of standard CPU computing in return for better, accelerated performance for such designated tasks.
The following is a short guide to some of the prominent areas of acceleration and their corresponding accelerators.
Hardware for AI and ML Workloads
Artificial Intelligence is changing how we compute – and, therefore, how we live. But the first AI analytics were forced to run on CPU chips which were far better suited to single-threaded jobs and certainly not designed for the parallel multitasking demanded by AI.
Enter: Graphics Processing Units (GPUs).
GPUs originated in the gaming industry to accelerate graphical workloads. A single GPU combines multiple specialized cores which run in tandem, allowing it to support parallel programs with a simple control flow. This is perfect for graphics workloads i.e., computer games, as they contain images with millions of pixels, which needed to be computed in parallel, independently. Processing these pixels also requires vectorized floating point multiplications which the GPU was designed to process extremely well.
The discovery that GPUs could also be used to process AI workloads opened new horizons for the way AI data is managed. Though the application is very different from graphical workloads, AI /Machine Learning (ML) workloads have, in many regards, similar computational demands, requiring efficient floating-point matrix multiplication. Over the last decade, as AI and ML workloads skyrocketed, GPUs have undergone substantial improvement to fit this burgeoning demand further.
Later, companies developed dedicated Application-Specific Integrated Circuits (ASICs) to address this important workload in attempts to usher in the second wave of AI acceleration. ASICs at the fore of AI acceleration include the TPU, Google’s tensor processing unit used mainly for inference; the IPU, Graphcore’s Intelligence processing unit; and the RDU, SambaNova’s Reconfigurable Dataflow Unit.
Data Processing Workloads
Data Processing Units (DPUs) are essentially Network Interface Controllers (NICs) – hardware that connects a given device to the digital network. These ASICs are explicitly designed to offload protocol networking functions from the CPU and higher layer processing like encryption or storage-related operations.
Companies have developed various DPUs, including Mellanox, which Nvidia acquired, and Persando, which AMD acquired. Although their architecture varies, and the exact networking protocol each offload differs, all DPU variations have the same end goal of speeding up data processing and offloading the network protocol from the CPU.
While Intel’s DPU was given its acronym – IPU (Infrastructure Processing Unit), it belongs to the DPU family. The IPU is designed to improve data center efficiency by offloading functions that would have traditionally run on a CPU, such as networking control, storage management, and security.
Big Data Analytics
Databases and analytical data processing is where big data truly yields actionable insights. As with the above workloads, CPUs were long seen as the standard. But as the scale of data analysis workloads continues to grow, these CPU functions have become exponentially less efficient.
Big data analytics workloads have many unique characteristics, which include data structure and format, data encoding, and processing operator types, as well as intermediate storage, IO, and memory requirements. This allows for a dedicated ASIC accelerator that is targeted to optimize workloads with these specific characteristics to provide significant acceleration at a cheaper cost than traditional CPUs. Despite this potential, no chip has emerged in the past ten years as the natural successor to the CPU for analytics workloads. The result: so far, dedicated accelerators have underserved big data analytics.
Analytical workloads are typically programmed with Structured Query Language (SQL), but other high-level languages are also very common. Analytical engines that process such workloads are abundant and include open-source engines such as Spark and Presto, as well as managed services like Databricks, Redshift and Big Query.
Speedata has created an Analytical Processing Unit (APU) to accelerate analytical workloads. With the explosion of data, the insights derived from such emerging tools have the potential to unlock incredible value across all industries.
See More: How Chatbots Simplify Data Analytics Consumption for Decision Makers
Respect the Process
There’s no “one size fits all” solution to today’s computing needs.
Instead, the once-ubiquitous CPU is evolving into a “system controller” that passes complex workloads – data analytics, AI/ML, graphics, video processing, etc. – off to specialized units and accelerators.
Companies, in turn, are adapting their data centers with such processing units strategically tailored to their workload needs. This heightened level of customization will not only improve the effectiveness and efficiency of data centers but will also minimize costs, reduce energy consumption, and reduce real estate needs.
For analytics, faster processing will also allow getting more insights on a larger amount of data, opening new opportunities. With more processing options and new opportunities, the big data age is just beginning.
How do you think dedicated processing can simplify the process of handling complex data workloads? Share with us on Facebook, Twitter, and LinkedIn. We’d love to know!
Image source: Shutterstock
MORE ON DATA ANALYTICS
