Akamai, Neural Magic team to bolster AI at the network edge

Akamai Technologies has partnered with Neural Magic to accelerate AI workloads on its cloud delivery platform, optimizing the performance of complex models on cost-efficient CPU-based servers rather than on expensive GPU resources.

As part of the deal, Neural Magic will join Akamai’s partner program, which makes services that are interoperable with Akamai’s platform easily accessible to Akamai customers.

Neural Magic offers software that makes it possible for AI inference models to run efficiently on commodity CPU-based hardware. The software accelerates AI inferencing using automated model sparsification technologies, available as a CPU inference engine, according to Akamai.

Akamai now will make Neural Magic’s software available alongside its distributed content-delivery platform.

“NeuralMagic’s solution, which is designed to help customers better run AI workloads, will be made available on Akamai’s distributed compute infrastructure,” Ramanath Iyer, chief strategist at Akamai, said via email. “Customers with large deep learning models will now be able to leverage a cost efficient, CPU-based platform at the edge to deploy data-intensive AI applications at scale with improved performance and lower latency.”

This means that Akamai’s customers will be able to capitalize on the benefits of edge computing – including lower latency and data residency – without needing to rely on scarce and costly GPU resources, he said.

“Additionally, Akamai’s hyper distributed edge network will make the Neural Magic solution available in remote edge locations as the platform expands, empowering more companies to scale AI-based workloads more widely across the globe,” Iyer said.

A case for deep learning at the edge?

The combination of technologies could solve a dilemma that AI poses: whether it’s worth it to put computationally intensive AI at the edge—in this case, Akamai’s own network of edge devices. Generally, network experts feel that it doesn’t make sense to invest in substantial infrastructure at the edge if it’s only going to be used part of the time.

Delivering AI models efficiently at the edge also “is a bigger challenge than most people realize,” said John O’Hara, senior vice president of engineering and COO at Neural Magic, in a press statement. “Specialized or expensive hardware and associated power and delivery requirements are not always available or feasible, leaving organizations to effectively miss out on leveraging the benefits of running AI inference at the edge.”

Using a less expensive processor to do this type of AI work, when it’s needed, may be easier for a company to justify.

AI made easier?

The partnership may serve to foster innovation around edge-AI inference across a host of industries, Iyer said.

“Fundamentally, our partnership with Neural Magic is focused solely on making inference more efficient,” he explained. “There will always be cases where organizations still need a GPU if they are training AI models or their AI workload requires a larger amount of compute/memory requirements; however, CPUs have a role to play as well.”

Running AI inference workloads on a CPU-based platform ultimately can help businesses use their scarce edge resources much more efficiently, keeping costs down and improving reliability, he added.

Another benefit of having better AI at the edge also could come in terms of cybersecurity, as “it opens the door for more sophisticated inspection of traffic at the network edge,” said Fernando Montenegro, senior principal analyst at Omdia.

“As we observe attacks shifting over time from not only exploiting very specific vulnerabilities but increasingly including more nuanced application-level abuse, having AI-aided anomaly detection capabilities can be helpful,” he said. “If partnerships such as this one open the door for increased use of deep learning and generative AI by more developers, I view this as positive.”