People in the wireless industry often rely on a form of “wishful thinking." We hoped that the applications for 5G would require low latency, because the guys in the lab had created a way to shorten latency in our radio links.
But wanting something to be true doesn’t make it true. The actual customers for 5G never wanted to pay for lower latency, so that aspect of the value proposition has been gathering dust on the shelf.
We’re doing it again with Physical AI. As I have been interviewing dozens of companies about their plans for robots, I notice a very different focus in different groups of people:
- Mobile operators and their suppliers focus on latency, presuming that decisions will be made in the network.
- Robotics developers focus on making all decisions on-board the robot.
The truth is somewhere between these two extremes. In fact, the applications with potential traction are not low-latency applications at all, but situations where an on-board model needs a “boost." I have run into some very interesting examples of robots that run 99% of their inferences on-board, but need a helping hand for the other 1%.
My team is starting to track 10 specific types of outdoor robots that have constrained cost, size and power profiles. Many of these robots need to make very sophisticated decisions, but they’re working with a mosquito brain in terms of horsepower.
Network-based AI Inference only plays a role in special cases where a decision must be made. Should a robot cross a busy street now, or wait? That’s similar to the decisions made by self-driving cars with 2,500 TOPS of compute power, but a food-delivery robot runs a smaller model with only 100 TOPS. One robot may need to outsource a few hundred inferences over the course of a day.
Here are some examples:
- Construction Robots: Several companies now sell robot products that can lay the bricks, or build the roof, or paint a wall. The labor-saving aspect of these robots creates huge value. But training can be very difficult for these construction robots, as every building has nuances and irregularities, and conditions could require some higher level reasoning. It’s okay for these machines to pause for a second, just like my father would pause long enough to say a few unprintable words.
- Food delivery robots: You’ve seen these friendly robots on the sidewalks of American cities…there are thousands of them now. These robots claim a 99.8% “delivery completion rate”. (They don’t advertise that they reach 99.8% success with a human taking over control of the robots occasionally). In this case, the little robot can pause for 500 milliseconds to get a more accurate inference from the network before it crosses the street…don’t worry, your burrito won’t get cold.
- Package delivery drones: Did you hear about the drone that dropped a woman’s package in the pool? Whoops. Again, most deliveries can be fully autonomous, but there are situations where the tiny brain in the drone is not smart enough. It’s better to hover for a second and get a second opinion when confidence is low.
- Self-driving lawn mowers are a thing! These are very intriguing products, but reviews indicate that the AI models in the lawn mower can be confused when the grass is too long…the model is unable to distinguish whether long grass is a hazard. Uploading occasional videos to a bigger AI model can allow the lawn mower to be a $2,000 product, instead of a $50,000 product.
- Humanoid robots are still using humans-in-the-loop for training purposes. That is fine for the early stages, but what will happen with the robot’s training is adequate for 95% of tasks? One more time, we point out that a humanoid that is washing dishes or folding laundry can pause for a moment, for a human or a bigger AI model to provide guidance.
You see the common theme here: many physical AI applications actually don’t need low latency. That’s because, in all of these cases, the on-board brain does pretty well with immediate decisions, and needs help with special situations.
My conclusion: Latency is not the primary factor in physical AI. Accuracy of the AI inference is the primary value, and waiting a few hundred milliseconds is much better than building expensive hardware into every robot.
Mobile Experts has been tracking the revenue for edge computing applications for years and clearly the bulk of the revenue is not latency-sensitive. Below is an example of our famous Edge Computing chart:
This leads me to a very different vision for the future of 5G and 6G networks, as well as the future for Satellite D2D networks. In my recent analysis, I propose that the next $250 billion-plus growth wave will come from physical AI in various forms, and the investment in compute resources can be inexpensive because it can be centralized and shared.
That’s exactly how the Cloud Computing market started. Compute power was centralized, and it was shared among a large number of customers, so the profit margin was Outstanding.
The remaining question is: Who will capture this $250 billion-plus prize? Can telcos move fast enough to offer Kinetic Tokens? Will SpaceX be able to leapfrog telcos and offer AI inference services directly to outdoor robots? Or will the hyperscalers and AI players dominate this market over the top?
Over the long term, I expect that these latency-tolerant applications will evolve. The security dog may take on new responsibilities, requiring it to be more dexterous. Latency requirements may tighten up over the next 20 years. That would be great…the investment can begin with a rack of GPUs in one central location, and then the investment can migrate to the edge as needed. That roadmap is a lot more acceptable to Wall Street than a single $100 billion investment in an “AI grid” that doesn’t really need to be distributed.
Let’s drop the wishful thinking and focus on the real applications that are in front of us.
Joe Madden is principal analyst at Mobile Experts, a network of market and technology experts that analyzes wireless markets.
Opinions from industry experts, analysts or our editorial staff do not represent the opinions of Fierce Network.