Physical AI needs a proving ground

Giving machines greater autonomy creates a problem that manufacturers cannot solve simply by building a better AI model. Before a robot, intelligent machine or autonomous system is allowed to respond to changing conditions in a live factory, engineers need evidence that it will behave as expected across situations that may be difficult, expensive or unsafe to reproduce physically.

That requirement is changing the role of the digital twin. Timo Kistner, EMEA Regional Industry Lead for Manufacturing and Industrial at NVIDIA, argues that manufacturers need to move beyond using twins primarily for visualization and planning toward physics-based environments capable of supporting the development and validation of physical AI.

“Every manufacturer needs to think not only about the AI pilot or use case they are running, but about how they are going to validate that use case,” Kistner says. “That validation needs a physics-based, accurate representation of the environment in which the system is going to operate.”

The distinction becomes more significant as AI moves from analyzing information to controlling machines. An incorrect recommendation on a dashboard can be challenged before anything happens, while a robot responding incorrectly to its surroundings can immediately affect equipment, production or people.

A twin must behave like the factory

Digital twins have existed in manufacturing for years, but the term covers everything from spreadsheets and process models to highly detailed 3D representations. For physical AI, visual similarity alone is not sufficient because the AI system must experience the same kinds of signals it will encounter in the real environment.

Kistner says NVIDIA’s emphasis on physics-based digital twins reflects the need to reproduce the behavior of the factory rather than simply its appearance. “What we see in Omniverse is the photorealistic visualization of that factory, but what we actually simulate is the full frequency range and the full spectrum of the sensors,” he says. “If we really want a simulation that tests the capability of the robot, meaning the capability of the model powering that robot, we need a physics-based simulation that reflects the behavior of those sensors.”

For engineers, the real test is whether the virtual environment is detailed enough to expose weaknesses in the system before deployment. A visually convincing model is of limited value if it cannot reproduce the conditions that determine how the machine will behave. “With that physics-based representation, we can give the exact feedback of, for example, the lidar sensor,” Kistner says. “Then we can simulate the behavior of the AI model of that machine.”

That changes simulation from a tool used mainly to review a planned installation into an environment where manufacturers can expose an intelligent system to different operating conditions before deployment. Robot cells can be commissioned virtually, autonomous systems can encounter different scenarios, and engineers can investigate how changes to the environment affect machine behavior without repeatedly interrupting physical production.

For manufacturers, the benefit is not simply that problems can be found earlier. The same virtual environment can become reusable infrastructure for subsequent AI projects, avoiding the need to construct a new validation process around every individual application.

Kistner believes that foundation will become increasingly important as physical AI expands. “You need to build that basis, the physics-based virtual representation, which you can then leverage to simulate not only that one use case but every use case you are going to implement in your factory in the future.”

Create the situations the factory cannot

Training physical AI is complicated by the uneven nature of real production data. Manufacturers may have abundant examples of normal operation while serious faults, unusual edge cases and potentially dangerous scenarios are poorly represented because they occur rarely or are deliberately prevented. Waiting for enough real-world examples to appear can therefore leave important gaps in the experience available to the model.

“There are many test cases that you do not want to try in a factory,” Kistner says. “Any security-related or safety-related situation is something you would not deliberately create physically, but synthetic data generation allows you to simulate those scenarios.”

NVIDIA is developing this approach through Cosmos, its family of world foundation models, which can be used to generate scenarios and data for physical AI development. The same principle applies beyond situations that would be unsafe to recreate physically, because manufacturers can also struggle to collect enough examples of events that are simply rare.

Quality inspection provides a more immediate example. A manufacturer may have thousands of images of normal production but relatively few examples of every defect it wants an AI model to recognize. Synthetic data can help fill those gaps, giving engineers a broader set of conditions against which to train and test the system before it is trusted in production.

“How often does a manufacturer really have all the datasets they need to comprehensively test quality?” Kistner says. “Very often there are gaps, and with synthetic data generation you can create and extend the dataset you have today for training the AI quality-control model.”

The approach does not remove the need for real-world evidence. The more important objective is to combine simulation and physical deployment in a way that progressively improves both the model and the virtual environment.

Concerns about whether behavior learned in simulation will transfer successfully into the factory have long accompanied synthetic data. Kistner argues that the answer lies partly in improving the realism of generated data, but also in abandoning the idea that the development process ends once the model has been deployed.

Deployment is part of the learning cycle

Once a system reaches production, manufacturers begin collecting evidence that could not be fully reproduced during development. Real operating conditions expose new edge cases, unexpected interactions and differences between the assumptions built into the model and what happens on the factory floor. “You train the model, simulate the model and, when you are confident that it is good enough, you deploy it,” Kistner says. “But that is not where it ends. There needs to be a closed loop.”

This continuous loop also changes the economics of building the digital twin. If the virtual factory exists only to support a single project, creating and maintaining a sufficiently detailed representation may be difficult to justify. If it becomes the common environment for validating successive robotics, inspection and physical AI applications, its value increases with every use case.

As physical AI spreads beyond individual pilots, manufacturers will have to decide whether every new application gets its own development and validation process, or whether the virtual factory becomes shared engineering infrastructure that can support successive use cases.

“If manufacturers do not start thinking today about the evolution of their digital twins, they will struggle to keep pace as there is more automation and more AI in these environments,” Kistner says. “You can implement an AI use case without that representation, but it requires a huge amount of manual effort. Once you have the digital representation, you can try each of those AI use cases in that space.”

The digital twin therefore becomes more than a representation of what the factory looks like or how it was designed to operate. For physical AI, its more important role may be providing a place where intelligent systems can fail, learn and improve before their decisions have consequences in the physical world.

As manufacturers give machines greater freedom to respond to their surroundings, that ability to test behavior before trusting it in production could become as important as the AI model itself.