Industrial AI cannot live in one place

As manufacturers move AI from isolated pilots into production, the infrastructure question becomes harder to avoid. Training a large model, running a low-latency inspection system and supporting an intelligent machine on the factory floor place very different demands on compute, connectivity and data. Treating all of them as the same workload risks creating an architecture that is either unnecessarily expensive or unable to respond quickly enough when production depends on it.

As AI moves into live manufacturing, infrastructure choices start to affect operational performance directly. Training workloads, real-time control and sensitive production data place very different demands on the underlying systems, so the location of compute can no longer be treated as a generic IT decision.

“It is a hybrid approach,” Kistner says. “You need to think about where you want to do the training, because the training data is large and requires a significant amount of compute. Then you have applications that require a certain latency, so those need to be at the edge. You also must consider the criticality of the data, what you want to keep on premises and what the process itself requires.”

That turns infrastructure into an operational decision rather than a simple IT choice. The location of an AI workload can affect response time, resilience, data movement and the ability of a production system to keep operating when connectivity is interrupted.

Put each workload where it belongs

Manufacturing AI is likely to span several layers of infrastructure because the work being performed changes across the lifecycle of the model. Training can demand large amounts of centralized compute, while inference may need to happen close enough to the process to support decisions measured in milliseconds.

A quality inspection system, for example, may need to reject a part before it leaves the station. A robotic application cannot wait for a round trip to a remote data center before responding to movement in its environment. Other workloads, particularly those involving model training, simulation or large historical datasets, may benefit from resources that are impractical to install beside every production line.

For Kistner, manufacturers should resist the temptation to standardize around a single answer. “Every manufacturer needs to think about a dedicated strategy for how to implement this,” he says. “It is not simply a case of saying that everything should be on premises, everything should be in the cloud, or everything should be at the edge. The requirements of the process will determine where the workload needs to sit.”

That strategy also must account for the sensitivity of industrial data. Some manufacturers will be comfortable moving selected information to cloud environments, while intellectual property, production recipes or other critical data may remain inside the plant. The architecture therefore has to support movement between different environments without assuming that every dataset can be treated in the same way.

The factory itself is becoming a more compute-intensive environment. Vision systems, intelligent machines and increasingly autonomous equipment are pushing processing closer to production, expanding the technology architecture from centralized IT infrastructure into machines, lines and industrial edge systems across the plant.

The result is less a migration from one location to another than a division of labor. Central infrastructure can support intensive training and development, while edge systems handle the decisions that cannot tolerate delay. The difficult part is keeping those layers connected strongly enough that models, data and operational context do not fragment as they move between them.

AI cannot become another silo

Most factories already contain a patchwork of systems built at different times for different purposes. MES, ERP, automation platforms, engineering software, quality systems and machine-level controls may all perform their own roles effectively while sharing information imperfectly. Adding AI without addressing that fragmentation could simply create another layer that operations teams must reconcile.

Kistner argues that manufacturers should think about AI as part of the integration problem rather than another isolated technology sitting beside the existing stack. “This is not a situation where there is one provider in the world and you get all the answers from that one provider,” he says. “Manufacturers are already using different tools and different ecosystem players. The question is how you leverage all those components and use the technology to integrate areas that today are still working in silos.”

That requires a different approach from building a sequence of disconnected AI proofs of concept. A successful vision application may solve one inspection problem, while another model improves robot behavior and a third assists engineering. If each project brings its own infrastructure, data pipeline and management approach, scaling AI across the plant quickly becomes another integration exercise.

Few manufacturers have the luxury of designing an AI architecture from scratch. New capabilities have to coexist with MES, automation, engineering and other systems already embedded in production, each with its own interfaces, data structures and ownership. “How do we make sure that AI does not become another silo, where you have your MES, another system and then AI sitting separately?” he says. “The AI element can become the binding part. Whether you call that physical AI or industrial AI is less important than using it to help break up those silos.”

The ecosystem becomes particularly important because industrial AI touches domains that no single technology supplier owns. Machine builders, automation vendors, software companies, cloud providers and manufacturers themselves all hold parts of the operational context required to make the system work.

NVIDIA’s role, in Kistner’s description, is therefore not to replace those industrial systems but to provide computing and AI capabilities that can operate across them. He points to partnerships with industrial software companies as part of that wider approach, with AI increasingly embedded into platforms manufacturers already use.

For manufacturers, the implication is that architecture decisions should begin with how information and decisions need to move through the process, rather than with a predetermined technology stack. The fewer new barriers AI introduces, the easier it becomes to extend a successful use case beyond the machine or line where it started.

Build for change, not replacement

Rapid advances in AI hardware create another concern for manufacturers accustomed to capital investments that remain in service for years. New GPU generations arrive quickly, models evolve and software capabilities change, raising an obvious question about whether infrastructure being installed today will become obsolete before the production system around it does.

Kistner argues that manufacturers should avoid treating AI infrastructure as a series of wholesale replacement cycles. The more practical route is to start with workloads that have clear operational value and expand capacity as demand grows. “The stack is built in a way that means you do not need to rip everything out,” he says. “You can operate it continuously. You start today with the use cases that are crucial and that you can integrate and scale quickly, and from there you start building out the infrastructure.”

Manufacturing AI will usually expand in stages rather than arrive as a single transformation. Early deployments may focus on a narrow operational problem, with additional workloads added as manufacturers build confidence, develop internal capability and identify where further investment can deliver value. The infrastructure therefore must support that growth without locking every future application into decisions made for the first use case.

“Whether there is a new chip generation does not change the fact that the older generation still works,” Kistner says. “The aim is interoperability across the stack, so manufacturers can continue to use what they have while adding new capability where it makes sense.”

That may prove particularly important as factories become more compute intensive. Physical AI, advanced inspection, simulation and autonomous systems all add workloads closer to production, while model development and training continue to consume substantial centralized resources.

The manufacturing AI architecture that emerges is therefore unlikely to resemble either a traditional cloud strategy or a conventional factory automation stack. It will span both, with workloads moving to the environment that best fits their latency, compute, security and process requirements.

For manufacturers, the central question is no longer whether AI belongs in the cloud or at the edge. It is whether the architecture can place each workload where it performs best, connect it to the industrial systems around it and continue evolving without creating another generation of isolated technology.

That is what will determine whether AI remains a collection of successful pilots or becomes part of the operating fabric of the factory.