AI workloads are accelerating development across the Kubernetes ecosystem, with incubating and sandbox projects addressing GPU scheduling, dataset management, and autoscaling for model training.
Oguzhan Coskun, Founder of Nsource, says community-driven innovation is what allows the platform to absorb new workload types quickly.
Organizations still face infrastructure decisions around cloud VMs versus bare metal, driven less by technical capability and more by regulatory constraints, data sovereignty, and cost.
The Kubernetes ecosystem is not short on options for teams building AI infrastructure. GPU workload management, pod autoscaling for model training, and dataset orchestration all have active projects in various stages of incubation. The operational question for most organizations is no longer whether Kubernetes can handle AI. It is how to evaluate a growing inventory of tools and integrations fast enough to make sound infrastructure decisions.
Oguzhan Coskun is a principal-level DevOps and cloud architect and AWS Certified Solutions Architect who runs Nsource, a London-based consultancy providing contract DevOps and cloud architecture services. He has worked in the Kubernetes ecosystem for more than five years, delivering over 30 production-grade EKS clusters for fintech clients and leading seven consecutive PCI-DSS audits with zero downtime. His work spans cost optimization, observability, and compliance across highly regulated environments.
"The real challenge for teams today isn't just running AI workloads. It's deciding which tools actually matter when the cloud-native ecosystem is constantly introducing new projects and technologies," Coskun says. The pace of development inside the ecosystem is what keeps Kubernetes viable as the foundation for AI infrastructure. Coskun points to the volume of sandbox and incubating projects now focused on AI workload management, from GPU handling to autoscaling around model cards and datasets. The speed is a direct product of open-source collaboration.
Community as engine: "The strength of Kubernetes has always been the community. New workloads like AI grow so quickly because the ecosystem keeps building the tools needed to support them," Coskun says. Projects like Cilium, which graduated from the CNCF, demonstrate how community-governed tooling matures fast enough to meet production demands. As AI workloads push Kubernetes toward more autonomous operations, community-backed projects are filling gaps that no single vendor could address alone.
Networking catches up: Observability and security in high-throughput AI environments benefit from kernel-level advancements. Coskun highlights eBPF-based networking through Cilium as a practical example. "When you run very heavy workloads and need to monitor your network traffic, eBPF solves critical problems. Previously, you needed to restart your operating system. Now you don't," he explains. For AI workloads generating massive transaction volumes, that kind of kernel-level monitoring without disruption is essential.
Cost as the real constraint: Coskun sees Kubernetes as technically ready for AI, but flags cost as the limiting factor. "Kubernetes is already becoming the foundation for AI infrastructure. Once you add GPU scheduling, dataset management, and autoscaling, the ecosystem can support almost every stage of the AI workflow," he says. The challenge is that model training and large-scale compute remain expensive, and organizations need disciplined cost management alongside technical capability.
Even with Kubernetes as the platform layer, infrastructure placement remains a decision shaped by factors outside of technology. For organizations handling financial data or operating under strict data residency rules, the choice between cloud VMs and bare metal is driven by regulation and geography. Data sovereignty mandates in the Middle East, for instance, can require that data stays in local data centers regardless of what the cloud can offer technically.
Project over preference: "When you're deciding between cloud VMs and bare metal, it's rarely just a technical decision. It comes down to the project requirements, regulatory constraints, and where your data is allowed to live," Coskun says. He adds that cloud providers already cover certifications like PCI-DSS and GDPR, but government and geographic restrictions can override those assurances. In his experience across fintech clients, the answer depends on what the project demands, not what the platform can do.
The broader challenge Coskun returns to is one of discipline. As new tools and integrations arrive constantly, teams tend to adopt what they know and resist switching, even when better options emerge. That creates a tension between ecosystem innovation and operational inertia that every organization building on Kubernetes must manage.
"People using the same tool want to continue with the same tool because the previous one already works for them. We have new solutions arriving every day, but teams don't want to change what they know," Coskun says. For organizations building AI-ready infrastructure on Kubernetes, the task is not just technical adoption. It is making deliberate choices about which tools earn a permanent place in the stack and which are noise.
