Your GPUs Are 95% Idle, and Buying More Won’t Fix It

Making Efficiency a Platform Capability

PANEL | When Domains Collide: AI, Security & FinOps Inside the Platform Engineering Stack

ROUNDTABLE | OpenTelemetry in Production

Pod Logs — June 2026

Platform Engineering and DevOps

Transforming optimization into an invisible platform capability

Stop Hiding Kubernetes Pod Inefficiencies With Horizontal Scaling

Stop Overprovisioning for Application Startup

Practical Policy-as-Code with OPA and a Managed Sidecar Container

Platform Engineering Untold Truths: Is It Just an Infrastructure Matter?

An Open-Source First Step to Greenify Your Cloud Environments

A Policy-as-Code Approach to RBAC Authorization

WORKSHOP | The Game of Platforms: A Guided Journey into the World of Digital Platforms

Challenges and Takeaways of Managing AI Workloads on Cloud

Finding Observability and DevEx Tranquility Sailing the Monitoring Data Seas

From DORA to the SPACE and Beyond: Metrics to Drive Platform Engineering Initiatives

When Platform Engineers Meet SREs: The Birth of Observability-as-a-Service Superpowers

Observability-as-a-Service: When Platform Engineers and SREs Join Forces

Platform Engineers Assemble! Create Your FaaS Platform Using WASM

The Kubernetes Compass: Navigating Space and Time for Energy Efficiency

How Platform Engineering Streamlines the End-to-End Delivery of a Self-Hosted Agentic Architecture

ROUNDTABLE | When Platform Engineers Meet SREs: The Birth of Observability-as-a-Service Superpowers

In API We Trust: Circular Design for Scalable Platforms

Another Cluster Bites the Dust... and That's Just Fine!

Another Cluster Bites the Dust… E Va Bene Così!

Can an AI Agent Give a Talk? Let's Find the Speaker in the Room!

SLMs and the Agents Revolution: Rethinking How Software Is Built

Serving LLMs on Kubernetes: A Golden Path for Platform Engineers

PANEL | The Future of Developer Portals

Cloud Native Live: Kubernetes v1.35 Release

Your Cluster is Lying to ArgoCD (And How to Catch It)

The Self-Improving Platform: Closing the Loop Between Telemetry and Tuning

Beyond Deployment: Mapping the Reality of Day 2 Operations in Platform Engineering

Making Observability Work at the Edge

The Next Chapter of Developer Experience: TAG DevEx in Action

Da Zero a Cento: Costruire un’Infrastruttura che si Auto-Ottimizza

The Efficiency Mission: Shrinking Java Pods without the Drama


[Published by Akamas](https://akamas.io/resources/gpu-utilization-llm-inference-optimization/)

AI is reshaping every industry, and agents are at the center of that shift. Many companies are racing to put them to work building assistants that handle customer requests around the clock, pipelines that classify and route data without human input or systems that reason across tools and act on their own. The appeal is obvious. What’s easy to miss in all the excitement is that every one of those agents, no matter how capable it looks from the outside, runs on a model, and that model runs on a GPU that is the engine keeping the whole thing moving.

Enterprises are pouring billions into GPUs to run LLMs and AI agents. For the first time in computing history, we are seeing data-center build-outs at a scale never witnessed before, where the GPU portion alone can account for as much as [70% of total investment](https://www.innovationendeavors.com/insights/future-data-centers), according to recent industry estimates.

Then the first real utilization reports come in. The number is hard to ignore: GPU usage as low as 5%. You might be spending millions on accelerators while [95% of that capacity sits idle](https://akamas.io/resources/gpu-utilization-llm-inference-optimization/#:~:text=AI%20is%20reshaping,the%20same%20hardware.) and yet almost every platform and engineering leader we talk to is trying to do the same thing: buy more GPUs to support the GenAI use cases moving into production this year. **The bottleneck, as it turns out, is usually not capacity but configuration.** On a stack as deep as the GenAI one, the gap between a default setup and a properly tuned one can easily reach 2x throughput or more on the same hardware.

![](https://akamas.io/wp-content/uploads/2026/06/AI-factory-ROI-equals-GPU-utilization.png)

This is the central tension we explored in our recent [webinar](https://akamas.io/events/webinar-genai-optimization-kubernetes/) with Daniele Zonca and Roland Huß from Red Hat, who quite literally wrote the book on running generative AI on Kubernetes.

## We’ve Already Seen This Pattern with CPUs

Here’s a number that should worry anyone treating GPUs the way they treated CPUs. Across one of our customers, with several dozen Kubernetes clusters and around 60,000 CPUs, average utilization over a full day was **5%**. At cloud prices, that adds up to roughly $15 million a year spent on resources that are almost entirely idle.

Those are CPUs. Now apply the same operational habits (over-provisioning, ignoring efficiency, assuming the answer to every capacity question is “add more”) to hardware that costs roughly 40x more per unit and is genuinely scarce. In much of Europe you simply can’t procure H100s on demand, no matter how big your budget is. If 5% utilization on cheap, abundant CPUs costs **$15 million a year**, the equivalent waste on GPUs is an order of magnitude worse. It can quietly block the ROI of an entire GenAI initiative.

## Why GPUs Are So Easy to Waste

GPUs are nothing like CPUs, and that difference is exactly why they’re hard to keep busy.

A **CPU is built for latency**: most of its silicon is dedicated to control logic and caches that predict what comes next, so that a handful of threads can execute general-purpose work as fast as possible otherwise a **GPU is built for throughput**: almost the entire chip is devoted to computation, with thousands of cores optimized for the matrix multiplications that LLM serving depends on. Think of a CPU as a fast motorbike and a GPU as a cargo ship.

The catch is feeding that cargo ship. Every computation requires data to move from high-bandwidth memory (HBM) into the cores, and that transfer frequently becomes the real bottleneck, which is the first reason GPU utilization ends up so low.

This gets sharper once you look at how inference actually runs, across two very different phases:

1. **Prefill** processes the entire input prompt in parallel to produce the first token. It is compute-bound and drives your time to first token (TTFT), which is the waiting time your user feels before anything appears on screen.
2. **Decode** generates tokens one at a time, each step reusing the KV cache built from all previous tokens. It is memory-bound and drives your time per output token (TPOT), which measures how fast the response streams out.

Because the same hardware handles two completely different access patterns, optimizing for one does not automatically improve the other. That mismatch is a core source of complexity in the whole stack.

## The Trade-off You’re Probably Losing On

NVIDIA’s Jensen Huang has popularized a chart worth understanding. When you plot total token throughput (your revenue proxy, since AI services are priced per token) against per-user interactivity (latency), you don’t get a single best point. You get a **Pareto front**: a curve of configurations where improving one dimension always costs you something on the other.

![](https://akamas.io/wp-content/uploads/2026/06/Extracting-2x-throughput-from-the-same-GPUs.png)

Push for maximum throughput and single-user latency suffers. Push for fast responses and you sacrifice a large share of GPU capacity.

Your use case fixes one constraint on that curve. A chatbot needs to emit tokens faster than human reading speed (around 200ms per token) or it feels sluggish and users leave. That means you have a fixed latency limit you cannot cross.

But along that line sit hundreds of possible configurations, and most teams are unknowingly sitting on a suboptimal one. A single misconfiguration can cut effective throughput in half, leading teams to conclude they’re maxed out and order more GPUs, when in reality they’re using only half of what they already paid for.

With proper tuning, doubling the throughput from existing hardware is not unusual. That is the real opportunity.

## The Playbook: Getting 2x from the GPUs You Already Have

The GenAI stack has hundreds of tuning knobs, from the model down to the kernel and Kubernetes itself. That depth is the real challenge. The skills needed to navigate it effectively are genuinely hard to find, which means most teams end up working with defaults that no one has had the time or expertise to question. Here are the levers with the highest payoff.

![](https://akamas.io/wp-content/uploads/2026/06/The-full-stack-LLM-inference-optimization-layers.png)

1. **Right-size the model, then quantize it**

Defaulting to the biggest model for every request is rarely the right call. Smaller and reasoning-tuned models can match quality on the right tasks, and a well-designed agent framework can maintain accuracy while running a smaller model. Quantization can compress a model to as little as 25–30% of its original size while preserving most of its accuracy, which is a significant reduction in resource use for almost no quality cost.

2. **Route intelligently and exploit cache locality**

Pick the right model per request, but also send follow-up messages to the replica that already holds the relevant KV cache. Multi-turn chat and agentic flows send the whole growing conversation back and forth, so cache-aware routing is critical to avoid recomputing work you’ve already done.

3. **Tune the inference engine and the parallelism strategy**

The engine layer around vLLM (or any other inference server) is evolving fast. It pays to match the parallelism strategy to your model:

- *Data parallelism* (replicas behind a load balancer) scales concurrency but not single-request latency.
- *Pipeline parallelism* splits a too-big model across nodes by layers.
- *Tensor parallelism* splits within a node using NVLink/NVSwitch to actually improve latency.

You can combine pipeline and tensor parallelism to get the best of both, but neither is free of trade-offs. Pipeline parallelism is constrained by inter-node network latency, which can be 25–50x slower than staying on a single node. Tensor parallelism introduces its own synchronization overhead across the NVLink fabric inside the node. The right combination always needs to be validated against your specific setup rather than assumed to work out of the box.

4. **Match hardware to the bottleneck**

If your constraint is in the decode phase, the real problem is memory bandwidth. Buying the GPU with the most teraflops will not help. Disaggregated serving (separating prefill and decode pools, as projects like llm-d enable) lets you scale each phase independently and route large inputs to dedicated prefill instances. As inference scales out, expect training-grade networking such as RDMA and InfiniBand to start mattering for inference as well.

5. **Partition GPUs with care**

If a GPU is allocated but underused, you may be able to host a second model on it, but GPUs have both a compute side and a bandwidth side. Partitioning can create a noisy-neighbor effect: spare capacity on paper does not guarantee you can add load without degrading both workloads. Always validate the impact before packing more onto a single card.

6. **Measure what you optimize**

None of these changes are safe to make without proper visibility. Track LLM-specific metrics (TTFT, TPOT), lean on the OpenTelemetry semantic conventions emerging for GenAI, and use tracing tools like MLflow to follow non-deterministic, multi-hop agentic requests. This also gives you the data for internal cost allocation across teams.

GPU-level metrics deserve special attention because they are significantly harder to interpret than application-level ones. Tools like NVIDIA’s DCGM give you utilization numbers, but those numbers are often too coarse to identify whether your actual bottleneck is compute, memory bandwidth, or something else. GPU memory bandwidth, for instance, cannot be directly measured from DCGM at all: vLLM can expose an estimation through the –enable-mfu-metrics flag, but it remains an approximation. To make things more confusing, “GPU utilization” means very different things depending on which layer of the stack you’re looking at (allocation, kernel execution, or model FLOP/s), and conflating them leads to wrong conclusions.

7. **Load test with your own traffic, not a vendor benchmark**

Traditional tools like k6, JMeter, and Locust work at the application level, but purpose-built tools like GuideLLM (part of the vLLM project) let you build realistic, record-and-replay workloads that reflect how your system is actually used. The most important point here is to always build your test dataset from real traffic, because a vendor’s benchmark on a foundation model says nothing about how the system will behave under your specific workload. The optimization space for GenAI is too workload-dependent for generic results to transfer.

## Too Many Knobs for a Human to Tune

With hundreds of knobs spread across three fundamentally different layers (the model, the inference server, and the GPU itself), millions of possible combinations, and a target that shifts with every new use case and every new model release, this is not a one-off exercise you do at launch. It is a continuous practice.

Each layer also requires different expertise. The people who understand model selection and quantization are rarely the same people who can tune vLLM internals or reason about GPU memory bandwidth, and finding someone who spans all three is genuinely uncommon. That makes this well beyond what any team can manage by hand.

The emerging answer is to use AI to optimize AI systems: autonomous tuning that explores the configuration space and finds the settings that maximize throughput while staying within your latency and cost targets. This is exactly the problem [Akamas](https://akamas.io/akamas-ai-infrastructure-optimization-launch/) was built to solve, first for traditional Kubernetes workloads and now for GenAI inference, and we have something significant coming on that front that we will be sharing very soon.

## The Bottom Line

As enterprises move AI agents into production and pour significant capital into the GPU infrastructure to support them, the pressure to show returns on that investment is growing fast. But utilization tells a different story, and the 5% figure is not a hardware problem. It is what happens when a genuinely complex stack gets deployed with default settings and no one has the time to tune it properly.

The good news is that the same depth that makes the GenAI stack hard to configure also creates real headroom. The difference between where most teams are parked today and what their existing hardware can actually deliver is large enough to change the ROI equation entirely.

You do not need more GPUs to get there. You need visibility into your actual bottleneck, a realistic picture of your workload, and a systematic way to explore the configuration space. Everything else in this article follows from those three things.







[Published by PlatformEngineering](https://platformengineering.org/blog/transforming-optimization-into-an-invisible-platform-capability)

In the high-stakes world of cloud-native delivery, organizations are frequently paralyzed by an "invisible wall" that fundamentally separates FinOps professionals, Site Reliability Engineers, and Developers. 

This structural friction strongly resembles a dysfunctional michelin-starred kitchen where the night’s success depends on characters who rarely see eye-to-eye on the operational priorities required to serve a flawless meal. 

![](https://cdn.prod.website-files.com/6489e23dd070ba71d41a33b2/6a0c8288c5dd8bfc13916685_Screenshot%202026-05-19%20at%2017.32.11.png)

Because these highly specialized teams speak entirely different languages regarding risk and reward, this dynamic creates a zero-sum game where the organization must continuously trade off cost for reliability or speed for efficiency. 

When a FinOps spreadsheet indicates that a Kubernetes pod is mostly idle, the SRE immediately perceives a threat to their carefully calculated reliability buffer, while the developers simply lack the contextual data to know how much compute space their specific application actually requires under realistic load. 

This operational paralysis is further exacerbated by the shared ownership model prevalent in many modern enterprises, where a lack of singular accountability means that reactive cost-cutting and conservative over-provisioning remain the default strategies.

## The day 2 operational gap

To break this destructive cycle, we must stop treating operational efficiency as a manual "spring cleaning" task and start recognizing it as an essential requirement that must be embedded deeply within modern platform architecture. 

While the platform engineering community has excelled at creating golden paths to streamline the initial developer experience and deployment pipelines, the lifecycle of applications reaching production has been largely neglected. Currently, our primary method of assisting developers with post-deployment optimization relies on providing access to passive monitoring dashboards rather than delivering automated, actionable intelligence. 

Organizations are striving to run highly performant applications while simultaneously maintaining strict cost controls and ensuring absolute reliability, but static golden paths are simply insufficient for managing this delicate trade-off. 

When building deployment pipelines, everyone shares the single goal of releasing software, which allows for strict and opinionated automation. However, production optimization requires a highly configurable approach because different applications inherently require dynamic tuning strategies tailored to their specific risk profiles and performance needs. For these reasons, static deployment pipelines inevitably fall short during Day 2 operations and lead to systemic resource waste.

## The full-stack optimization puzzle

True optimization represents a complex multi-layered puzzle that extends far beyond simply resizing Kubernetes pods or shutting down idle development environments after business hours. 

Deep efficiency must begin at the application runtime level by tuning elements like the Java Virtual Machine heap size or garbage collection settings to ensure the core workload operates at peak health. Once the runtime is properly configured and understood, engineers can accurately size the container requests and limits to perfectly encase that specific application without causing artificial throttling or fatal memory errors. 

This layered approach must also carefully incorporate how you scale horizontally and the underlying node groups to ensure the entire system remains stable when infrastructure density increases. Without looking at this vertical stack of dependencies, scaling policies might rely on superficial metrics that do not align with the newly established pod boundaries.

![](https://cdn.prod.website-files.com/6489e23dd070ba71d41a33b2/6a0c82b03624c5b2b527b085_Screenshot%202026-05-19%20at%2017.32.51.png)

This inevitably causes the system to fluctuate unpredictably and degrade the performance of your applications. You cannot effectively optimize a system if you treat the container as an opaque black box, yet industry data shows that very few companies actually attempt to tune the container and the runtime in tandem.

### Resolving the kitchen conflict

To solve this puzzle, we must address the conflicting motivations of the specialized roles mentioned earlier. The "invisible wall" persists because each player in the kitchen has a different definition of success:

- The **FinOps** team acts as the restaurant owner who scrutinizes every cloud expenditure and demands higher utilization rates to justify the infrastructure investments.
- The **Site Reliability Engineers** operate as the executive chefs who prioritize absolute uptime and system stability, often demanding massive resource buffers to absorb unexpected traffic spikes.
- The **platform team** serves as the overwhelmed manager attempting to standardize infrastructure while keeping the peace.
- The **developers** act as line cooks who desperately try to innovate without clear guidance on the resource constraints they must operate within.

## Integrating efficiency into the GitOps workflow

By integrating performance gates directly into the continuous integration and continuous deployment pipeline, the platform team establishes a controlled environment where efficiency becomes a core component of the software release lifecycle. 

These purpose-built platform capabilities can continuously analyze historical observability data to generate intelligent recommendations based on predefined tuning profiles that strictly respect the safety limits established by the SRE team.

![](https://cdn.prod.website-files.com/6489e23dd070ba71d41a33b2/6a0c82cefc9e4e1f6cdac159_Screenshot%202026-05-19%20at%2017.33.24.png)

The most effective way to implement these data-driven recommendations involves establishing a human-in-the-loop workflow driven entirely through established GitOps principles. Whenever the platform identifies a more efficient configuration that satisfies both cost constraints and performance requirements, it seamlessly opens a pull request directly in the developer's repository containing the evidence-based sizing adjustments. 

Developers can easily review these insights within their existing tools and approve the new configuration with absolute confidence, knowing that the proposed changes have been mathematically validated against the organizational reliability standards. 

By embedding these automated right-sizing mechanisms directly into the golden path, the platform truly becomes a unified bridge that eliminates the historical friction between engineering departments. This approach moves the organization beyond simple infrastructure provisioning to create a self-improving ecosystem where cost-awareness and performance are continuously balanced without requiring constant human intervention.

By transforming optimization from a manual chore into an invisible platform capability, we solve the "Michelin kitchen" dilemma. This shift allows developers to focus entirely on innovation and feature delivery while the underlying system ensures the organizational menu is always perfectly optimized for both the business and the end user. When efficiency is automated, the invisible wall disappears, replaced by a seamless path toward sustainable, high-performance delivery.


[Published by CNCF co-authored with Hélia Barroso, Alessandro Pomponio, Sonali Srivastava](https://www.cncf.io/blog/2026/04/29/the-state-of-ai-in-cncf-projects-a-first-look-at-the-data/)

At **CNCF TAG Developer Experience**, we recently set out to understand how Artificial Intelligence is shaping open-source development. The response from the community has been impressive in its scale, with nearly half of our initial responses arriving within the first week alone. This immediate engagement highlights the urgency of the topic and the community’s desire for shared guidelines for AI-assisted development.

The data collected from 133 respondents so far represents **nearly 100 unique projects**, giving us confidence that these findings reflect the cloud-native ecosystem at large rather than a narrow subset of projects. This article serves as a sneak peek into our initial findings; a more comprehensive analysis will follow as we process further data.

### Who is answering?

The feedback primarily reflects the perspectives of those on the front lines. The vast majority of participants are code-centric contributors focusing on submission, CI/CD, and infrastructure, while approximately 20% combine engineering with critical roles like release management and documentation.

### How maintainers use AI: Tools and workflows

Modern AI tools have moved beyond simple web chatbots and are now deeply integrated into daily routines. Nearly half of respondents actively use AI assistants directly within their IDEs or command-line interfaces.

![A blue bar graph showing the levels of basic, advanced, other and intermediate. ](https://www.cncf.io/wp-content/uploads/2026/04/image-14.png)

A blue bar graph showing the levels of basic, advanced, other and intermediate.

When it comes to tooling, Claude Code and Github Copilot emerge as clear leaders in the space. Interestingly, only a small fraction (roughly 10%) of contributors still rely on basic chatbots via manual copy-pasting. Meanwhile, a similar percentage of advanced users has already moved toward “high-level integration,” where AI is built directly into project automation for PR reviews and issue triaging.

### Where AI helps the most

Contributors are seeing the most significant boosts in productivity within a few specific areas:

- Writing and refactoring code.
- Improving documentation and debugging.
- Understanding unfamiliar codebases.
- Analyzing Pull Requests.

The high ranking of “understanding the codebase” suggests that AI is acting as a knowledgeable guide, helping developers navigate the inherent complexity of large-scale projects.

### The gap between AI use and official policies

One of our most striking findings is the disconnect between individual AI usage and formal project governance. While local AI use is widespread, the adoption of official policies has lagged behind.

Roughly two-thirds of respondents are either unaware of any specific AI guidelines or confirmed that no official policies exist in their main repositories. Furthermore, the vast majority of projects make no mention of AI usage in their public-facing documentation or contributing guides. While a few pioneering projects are setting the pace with clear policies, the ecosystem is largely operating in an environment that is still figuring out how to govern automated code generation.

![An orange bar graph showing levels of clear policies, informal acceptance and unaware/no policy ](https://www.cncf.io/wp-content/uploads/2026/04/image-15.png)

An orange bar graph showing levels of clear policies, informal acceptance and unaware/no policy

### Community sentiment and code reviews

Despite the lack of formal rules, the general “vibe” toward AI is open and accepting. Roughly one-third of contributors noted that AI usage is generally allowed. Conversely, only a tiny minority (less than 4%) reported that AI usage is explicitly prohibited in their environments.

![A green bar graph showing levels of 'unsure', 'not required', 'specific cases only' and 'always require disclosure.' ](https://www.cncf.io/wp-content/uploads/2026/04/image-16.png)

A green bar graph showing levels of 'unsure', 'not required', 'specific cases only' and 'always require disclosure.'

This pragmatic approach extends to how maintainers handle AI-generated contributions:

- A solid majority follow their standard review process without applying special filters.
- Over a quarter of maintainers prefer a collaborative approach, asking contributors to refine AI-generated code to meet quality standards rather than rejecting it.
- Only a nominal percentage automatically reject suspected AI PRs.

### Top concerns and the call for transparency

While the community is optimistic, several valid concerns remain. Maintainers are particularly worried about:

1. Security vulnerabilities
2. License compliance.
3. The burden on reviewers, caused by **a potential flood of low-effort PRs**.

To mitigate these risks, there is a strong collective desire for visibility. Over half of respondents believe that AI-assisted contributions should always require formal disclosure (such as an “AI-authored” tag). An additional 20% feel disclosure should be required in specific cases. This suggests that while maintainers are willing to accept AI-generated code, they want the transparency necessary to adjust their review efforts accordingly.

### Wrapping up

This first batch of data confirms that AI integration is no longer a trend, it is a core part of the modern workflow. As we move forward, the challenge for the cloud-native community will be balancing this new productivity with the high standards of security and manual oversight that enterprise-grade open source requires.

We aren’t finished yet! To ensure our final report truly reflects the diverse cloud-native landscape, we need your voice. The survey will remain open until Monday, May 18 (End of Day, Anywhere on Earth).If you haven’t shared your experience yet, we invite you to [**contribute to the survey**](https://forms.gle/tBjQp9sFnBF3CXE17) and help us build a more accurate and comprehensive picture of AI’s role in our community.


[Published by Akamas](https://akamas.io/resources/bridging-finops-sre-platform-teams-kubernetes/)

In the world of fine dining, the tension in the kitchen is palpable long before the first guest arrives. Imagine a prestigious Michelin-starred restaurant where the night’s success depends on **four formidable characters who rarely see eye-to-eye**. There is the Owner, who meticulously tracks the cost of every truffle and the electricity bill of the walk-in freezer; there is the Executive Chef, who would rather throw away ten gallons of perfectly good consommé than risk serving a single plate that is less than transcendent; there is the restaurant manager, who struggles to coordinate the chaotic flow between the ovens and tables using a reservation system that feels increasingly outdated; and finally, there are the Line Cooks, the ones actually preparing the dishes.  
  
If the Owner cuts the budget too thin, the quality of the food collapses. If the Chef over-prepares to ensure they never run out of ingredients, the restaurant bleeds money through food waste. If the Manager fails to sync the two, the entire experience becomes a slow-motion disaster. The Line Cooks are caught in the middle: they want to innovate and serve great food, but they lack the data to know if they are using too many expensive ingredients or if their “stove” is set to the right temperature.

In the digital landscape of cloud-native enterprises, our Kubernetes clusters are the kitchens, and our microservices are the delicate dishes being served to millions of hungry users. The **FinOps** team acts as the Owner, scrutinizing the cloud bill and demanding that we stop paying for “tables” that sit empty during off-peak hours. The **Site Reliability Engineers (SREs)** are our Chefs, their reputation is built on uptime and performance, and for them, an over-provisioned cluster is not waste: it is the extra pantry stock that prevents a catastrophic “sold out” sign during a traffic surge. Meanwhile, the Platform Team is the Manager, trying to build a standardized infrastructure that keeps the peace while the tickets pile up. In this scenario, the Developers are the Line Cooks creating the recipes: without clear guidance, they end up requesting more fridge space than necessary, fearing that the Chef (SRE) or the Owner (FinOps) will take away the “heat” they need to finish the service.

## The Invisible Wall

This structural gap is what we call the ***Invisible Wall***. It is a byproduct of specialization where everyone is doing their job perfectly, yet the organization as a whole is failing to achieve true efficiency. According to the [2026 FinOps Foundation’s State of FinOps](https://data.finops.org/#priorities) report, the primary challenge for organizations has shifted toward reducing waste and over-provisioning while still struggling with “empowering engineers to take action”.  
  
This isn’t because engineers are lazy; it is because the Merchant, the Chef and the Line Cooks speak different languages. When a FinOps spreadsheet says a pod is 80% idle, the SRE hears a threat to their reliability buffer while Developers simply lack the data to know how much space their specific application actually requires. Without a shared source of truth that translates “dollars saved” into “risk managed”, these teams remain locked in a cycle of conservative over-provisioning and reactive cost-cutting.

![Diagram showing how FinOps, SRE, Developers and Platform teams' conflicting priorities lead to over-provisioning and friction in Kubernetes environments](https://akamas.io/wp-content/uploads/2026/04/invisible-wall-finops-sre-devs-platform-kubernetes.png.png)

Diagram showing how FinOps, SRE, Developers and Platform teams' conflicting priorities lead to over-provisioning and friction in Kubernetes environments

The tragedy of this friction is that it treats optimization as a zero-sum game, a world where you must choose between a healthy bottom line and a reliable service. Ironically, this defensive over-provisioning doesn’t just inflate the bill; it often results in suboptimal performance and reliability issues, such as unexpected Out-Of-Memory (OOM) errors, as teams lack the precision to balance resources correctly. However, the reality of modern cloud-native architecture is far more nuanced. True efficiency is not about buying cheaper ingredients: it is about the fit of the resource to the task. That is the very principle on which [Akamas Insights](https://akamas.io/insights/) was born. To achieve adequate **fit** for purpose, [Akamas Insights](https://akamas.io/insights/) introduces the concept of the ***Tuning Profiles***. Think of it as the “House Rules” or the “Recipe Standard” for the kitchen. A Tuning Profile allows the Platform Team and the SREs to define the boundaries of optimization specifying which objective matters most (such as cost reduction or latency improvement) and which constraints must be respected (such as minimum replicas or memory headroom). By establishing these profiles, the Chef (SRE) can trust that the “Owner’s” demands will never violate the fundamental safety limits of the service.

![Diagram showing how Akamas Insights aligns FinOps, SRE, and Platform teams around a shared view to achieve high reliability and low waste in Kubernetes](https://akamas.io/wp-content/uploads/2026/04/akamas-insights-shared-truth-finops-sre-platform.png-2048x685.png)

Diagram showing how Akamas Insights aligns FinOps, SRE, and Platform teams around a shared view to achieve high reliability and low waste in Kubernetes

## The Full-Stack Recipe

Instead of looking at a Kubernetes cluster as a collection of static boxes, Akamas Insights treats the environment as a living, breathing organism. By bridging the gap between infrastructure metrics and application runtime behavior, it allows the Owner, the Chef, the Manager and the Line Cooks to finally look at the same dashboard and see the same truth.

When we talk about deep, **full-stack optimization**, we are looking at the vertical stack of dependencies. Optimization is a multi-layered puzzle that starts at the Application Runtime (tuning JVM heap or Go GC) to ensure the engine is healthy. Only then do we move to the Pods, sizing requests and limits to perfectly encase that runtime. Crucially, this is where we also align the Horizontal Pod Autoscaler (HPA). If an HPA is scaling based on a metric that doesn’t align with the Pod’s new limits, the system will “flap” and become unstable. Akamas Insights coordinates these layers to ensure that when we increase density, the contents still have room to breathe.

Finally, we size the Node Group accordingly, ensuring the underlying hardware is the right vessel for these optimized and correctly-scaled containers**.** By following this sequence, the infrastructure becomes a direct reflection of the application’s actual needs, rather than a collection of oversized safety nets.

![Three-layer model showing how full-stack Kubernetes optimization spans App Runtime (JVM, concurrency), Pods and HPA (CPU, memory, scaling), and Node Groups to achieve optimal density](https://akamas.io/wp-content/uploads/2026/04/full-stack-kubernetes-optimization-layers-jvm-pods-nodes.png-2048x545.png)

Three-layer model showing how full-stack Kubernetes optimization spans App Runtime (JVM, concurrency), Pods and HPA (CPU, memory, scaling), and Node Groups to achieve optimal density

## Action over Analysis

This level of insight transforms the conversation from one of ***cutting*** to one of ***tuning***. In our restaurant metaphor, this is the equivalent of a smart inventory system that knows exactly how many guests are coming, their dietary preferences, and the precise temperature each oven needs to be to minimize energy without losing a degree of heat. When the Platform Team can present an optimization plan **backed by a verified Tuning Profile** that accounts for JVM heap settings, CPU cores, and node affinity, the SRE no longer feels the need to hoard resources “just in case”. They can see, through data-driven and historical analysis, that the new configuration is actually more stable than the old, bloated one.

Furthermore, the industry is beginning to realize that the “Spring Cleaning” approach to cloud costs is fundamentally broken. You cannot achieve a Michelin star by cleaning your kitchen once a year: it requires constant, meticulous attention to detail. Yet, many organizations still treat FinOps as a quarterly audit and a reactive “Crawl” stage behavior. According to the [FinOps Foundation’s Workload Optimization](https://www.finops.org/framework/capabilities/workload-optimization/) capability, true maturity (the “Run” stage) is defined by a shift toward **Continuous Optimization**.

However, the path to this maturity is not about handing the keys of the kitchen over to an unpredictable autonomous agent. In a world where reliability is paramount, a “black-box” real-time approach can introduce more instability and “flapping” than it solves. True excellence requires a **Human-in-the-Loop** model where the Developers (the Line Cooks) are the essential piece. Instead of chasing every micro-fluctuation, the most effective optimization happens in lockstep with the **release cycle**.

![Workflow diagram showing the continuous optimization cycle with Akamas Insights: deploy, analyze telemetry, generate AI-powered recommendations, human review, and apply to next release](https://akamas.io/wp-content/uploads/2026/04/akamas-insights-release-analyze-recommend-apply-loop.png-2048x336.png)

Workflow diagram showing the continuous optimization cycle with Akamas Insights: deploy, analyze telemetry, generate AI-powered recommendations, human review, and apply to next release

Every time a new version of the “menu” is released, Akamas Insight performs a deep analysis of the previous cycle’s telemetry. It then generates intelligent, evidence-based recommendations tailored to the Tuning Profile. These recommendations are presented to the Developers for review. This is the “aha!” moment: the developer sees exactly how their code performs and can approve the new configuration with one click, knowing it aligns with the safety standards set by the SRE. This ensures that resource efficiency is balanced against performance and reliability within a controlled, predictable workflow: **Release, Analyze, Recommend, and Apply**. By connecting directly to the telemetry data teams already have, Akamas Insights turns the release cycle into a self-improving loop of continuous harmony.

## The Michelin Standard of Engineering

This transition allows the Platform Team to fulfill its true potential. Instead of being the “Manager” who spends all night mediating between the “Owner” and the “Chef”, they become the architects of a system that empowers Developers to be efficient by default. This creates a culture of “Unit Economics”, where the cost of a transaction is as much a part of the performance metric as latency or error rate. When everyone understands that a 10-millisecond improvement in code efficiency translates directly to a specific reduction in the cloud bill, the silos finally begin to crumble.

The beauty of this harmony is that it liberates the human talent within the organization. Engineers who used to spend their weeks manually tweaking YAML files and arguing over resource limits are suddenly free to innovate. The FinOps team moves from being the “budget police” to being strategic advisors who can predict how a new product launch will impact the bottom line with surgical precision. The SREs can sleep through the night, knowing their clusters are not just “big enough”, but “accurate enough”.

As we look toward the future of cloud-native technology, the goal is no longer just “getting to the cloud”. We are already there. The goal is to live in the cloud with grace and precision. We must move beyond the blunt instruments of the past and embrace a more sophisticated way of managing our digital kitchens. The “Michelin Star” of engineering is awarded to those who can deliver a flawless user experience with the elegance of a perfectly managed budget. By using intelligent, full-stack insights to bridge the gap between our teams, we stop guessing, stop wasting, and finally start cooking with fire.


[Published by DZONE](https://dzone.com/articles/observability-otel-fluentbit)

When we design observability pipelines for modern cloud environments, we implicitly rely on a set of luxurious guarantees: limitless bandwidth, highly available networks, practically infinite storage, and abundant computing power. But when you move these workloads to the edge, think of a maritime vessel navigating the mid-Atlantic or a remote wind turbine, those guarantees vanish. Edge environments are constrained by intermittent connectivity, severe limits on CPU and RAM, and a lack of persistent storage guarantees. You simply cannot run a full, traditional observability stack locally, nor can you stream everything to the cloud without exhausting limited satellite bandwidth.

The engineering challenge becomes clear: how do we build a pipeline that reliably captures traces, metrics, and logs, survives unpredictable network outages, and perfectly correlates signals without saturating edge constraints? A highly compelling, production-realistic solution to this problem was showcased for KubeCon EU 2026, demonstrating a fully correlated observability pipeline built for constrained edge environments using [OpenTelemetry](https://dzone.com/articles/opentelemetry-ending-era-of-fragmented-visibility) and Fluent Bit. You can explore the complete implementation in the [graz-dev/observability-on-edge](https://github.com/graz-dev/observability-on-edge) repository.

This article dives deep into the architecture, the technical trade-offs, and the specific configurations required to make observability work reliably when the network itself is your biggest enemy.

## Taming the Bandwidth With Tail-Based Sampling

The fundamental problem with [distributed tracing](https://dzone.com/articles/distributed-tracing-a-full-guide) is the sheer volume of data it generates. A single HTTP request traversing various middleware and downstream services can easily produce up to 50 individual spans. At a relatively modest load of ten requests per second, you are suddenly dealing with hundreds of gigabytes of trace data over a year. In a cloud environment, you might simply scale up your storage. On a maritime vessel connected via an expensive, low-bandwidth satellite link, sending all of this is economically and technically impossible.

To solve this, we must aggressively sample the data, dropping the noise and keeping only what is actionable. The most naive approach is **head-based sampling**, where the system makes a keep-or-drop decision at the very first span of a trace. While head-based sampling adds almost no computational latency, it is entirely blind to the outcome of the request. If you decide to drop a trace at its inception, and that request subsequently fails or experiences a massive latency spike, that crucial diagnostic information is lost forever. In edge deployments where errors might be rare but highly critical, this is unacceptable.

The solution is **tail-based sampling**. By buffering all spans of a trace in memory within the OpenTelemetry Collector, the pipeline waits for the trace to fully complete before making a decision based on the final outcome. In this implementation, the tail-sampling policy is strictly configured to keep 100% of traces that contain an error, and 100% of traces where the duration exceeds 200 milliseconds. Normal, fast, successful traces are discarded entirely. The observed result is a massive reduction in bandwidth overhead, dropping roughly 80% of all spans before they are ever exported over the network.

To make this bandwidth reduction truly effective, we must apply the same philosophy to our logs. If we filter traces but send every single log line, we defeat the purpose. For log processing, Fluent Bit runs as a DaemonSet on the edge nodes, tailing container logs. Rather than using Fluent Bit's native grep filters, which struggle with complex multi-field conditional logic, a custom Lua filter is injected into the pipeline. This Lua script precisely mirrors the OpenTelemetry tail-sampling criteria, evaluating each log record and keeping only those with an error level or a duration exceeding 200 milliseconds. By performing this logic at the absolute edge, before the log data even leaves the node, the pipeline drops approximately 86% of log volume at the source, preventing unnecessary network I/O.

## Persistent Queuing for Intermittent Connectivity

Aggressive sampling solves the bandwidth issue during steady-state operations, but what happens when the satellite link inevitably fails? If the OpenTelemetry Collector cannot reach the central hub, it will quickly exhaust its in-memory retry buffers, and all telemetry accumulated during the outage will be permanently lost.

To survive these disconnections, the pipeline implements a file-backed persistent queue utilizing the `file_storage` extension within the OpenTelemetry Collector. This provides a bbolt (BoltDB) key-value store directly on the edge node's local disk. When the exporter's sending queue is configured to use this file storage, outgoing telemetry batches are serialized and written safely to the crash-safe bbolt database before dispatch. These items remain safely on disk until the collector receives a successful acknowledgment from the remote endpoints.

Configuring this queue correctly requires understanding a critical nuance of the OpenTelemetry Collector's internal architecture regarding consumers. By default, the exporter uses four concurrent consumer goroutines to claim batches from the queue. Because the processing pipeline generally produces batches relatively slowly at edge traffic volumes, these four consumers will claim batches almost instantly, holding them in their in-memory retry buffers rather than leaving them in the bbolt queue.

Consequently, your queue depth metrics will deceptively report zero even during an active network outage, blinding you to the growing backlog. By deliberately setting the `num_consumers` configuration to exactly one, only a single batch is ever held in flight in memory. All subsequent batches safely queue in bbolt, allowing the metric to accurately reflect the growing backlog during an outage.

## The Reconnection and the Time Travel Problem

When the satellite link is eventually restored, the real chaos begins. The OpenTelemetry Collector detects the restored connection and immediately begins draining the accumulated bbolt queue at maximum network speed. However, the data stored in this queue possesses timestamps from minutes or hours in the past. We are essentially attempting to push historical, out-of-order data into our central observability backends. Different backends handle this "time travel" problem differently. Jaeger, our distributed trace storage, handles it gracefully by design. Its storage model is append-only, possessing no concept of out-of-order rejection. Traces originating from the failure window simply appear in the user interface precisely where they belong chronologically.

[Loki](https://dzone.com/articles/grafana-loki-fundamentals-and-architecture), handling our logs, is much stricter. By default, Loki expects log entries for a given stream to arrive in roughly chronological order, and it will forcefully reject significantly older timestamps with HTTP 400 errors. If left unconfigured, the OpenTelemetry Collector would receive these errors continuously upon queue drain, leading to the permanent loss of all logs generated during the outage. To prevent this disaster, we must explicitly configure `unordered_writes: true` in the Loki settings. This crucial parameter disables the strict per-stream ordering requirement, allowing the massive burst of queued, historical log entries to be ingested successfully.

Metrics ingestion presents an even harsher reality. In this architecture, metrics are exported to Prometheus using the `prometheusremotewrite` exporter. Unlike logs and traces, the OpenTelemetry Collector library supporting this exporter lacks support for the file-backed bbolt queue, leaving the metrics queue strictly in-memory.

Furthermore, when the link restores, any old metrics held in memory are sent to Prometheus, but Prometheus natively rejects out-of-order samples. While there are alternative protocols like OTLP HTTP for metrics, utilizing OTLP for Prometheus ingestion results in aggressive HTTP 400 rejections for out-of-order data. This causes the exporter to retry indefinitely, permanently blocking the queue and grinding the entire metrics pipeline to a halt.

It is crucial to note that this specific limitation — the inability to use file-backed queues for metrics — is a known library constraint tied directly to the OpenTelemetry Collector versions (v0.95 and v0.96) utilized in this repository. Because these specific builds do not support `sending_queue.storage: file_storage` for the Prometheus remote-write exporter, the architecture is forced to keep the metrics queue in RAM. The architectural decision here is a deliberate and calculated engineering trade-off: by using the `prometheusremotewrite` exporter targeting the remote-write endpoint, Prometheus silently skips the out-of-order samples by returning an HTTP 204 status with zero written. The pipeline queue drains cleanly and unblocks, but the metric data generated during the actual outage window is intentionally sacrificed. At the edge, maintaining pipeline integrity is often prioritized over absolute metric continuity.

## Achieving Deterministic Signal Correlation

An observability stack is only as valuable as its ability to correlate signals. A latency spike on a dashboard must lead seamlessly to the exact distributed trace, which must seamlessly transition to the specific log lines emitted during that exact request. In this edge architecture, achieving perfect correlation is not reliant on best-effort timestamp matching; it is structurally guaranteed.

The process begins in the application code, which extracts the OpenTelemetry `trace_id` and `span_id`  
from the context of every incoming HTTP request and structurally injects them into every log line via a JSON logger. Because the Fluent Bit Lua filter and the OpenTelemetry tail-sampling processor utilize the exact same logic, we achieve a deterministic alignment. Every trace that survives the sampler will have a corresponding log line surviving the Lua filter, and conversely, no log line will exist without a parent trace. There are no orphaned traces and no orphaned logs. To link our high-level metrics directly to these traces, the architecture employs the OpenTelemetry `spanmetrics` connector. This connector reads the sampled spans and generates Prometheus histogram metrics regarding request rates and latencies. Crucially, it attaches an exemplar to each histogram bucket, a sparse metadata annotation carrying the specific `trace_id` that contributed to that latency measurement.

The placement of this connector is paramount. In the configuration pipeline, `spanmetrics` is wired strictly after the tail sampling processor. Because it runs post-sampling, every single `trace_id` it embeds into a metric exemplar is absolutely guaranteed to have survived the sampling process and exist in Jaeger. When operators view their Grafana dashboards, they see diamond markers on their latency graphs representing these exemplars; clicking a marker reliably drops them directly into the exact failing trace with zero dead links.

## Performance Testing

To prove this architecture works under realistic conditions, the project doesn't just send a few manual requests. Instead, it employs the k6 Operator, a Kubernetes-native load test runner, to generate a continuous, high-volume telemetry stream from within the cluster hub. The load generator utilizes a custom `TestRun` resource that spins up 500 virtual users over a 40-minute period.

This performance test follows a deliberate ramp-up profile: it scales from zero to 100 virtual users in the first 30 seconds, climbs to 250 in the next 30 seconds, reaches 500 at the one-minute mark, and sustains that peak for a 40-minute steady state. At its peak, this setup generates approximately 2,500 spans per second. The traffic is intelligently distributed across four distinct API endpoints simulating a vessel's systems, each with specific latency and error profiles, perfectly exercising the tail-sampling and Lua filtering logic.

While the k6 load test validates the pipeline's throughput, the underlying reality is that rigorous performance testing was essential for a much more critical goal: drastically minimizing the collector's resource footprint. In constrained edge environments, every megabyte of RAM and CPU cycle consumed by observability is stolen directly from the primary application workloads. Through an extensive, automated performance tuning campaign, we analyzed the complex interactions between the Go runtime and the collector's internal processors. The findings revealed that optimizing an edge node requires surgically tuning both the OTel configuration and the underlying Go environment rather than simply guessing at limits. By methodically testing various permutations, we discovered the exact "sweet spot" to maximize performance while shrinking the footprint.

The most impactful findings from these tests led to highly specific internal calibrations. For example, the `memory_limiter` processor was precisely tuned to enforce a soft limit of 320 MiB and a hard limit of 400 MiB. This was paired with a batch processor rigorously configured to accumulate exactly 512 spans or wait for a maximum 5-second timeout. Furthermore, these tests demonstrated that throttling the exporter queue to a single consumer (`num_consumers: 1`) was critical. It not only provided accurate backpressure metrics during a simulated satellite outage but structurally prevented the Go runtime's garbage collector from thrashing when the connection was restored and massive historical queues suddenly drained.

The results of this optimization campaign are striking. Stripped of the bloated default components via the OpenTelemetry Collector Builder, the resulting 30 MB binary operates seamlessly under intense pressure. It continuously processes thousands of spans per second while consuming merely 1% to 5% of a single CPU core and hovering predictably between 80 MiB and 150 MiB of active memory. This definitively proves that with proper performance testing and exact Go runtime calibrations, you do not have to choose between rich telemetry and edge node stability.

## Validating Resilience Through Network Chaos

Proving that resilience actually works requires simulating harsh physical realities within a controlled Kubernetes environment. Relying on high-level Kubernetes NetworkPolicies is insufficient for this testing, as they do not provide the surgical, instantaneous, and reversible IP-layer control needed to simulate an abrupt satellite drop. Instead, the project utilizes a privileged DaemonSet running `netshoot`, a network debugging container. Operating in the host network namespace, this pod can directly manipulate the edge node's kernel routing rules using `iptables`. A dedicated chaos script surgically inserts DROP rules into the FORWARD chain, specifically targeting the outbound ports for Jaeger, Loki, and Prometheus.

A critical detail in this simulation is the behavior of the Linux kernel's connection tracking framework, `conntrack`. Modern kernels maintain state for established TCP connections, allowing them to bypass newly inserted DROP rules. If you apply an `iptables` drop rule without further action, existing gRPC connections between the collector and the hub will simply continue to flow unaffected. The chaos script explicitly executes a `conntrack` flush command targeting the collector's IP address. This violently terminates the established states, forcing the client to initiate a new TCP handshake, which is immediately blocked by the new rules. This accurately triggers the failure state: the OpenTelemetry exporter begins failing, batches begin piling up in the bbolt database, and the queue depth metrics steadily climb. Removing the rules simulates link restoration, triggering the massive, satisfying spike in export throughput as the resilient queue drains historical data into the backends.

## Conclusion

Observability at the edge forces engineering teams to abandon the comfortable defaults of cloud-native computing. We cannot afford to transmit every metric, log line, and trace span. By combining aggressive tail-based and source-based sampling, highly localized persistent queuing, out-of-order gap-filling configurations, and meticulous correlation through exemplars, it is completely possible to maintain deep, actionable visibility into remote, constrained environments.

The implementation presented in the [graz-dev/observability-on-edge](https://github.com/graz-dev/observability-on-edge) repository serves as an example of these techniques. It proves that with strict resource management and a deep understanding of network behavior, robust edge observability is not just a theoretical concept, but a highly achievable engineering reality.


[Published by Akamas](https://akamas.io/resources/the-state-of-cloud-native-optimization-2026/)

A critical realization has emerged within the platform engineering community: while the industry has excelled at creating **golden paths** to streamline the developer experience, we have largely neglected the lifecycle of applications once they reach production. We have successfully established robust pipelines, implemented extensive automation, and delivered reusable services that empower developers to build and deploy software with unprecedented speed. However, a significant gap remains in our operational maturity.

Currently, our primary method of assisting developers with post-deployment optimization is simply providing access to monitoring dashboards. **We have not yet developed the automated processes required to help them actively tune their applications based on fluctuating real-world demand.** This missing link between deployment and continuous operational efficiency is precisely where the next evolution of cloud-native infrastructure must focus.

## The Organizational Landscape and the Ownership Dilemma

To understand the magnitude of this challenge, we conducted an extensive analysis based on in-depth interviews with over 50 organizations across various sectors, including large telecommunications, healthcare, finance, defense, and insurance.

![Cloud native optimization 2026 geographic distribution of surveyed organizations](https://akamas.io/wp-content/uploads/2026/04/20.svg)

Cloud native optimization 2026 geographic distribution of surveyed organizations

The geographic distribution shows that almost **70% of these enterprises are based in Europe**, while 21.9% operate in the United States. Furthermore, the dataset represents several market players, with 38.3% classified as Large organizations and 25.5% as Enterprise-scale entities.

A close examination of how these companies organize their technical teams reveals a highly fragmented environment.

![Cloud native optimization 2026 breakdown of organizations by size](https://akamas.io/wp-content/uploads/2026/04/21.svg)

Cloud native optimization 2026 breakdown of organizations by size

Dedicated platform teams are present in approximately 43% of the organizations. Companies without these dedicated teams often attempt to maintain their platforms through disjointed efforts, typically delivering superficial automations that fail to address the comprehensive developer workflow.

The roles of Site Reliability Engineering and FinOps present an even greater organizational challenge. SREs frequently operate in silos within individual application teams, lacking a standardized baseline for reliability across the entire company. Meanwhile, **dedicated FinOps teams exist in only about 20% of the interviewed organizations**. For many companies, financial optimization is either outsourced or treated as a quarterly, reactive cost-cutting exercise rather than a continuous discipline.

This fragmentation inevitably raises a fundamental question about accountability regarding who actually owns the optimization process.

![Platform engineering adoption across organizations in cloud native optimization study](https://akamas.io/wp-content/uploads/2026/04/22.svg)

Platform engineering adoption across organizations in cloud native optimization study

The data indicates that a **shared ownership model dominates, accounting for 52% of the surveyed organizations**. While shared responsibility sounds excellent in theory, the practical reality is that when everyone is technically responsible, nobody takes decisive action.

This operational paralysis stems from fundamentally conflicting priorities. Developers are driven by the need to ship new features quickly and lack the time to revisit legacy deployments. SREs prioritize system stability, frequently overprovisioning resources to create a massive safety buffer against traffic spikes to avoid costly outages. Conversely, FinOps professionals view this overprovisioning as unacceptable financial waste and push relentlessly to reduce the cloud bill. **The developer ultimately remains trapped in the middle of this structural conflict.**

## Navigating the Trade-Offs of Modern Infrastructure

These internal frictions align perfectly with the primary pain points reported at the organizational level.

![Distribution of SRE and FinOps teams in cloud native environments](https://akamas.io/wp-content/uploads/2026/04/23.svg)

Distribution of SRE and FinOps teams in cloud native environments

Companies are equally divided between optimizing for costs and maximizing performance, with **cost concerns cited by 38.6% of respondents and performance issues closely following at 36.1%**. Organizations are striving to run highly performant applications while simultaneously maintaining strict cost controls and ensuring absolute reliability.

This delicate trade-off demonstrates exactly why static golden paths are insufficient for Day 2 operations. When building deployment pipelines, everyone shares the single goal of releasing software, allowing for strict and opinionated automation. However, production optimization requires a highly configurable approach, as different applications inherently require dynamic tuning strategies tailored to their specific risk profiles, tech stack and performance needs.

![Shared ownership model in cloud native optimization across teams](https://akamas.io/wp-content/uploads/2026/04/24.svg)

Shared ownership model in cloud native optimization across teams

Examining the technology stack, standard Kubernetes has solidified its position as the industry baseline, utilized by 74.5% of the organizations. When analyzing the application runtimes operating within these clusters, **the Java Virtual Machine clearly dominates, powering 53.1% of all workloads**, followed by.NET at 18.4% and Python at 10.2%.

![Cloud optimization priorities comparing cost and performance trade-offs](https://akamas.io/wp-content/uploads/2026/04/25.svg)

Cloud optimization priorities comparing cost and performance trade-offs

Understanding the runtime is absolutely crucial because infrastructure optimization is inherently a layered problem. You cannot effectively optimize a system if you treat the container as an opaque black box but the data shows that only **18% of companies attempt to tune the container and runtime in tandem**.

On Kubernetes, full-stack visibility isn’t just a best practice, it is the only way to realize ROI. Organizations often fall into one of two traps:

1. **The Bottom-Up Failure**: They optimize the node and cluster layer, but the individual workloads remain poorly rightsized. The result? You are simply running inefficient applications on cheaper hardware, but still paying for “ghost” resources that provide no value.
2. **The Top-Down Failure**: They “rightsize” pods and containers by cutting limits, but don’t touch the underlying cluster configuration. The result? The cloud bill doesn’t decrease because the nodes remain underutilized, and the cloud provider still charges for the full capacity of the instance.
![Runtime distribution showing JVM, .NET and Python usage in Kubernetes workloads](https://akamas.io/wp-content/uploads/2026/04/27.svg)

Runtime distribution showing JVM,.NET and Python usage in Kubernetes workloads

The first layer involves the container limits, which are relatively simple to adjust. However, the second layer, the **application runtime**, is where true efficiency is determined. If you allocate a large container but run a JVM workload with default settings, the runtime may be artificially constrained to use only a small fraction of the available memory, leading to severe resource waste. Conversely, misaligned settings can quickly trigger fatal out-of-memory errors if the runtime attempts to allocate memory beyond the container’s physical boundaries.

The subsequent layer involves the **Horizontal Pod Autoscaler**. While horizontal scaling is frequently viewed as a universal remedy, applying it to misconfigured pods simply multiplies underlying inefficiencies. Our analysis demonstrates that, for JVM applications, allocating more resources to a single, properly tuned pod often yields vastly superior performance and lower latency compared to scaling out horizontally with numerous smaller, inefficient pods. Finally, the **node layer** is where organizations can realize the most substantial cost savings, provided that the foundational pod and scaling layers are already optimized to maximize cluster density and enable accurate bin packing.

## Moving Beyond Manual Tuning

Faced with the complex layers of modern infrastructure, organizations are searching for a way to escape the manual burden of system tuning. However, our research reveals that most remain stuck in an operational “no-man’s-land”.

![Kubernetes usage across organizations in cloud native optimization 2026](https://akamas.io/wp-content/uploads/2026/04/26.svg)

Kubernetes usage across organizations in cloud native optimization 2026

This maturity gap is two-dimensional:

- Method: An overwhelming **72.6% of companies still manage optimization entirely manually**, relying on human engineers to adjust limits and parameters.
- Mindset: **17.6% of organizations remain trapped in a purely reactive “firefighting”** mode, only touching configurations after a performance degradation or OOM error occurs.

Even those who attempt to be proactive often remain manual, spending hundreds of engineering hours on trial-and-error tuning. This approach is fundamentally unscalable; human intuition cannot keep pace with the dynamic, fluctuating demands of a cloud-native environment. To move forward, organizations must shift from these “best-guess” manual adjustments toward a deterministic model.

Effective optimization requires a system that isn’t just suggesting changes, but one that calculates them based on precise, mathematical resource configurations. By removing the unpredictability of manual tuning, teams can establish safe, reliable scaling boundaries that protect critical production workloads while simultaneously driving down costs.

## The Path to Autonomous Optimization

To truly resolve this systemic failure, the industry must transition toward an intelligent, mathematical optimization model specifically engineered for performance.

![Cloud native optimization maturity showing manual tuning and reactive approaches](https://akamas.io/wp-content/uploads/2026/04/28.svg)

Cloud native optimization maturity showing manual tuning and reactive approaches

This modern approach begins with a foundation of universal observability. After decades of struggling with proprietary agents and custom telemetry silos, OpenTelemetry (OTel) has finally provided a standardized baseline we can build upon. By leveraging OTel’s semantic conventions, organizations can now collect unified, context-rich data across the entire stack from the application runtime down to the cloud infrastructure.

This standardized telemetry is the fuel for an advanced **optimization engine**. Instead of relying on manual metric analysis or trial-and-error configurations, organizations require an engine capable of understanding the intricate relationships between runtimes, containers, scaling policies, and underlying nodes.

This engine bridges the gap by transforming raw observability data into **actionable, intelligent insights**. By leveraging sophisticated analysis guided by specific tuning profiles where SREs define clear optimization goals and failure scenario simulations this platform capability continuously determines the most efficient configurations.

The system then seamlessly **integrates into the developer workflow** by opening a pull request directly in the **GitOps** repository. Crucially, these recommendations are backed by full **explainability**. By providing the data-driven “why” behind every change, the platform resolves the ownership dilemma, giving SREs, developers, and FinOps teams the confidence to review and merge insights right where they expect them, without disrupting their primary objectives.

**System optimization must evolve into an invisible, insight-generating platform capability**. By providing a deterministic bridge between telemetry and action, we can finally resolve the structural conflict between performance, reliability, and cost enabling all technical teams to achieve their goals simultaneously.

Making cloud native easier for the humans inside it. OSS Contributor, Co-Chair @ CNCF TAG DevEx, DevRel Engineer at Akamas, Kubernetes Blog Maintainer.

Graziano Casto — OSS Contributor & Co-Chair @ CNCF TAG DevEx

Graziano Casto

OSS Contributor & Builder (K8s), Co-Chair of the CNCF TAG Developer Experience, DevRel Engineer at Akamas, Kubernetes Blog Maintainer.

DevRel Engineer

Akamas

Making cloud native easier
for the humans inside it.

Latest Activity

Featured Articles

Upcoming Events

Making cloud native easierfor the humans inside it.

Latest Activity

Featured Articles

Upcoming Events

Making cloud native easier
for the humans inside it.