Skip to content

Agent Infrastructure Debt

Summary

A framework mapping seven blocks of hidden infrastructure debt that organizations face when moving AI agents from local development into production systems. First articulated by Port (April 2026) as an analogy to Google's 2015 "Hidden Technical Debt in Machine Learning Systems" paper. The framework identifies integrations, context lake, agent registry, measurement, human-in-the-loop, governance, and orchestration as the areas where infrastructure debt accumulates when agents scale across an engineering organization.

Details

The Premise

Google's 2015 paper "Hidden Technical Debt in Machine Learning Systems" helped ML engineers understand why building ML models was easy but running them at scale was hard. The same pattern is emerging with AI agents: they are trivial to build locally but require substantial infrastructure to run safely at production scale. Port's April 2026 analysis maps seven infrastructure blocks where this debt accumulates.

The Seven Blocks

Block 1 — Integrations Without centralized integration management, teams wire their own agent connections independently, creating hundreds of expiring auth tokens, different data views across agents, and duplicated debugging when upstream APIs change. MCP (Model Context Protocol) provides a connection mechanism but is not an integration management solution. The debt here is in the handoffs between agents and external systems — not in the agents themselves.

Block 2 — Context Lake Agents need runtime context: live, accurate service ownership and dependency data, plus decision traces (history of what was done and why). Static markdown files decay as a context source because they cannot keep pace with constant organizational churn. The debt accumulates when agents make decisions based on stale or incorrect organizational knowledge.

Block 3 — Agent Registry The org chart is expanding to 5-10x people in agents. Without visibility, teams build duplicate agents with overlapping responsibilities, conflicting behaviors, and invisible dependencies. Agent creation needs standardized templates with owner, lifecycle state, tools, and service connections. Platform teams must enable agent creation from workstations while enforcing standards.

Block 4 — Measurement Four distinct measurement needs that most setups ignore: - Observability: what did the agent do? - Evals: is it getting better/worse after changes? - Business ROI: cost vs. impact? - Feedback loops: human acceptance/correction of agent output?

Block 5 — Human-in-the-Loop Approval checkpoints must be centrally configurable, not hard-coded into individual agents. Orchestration of approvals across different channels (Slack, email, custom UI) itself becomes major tech debt. Engineers need a control plane to see what agents are doing in real time and be able to intervene.

Block 6 — Governance Agent access permissions are often unrestricted (inherited from the creator's credentials). Governance requires: specific rules defined centrally, one-click tool disabling across all agents, audit trails of agent actions, and cost governance per agent/team/use-case. Hidden debt includes agents accessing data they shouldn't, publishing sensitive information to shared channels, and no audit trail of agent decisions.

Block 7 — Orchestration Agent workflows mix non-deterministic agents with deterministic tools. The debt is in the handoffs between nodes (routing, failure handling, ownership), not individual steps. No one owns workflows that span teams, and model/prompt changes in one agent silently break downstream agents. Traditional workflow orchestration is deterministic (Step A → known output → Step B); agent workflows introduce non-determinism into chains that previously had none.

When the Debt Hits

Port identifies three phases where this debt becomes painful:

  1. Exploration stage (1 agent, 1 engineer): No debt visible. Works fine.
  2. Multi-team stage (multiple teams running agents independently): Debt piles up rapidly. Agent registry, measurement, and human-in-the-loop all surface at once. Approximately 50% of a team's capacity goes to building infrastructure around agents.
  3. Production scale (agents embedded across most of an engineering org): Governance and orchestration become the priority. Some companies see it coming; others learn the hard way.

Platform Engineering's Evolving Role

Platform engineering used to be a velocity initiative: self-service, reducing ticket load, scaffolding new services. With agents, platform teams are playing catch-up. Engineers create agents in Cursor or Claude Code whenever needed. The platform team's first job is identifying existing agents and getting them under control — only then can they do what they've always done: make it faster, safer, and easier for everyone else to create and use them.

What to Do About It

  • Start with visibility: audit GitHub org for AI-related workflows/actions, check active API tokens on Claude/OpenAI/Bedrock, review workflow tools for AI nodes.
  • Define what counts as an agent (GitHub Actions automation? Claude Code task? n8n workflow with AI node?).
  • Decide centralized vs. democratized: platform team builds everything, or provides guardrails while teams build their own?
  • The choice is whether to build infrastructure before the pain or after. You'll build it either way.

Key Claims & Data Points

  • Organizations deploying agents at scale will spend approximately 50% of team capacity on infrastructure rather than features — [source: thenewstack-hidden-agentic-technical-debt-2026]
  • Agent registry is needed because the org chart is expanding to 5-10x people in agents — [source: thenewstack-hidden-agentic-technical-debt-2026]
  • Agents running locally inherit their creator's unrestricted credentials, creating silent governance debt — [source: thenewstack-hidden-agentic-technical-debt-2026]
  • Non-determinism in agent workflows makes traditional testing approaches (test every path) impossible — [source: thenewstack-hidden-agentic-technical-debt-2026]
  • Platform teams' first job with agents is identifying existing agents, not building new ones — [source: thenewstack-hidden-agentic-technical-debt-2026]

Open Questions

  • Does Port's product specifically address all seven infrastructure blocks, or only a subset?
  • At what agent count does an organization's infrastructure spending shift from 5% to 50% of capacity? The 50% figure applies at the multi-team stage — but there is no data on the intermediate curve.
  • How do organizations track tokens/costs per agent in practice? The article acknowledges the need but doesn't provide solutions.
  • When does agent sprawl become an operational crisis vs. manageable? No specific thresholds are given beyond "about 50% of team capacity goes to infrastructure."
  • What does a working definition of "agent" look like for organizations trying to inventory their agents? The gap between "agent" and "automation workflow" is acknowledged but not resolved.

Sources

  • The hidden technical debt of agentic engineering — Zohar Einy, The New Stack, April 2, 2026. Sponsored by Port. Maps seven infrastructure blocks (integrations, context lake, registry, measurement, human-in-the-loop, governance, orchestration) for agent infrastructure scaling.