AI Technical Debt

AI technical debt is harder to see than ordinary code debt. It hides in prompts nobody owns, evaluations nobody runs, stale embeddings, unpinned model behavior, and workflows where every small change feels risky.

This hub connects the older technical debt writing with the newer AI-specific debt pattern: invisible system behavior that keeps returning 200 OK while quality quietly degrades.

Start Here

Common Debt Types

Prompt Debt

Prompts become legacy code when they are copied from demos, edited without review, and never tied to expected behavior.

Evaluation Debt

Teams without evals argue about quality using anecdotes. That makes every model, prompt, or retrieval change risky.

Data Pipeline Debt

Stale embeddings, missing documents, drifting labels, and weak source tracking create confident wrong answers.

Architecture Debt

Provider-specific logic and tool calls buried in application code make upgrades feel like surgery.

How to Pay It Down

Start with the highest-leverage stabilizers:

  1. Put prompts and retrieval configs in version control.
  2. Add a small evaluation set for the highest-risk workflow.
  3. Track model versions, prompt versions, and data freshness together.
  4. Add cost attribution by feature or workflow.
  5. Decouple model access from product logic before switching vendors.

Supporting Reads

References