Cheat Sheet: Capstone Architecture & Best Practices

Internal Developer Platform (IDP) Core Components

Developer Portal: The UI (e.g., Backstage) for service discovery and scaffolding.
Software Templates: Golden paths for creating new services.
GitOps Engine: ArgoCD or Flux for syncing Git state to the cluster.
Infrastructure Provisioner: Terraform or Crossplane for managing cloud resources.

Serving Engine: vLLM, Ollama, or Triton.
Key Optimization: Continuous batching to maximize GPU utilization.
Autoscaling: Use KEDA to scale based on token throughput or request queue length, not just CPU/Memory.
Vector Database: Qdrant, Milvus, or pgvector for RAG architectures.

Security: RBAC configured with least privilege?
Security: Secrets encrypted in transit and at rest (SOPS/External Secrets)?
Reliability: Multi-zone node pools configured?
Reliability: GitOps automated recovery tested?
Observability: Prometheus/Grafana installed and collecting metrics?
Observability: Alerts configured for GPU starvation and high latency?
Automation: CI/CD pipelines enforcing linting, testing, and vulnerability scanning?