The mega-capstone. One developer action in Backstage triggers the entire platform - Terraform provisions infrastructure, ArgoCD deploys the application, Prometheus monitors it, Kyverno validates policies, Kubecost tracks spend. Everything from Capstones 1-5 working together as one complete Platform Engineering system.
Five capstones. Here is what you built in each one: * Capstone 1 — a cloud-native application: Node.js backend, React frontend, PostgreSQL, Redis, running in Kubernetes with health probes, HPA, and PDB * Capstone 2 — production AWS infrastructure: EKS cluster, VPC, RDS, ElastiCache, all provisioned with Terraform, monitored with Prometheus and Grafana, cost-tracked with Kubecost * Capstone 3 — a GitOps delivery platform: ArgoCD with App of Apps, Kustomize overlays, staging-to-production promotion via pull request, canary deployments with Argo Rollouts * Capstone 4 — a developer portal: Backstage software catalog with GitHub auto-discovery, Kubernetes integration, and a Golden Path Template that creates a new service in 10 minutes * Capstone 5 — production operations: six real failure scenarios injected and fixed, SLOs defined, burn rate alerts configured, master debugging playbook written Every one of those capstones built one layer of the platform in isolation. The application did not know about the delivery platform. The delivery platform did not know about the developer portal. The developer portal did not know about the cost tracking. Each piece worked on its own. This is where most companies stop. They have all the tools — Terraform, ArgoCD, Prometheus, Kyverno, Backstage, Kubecost — but the tools are islands. A developer still needs to open five different systems, run commands in three terminals, and file two tickets to create a new service. The tools exist. The platform does not. **Wiring the tools together is the actual Platform Engineering work.** This capstone does exactly that. In this capstone a developer opens Backstage, fills in one form, and clicks Create. By the time they make a cup of chai and come back, their service is provisioned, deployed, monitored, policy-validated, and cost-tracked. They did not write a single manifest. They did not run a single kubectl command. They did not open the AWS console. The platform did everything. Here is what happens under the hood: ``` Developer opens Backstage │ ▼ Fills Golden Path Template: Service name: order-analytics Team: data-team Environment: staging Database: enabled │ ▼ Backstage Step 1: fetch:template Creates GitHub repository with: - src/index.js (Express + Prometheus metrics) - Dockerfile (multi-stage, non-root) - k8s/deployment.yaml - k8s/service.yaml - k8s/hpa.yaml - k8s/pdb.yaml - k8s/networkpolicies.yaml - k8s/servicemonitor.yaml - k8s/database-claim.yaml (Crossplane) - argocd/application.yaml - catalog-info.yaml - docs/index.md - .github/workflows/ci.yaml │ ▼ Backstage Step 2: publish:github Creates private GitHub repository │ ▼ Backstage Step 3: github:actions:dispatch Triggers Terraform provisioning workflow: - Creates Kubernetes namespace - Creates ResourceQuota and LimitRange - Creates ECR repository - Creates IRSA role for ECR access Waits for workflow completion (timeout: 10m) │ ▼ Backstage Step 4: fetch:plain Reads Terraform outputs: - ECR repository URL - IRSA role ARN │ ▼ Backstage Steps 5-6: github:file:push Updates k8s/deployment.yaml with real ECR URL Pushes argocd/application.yaml to GitOps repo │ ▼ Backstage Step 7: catalog:register Registers service in Backstage catalog │ ▼ ArgoCD detects new Application manifest Deploys all k8s/ resources to the cluster │ ▼ Crossplane creates RDS database from claim │ ▼ Prometheus auto-discovers metrics endpoint │ ▼ Kyverno validates every resource on admission │ ▼ Kubecost begins tracking namespace cost │ ▼ Developer returns to Backstage: - Service visible in catalog - Kubernetes tab: live pod status - Docs tab: TechDocs rendered - Cost visible in 15 minutes ``` **Time to complete:** 5-6 hours. **Prerequisites:** Capstones 1-5 completed, or their outputs available — EKS cluster running, ArgoCD installed, Backstage running, Prometheus and Kubecost deployed, Kyverno installed. ---
### Why these tools integrate this way Before writing any code, understand the integration decisions. Each tool has a specific role. The connections between them are deliberate. **Backstage is the orchestrator, not the executor.** Backstage does not run Terraform. It does not deploy to Kubernetes directly. It creates repositories, dispatches workflows, and registers catalog entries. Every action with a side effect in an external system is delegated to the system that owns that side effect. This is the single responsibility principle applied to platform tooling. **GitHub Actions runs Terraform, not Backstage.** Backstage templates can run shell commands, but that would mean embedding AWS credentials in Backstage and running infrastructure provisioning inside the portal process. GitHub Actions has better secrets management, better audit logging, native Terraform state locking, and runs in isolation. When provisioning fails, the GitHub Actions log shows exactly which Terraform resource failed and why. If you ran Terraform inside Backstage, the error would be buried in a Backstage task log with no context. **ArgoCD deploys the application, not GitHub Actions.** The CI workflow builds the image and updates the image tag. That is where CI's responsibility ends. ArgoCD watches the Git repository and deploys what it finds. This separation means deployments are always from Git — never directly from a CI pipeline. Every deployment is auditable, reversible, and drift-detected. **Crossplane provisions databases, not Terraform.** Long-lived infrastructure like the EKS cluster, VPC, and RDS master instance is managed by Terraform because it changes rarely and needs careful state management. Per-service databases are different — they are created and destroyed frequently, scoped to a service lifecycle, and should be declared alongside the service's Kubernetes manifests. Crossplane manages AWS resources as Kubernetes custom resources, which means ArgoCD can manage them the same way it manages Deployments and Services. **Kyverno validates at admission, not in CI.** Running policy checks in CI is good but not sufficient — someone with kubectl access can deploy a non-compliant manifest directly, bypassing CI entirely. Kyverno runs as a Kubernetes admission webhook, which means every resource — whether deployed by ArgoCD, CI, or kubectl — goes through the same policy enforcement. There is no bypass. ### The integration map ``` ┌─────────────────────────────────────────────┐ │ Backstage Portal │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │ Catalog │ │ Template │ │ TechDocs │ │ │ └────┬─────┘ └────┬─────┘ └──────────┘ │ └───────┼─────────────┼───────────────────────┘ │ │ │ reads │ triggers ▼ ▼ ┌──────────────┐ ┌──────────────────────────┐ │ GitHub │ │ GitHub Actions │ │ Repository │ │ Terraform Provisioning │ └──────┬───────┘ └────────────┬─────────────┘ │ │ │ ArgoCD watches │ creates ▼ ▼ ┌──────────────┐ ┌────────────────────────────┐ │ ArgoCD │ │ AWS (EKS namespace, │ │ GitOps sync │ │ ECR repo, IRSA role) │ └──────┬───────┘ └────────────────────────────┘ │ │ deploys to ▼ ┌───────────────────────────────────────────┐ │ Kubernetes Cluster │ │ ┌──────────┐ ┌──────────┐ │ │ │ Kyverno │ │Crossplane│ │ │ │ policies │ │ claims │ │ │ └──────────┘ └────┬─────┘ │ │ │ creates │ │ ┌──────────┐ ▼ │ │ │Prometheus│ ┌──────────┐ │ │ │ scrapes │ │ AWS RDS │ │ │ └──────────┘ └──────────┘ │ │ ┌──────────┐ │ │ │ Kubecost │ │ │ │ tracks $ │ │ │ └──────────┘ │ └───────────────────────────────────────────┘ ``` ---
### 2.1 GitHub Actions Workflow for Terraform Provisioning This workflow is the bridge between Backstage and AWS. When Backstage dispatches it, it provisions everything the new service needs in AWS and Kubernetes before ArgoCD deploys the application. The workflow uses `workflow_dispatch` with inputs rather than a push trigger because it is called programmatically by Backstage — not by a code change. This distinction matters: `workflow_dispatch` workflows are designed to be triggered externally, support structured inputs with types and defaults, and are visible in the GitHub Actions UI as a separate trigger category. When you look at the workflow history, you can immediately see which runs were triggered by Backstage versus by code pushes. ```yaml ## .github/workflows/provision-namespace.yml ## Lives in the platform infrastructure repository from Capstone 2 ## Triggered by Backstage when a developer creates a new service name: Provision Service Namespace on: workflow_dispatch: inputs: service_name: description: 'Service name (lowercase, hyphen-separated)' required: true type: string team: description: 'Owning team name' required: true type: string namespace: description: 'Kubernetes namespace to create' required: true type: string environment: description: 'Target environment' required: true type: choice options: - staging - production ## Prevent concurrent runs for the same service ## Two simultaneous provisions would cause Terraform state conflicts concurrency: group: provision-${{ github.event.inputs.service_name }} cancel-in-progress: false jobs: provision: name: Provision ${{ github.event.inputs.service_name }} runs-on: ubuntu-latest ## Write permissions needed to set repository variables (ECR URL output) permissions: contents: read id-token: write ## needed for OIDC auth to AWS — no static credentials ## Expose outputs so Backstage can read the Terraform results outputs: ecr_repository_url: ${{ steps.terraform-output.outputs.ecr_repository_url }} irsa_role_arn: ${{ steps.terraform-output.outputs.irsa_role_arn }} namespace_name: ${{ steps.terraform-output.outputs.namespace_name }} steps: - name: Checkout platform infrastructure repository uses: actions/checkout@v4 - name: Configure AWS credentials via OIDC uses: aws-actions/configure-aws-credentials@v4 with: role-to-assume: ${{ secrets.TERRAFORM_ROLE_ARN }} aws-region: ap-south-1 ## OIDC means no static AWS keys stored in GitHub secrets ## The role is assumed temporarily for this workflow run only - name: Setup Terraform uses: hashicorp/setup-terraform@v3 with: terraform_version: '1.7.0' terraform_wrapper: false ## wrapper adds noise to outputs — disable it - name: Terraform init working-directory: modules/service-namespace run: | terraform init \ -backend-config="bucket=${{ secrets.TF_STATE_BUCKET }}" \ -backend-config="key=services/${{ github.event.inputs.service_name }}/terraform.tfstate" \ -backend-config="region=ap-south-1" \ -backend-config="dynamodb_table=${{ secrets.TF_LOCK_TABLE }}" ## Each service gets its own state file — no risk of one service's ## Terraform run affecting another service's state - name: Terraform plan working-directory: modules/service-namespace run: | terraform plan \ -var="service_name=${{ github.event.inputs.service_name }}" \ -var="team=${{ github.event.inputs.team }}" \ -var="namespace=${{ github.event.inputs.namespace }}" \ -var="environment=${{ github.event.inputs.environment }}" \ -var="cluster_name=${{ secrets.EKS_CLUSTER_NAME }}" \ -out=tfplan ## Plan first, apply second — never apply without a plan in CI - name: Terraform apply working-directory: modules/service-namespace run: terraform apply -auto-approve tfplan - name: Read Terraform outputs id: terraform-output working-directory: modules/service-namespace run: | ## Read outputs and expose them as step outputs ## Backstage reads these via the GitHub API after workflow completes ECR_URL=$(terraform output -raw ecr_repository_url) IRSA_ARN=$(terraform output -raw irsa_role_arn) NS_NAME=$(terraform output -raw namespace_name) echo "ecr_repository_url=${ECR_URL}" >> $GITHUB_OUTPUT echo "irsa_role_arn=${IRSA_ARN}" >> $GITHUB_OUTPUT echo "namespace_name=${NS_NAME}" >> $GITHUB_OUTPUT echo "✅ Provisioning complete" echo "ECR: ${ECR_URL}" echo "IRSA: ${IRSA_ARN}" echo "Namespace: ${NS_NAME}" - name: Notify on failure if: failure() uses: slackapi/slack-github-action@v1.26.0 with: payload: | { "text": "❌ Namespace provisioning failed for ${{ github.event.inputs.service_name }}", "blocks": [ { "type": "section", "text": { "type": "mrkdwn", "text": "*Provisioning failed*\nService: `${{ github.event.inputs.service_name }}`\nTeam: ${{ github.event.inputs.team }}\n<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View logs>" } } ] } env: SLACK_WEBHOOK_URL: ${{ secrets.SLACK_PLATFORM_WEBHOOK }} ``` ### 2.2 Terraform Service Namespace Module This module creates exactly what a new service needs in AWS and Kubernetes. It is intentionally separate from the main EKS Terraform module from Capstone 2. The reason for separation is state isolation. The EKS cluster is created once and lives for months or years. Service namespaces are created every time a team creates a new service — potentially dozens per month. If both were in the same Terraform state file, a `terraform apply` for a new namespace would lock the state for the entire cluster configuration. Worse, a Terraform state corruption for one service's namespace could affect the entire cluster state. Separate modules means separate state files means zero risk of cross-contamination. ```hcl ## modules/service-namespace/variables.tf variable "service_name" { description = "Name of the service being provisioned" type = string validation { ## Enforce the same naming convention as the Backstage template form condition = can(regex("^[a-z][a-z0-9-]*[a-z0-9]$", var.service_name)) error_message = "service_name must be lowercase, start with a letter, and contain only letters, numbers, and hyphens." } } variable "team" { description = "Team that owns this service" type = string } variable "namespace" { description = "Kubernetes namespace to create" type = string } variable "environment" { description = "Target environment" type = string validation { condition = contains(["staging", "production"], var.environment) error_message = "environment must be staging or production." } } variable "cluster_name" { description = "Name of the EKS cluster from Capstone 2" type = string } ``` ```hcl ## modules/service-namespace/main.tf terraform { required_providers { aws = { source = "hashicorp/aws" version = "~> 5.0" } kubernetes = { source = "hashicorp/kubernetes" version = "~> 2.0" } } } ## Read the EKS cluster data from Capstone 2 ## This references the existing cluster without managing it data "aws_eks_cluster" "platform" { name = var.cluster_name } data "aws_eks_cluster_auth" "platform" { name = var.cluster_name } data "aws_caller_identity" "current" {} ## ── Kubernetes Resources ────────────────────────────────────────── resource "kubernetes_namespace" "service" { metadata { name = var.namespace labels = { ## Labels used by Kyverno policies for namespace-scoped rules "app.kubernetes.io/managed-by" = "terraform" "platform.devops-network.io/team" = var.team "platform.devops-network.io/environment" = var.environment "platform.devops-network.io/service" = var.service_name } annotations = { ## Tells Kubecost which team to attribute costs to "kubecost.com/team" = var.team } } } resource "kubernetes_resource_quota" "service" { ## ResourceQuota caps total resource consumption in this namespace ## Prevents one service from consuming the entire cluster metadata { name = "service-quota" namespace = kubernetes_namespace.service.metadata[0].name } spec { hard = { "pods" = "20" "requests.cpu" = "4" "requests.memory" = "8Gi" "limits.cpu" = "8" "limits.memory" = "16Gi" } } } resource "kubernetes_limit_range" "service" { ## LimitRange sets default requests on pods that do not specify them ## Ensures every pod has resource requests for Kubecost attribution metadata { name = "service-defaults" namespace = kubernetes_namespace.service.metadata[0].name } spec { limit { type = "Container" default_request = { cpu = "100m" memory = "128Mi" } default = { cpu = "500m" memory = "512Mi" } min = { cpu = "10m" memory = "32Mi" } } } } ## ── AWS Resources ───────────────────────────────────────────────── resource "aws_ecr_repository" "service" { name = var.service_name image_tag_mutability = "MUTABLE" ## allows overwriting tags like 'latest' image_scanning_configuration { scan_on_push = true ## automatically scan every pushed image for CVEs } encryption_configuration { encryption_type = "AES256" } tags = { Service = var.service_name Team = var.team Environment = var.environment ManagedBy = "terraform" } } resource "aws_ecr_lifecycle_policy" "service" { ## Keep the last 10 tagged images and remove untagged images after 1 day ## Prevents ECR storage costs from growing unbounded repository = aws_ecr_repository.service.name policy = jsonencode({ rules = [ { rulePriority = 1 description = "Remove untagged images after 1 day" selection = { tagStatus = "untagged" countType = "sinceImagePushed" countUnit = "days" countNumber = 1 } action = { type = "expire" } }, { rulePriority = 2 description = "Keep last 10 tagged images" selection = { tagStatus = "tagged" tagPrefixList = ["v"] countType = "imageCountMoreThan" countNumber = 10 } action = { type = "expire" } } ] }) } data "aws_iam_policy_document" "irsa_trust" { ## IRSA (IAM Roles for Service Accounts) trust policy ## Allows the Kubernetes service account to assume this IAM role ## Scoped to the exact namespace and service account — least privilege statement { effect = "Allow" actions = ["sts:AssumeRoleWithWebIdentity"] principals { type = "Federated" identifiers = [ "arn:aws:iam::${data.aws_caller_identity.current.account_id}:oidc-provider/${replace(data.aws_eks_cluster.platform.identity[0].oidc[0].issuer, "https://", "")}" ] } condition { test = "StringEquals" variable = "${replace(data.aws_eks_cluster.platform.identity[0].oidc[0].issuer, "https://", "")}:sub" values = ["system:serviceaccount:${var.namespace}:${var.service_name}"] } } } resource "aws_iam_role" "service_irsa" { name = "${var.service_name}-irsa-${var.environment}" assume_role_policy = data.aws_iam_policy_document.irsa_trust.json tags = { Service = var.service_name Team = var.team Environment = var.environment } } resource "aws_iam_role_policy" "ecr_access" { ## The service needs to pull its own image from ECR ## Scoped to this service's ECR repository only name = "${var.service_name}-ecr-access" role = aws_iam_role.service_irsa.id policy = jsonencode({ Version = "2012-10-17" Statement = [ { Effect = "Allow" Action = [ "ecr:GetDownloadUrlForLayer", "ecr:BatchGetImage", "ecr:BatchCheckLayerAvailability", "ecr:GetAuthorizationToken" ] Resource = aws_ecr_repository.service.arn } ] }) } ``` ```hcl ## modules/service-namespace/outputs.tf output "ecr_repository_url" { description = "ECR repository URL for the service — used in k8s/deployment.yaml" value = aws_ecr_repository.service.repository_url } output "irsa_role_arn" { description = "IRSA role ARN — annotated on the Kubernetes ServiceAccount" value = aws_iam_role.service_irsa.arn } output "namespace_name" { description = "The created Kubernetes namespace name" value = kubernetes_namespace.service.metadata[0].name } ``` ### 2.3 Update Backstage to Dispatch GitHub Actions Backstage has a built-in `github:actions:dispatch` action that triggers a `workflow_dispatch` event and optionally waits for the workflow to complete. Add this to the Backstage backend configuration to allow the template to use it. ```typescript // packages/backend/src/plugins/scaffolder.ts // Register the GitHub Actions dispatch action import { createBuiltinActions } from '@backstage/plugin-scaffolder-backend'; import { ScmIntegrations } from '@backstage/integration'; export default async function createPlugin(env) { const integrations = ScmIntegrations.fromConfig(env.config); // createBuiltinActions includes github:actions:dispatch // It uses the GitHub integration token from app-config.yaml const builtinActions = createBuiltinActions({ integrations, catalogClient: env.catalogClient, config: env.config, reader: env.reader, }); return await createRouter({ logger: env.logger, config: env.config, database: env.database, reader: env.reader, catalogClient: env.catalogClient, integrations, actions: [...builtinActions], }); } ``` > ⚠️ **Security:** The GitHub Personal Access Token used by Backstage needs the `workflow` scope to dispatch GitHub Actions workflows. Audit your token scopes — a token with `workflow` scope can trigger any workflow in any repository the token has access to. Use a dedicated machine account or GitHub App with minimal repository permissions. ---
### Why this template exists Every section of this template answers a real developer problem. The form fields map to real infrastructure decisions. The steps map to real systems. Nothing in this template is ceremonial. A developer at Hotstar who creates a new feature service should not need to know: * What ECR repository naming conventions are * How to configure IRSA for ECR access * What ResourceQuotas their namespace should have * How to write a Crossplane database claim * How to structure an ArgoCD Application manifest They should know: what the service is called, which team owns it, what environment it targets, and whether it needs a database. Everything else is the platform's job. ### The complete template ```yaml ## templates/platform-engineering-challenge/template.yaml ## The mega-template that wires everything together apiVersion: scaffolder.backstage.io/v1beta3 kind: Template metadata: name: platform-engineering-challenge title: Platform Engineering — Production Service description: Creates a complete production-ready service with infrastructure provisioning, GitOps delivery, monitoring, policy validation, and cost tracking tags: - nodejs - kubernetes - gitops - terraform - recommended annotations: ## Link to the TechDocs for this template backstage.io/techdocs-ref: dir:. spec: owner: group:platform-team type: service ## ── Parameters (form pages) ────────────────────────────────────── parameters: ## Page 1: Service identity - title: Service Details required: - name - description - owner properties: name: title: Service Name type: string description: Lowercase, hyphen-separated (e.g. order-analytics) pattern: '^[a-z][a-z0-9-]*[a-z0-9]$' ui:autofocus: true description: title: Description type: string description: What does this service do? (shown in the catalog and repository) ui:widget: textarea ui:options: rows: 3 owner: title: Owner Team type: string ui:field: OwnerPicker ui:options: allowedKinds: - Group ## Page 2: Infrastructure - title: Infrastructure required: - namespace - environment - instanceSize properties: namespace: title: Kubernetes Namespace type: string description: Namespace for this service (e.g. order-analytics-staging) pattern: '^[a-z][a-z0-9-]*[a-z0-9]$' environment: title: Environment type: string enum: - staging - production default: staging instanceSize: title: Service Size type: string description: Controls resource requests and limits enum: - small - medium - large enumNames: - 'Small (100m CPU, 128Mi memory)' - 'Medium (250m CPU, 256Mi memory)' - 'Large (500m CPU, 512Mi memory)' default: small ## Page 3: Database (optional) - title: Database properties: enableDatabase: title: Enable PostgreSQL Database type: boolean default: false description: Provisions an RDS PostgreSQL database via Crossplane storageSize: title: Storage Size type: string enum: - 20Gi - 50Gi - 100Gi default: 20Gi ## Only shown when enableDatabase is true ui:widget: select databaseTier: title: Database Tier type: string enum: - dev - staging - production default: staging ## dev = db.t3.micro, staging = db.t3.small, production = db.t3.medium ## Page 4: Review (read-only summary — shown before creation) - title: Review properties: reviewNote: title: ' ' type: string ui:widget: markdown default: | Review your selections before creating. Once created, the service will be provisioned automatically. Infrastructure creation takes approximately 3-5 minutes. ## ── Steps ──────────────────────────────────────────────────────── steps: ## Step 1: Render the skeleton files with template variables ## This creates a local working copy with all placeholders replaced - id: fetch-template name: Render Service Files action: fetch:template input: url: ./skeleton values: name: ${{ parameters.name }} description: ${{ parameters.description }} owner: ${{ parameters.owner }} namespace: ${{ parameters.namespace }} environment: ${{ parameters.environment }} instanceSize: ${{ parameters.instanceSize }} enableDatabase: ${{ parameters.enableDatabase }} storageSize: ${{ parameters.storageSize }} databaseTier: ${{ parameters.databaseTier }} ## These placeholders are filled in Step 5 after Terraform runs ecrRepositoryUrl: 'PLACEHOLDER_REPLACED_BY_TERRAFORM' irsaRoleArn: 'PLACEHOLDER_REPLACED_BY_TERRAFORM' ## Step 2: Create the GitHub repository ## Branch protection and topics are set here - id: publish-github name: Create GitHub Repository action: publish:github input: allowedHosts: ['github.com'] description: ${{ parameters.description }} repoUrl: github.com?owner=your-org&repo=${{ parameters.name }} repoVisibility: private defaultBranch: main requireCodeOwnerReviews: true ## requires review from catalog owner bypassPullRequestAllowances: apps: ['argocd'] ## ArgoCD can push image tag updates without review topics: - nodejs - kubernetes - ${{ parameters.environment }} - team-${{ parameters.owner | replace("group:", "") }} ## Step 3: Trigger Terraform provisioning ## Dispatches the GitHub Actions workflow from Part 2 ## Waits for completion before proceeding — if it fails, the template fails - id: provision-infrastructure name: Provision Infrastructure action: github:actions:dispatch input: repoUrl: github.com?owner=your-org&repo=platform-infrastructure workflowId: provision-namespace.yml branchOrTagName: main workflowInputs: service_name: ${{ parameters.name }} team: ${{ parameters.owner | replace("group:", "") }} namespace: ${{ parameters.namespace }} environment: ${{ parameters.environment }} ## Step 4: Fetch Terraform outputs from the completed workflow ## The workflow wrote ECR URL and IRSA ARN to its outputs ## This step reads them so Step 5 can use them - id: fetch-terraform-outputs name: Read Infrastructure Outputs action: github:actions:workflow:output input: repoUrl: github.com?owner=your-org&repo=platform-infrastructure runId: ${{ steps['provision-infrastructure'].output.runId }} outputNames: - ecr_repository_url - irsa_role_arn ## Step 5: Update the deployment manifest with the real ECR URL ## Step 1 put a placeholder — now we replace it with the actual URL - id: update-deployment-image name: Update Deployment with ECR URL action: github:file:update input: repoUrl: github.com?owner=your-org&repo=${{ parameters.name }} branchName: main path: k8s/deployment.yaml ## Replace the placeholder ECR URL with the real one from Terraform token: ${{ secrets.GITHUB_TOKEN }} commitMessage: 'chore: set ECR repository URL from Terraform provisioning' content: | ${{ steps['fetch-template'].output.fileContents['k8s/deployment.yaml'] | replace('PLACEHOLDER_REPLACED_BY_TERRAFORM', steps['fetch-terraform-outputs'].output.ecr_repository_url) }} ## Step 6: Push the ArgoCD Application manifest to the GitOps repository ## This is what triggers ArgoCD to start watching the new service ## The GitOps repository is separate from the service repository - id: push-argocd-application name: Register with ArgoCD action: github:file:create input: repoUrl: github.com?owner=your-org&repo=gitops-platform branchName: main path: apps/${{ parameters.environment }}/${{ parameters.name }}-app.yaml commitMessage: 'feat: add ArgoCD Application for ${{ parameters.name }}' content: | apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: ${{ parameters.name }}-${{ parameters.environment }} namespace: argocd finalizers: - resources-finalizer.argocd.argoproj.io spec: project: default source: repoURL: https://github.com/your-org/${{ parameters.name }} targetRevision: HEAD path: k8s/ destination: server: https://kubernetes.default.svc namespace: ${{ parameters.namespace }} syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=false ## Step 7: Register the service in the Backstage catalog ## The developer can now find it at localhost:3000/catalog - id: register-catalog name: Register in Catalog action: catalog:register input: repoContentsUrl: ${{ steps['publish-github'].output.repoContentsUrl }} catalogInfoPath: '/catalog-info.yaml' ## ── Output links shown after creation ──────────────────────────── output: links: - title: GitHub Repository url: ${{ steps['publish-github'].output.remoteUrl }} icon: github - title: Open in Catalog icon: catalog entityRef: ${{ steps['register-catalog'].output.entityRef }} - title: ArgoCD Application url: https://argocd.your-domain.com/applications/${{ parameters.name }}-${{ parameters.environment }} icon: dashboard - title: Grafana Dashboard url: https://grafana.your-domain.com/d/k8s-namespace?var-namespace=${{ parameters.namespace }} icon: dashboard - title: Kubecost Namespace Cost url: https://kubecost.your-domain.com/detail?namespace=${{ parameters.namespace }} icon: monetization_on ``` ### The skeleton directory — every file explained #### `catalog-info.yaml` The catalog entry that registers this service in Backstage. The Kubernetes annotation links it to live pod data. The TechDocs annotation links it to the docs folder. ```yaml ## templates/platform-engineering-challenge/skeleton/catalog-info.yaml apiVersion: backstage.io/v1alpha1 kind: Component metadata: name: ${{ values.name }} description: ${{ values.description }} annotations: github.com/project-slug: your-org/${{ values.name }} backstage.io/kubernetes-label-selector: 'app=${{ values.name }}' backstage.io/techdocs-ref: dir:. ## Kubecost annotation — links catalog page to cost data kubecost.com/namespace: ${{ values.namespace }} tags: - nodejs - ${{ values.environment }} spec: type: service lifecycle: experimental owner: ${{ values.owner }} dependsOn: ${{ if values.enableDatabase }} - resource:default/${{ values.name }}-database ${{ endif }} ``` #### `src/index.js` A minimal Express server that is immediately functional after the template runs. It includes the three endpoints every service on this platform needs: liveness probe, readiness probe, and Prometheus metrics. ```javascript // templates/platform-engineering-challenge/skeleton/src/index.js // Minimal Express server — replace with your application logic const express = require('express'); const { Registry, collectDefaultMetrics, Counter, Histogram } = require('prom-client'); const app = express(); const register = new Registry(); // Collect default Node.js metrics (CPU, memory, event loop lag) // These appear automatically in Prometheus and Grafana collectDefaultMetrics({ register }); // Custom metrics for this service // Prometheus uses these to power SLO alerting from Capstone 5 const httpRequestsTotal = new Counter({ name: 'http_requests_total', help: 'Total number of HTTP requests', labelNames: ['method', 'route', 'status'], registers: [register], }); const httpRequestDuration = new Histogram({ name: 'http_request_duration_seconds', help: 'HTTP request duration in seconds', labelNames: ['method', 'route', 'status'], buckets: [0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5], registers: [register], }); // Middleware to record metrics for every request app.use((req, res, next) => { const end = httpRequestDuration.startTimer(); res.on('finish', () => { httpRequestsTotal.inc({ method: req.method, route: req.route?.path || req.path, status: res.statusCode, }); end({ method: req.method, route: req.route?.path || req.path, status: res.statusCode }); }); next(); }); // Liveness probe — returns 200 as long as the process is running // Kubernetes kills and restarts the pod if this fails 3 times app.get('/health/live', (req, res) => { res.json({ status: 'ok', service: '${{ values.name }}' }); }); // Readiness probe — returns 200 when the service can handle traffic // Add database connection check here when you add a database app.get('/health/ready', (req, res) => { res.json({ status: 'ok', service: '${{ values.name }}' }); }); // Prometheus metrics endpoint — scraped by Prometheus every 15 seconds app.get('/metrics', async (req, res) => { res.set('Content-Type', register.contentType); res.end(await register.metrics()); }); // Your application routes go here app.get('/', (req, res) => { res.json({ service: '${{ values.name }}', version: process.env.SERVICE_VERSION || 'dev' }); }); const PORT = process.env.PORT || 4000; app.listen(PORT, () => { console.log(`${{ values.name }} listening on port ${PORT}`); }); ``` #### `Dockerfile` Multi-stage build, non-root user, matches the security context in the Kubernetes manifest exactly. ```dockerfile ## templates/platform-engineering-challenge/skeleton/Dockerfile FROM node:20-alpine AS builder WORKDIR /app COPY package*.json ./ ## npm ci installs exact versions from package-lock.json — reproducible builds RUN npm ci --only=production FROM node:20-alpine AS production ## Create the user and group that match runAsUser: 1001 in deployment.yaml RUN addgroup -g 1001 -S appgroup && \ adduser -u 1001 -S appuser -G appgroup WORKDIR /app ## Copy only the production dependencies from the builder stage COPY --from=builder /app/node_modules ./node_modules COPY --chown=appuser:appgroup . . ## Switch to non-root user before the final CMD USER appuser EXPOSE 4000 ## HEALTHCHECK allows Docker to report container health (not just running) HEALTHCHECK --interval=10s --timeout=3s --start-period=15s --retries=3 \ CMD wget -qO- http://localhost:4000/health/live || exit 1 CMD ["node", "src/index.js"] ``` #### `k8s/deployment.yaml` The ECR URL is a placeholder that Step 5 of the template replaces with the real URL from Terraform outputs. Every security and reliability pattern from Capstone 1 is pre-configured. ```yaml ## templates/platform-engineering-challenge/skeleton/k8s/deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: ${{ values.name }} namespace: ${{ values.namespace }} labels: app: ${{ values.name }} app.kubernetes.io/managed-by: backstage-golden-path team: ${{ values.owner }} spec: ## replicas omitted — HPA controls this selector: matchLabels: app: ${{ values.name }} strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 0 ## zero-downtime deploys — never kill old pods before new ones are ready maxSurge: 1 template: metadata: labels: app: ${{ values.name }} team: ${{ values.owner }} annotations: ## Prometheus auto-discovers this endpoint for scraping prometheus.io/scrape: "true" prometheus.io/port: "4000" prometheus.io/path: "/metrics" spec: serviceAccountName: ${{ values.name }} ## has IRSA annotation for ECR access securityContext: runAsNonRoot: true runAsUser: 1001 ## matches the user created in the Dockerfile fsGroup: 1001 containers: - name: ${{ values.name }} ## PLACEHOLDER_REPLACED_BY_TERRAFORM is substituted in template Step 5 image: PLACEHOLDER_REPLACED_BY_TERRAFORM/${{ values.name }}:latest ports: - containerPort: 4000 env: - name: PORT value: "4000" - name: SERVICE_VERSION valueFrom: fieldRef: fieldPath: metadata.labels['app.kubernetes.io/version'] ${{ if values.enableDatabase }} envFrom: - secretRef: ## Created by Crossplane when the database claim is provisioned name: ${{ values.name }}-db-credentials ${{ endif }} ## Resource sizes controlled by the instanceSize parameter from the form ${{ if values.instanceSize == 'small' }} resources: requests: cpu: "100m" memory: "128Mi" limits: cpu: "500m" memory: "512Mi" ${{ endif }} ${{ if values.instanceSize == 'medium' }} resources: requests: cpu: "250m" memory: "256Mi" limits: cpu: "1000m" memory: "1Gi" ${{ endif }} ${{ if values.instanceSize == 'large' }} resources: requests: cpu: "500m" memory: "512Mi" limits: cpu: "2000m" memory: "2Gi" ${{ endif }} livenessProbe: httpGet: path: /health/live port: 4000 initialDelaySeconds: 15 periodSeconds: 10 failureThreshold: 3 readinessProbe: httpGet: path: /health/ready port: 4000 initialDelaySeconds: 10 periodSeconds: 5 failureThreshold: 2 --- ## ServiceAccount with IRSA annotation — allows ECR image pulls without static credentials apiVersion: v1 kind: ServiceAccount metadata: name: ${{ values.name }} namespace: ${{ values.namespace }} annotations: ## IRSA_ROLE_ARN is also replaced in Step 5 eks.amazonaws.com/role-arn: PLACEHOLDER_REPLACED_BY_TERRAFORM_IRSA ``` #### `k8s/hpa.yaml` ```yaml ## templates/platform-engineering-challenge/skeleton/k8s/hpa.yaml apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: ${{ values.name }}-hpa namespace: ${{ values.namespace }} spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: ${{ values.name }} minReplicas: 2 ## always at least 2 for availability maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 80 ``` #### `k8s/pdb.yaml` ```yaml ## templates/platform-engineering-challenge/skeleton/k8s/pdb.yaml ## PodDisruptionBudget: ensures at least 1 pod is available during node drains ## Without this, a rolling node upgrade could take down all pods simultaneously apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: ${{ values.name }}-pdb namespace: ${{ values.namespace }} spec: minAvailable: 1 selector: matchLabels: app: ${{ values.name }} ``` #### `k8s/networkpolicies.yaml` Pre-configured with the correct policies from the Scenario 6 fix in Capstone 5. Every new service starts with working DNS and same-namespace communication — no manual NetworkPolicy debugging on day one. ```yaml ## templates/platform-engineering-challenge/skeleton/k8s/networkpolicies.yaml ## Three policies that form a secure baseline — learned from Capstone 5 Scenario 6 ## Policy 1: Allow DNS so service discovery works apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-dns namespace: ${{ values.namespace }} spec: podSelector: {} policyTypes: - Egress egress: - ports: - port: 53 protocol: UDP - port: 53 protocol: TCP --- ## Policy 2: Allow communication within this namespace apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-same-namespace namespace: ${{ values.namespace }} spec: podSelector: {} policyTypes: - Ingress - Egress ingress: - from: - podSelector: {} egress: - to: - podSelector: {} --- ## Policy 3: Allow ingress controller to reach this service apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-ingress-controller namespace: ${{ values.namespace }} spec: podSelector: matchLabels: app: ${{ values.name }} policyTypes: - Ingress ingress: - from: - namespaceSelector: matchLabels: kubernetes.io/metadata.name: ingress-nginx ``` #### `k8s/servicemonitor.yaml` The ServiceMonitor tells Prometheus to scrape this service's metrics endpoint. Without it, Prometheus only discovers services with the legacy annotation-based discovery. ServiceMonitor is more reliable and supports authentication. ```yaml ## templates/platform-engineering-challenge/skeleton/k8s/servicemonitor.yaml apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: ${{ values.name }} namespace: ${{ values.namespace }} labels: ## Must match the Prometheus operator's serviceMonitorSelector release: monitoring spec: selector: matchLabels: app: ${{ values.name }} endpoints: - port: http path: /metrics interval: 15s ## scrape every 15 seconds — standard for most services ``` #### `k8s/database-claim.yaml` Only included in the skeleton when `enableDatabase` is true. Crossplane reads this manifest and provisions an RDS instance. ArgoCD manages it the same way it manages a Deployment. ```yaml ## templates/platform-engineering-challenge/skeleton/k8s/database-claim.yaml ## Only rendered when enableDatabase=true in the template form ${{ if values.enableDatabase }} apiVersion: database.example.org/v1alpha1 kind: PostgreSQLInstance metadata: name: ${{ values.name }}-database namespace: ${{ values.namespace }} spec: parameters: storageGB: ${{ values.storageSize | replace("Gi", "") | int }} ## Tier maps to RDS instance class ## dev=db.t3.micro, staging=db.t3.small, production=db.t3.medium tier: ${{ values.databaseTier }} version: "15" multiAZ: ${{ values.databaseTier == 'production' }} ## Crossplane writes the connection string to this Secret ## The Deployment's envFrom reads from this Secret writeConnectionSecretToRef: name: ${{ values.name }}-db-credentials ${{ endif }} ``` #### `.github/workflows/ci.yaml` The CI pipeline that builds the Docker image and updates the image tag in the Kubernetes manifest. Following the GitOps pattern from Capstone 3 — CI builds, Git stores the desired state, ArgoCD deploys. ```yaml ## templates/platform-engineering-challenge/skeleton/.github/workflows/ci.yaml name: Build and Deploy on: push: branches: [main] pull_request: branches: [main] jobs: build: runs-on: ubuntu-latest permissions: contents: write ## needed to push the image tag update back to the repo id-token: write ## needed for OIDC auth to AWS ECR steps: - uses: actions/checkout@v4 - name: Configure AWS credentials uses: aws-actions/configure-aws-credentials@v4 with: role-to-assume: ${{ secrets.DEPLOY_ROLE_ARN }} aws-region: ap-south-1 - name: Login to ECR id: login-ecr uses: aws-actions/amazon-ecr-login@v2 - name: Build and push Docker image env: ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }} ## Use commit SHA as image tag — immutable, traceable, never ambiguous IMAGE_TAG: ${{ github.sha }} run: | docker build -t ${ECR_REGISTRY}/${{ values.name }}:${IMAGE_TAG} . docker push ${ECR_REGISTRY}/${{ values.name }}:${IMAGE_TAG} echo "IMAGE=${ECR_REGISTRY}/${{ values.name }}:${IMAGE_TAG}" >> $GITHUB_ENV - name: Update image tag in deployment manifest ## GitOps pattern: CI commits the new image tag, ArgoCD deploys it ## The commit here is what triggers the ArgoCD sync run: | sed -i "s|image: .*/${{ values.name }}:.*|image: ${IMAGE}|" \ k8s/deployment.yaml git config user.name "github-actions[bot]" git config user.email "github-actions[bot]@users.noreply.github.com" git add k8s/deployment.yaml git diff --staged --quiet || \ git commit -m "deploy: update ${{ values.name }} to ${GITHUB_SHA::8}" git push if: github.ref == 'refs/heads/main' ## only update tag on main branch ``` ---
### 4.1 Pre-flight checks Before running the template, verify every component the template depends on is working. A failed pre-flight check here saves 15 minutes of debugging a half-created service. ```bash ## ─── Backstage ─────────────────────────────────────────────────── curl -s http://localhost:7007/api/catalog/entities?kind=Template | \ jq '.[].metadata.name' | grep platform-engineering ## Should return: "platform-engineering-challenge" ## ─── GitHub Actions workflow exists ────────────────────────────── curl -s -H "Authorization: token ${GITHUB_TOKEN}" \ "https://api.github.com/repos/your-org/platform-infrastructure/actions/workflows" | \ jq '.workflows[] | select(.name == "Provision Service Namespace") | .state' ## Should return: "active" ## ─── ArgoCD is watching the GitOps repository ──────────────────── argocd app list | grep root-staging ## Should show: Synced Healthy ## ─── Kyverno policies are active ───────────────────────────────── kubectl get clusterpolicies ## Should show at least: require-resource-requests, require-non-root-user ## ─── Kubecost is running ───────────────────────────────────────── kubectl get pods -n kubecost | grep cost-analyzer ## Should show: Running ## ─── Prometheus is running ─────────────────────────────────────── kubectl get pods -n monitoring | grep prometheus ## Should show: prometheus-monitoring-kube-prometheus-prometheus-0 Running ## ─── Crossplane is installed (only if using database option) ───── kubectl get pods -n crossplane-system 2>/dev/null | grep crossplane ## Should show: Running — if not installed, disable enableDatabase in the form echo "✅ All pre-flight checks passed — ready to run template" ``` ### 4.2 Run the template 1. Open `http://localhost:3000/create` 2. Find **Platform Engineering — Production Service** (it has the "recommended" badge) 3. Click **Choose** 4. **Page 1 — Service Details:** * Service Name: `analytics-service` * Description: `Real-time analytics aggregation for order and delivery metrics` * Owner: `data-team` 5. **Page 2 — Infrastructure:** * Namespace: `analytics-service-staging` * Environment: `staging` * Instance Size: `medium` 6. **Page 3 — Database:** * Enable PostgreSQL: `true` * Storage Size: `20Gi` * Database Tier: `staging` 7. **Page 4 — Review:** verify all selections then click **Create** Watch the task log in Backstage. Each step shows its status in real time: ``` ✅ Step 1: Render Service Files ~5 seconds ✅ Step 2: Create GitHub Repository ~10 seconds ⏳ Step 3: Provision Infrastructure ~3-5 minutes (Terraform running) ✅ Step 4: Read Infrastructure Outputs ~2 seconds ✅ Step 5: Update Deployment with ECR URL ~5 seconds ✅ Step 6: Register with ArgoCD ~5 seconds ✅ Step 7: Register in Catalog ~5 seconds ``` Total time from clicking Create to all steps complete: approximately 4-5 minutes. ### 4.3 Verify every integration point ```bash ## ─── GitHub: repository created ────────────────────────────────── curl -s -H "Authorization: token ${GITHUB_TOKEN}" \ https://api.github.com/repos/your-org/analytics-service | \ jq '{name: .name, visibility: .visibility, default_branch: .default_branch}' ## Expected: {"name":"analytics-service","visibility":"private","default_branch":"main"} ## ─── GitHub: all skeleton files present ────────────────────────── curl -s -H "Authorization: token ${GITHUB_TOKEN}" \ "https://api.github.com/repos/your-org/analytics-service/git/trees/main?recursive=1" | \ jq '.tree[].path' ## Expected: catalog-info.yaml, Dockerfile, src/index.js, k8s/deployment.yaml, ## k8s/hpa.yaml, k8s/pdb.yaml, k8s/networkpolicies.yaml, ## k8s/servicemonitor.yaml, k8s/database-claim.yaml, ## argocd/application.yaml, docs/index.md, mkdocs.yml ## ─── GitHub Actions: Terraform workflow completed ──────────────── curl -s -H "Authorization: token ${GITHUB_TOKEN}" \ "https://api.github.com/repos/your-org/platform-infrastructure/actions/runs?event=workflow_dispatch&status=completed" | \ jq '.workflow_runs[0] | {name: .name, conclusion: .conclusion, created_at: .created_at}' ## Expected: {"name":"Provision Service Namespace","conclusion":"success",...} ## ─── Terraform: namespace created ──────────────────────────────── kubectl get namespace analytics-service-staging ## Expected: STATUS Active kubectl get resourcequota -n analytics-service-staging ## Expected: service-quota with hard limits set kubectl get limitrange -n analytics-service-staging ## Expected: service-defaults with container defaults set ## ─── Terraform: ECR repository created ─────────────────────────── aws ecr describe-repositories --repository-names analytics-service \ --region ap-south-1 | jq '.repositories[0].repositoryUri' ## Expected: "YOUR_ACCOUNT.dkr.ecr.ap-south-1.amazonaws.com/analytics-service" ## ─── ArgoCD: Application detected and syncing ──────────────────── argocd app get analytics-service-staging ## Expected: Status Synced, Health Healthy (or Progressing while deploying) kubectl get application analytics-service-staging -n argocd \ -o jsonpath='{.status.sync.status}{" "}{.status.health.status}{"\n"}' ## Expected: Synced Healthy ## ─── Kubernetes: pods running ──────────────────────────────────── kubectl get pods -n analytics-service-staging ## Expected: analytics-service-xxx 1/1 Running 0 kubectl get pods -n analytics-service-staging \ -o jsonpath='{.items[*].status.containerStatuses[*].ready}' ## Expected: true true (one per pod) ## ─── Kyverno: policies passed ──────────────────────────────────── kubectl get policyreport -n analytics-service-staging ## Expected: PASS status — no violations ## If there are violations, this shows which policy failed: kubectl get policyreport -n analytics-service-staging -o json | \ jq '.results[] | select(.result == "fail") | {policy: .policy, message: .message}' ## ─── Prometheus: service is being scraped ──────────────────────── ## Query Prometheus for the new service's metrics curl -s "http://localhost:9090/api/v1/query?query=up{namespace='analytics-service-staging'}" | \ jq '.data.result[] | {job: .metric.job, value: .value[1]}' ## Expected: {"job":"analytics-service","value":"1"} (1 = up) ## ─── Kubecost: tracking namespace cost ─────────────────────────── kubectl port-forward -n kubecost deployment/kubecost-cost-analyzer 9090:9090 & curl -s "http://localhost:9090/model/allocation?window=1h&namespace=analytics-service-staging" | \ jq '.data[0]["analytics-service-staging"].totalCost' ## Expected: a small number (may be 0 for the first 15 minutes) ## ─── Crossplane: database provisioning ─────────────────────────── kubectl get postgresqlinstance -n analytics-service-staging ## Expected: READY True (may take 5-10 minutes for RDS to provision) kubectl get secret analytics-service-db-credentials -n analytics-service-staging ## Expected: secret exists once Crossplane writes the connection string echo "✅ All integration points verified" ``` ### 4.4 The developer's view The developer returns to Backstage after their chai. Here is what they see: On the catalog page for `analytics-service`: * **Overview tab:** service name, description, owner (data-team), lifecycle (experimental), links to GitHub, ArgoCD, Grafana, and Kubecost * **Kubernetes tab:** 2 pods Running, HPA active with min/max replicas shown, Deployment health Healthy * **Docs tab:** TechDocs rendered from `docs/index.md` — the architecture section, API reference, and deployment instructions are all pre-populated from the skeleton * **Cost tab** (if Kubecost plugin is installed): initial cost tracking started, will show accurate numbers within 15 minutes The developer pushes their first line of application code to the repository. The CI workflow builds the image, pushes it to ECR, commits the new image tag to `k8s/deployment.yaml`, and ArgoCD deploys it. The developer never touched the AWS console, never ran kubectl, and never filed a ticket. ---
These three scenarios have no step-by-step instructions. Use everything from Capstones 1-5. ### Scenario A — Deploy a second service with a database **Goal:** Use the golden path template to deploy a service called `payment-reconciler` with a PostgreSQL database. Verify four things when complete: * The RDS database is provisioned by Crossplane and in a Ready state * The `payment-reconciler` pod is running and the readiness probe passes * The database connection string is in a Kubernetes Secret named `payment-reconciler-db-credentials` — confirm no database credentials exist anywhere in the GitHub repository or in environment variables set directly in the Deployment * Kubecost shows cost for both `analytics-service-staging` and `payment-reconciler-staging` namespaces **Success criteria:** ```bash ## Run these — all must pass kubectl get postgresqlinstance -n payment-reconciler-staging ## READY: True kubectl get pods -n payment-reconciler-staging ## 1/1 Running kubectl get secret payment-reconciler-db-credentials -n payment-reconciler-staging ## Secret exists kubectl get deployment payment-reconciler -n payment-reconciler-staging -o json | \ jq '.spec.template.spec.containers[0].env[] | select(.name == "DB_PASSWORD")' ## Must return empty — no hardcoded credentials in the Deployment ``` ### Scenario B — Break and fix a Kyverno policy violation **Goal:** Deliberately introduce a policy violation, observe it being blocked, and fix it. Modify the `analytics-service` Deployment manifest in the GitHub repository to run as root: ```yaml ## Change this in k8s/deployment.yaml: securityContext: runAsNonRoot: false runAsUser: 0 ``` Push the change to the main branch. Watch what happens in ArgoCD. The deployment should fail with a Kyverno policy violation. Read the error message in the ArgoCD UI — it tells you exactly which policy was violated. Fix the manifest. Push the fix. Watch ArgoCD succeed. **Success criteria:** ```bash ## After introducing the violation: argocd app get analytics-service-staging ## Health: Degraded — sync error visible with Kyverno policy name kubectl get policyreport -n analytics-service-staging -o json | \ jq '.results[] | select(.result == "fail")' ## Shows the violation details ## After fixing: argocd app get analytics-service-staging ## Health: Healthy kubectl get policyreport -n analytics-service-staging -o json | \ jq '[.results[] | select(.result == "fail")] | length' ## Must return: 0 ``` ### Scenario C — Reduce analytics-service cost by 40% **Goal:** The `analytics-service` was deployed with `instanceSize: medium` (250m CPU, 256Mi memory requests). Kubecost shows its efficiency score is below 30% — it is using far less than it requested. Using VPA recommendations and Kubecost data, reduce the service's infrastructure cost by at least 40% without violating its SLOs. Work through these independently: * Check what the service is actually consuming with `kubectl top pods` * Install VPA in recommendation mode and wait for it to suggest correct sizes * Check the Kubecost efficiency score before and after * Update the resource requests in `k8s/deployment.yaml` * Push the change through GitOps * Verify the SLO recording rules from Capstone 5 still show availability above 99.9% **Success criteria:** ```bash ## Before: note the Kubecost cost for analytics-service-staging ## After: kubectl get vpa analytics-service-vpa -n analytics-service-staging \ -o json | jq '.status.recommendation.containerRecommendations[0].target' ## VPA target should match your new resource requests ## Kubecost cost after change must be at least 40% lower than before curl -s "http://localhost:9090/model/allocation?window=24h&namespace=analytics-service-staging" | \ jq '.data[0]["analytics-service-staging"].totalCost' ## SLO must still be healthy curl -s "http://localhost:9090/api/v1/query?query=job:http_availability:rate5m" | \ jq '.data.result[0].value[1]' ## Must be >= 0.999 ``` ---
Five capstones. Here is what you built in each one: Capstone 1 — a cloud-native application: Node.js backend, React fron...
Why these tools integrate this way Before writing any code, understand the integration decisions. Each tool has a specif...
2.1 GitHub Actions Workflow for Terraform Provisioning This workflow is the bridge between Backstage and AWS. When Backs...
Why this template exists Every section of this template answers a real developer problem. The form fields map to real in...
4.1 Pre-flight checks Before running the template, verify every component the template depends on is working. A failed p...
These three scenarios have no step-by-step instructions. Use everything from Capstones 1-5. Scenario A — Deploy a second...
This capstone is attached to Step 14: System Design and Interview Prep. After building the complete integrated platform,...
---...
❌ Wiring everything together before each piece works independently 💥 A team decides to build the full integrated platfo...
Template step fails at GitHub Actions dispatch Terraform provisioning fails ArgoCD not detecting the new Application Cro...
Six capstones. One complete platform. Here is what your platform does in a world where it does not exist versus a world ...
Aligns directly with DevOps, Site Reliability (SRE), and Platform Engineering job descriptions.