Build a Mini Internal Developer Platform
Extend Backstage into a self-service platform: a golden path that creates a repo, CI pipeline and ArgoCD app, plus live Kubernetes status.
Blueprint Walkthrough
Architecture Overview & Problem Statement
This project turns a Backstage portal into a self-service platform. A developer fills in a form, and the golden path template creates a GitHub repository with a working Node.js service, a CI pipeline that builds and pushes the image, Kubernetes manifests, and a catalog entry. An Argo CD ApplicationSet then notices the new repository and deploys it with no human step, and a read-only Kubernetes plugin shows the live pods on the service's page in Backstage. You finish with a measured "time from form to running pods" and a template repository you can show in a portfolio.
This is the second of three connected platform projects. It assumes the portal from Build an Internal Developer Portal with Backstage is running, so it does not re-teach installing Backstage, the catalog, or TechDocs. It also assumes a Kubernetes cluster with Argo CD, as built in Build a GitOps Delivery Platform with ArgoCD. Here you build the platform behind the portal: the automation that makes the Create button deliver a running service. The final project, The Platform Engineering Challenge, builds on both.
Platform versus portal
The platform is the engine: Kubernetes, CI, Argo CD, and the automation that deploys and runs services. It exists whether or not anyone opens a web page. The portal is the front door that makes the platform usable without expert knowledge. Backstage is the portal, and this project connects it to a real platform so the front door opens onto something.
The problem this solves
Without a platform, a new service means asking for a repository, a namespace, a pipeline, and a deployment, usually through four separate requests. Each wait is small, and together they take days. With a golden path, the developer fills in a form and the platform does the rest. Do not trust any number you read about this: time it yourself in the validation step and record it.
Treat the platform as a product. Before building, talk to three to five developers and ask what the most annoying part of creating a service is. Build that first, and keep the adoption numbers, because a platform nobody uses has failed however good the technology is.
What you build:
- A read-only Kubernetes integration so Backstage shows live pod status
- An Argo CD ApplicationSet that deploys any repository carrying the
golden-pathtopic - A service skeleton with a Dockerfile, Kustomize manifests, a CI workflow, and docs
- A Backstage template that creates the repository and registers the service
topic: golden-path] BS -->|registers| CAT[Catalog entry] GH -->|push| CI[GitHub Actions
build and tag image] CI -->|image| REG[GHCR] CI -->|commit new tag| GH GH -->|discovered by| AS[Argo CD ApplicationSet] AS -->|syncs| K8S[Kubernetes
one namespace per service] K8S -->|live status| BS classDef portal fill:#F97316,color:#fff classDef src fill:#475569,color:#fff classDef ci fill:#8B5CF6,color:#fff classDef cluster fill:#326CE5,color:#fff class BS,CAT portal class GH,REG src class CI ci class AS,K8S cluster
Milestone 1: Check the Prerequisites
This milestone confirms the three things you build on, so a failure later points at this project and not at a missing foundation.
- A working Backstage app from the portal project, with a
GITHUB_TOKENthat can read and create repositories (classic token with therepoandworkflowscopes for this lab). - A Kubernetes cluster such as kind, minikube, or the EKS cluster from the infrastructure project, with
kubectlaccess. - Argo CD installed in the
argocdnamespace.
kubectl get nodeskubectl get pods -n argocd# Expected: all Argo CD pods Running, including argocd-applicationset-controller # In the Backstage appyarn start# Expected: the portal opens and the catalog loadsCommon MistakeUsing a very old Argo CD install. The ApplicationSet controller ships with Argo CD from version 2.3 onwards. If
argocd-applicationset-controlleris missing, upgrade Argo CD first.
Milestone 2: Give Backstage Read-Only Access to the Cluster
This milestone lets developers see whether their service is healthy without cluster access of their own. Backstage reads the cluster through a dedicated service account that can only get, list, and watch. It never gets write access, because a portal that can change the cluster becomes an attack path.
Create k8s/backstage-reader.yaml:
apiVersion: v1kind: Namespacemetadata: name: platform---apiVersion: v1kind: ServiceAccountmetadata: name: backstage-reader namespace: platform---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata: name: backstage-readerrules: - apiGroups: [""] resources: ["pods", "services", "configmaps", "namespaces", "nodes", "limitranges", "resourcequotas"] verbs: ["get", "list", "watch"] - apiGroups: ["apps"] resources: ["deployments", "replicasets", "statefulsets", "daemonsets"] verbs: ["get", "list", "watch"] - apiGroups: ["autoscaling"] resources: ["horizontalpodautoscalers"] verbs: ["get", "list", "watch"] - apiGroups: ["batch"] resources: ["jobs", "cronjobs"] verbs: ["get", "list", "watch"] - apiGroups: ["networking.k8s.io"] resources: ["ingresses"] verbs: ["get", "list", "watch"]---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata: name: backstage-readerroleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: backstage-readersubjects: - kind: ServiceAccount name: backstage-reader namespace: platform---apiVersion: v1kind: Secretmetadata: name: backstage-reader-token namespace: platform annotations: kubernetes.io/service-account.name: backstage-readertype: kubernetes.io/service-account-tokenKubernetes 1.24 and later no longer create a token for a service account automatically, which is why the last object requests one explicitly.
kubectl apply -f k8s/backstage-reader.yaml # Confirm it can read, and cannot writekubectl auth can-i list pods --as=system:serviceaccount:platform:backstage-reader -Akubectl auth can-i delete pods --as=system:serviceaccount:platform:backstage-reader -A# Expected: yes, then noLoad the connection details into environment variables without printing the token:
export K8S_CLUSTER_URL=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')export K8S_SERVICE_ACCOUNT_TOKEN=$(kubectl get secret backstage-reader-token -n platform \ -o jsonpath='{.data.token}' | base64 -d)export K8S_CA_DATA=$(kubectl get secret backstage-reader-token -n platform \ -o jsonpath='{.data.ca\.crt}')Install the plugin and tell Backstage about the cluster:
yarn --cwd packages/app add @backstage/plugin-kubernetesyarn --cwd packages/backend add @backstage/plugin-kubernetes-backendIn packages/backend/src/index.ts:
backend.add(import('@backstage/plugin-kubernetes-backend'));In app-config.yaml:
kubernetes: serviceLocatorMethod: type: 'multiTenant' clusterLocatorMethods: - type: 'config' clusters: - name: lab-cluster url: ${K8S_CLUSTER_URL} authProvider: 'serviceAccount' skipTLSVerify: false serviceAccountToken: ${K8S_SERVICE_ACCOUNT_TOKEN} caData: ${K8S_CA_DATA}The Kubernetes tab on a service page also needs frontend wiring that depends on your Backstage version. Follow the Kubernetes plugin documentation for your frontend system. With the older frontend, the change is a route on the service entity page:
import { EntityKubernetesContent, isKubernetesAvailable,} from '@backstage/plugin-kubernetes'; <EntityLayout.Route path="/kubernetes" title="Kubernetes" if={isKubernetesAvailable}> <EntityKubernetesContent refreshIntervalMs={30000} /></EntityLayout.Route>A service appears in that tab only when its catalog-info.yaml carries the backstage.io/kubernetes-label-selector annotation. You add it through the skeleton in Milestone 4.
SecurityThe token can read every pod in the cluster. Keep it in environment variables or a secret store, never in
app-config.yaml, and never print it into a terminal log. Rotate it if it leaks.
Milestone 3: Let Argo CD Discover New Services
This milestone makes deployment automatic. An Argo CD ApplicationSet is a controller feature that generates Applications from a list. Its GitHub SCM generator scans an organisation, finds repositories that match a filter, and creates one Application per repository. When the template creates a repository with the golden-path topic, an Application appears without anyone applying a manifest.
Create two secrets: one so the generator can list repositories, and one so Argo CD can clone them if they are private.
kubectl create secret generic github-scm-token -n argocd \ --from-literal=token=<your token> kubectl create secret generic github-repo-creds -n argocd \ --from-literal=type=git \ --from-literal=url=https://github.com/YOUR_GITHUB_ORG \ --from-literal=username=git \ --from-literal=password=<your token>kubectl label secret github-repo-creds -n argocd argocd.argoproj.io/secret-type=repo-credsCreate argocd/golden-path-appset.yaml:
apiVersion: argoproj.io/v1alpha1kind: ApplicationSetmetadata: name: golden-path-services namespace: argocdspec: generators: - scmProvider: github: organization: YOUR_GITHUB_ORG tokenRef: secretName: github-scm-token key: token cloneProtocol: https filters: - labelMatch: ^golden-path$ pathsExist: - k8s/kustomization.yaml requeueAfterSeconds: 60 template: metadata: name: '{{repository}}' spec: project: default source: repoURL: '{{url}}' targetRevision: '{{branch}}' path: k8s destination: server: https://kubernetes.default.svc namespace: '{{repository}}' syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=truekubectl apply -f argocd/golden-path-appset.yamlEach service gets its own namespace named after its repository. That convention is what lets the Kubernetes plugin find it later. The generator checks every 60 seconds here so the lab is quick. The default is much slower, so in production consider a webhook instead of polling.
Common MistakePutting an Argo CD
Applicationfile inside each new repository and expecting it to deploy. Nothing applies that file. Argo CD only deploys Applications that exist in the cluster, so something must create them, and here the ApplicationSet does.
Milestone 4: Build the Service Skeleton
This milestone creates the files every new service starts with. The skeleton enforces the non-negotiables: a non-root user, resource limits, health probes, and a catalog entry with an owner. Everything else a team can change after creation.
Create the skeleton in templates/nodejs-microservice/skeleton/:
skeleton/ catalog-info.yaml Dockerfile package.json src/index.js k8s/deployment.yaml k8s/service.yaml k8s/kustomization.yaml .github/workflows/ci.yml mkdocs.yml docs/index.mdcatalog-info.yaml registers the service and links it to its cluster resources:
apiVersion: backstage.io/v1alpha1kind: Componentmetadata: name: ${{ values.name }} description: ${{ values.description }} annotations: github.com/project-slug: ${{ values.destination.owner }}/${{ values.name }} backstage.io/kubernetes-label-selector: 'app=${{ values.name }}' backstage.io/kubernetes-namespace: ${{ values.name }} backstage.io/techdocs-ref: dir:. tags: - nodejsspec: type: service lifecycle: experimental owner: ${{ values.owner }}src/index.js is a small service that already exposes the health and metrics endpoints the manifests expect, so the pods become ready on their own:
const express = require('express');const client = require('prom-client'); const app = express();client.collectDefaultMetrics(); app.get('/', (_req, res) => res.send('${{ values.name }} is running'));app.get('/health/live', (_req, res) => res.send('ok'));app.get('/health/ready', (_req, res) => res.send('ok'));app.get('/metrics', async (_req, res) => { res.set('Content-Type', client.register.contentType); res.send(await client.register.metrics());}); app.listen(4000, () => console.log('listening on 4000'));package.json:
{ "name": "${{ values.name }}", "version": "1.0.0", "main": "src/index.js", "scripts": { "start": "node src/index.js" }, "dependencies": { "express": "^4.19.2", "prom-client": "^15.1.0" }}Dockerfile uses a numeric user so Kubernetes can verify the container is not root:
FROM node:22-alpine AS builderWORKDIR /appCOPY package.json ./RUN npm install --omit=dev FROM node:22-alpineRUN addgroup -g 1001 -S appgroup && adduser -u 1001 -S appuser -G appgroupWORKDIR /appCOPY --from=builder /app/node_modules ./node_modulesCOPY --chown=appuser:appgroup src ./srcUSER 1001EXPOSE 4000CMD ["node", "src/index.js"]k8s/deployment.yaml carries the platform's standards:
apiVersion: apps/v1kind: Deploymentmetadata: name: ${{ values.name }} labels: app: ${{ values.name }}spec: replicas: ${{ values.replicas }} selector: matchLabels: app: ${{ values.name }} template: metadata: labels: app: ${{ values.name }} spec: securityContext: runAsNonRoot: true runAsUser: 1001 containers: - name: ${{ values.name }} image: ghcr.io/${{ values.destination.owner }}/${{ values.name }} ports: - containerPort: 4000 securityContext: allowPrivilegeEscalation: false resources: requests: cpu: 100m memory: 128Mi limits: cpu: 500m memory: 256Mi livenessProbe: httpGet: path: /health/live port: 4000 initialDelaySeconds: 10 periodSeconds: 10 readinessProbe: httpGet: path: /health/ready port: 4000 initialDelaySeconds: 5 periodSeconds: 5k8s/service.yaml:
apiVersion: v1kind: Servicemetadata: name: ${{ values.name }}spec: selector: app: ${{ values.name }} ports: - name: http port: 4000 targetPort: 4000k8s/kustomization.yaml holds the image tag, which is the one value CI changes on every release:
apiVersion: kustomize.config.k8s.io/v1beta1kind: Kustomizationresources: - deployment.yaml - service.yamlimages: - name: ghcr.io/${{ values.destination.owner }}/${{ values.name }} newTag: bootstrap.github/workflows/ci.yml builds the image, tags it with the commit SHA, and commits the new tag into the manifests. Argo CD sees that commit and deploys it. Using a unique tag per build matters: with latest, Argo CD sees no change in Git and never redeploys.
name: CI on: push: branches: [main] paths-ignore: - 'k8s/**' permissions: contents: write packages: write jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: docker/login-action@v3 with: registry: ghcr.io username: {% raw %}${{ github.actor }}{% endraw %} password: {% raw %}${{ secrets.GITHUB_TOKEN }}{% endraw %} - uses: docker/build-push-action@v6 with: context: . push: true tags: ghcr.io/${{ values.destination.owner }}/${{ values.name }}:{% raw %}${{ github.sha }}{% endraw %} - name: Point the manifests at the new image run: | cd k8s kustomize edit set image ghcr.io/${{ values.destination.owner }}/${{ values.name }}=ghcr.io/${{ values.destination.owner }}/${{ values.name }}:{% raw %}${{ github.sha }}{% endraw %} git config user.name "platform-bot" git config user.email "platform-bot@users.noreply.github.com" git commit -am "deploy: {% raw %}${{ github.sha }}{% endraw %}" git pushThe paths-ignore rule stops the bot's own commit from starting another build. Finally, add a mkdocs.yml and a docs/index.md with a short description, how to run the service locally, and an on-call section, as in the portal project.
Common MistakeCopying a GitHub Actions workflow into a Backstage skeleton unchanged. Both tools use
${{ }}, so Backstage tries to fill ingithub.shaand fails. Wrap every GitHub expression in{% raw %}and{% endraw %}, as above.
TipThe first build can only run after the repository exists, so for a minute or two the pods show
ImagePullBackOffwhile CI builds the first image. That is normal. If it stays, the container package may be private. Make it public in the package settings, or add animagePullSecretto the Deployment.
Milestone 5: Write the Golden Path Template
This milestone writes the form and the automation behind it. The template collects four fields, copies the skeleton with the answers filled in, creates the repository with the golden-path topic that the ApplicationSet watches, and registers the service in the catalog.
Create templates/nodejs-microservice/template.yaml:
apiVersion: scaffolder.backstage.io/v1beta3kind: Templatemetadata: name: nodejs-microservice title: Node.js Microservice description: Creates a repository, CI pipeline, manifests and catalog entry, then deploys through Argo CD tags: - nodejs - kubernetes - recommendedspec: owner: group:platform-team type: service parameters: - title: Service details required: - name - description - owner properties: name: title: Service name type: string description: Lowercase and hyphen-separated, for example notification-service pattern: '^[a-z][a-z0-9-]*[a-z0-9]$' ui:autofocus: true description: title: Description type: string owner: title: Owner team type: string ui:field: OwnerPicker ui:options: catalogFilter: kind: Group replicas: title: Initial replicas type: integer default: 2 minimum: 1 maximum: 5 - title: Repository required: - repoUrl properties: repoUrl: title: Repository location type: string ui:field: RepoUrlPicker ui:options: allowedHosts: - github.com steps: - id: fetch-template name: Fetch skeleton action: fetch:template input: url: ./skeleton values: name: ${{ parameters.name }} description: ${{ parameters.description }} owner: ${{ parameters.owner }} replicas: ${{ parameters.replicas }} destination: ${{ parameters.repoUrl | parseRepoUrl }} - id: publish-github name: Create GitHub repository action: publish:github input: repoUrl: ${{ parameters.repoUrl }} description: ${{ parameters.description }} repoVisibility: public defaultBranch: main topics: - golden-path - nodejs - id: register-catalog name: Register in catalog action: catalog:register input: repoContentsUrl: ${{ steps['publish-github'].output.repoContentsUrl }} catalogInfoPath: '/catalog-info.yaml' output: links: - title: Repository url: ${{ steps['publish-github'].output.remoteUrl }} - title: Open in catalog icon: catalog entityRef: ${{ steps['register-catalog'].output.entityRef }}Repositories are public here to keep the lab simple: no clone credentials and no pull secret. For private repositories, keep the repo-creds secret from Milestone 3 and add an imagePullSecret.
Make sure the golden-path topic and the repository name match what the ApplicationSet expects, because that is the only link between the template and the deployment.
Common MistakeMaking the template so rigid that nobody can use it. Enforce only what is truly non-negotiable, such as the security context, resource limits, probes, and an owner. Leave language, port, and database to the team, and add a second template for a different kind of service rather than adding exceptions to this one.
Milestone 6: Register the Template and Run It
This milestone makes the template appear under Create. Add it to the catalog locations in app-config.yaml, using a file target locally:
catalog: locations: - type: file target: ../../templates/nodejs-microservice/template.yaml rules: - allow: [Template]Restart Backstage, open Create, and confirm Node.js Microservice is listed. Make sure the platform-team group and at least one team to own services exist in your catalog from the portal project, because the owner picker lists catalog groups.
export GITHUB_TOKEN=<your token>yarn startValidation & Testing
Run the whole path and time it. Start a timer when you click Create.
- In Backstage, open Create, choose Node.js Microservice, and fill in the form:
notification-service, a description, an owner team,2replicas, and your organisation as the repository owner. Click Create and watch the three steps finish. - Check the pieces the platform built:
# The repository exists with the right topicgh repo view YOUR_GITHUB_ORG/notification-service --json name,repositoryTopics # CI built the image and committed the new taggh run list --repo YOUR_GITHUB_ORG/notification-service # Argo CD created the Application on its ownkubectl get applications -n argocd# Expected: notification-service, Synced and Healthy # The pods are running in their own namespacekubectl get pods -n notification-service# Expected: 2 pods Running- In Backstage, open the service from the catalog. Expected: the owner, the Kubernetes tab with 2 healthy pods, the image tag that is deployed, and the docs.
- Stop the timer when the pods are Running and visible in Backstage, and write the time down.
- Prove the loop: change the message in
src/index.jsand push. Expected: CI runs, adeploy:commit changes the tag, Argo CD syncs, and the Kubernetes tab shows the new tag. - Delete one pod with
kubectl delete pod -n notification-service <pod>. Expected: it is replaced, and the Kubernetes tab reflects it.
If something fails, check the first broken link in the chain, in this order: the Backstage task log, the repository and its topics, the GitHub Actions run, the ApplicationSet (kubectl describe applicationset golden-path-services -n argocd), and then the Application's sync status. If the Kubernetes tab is empty, confirm the label selector matches the pods with kubectl get pods -n notification-service -l app=notification-service, and that the token has not expired.
Clean up:
gh repo delete YOUR_GITHUB_ORG/notification-service --yeskubectl delete namespace notification-serviceDeleting the repository makes the ApplicationSet drop the Application on its next scan.
For your portfolio, add a README with the architecture diagram, screenshots of the form, the catalog page, and the Kubernetes tab, and a timing table: how long creating a service takes by hand compared with the template. Add a short "platform as product" note listing who you interviewed, what they asked for, and what you would measure after launch, such as adoption and time to first deploy.