Build an Internal Developer Platform from first principles - understand why IDPs exist, what problems they solve, how to implement a software catalog and golden paths with Backstage, and how to measure platform success with DORA metrics.
Picture this. A new engineer joins Swiggy's platform team. They need to create a new microservice. Here is what happens without an Internal Developer Platform: ``` Day 1: "How do I create a new service repo?" → asks senior engineer (interrupts their work) Day 2: "Which Dockerfile template do we use?" → asks a different senior engineer Day 3: "How do I set up CI/CD for this service?" → opens a ticket to the DevOps team Day 5: "What's the standard for Kubernetes manifests?" → reads outdated wiki, gets it wrong Day 8: "How do I add monitoring?" → another ticket, another wait Week 3: Service finally deployed, but inconsistently configured, missing observability, using a non-standard Docker base image ``` This is not a hypothetical. Spotify measured it — new engineers took over 60 days to merge their tenth pull request. That is two months of partial productivity. And it was not because the engineers were slow. It was because the information was scattered, the processes were tribal knowledge, and every standard had to be discovered rather than provided. **An Internal Developer Platform (IDP) is the solution.** It is the toolchain, workflows, and self-service capabilities that allow any developer — new or experienced — to build, deploy, and operate services without constantly interrupting platform engineers or waiting for tickets. After implementing Backstage, Spotify engineers could onboard in days, not months. Over 80% of internal users reported satisfaction. The platform team's ticket volume dropped dramatically because developers could self-serve. ---
The term gets used loosely. Two important distinctions before going further. ### Internal Developer Platform vs Internal Developer Portal These are related but different: ``` Internal Developer Platform (the engine): The set of tools, APIs, and automation that enables self-service. Examples: CI/CD pipelines, Kubernetes clusters, Terraform modules, secrets management, observability stack, deployment tooling. → This is the infrastructure of self-service Internal Developer Portal (the front door): The UI through which developers discover and use the platform. Examples: Backstage, Port, OpsLevel → This is the interface to the platform Analogy: Platform = the kitchen (where food is made) Portal = the restaurant menu (how customers order) ``` Backstage is primarily a portal — but most teams use the terms interchangeably because the portal is what developers actually interact with. ### Platform as a Product — The Mental Shift That Changes Everything The single most important concept in platform engineering is treating the platform as a product, not as an infrastructure ticket system. **Ticket-ops model (wrong):** ``` Developer: "I need a new namespace" Platform team: creates Jira ticket → waits → fulfils ticket Developer: "I need a database" Platform team: creates another ticket → waits → fulfils ticket Result: Platform team is a bottleneck. Developers are blocked. ``` **Platform as a product (right):** ``` Platform team asks: "What does a developer need to create a new service end-to-end?" Platform team builds: a self-service template that does all of it Developer: runs the template, gets everything in 5 minutes Result: Platform team enables velocity instead of bottlenecking it. ``` The developers are the customers. Their pain points are the product roadmap. Their adoption is the success metric. This mindset shift, described in Team Topologies by Matthew Skelton and Manuel Pais, is what separates platform teams that succeed from those that become shadow DevOps teams. ### The Three Characteristics of a Good Platform According to Team Topologies and confirmed by DORA research: **Self-service** — changes must be possible on-demand without waiting for another team. If a developer needs to raise a ticket to provision anything standard, the platform is not self-service. **Compelling** — the platform must be the easiest path, not the mandated one. At Spotify, the rule was: if developers are not choosing to use it, the platform is not good enough yet. Make it better. **Thinnest viable** — build the minimum that meaningfully reduces developer friction. A platform trying to do everything before releasing anything will miss the mark. Start with the most common workflow, make that excellent, then iterate. ---
DORA's 2025 research found that 90% of organisations now have some form of internal developer platform and 76% have dedicated platform teams. But the more important finding was this: > When platform quality is high, AI adoption drives strong performance improvements. When platform quality is low, AI adoption has negligible impact. AI tools make individual developers faster at writing code. But that speed is lost at the bottlenecks — inconsistent deployments, manual security reviews, complex debugging processes. A high-quality platform eliminates those bottlenecks, letting AI speed flow through to production. The four DORA metrics that matter most: ``` Deployment Frequency: How often does your team deploy to production? Elite: Multiple times per day Platform role: automated deployment pipelines, self-service Lead Time for Changes: How long from commit to production? Elite: Less than 1 hour Platform role: golden paths, automated testing, no ticket waits Mean Time to Recovery (MTTR): How long to recover from a production failure? Elite: Less than 1 hour Platform role: observability integration, runbooks in the portal Change Failure Rate: What % of deployments cause production issues? Elite: 0-15% Platform role: policy enforcement, testing gates in templates ``` Measuring these four metrics tells you objectively whether your platform is improving developer velocity or just generating work. ---
Backstage is the open-source developer portal framework that Spotify built, then open-sourced in 2020, then donated to the CNCF. It is now the de facto standard for building internal developer portals. What Backstage provides out of the box: * **Software Catalog** — a searchable registry of every service, library, website, data pipeline, and ML model in your organisation, with ownership, documentation, and health status * **Software Templates (Scaffolder)** — self-service wizards that create new services from templates, pre-configured with your standards * **TechDocs** — documentation-as-code, written in Markdown alongside the service code, rendered in Backstage * **Search** — unified search across catalog, docs, and external systems * **Plugin system** — integrations for Kubernetes, GitHub, PagerDuty, Datadog, Grafana, and hundreds more ### Installing Backstage ```bash ## Prerequisites: Node.js 18+, Yarn, Docker ## Create a new Backstage app npx @backstage/create-app@latest ## Follow the prompts: ## Enter a name for the app: devops-network-portal cd devops-network-portal ## Install dependencies yarn install ## Start development server yarn dev ## Open: http://localhost:3000 ``` The default Backstage app includes a demo catalog. Next step is connecting it to your real systems. ### Configuring Backstage — app-config.yaml ```yaml ## app-config.yaml app: title: DevOps Network Portal baseUrl: http://localhost:3000 organization: name: DevOps Network backend: baseUrl: http://localhost:7007 listen: port: 7007 ## GitHub integration — reads catalog-info.yaml from repos integrations: github: - host: github.com token: ${GITHUB_TOKEN} ## Auto-discover services from your GitHub org catalog: providers: github: myOrg: organization: your-github-org catalogPath: /catalog-info.yaml ## file in each repo filters: branch: main schedule: frequency: { minutes: 30 } timeout: { minutes: 3 } ## Kubernetes plugin — shows deployments per service kubernetes: serviceLocatorMethod: type: multiTenant clusterLocatorMethods: - type: config clusters: - url: https://kubernetes.default.svc name: production authProvider: serviceAccount serviceAccountToken: ${K8S_SA_TOKEN} skipTLSVerify: true ## Authentication (GitHub OAuth) auth: providers: github: development: clientId: ${GITHUB_CLIENT_ID} clientSecret: ${GITHUB_CLIENT_SECRET} ``` ---
The software catalog is the foundation of Backstage. Every service in your organisation registers itself by adding a `catalog-info.yaml` file to its repository. ### The catalog-info.yaml File ```yaml ## catalog-info.yaml — add this to every service repo root apiVersion: backstage.io/v1alpha1 kind: Component metadata: name: payment-service description: Handles all payment processing for Razorpay checkout annotations: ## Link to GitHub repository github.com/project-slug: razorpay/payment-service ## Link to Kubernetes deployments backstage.io/kubernetes-id: payment-service ## Link to monitoring grafana/dashboard-selector: payment-service ## Link to on-call pagerduty.com/integration-key: ${PAGERDUTY_KEY} tags: - payments - critical - golang links: - url: https://grafana.internal/d/payment-service title: Grafana Dashboard icon: dashboard - url: https://runbooks.internal/payment-service title: Runbook icon: docs spec: type: service lifecycle: production ## production, experimental, deprecated owner: payments-team ## team that owns this service system: payment-platform ## logical grouping of related services ## What this service depends on dependsOn: - resource:default/payment-db - component:default/user-service ## APIs this service provides providesApis: - payment-api ``` ```yaml ## catalog-info.yaml for a database resource apiVersion: backstage.io/v1alpha1 kind: Resource metadata: name: payment-db description: PostgreSQL database for payment service spec: type: database owner: payments-team system: payment-platform ``` ```yaml ## catalog-info.yaml for a team apiVersion: backstage.io/v1alpha1 kind: Group metadata: name: payments-team description: Team responsible for all payment processing spec: type: team profile: displayName: Payments Team email: payments@razorpay.com parent: engineering children: [] members: - arjun.sharma - priya.nair - vikram.das ``` Once these files exist in repos and GitHub discovery is configured, Backstage automatically populates the catalog. Engineers can search for any service, see who owns it, find its documentation, check its Kubernetes deployments, and view its dependencies — all from one place. > 💡 **Tip:** The catalog solves the "who owns this service?" problem immediately. In any incident, you can find the owning team in seconds. This alone justifies the Backstage setup time. ---
A golden path is a pre-built, approved workflow that developers follow to create new services. The term comes from the idea of a paved road — you can still go off-road, but the paved path is faster and easier. Without golden paths: ``` Developer creates new service → picks their own Docker base image (maybe insecure) → writes their own CI pipeline (may miss security scans) → writes their own Kubernetes manifests (may miss resource limits) → adds monitoring manually (often forgotten) → documentation? (almost always skipped) Result: 20 services, 20 different configurations, 20 different problems ``` With golden paths: ``` Developer runs the Backstage template → secure, approved Docker base image included → CI pipeline with tests and security scan included → Kubernetes manifests with resource limits included → Prometheus monitoring annotations included → TechDocs structure included Result: 20 services, 1 consistent standard, problems caught before production ``` ### Building a Software Template in Backstage ```yaml ## templates/microservice/template.yaml apiVersion: scaffolder.backstage.io/v1beta3 kind: Template metadata: name: golang-microservice title: Go Microservice description: > Creates a production-ready Go microservice with CI/CD, Kubernetes manifests, monitoring, and documentation. tags: - golang - recommended - production-ready spec: owner: platform-team type: service ## Form that developer fills in Backstage UI parameters: - title: Service Details required: - serviceName - owner - description properties: serviceName: title: Service Name type: string description: Lowercase, hyphens only (e.g. payment-processor) pattern: '^[a-z][a-z0-9-]*$' ui:autofocus: true owner: title: Owner Team type: string description: Which team owns this service? ui:field: OwnerPicker ui:options: allowedKinds: [Group] description: title: Description type: string description: What does this service do? repoVisibility: title: Repository Visibility type: string enum: [private, internal, public] default: private - title: Infrastructure properties: namespace: title: Kubernetes Namespace type: string description: Which namespace to deploy to? enum: - payments-production - orders-production - users-production - platform-production cpuRequest: title: CPU Request type: string default: "100m" memoryRequest: title: Memory Request type: string default: "128Mi" ## Steps executed when developer clicks "Create" steps: ## Step 1 — Copy the skeleton template files - id: fetch-template name: Fetch Template action: fetch:template input: url: ./skeleton values: serviceName: ${{ parameters.serviceName }} owner: ${{ parameters.owner }} description: ${{ parameters.description }} namespace: ${{ parameters.namespace }} cpuRequest: ${{ parameters.cpuRequest }} memoryRequest: ${{ parameters.memoryRequest }} ## Step 2 — Create GitHub repository with the files - id: create-repo name: Create GitHub Repository action: publish:github input: allowedHosts: ['github.com'] description: ${{ parameters.description }} repoUrl: > github.com?owner=yourorg&repo=${{ parameters.serviceName }} repoVisibility: ${{ parameters.repoVisibility }} defaultBranch: main deleteBranchOnMerge: true protectDefaultBranch: true ## Step 3 — Register the new service in the Backstage catalog - id: register-catalog name: Register in Catalog action: catalog:register input: repoContentsUrl: ${{ steps['create-repo'].output.repoContentsUrl }} catalogInfoPath: /catalog-info.yaml ## What the developer sees after completion output: links: - title: Repository url: ${{ steps['create-repo'].output.remoteUrl }} - title: Open in Backstage icon: catalog entityRef: ${{ steps['register-catalog'].output.entityRef }} ``` ### The Template Skeleton — What Gets Created The skeleton is the actual file structure that gets copied into the new repo. For a Go microservice template: ``` templates/microservice/skeleton/ .github/ workflows/ ci.yaml ← pre-configured GitHub Actions with tests + security scan k8s/ deployment.yaml ← Kubernetes Deployment with resource limits, health checks service.yaml ← Kubernetes Service hpa.yaml ← HorizontalPodAutoscaler docs/ index.md ← TechDocs template catalog-info.yaml ← pre-filled with template variables Dockerfile ← approved base image, non-root user Makefile ← standard build targets README.md ``` ```yaml ## skeleton/k8s/deployment.yaml ## Pre-filled template file — ${{ values.X }} gets replaced apiVersion: apps/v1 kind: Deployment metadata: name: ${{ values.serviceName }} namespace: ${{ values.namespace }} labels: app: ${{ values.serviceName }} owner: ${{ values.owner }} spec: replicas: 2 selector: matchLabels: app: ${{ values.serviceName }} template: metadata: labels: app: ${{ values.serviceName }} owner: ${{ values.owner }} annotations: ## Prometheus scraping pre-configured prometheus.io/scrape: "true" prometheus.io/port: "8080" prometheus.io/path: "/metrics" spec: securityContext: runAsNonRoot: true runAsUser: 1000 containers: - name: ${{ values.serviceName }} image: yourregistry/${{ values.serviceName }}:latest ports: - containerPort: 8080 resources: requests: cpu: "${{ values.cpuRequest }}" memory: "${{ values.memoryRequest }}" limits: cpu: "1" memory: "512Mi" livenessProbe: httpGet: path: /health/live port: 8080 initialDelaySeconds: 15 periodSeconds: 10 readinessProbe: httpGet: path: /health/ready port: 8080 initialDelaySeconds: 10 periodSeconds: 5 ``` When a developer fills the form in Backstage and clicks Create, they get a fully configured repository with everything above — in under 5 minutes, compared to days of manual setup. ---
Picture this. A new engineer joins Swiggy's platform team. They need to create a new microservice. Here is what happens ...
The term gets used loosely. Two important distinctions before going further. Internal Developer Platform vs Internal Dev...
DORA's 2025 research found that 90% of organisations now have some form of internal developer platform and 76% have dedi...
Backstage is the open-source developer portal framework that Spotify built, then open-sourced in 2020, then donated to t...
The software catalog is the foundation of Backstage. Every service in your organisation registers itself by adding a cat...
A golden path is a pre-built, approved workflow that developers follow to create new services. The term comes from the i...
Backstage handles the developer portal layer. But what about actual infrastructure — databases, queues, S3 buckets? Deve...
The complete self-service flow combines all three tools: This is what "golden path" means in practice — not just a templ...
A platform without measurement is a guess. Track these metrics to know if your platform is actually helping. DORA Metric...
❌ Building an IDP nobody uses — the "ivory tower" mistake 💥 A platform team spends 6 months building a comprehensive ID...
When Backstage catalog is not showing new services: When a software template fails: When DORA metrics show no improvemen...
This project builds a working Backstage instance connected to a GitHub organisation, with a software catalog, a service ...
Aligns directly with DevOps, Site Reliability (SRE), and Platform Engineering job descriptions.