Pipeline

CI/CD hub

A CI/CD pipeline is an automated sequence of stages that takes source code from a Git commit through building, testing, scanning, and deploying to production — eliminating manual steps and ensuring every change follows the same verified path to release.

Understanding CI/CD Pipelines

What Is a Pipeline in Simple Terms

A pipeline is an assembly line for software. Raw material enters at one end (a Git commit), passes through a series of automated checks and transformations (build, test, scan, deploy), and comes out the other end as running software in production. Every car that comes off an assembly line went through the same process. Every deployment that reaches production went through the same pipeline.

Before CI/CD pipelines, deploying software at Swiggy meant someone running scripts manually, hoping the steps were in the right order, discovering broken dependencies in production. Pipelines made this process repeatable, auditable, and automatic.

How It Works

Bash

+------------------------------------------+
| Trigger: git push to main branch         |
+------------------------------------------+
                    |
                    v
+------------------------------------------+
| Stage 1: Build                           |
| Compile code, build Docker image,        |
| push to container registry               |
+------------------------------------------+
                    |
              pass or fail
                    |
                    v
+------------------------------------------+
| Stage 2: Test                            |
| Unit tests, integration tests,           |
| coverage gate enforcement                |
+------------------------------------------+
                    |
                    v
+------------------------------------------+
| Stage 3: Scan                            |
| Image vulnerability scan (Trivy),        |
| SAST (Semgrep), dependency audit         |
+------------------------------------------+
                    |
                    v
+------------------------------------------+
| Stage 4: Deploy                          |
| Dev: automatic                           |
| Staging: automatic                       |
| Production: manual approval gate         |
+------------------------------------------+

Pipeline as code:

YAML

## .github/workflows/deploy.yaml
name: CI/CD Pipeline
 
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
 
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build Docker image
        run: docker build -t payment-api:${{ github.sha }} .
 
  test:
    needs: build    ## runs after build completes
    runs-on: ubuntu-latest
    steps:
      - name: Run tests
        run: npm test
 
  deploy:
    needs: test     ## runs after test completes
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - name: Deploy to staging
        run: ./scripts/deploy.sh staging

Practical Commands

Bash

## GitHub CLI -- trigger a pipeline manually
gh workflow run deploy.yaml
 
## GitHub CLI -- view pipeline run status
gh run list --workflow=deploy.yaml
gh run view RUN_ID
 
## GitLab -- trigger pipeline via API
curl -X POST \
  -F token=PIPELINE_TRIGGER_TOKEN \
  -F ref=main \
  https://gitlab.com/api/v4/projects/PROJECT_ID/trigger/pipeline
 
## Jenkins -- trigger build via CLI
java -jar jenkins-cli.jar -s http://jenkins.internal build payment-api-pipeline
 
## View pipeline logs
gh run view RUN_ID --log

Troubleshooting

Symptom	First Check	What to Look For
Pipeline not triggering	Check branch protection and trigger config	Branch name matches trigger filter
Stage fails silently	Check job logs	Exit code and last command output
Pipeline too slow	Profile each stage duration	Parallelise independent jobs
Flaky tests causing failures	Track failure patterns	Tests depending on external state

Tip
Store your pipeline configuration as code in the same repository as the application. When a pipeline change breaks the build, you can trace it through the same Git history as the application code — and revert both together.

Remember
Artifact promotion is the most important pipeline discipline: the Docker image built and tested in Stage 1 must be the exact same image deployed in Stage 4. Never rebuild the image for each environment — rebuilding can introduce differences that invalidate everything the tests verified.

Security
A CI/CD pipeline has access to production deployment credentials, container registries, and cloud accounts. Treat pipeline configuration with the same security discipline as application code. Never print secrets to logs, always use secret masking, and audit who can approve production deployments.

Common Mistake
Adding more stages and checks to a pipeline that is already slow instead of making existing stages faster. A pipeline that takes 45 minutes trains engineers to stop watching it and context-switch away. Aim for under 10 minutes to staging and under 20 minutes to production.

Frequently Asked Questions

What's the difference between a pipeline and a single CI build job?

A build job runs one task — compile, test, whatever — and stops. A pipeline chains multiple stages (build, test, scan, deploy) with defined dependencies and gates between them, so a failure at any stage halts progression and nothing reaches production without passing everything before it. Pipelines emerged from the need to make releases repeatable: instead of an engineer manually running a checklist before each deploy, the same automated sequence runs identically for every commit.

What's a common mistake teams make when designing a pipeline's stages?

Putting slow stages first. If your pipeline runs a 15-minute integration test suite before a 10-second lint check, every trivial syntax error costs 15 minutes to surface. Order stages fastest-and-cheapest to slowest-and-most-expensive — lint, then unit tests, then build, then integration tests, then deploy — so failures are caught as early as possible and engineers get feedback in seconds, not minutes, for the mistakes that are easiest to make.