Deploy an ML Model with FastAPI, Docker and AWS

Train a fraud model, serve it with FastAPI, ship it in Docker to AWS EC2 via ECR, then load test with Locust and fix p95 latency.

Architecture Overview & Problem Statement

This project takes a fraud model from a training script to a live test endpoint on AWS: you train a scikit-learn model with a validation gate, serve it with FastAPI, package it in a Docker image, push the image to Amazon ECR, run it on one EC2 instance, and load test it with Locust. You record p50, p95 and error rate before and after one real fix, then delete every AWS resource with one script.

It is the first portfolio piece in the MLOps roadmap and uses only Step 1 to 3 skills: Python, Git, Docker, one cloud and GitHub Actions. No model registry and no Kubernetes yet. Those come later.

The problem

The acme-fraud team at the fictional payments company acme-shop has a model that scores well in a notebook. Nobody else can call it. A teammate's laptop gives a different number, the "API" reloads the model on every call, and the last cloud demo left an instance running for three weeks. Swiggy, Razorpay and Stripe all hit this gap between "model trained" and "model answering requests", and it is the gap this project closes.

What you build

flowchart TD subgraph Laptop["Your laptop"] A([params.yaml]) --> B[Train + gate] B --> C[(model.joblib
+ metadata)] C --> D[Docker image] L[Locust] end subgraph AWS["AWS ap-south-1"] E[(ECR repo)] --> F[EC2 + Docker] F --> G[FastAPI :8000] end D -->|push by SHA| E L -->|load test| G classDef code fill:#DBEAFE,color:#1E293B,stroke:#326CE5 classDef store fill:#EDE9FE,color:#1E293B,stroke:#8B5CF6 classDef cloud fill:#F97316,color:#FFFFFF,stroke:#F97316 class A,B,D,L code class C,E store class F,G cloud

The flow of trust matters as much as the flow of data. Every number has one place it comes from:

Decision	Made on	Why
Decision threshold	Validation set	Tuning on test leaks it
Ship or not (gate)	Validation PR-AUC	Floor fixed before training
Final quality claim	Test set, once	Recorded in a file you commit

Repository layout

◈ DIAGRAM

acme-fraud-api/
├── params.yaml                  # every knob, one file
├── requirements.txt             # pinned runtime deps (go into the image)
├── requirements-dev.txt         # + pytest, httpx2, locust
├── pytest.ini
├── src/acme_fraud_api/
│   ├── data.py                  # make or load data, seeded 64/16/20 split
│   ├── train.py                 # train, pick threshold, gate, save
│   ├── evaluate_test.py         # score the test set once, safely
│   └── app.py                   # FastAPI service
├── tests/                       # test_train.py, test_api.py
├── loadtest/                    # locustfile.py, bench.sh, summary.py
├── deploy/                      # aws_up.sh, aws_down.sh, user_data.sh
├── reports/                     # test_report.json + load test CSVs (committed)
├── Dockerfile  .dockerignore  compose.yaml
└── .github/workflows/ci.yml

Prerequisites (by module title)

Python for AI Engineering, Git and GitHub, Linux Fundamentals.
Machine Learning for AI Engineers and Reproducible ML Training and Project Structure.
Docker Containerization, AWS, and GitHub Actions.
For Milestones 7 and 8 only: the basics of AWS IAM roles, EC2, ECR and security groups (the AWS module covers them), an AWS account you can bill, and a region with a default VPC.
Tools: Python 3.12, Docker Desktop or Docker Engine with Compose v2, Git, AWS CLI v2, bc and curl (for the benchmark script).

Versions used

Tool	Version	Checked
FastAPI / Uvicorn / Pydantic	0.162.0 / 0.54.0 / 2.14.2	PyPI, 5 Nov 2026
scikit-learn / pandas / numpy	1.9.1 / 3.1.1 / 2.5.4	PyPI, 5 Nov 2026
Locust / pytest / httpx2	2.47.0 / 9.2.0 / 2.16.0	PyPI, 5 Nov 2026
actions/checkout / setup-python	v7.0.1 / v7.1.0 (pinned by SHA)	GitHub tags, 5 Nov 2026

Tip
What was actually run for this guide: Milestones 1, 2, 3 and 5, the Python steps of Milestone 6, all tests and the break-it steps ran on Python 3.12 in a 2 vCPU Linux sandbox, and every output block marked "real run" is pasted from those runs. The Docker image build, the GitHub Actions run and everything on AWS were not run here (no Docker Hub, GitHub or AWS access from the sandbox). Those steps say so where they appear, and the AWS scripts were checked with bash -n and shellcheck only. Treat the project as locally verified, not fully validated, until you have run one real deploy and teardown yourself.

Milestone 1: Set Up the Repository and Train a Gated Model

This milestone gives you one command that trains the model the same way every time, picks the threshold on validation data, and refuses to save a model that misses a quality floor you set in advance.

Why a gate before anything else

A deployment pipeline is only as good as the thing it ships. If training can quietly produce a weak model, every later step (image, deploy, load test) faithfully delivers a weak model. The gate is a single if that stops the chain. The floor (min_val_pr_auc: 0.20) is illustrative and was written into params.yaml before the first training run. You never lower it after seeing a result.

Step 1: Create the project and environment

Bash

mkdir acme-fraud-api && cd acme-fraud-api
git init
python3.12 -m venv .venv
source .venv/bin/activate
mkdir -p src/acme_fraud_api tests loadtest deploy reports .github/workflows
touch src/acme_fraud_api/__init__.py
export PYTHONPATH=src          # so "python -m acme_fraud_api.train" finds the package

requirements.txt (only what the running service needs, all pinned):

TEXT

fastapi==0.162.0
uvicorn==0.54.0
pydantic==2.14.2
scikit-learn==1.9.1
pandas==3.1.1
numpy==2.5.4
joblib==1.6.0
pyyaml==6.0.3

requirements-dev.txt (tools for your laptop and CI, never baked into the image):

TEXT

-r requirements.txt
pytest==9.2.0
httpx2==2.16.0
locust==2.47.0

Bash

pip install -r requirements-dev.txt

Tip
httpx2 is the client Starlette's TestClient now asks for. With the older httpx your tests still pass but print a deprecation warning on every run.

Step 2: Put every knob in params.yaml

YAML

seed: 42
data:
  path: data/payments.csv
  rows: 20000
  fraud_rate: 0.03
split:
  test_size: 0.20      # 20 percent, looked at once
  val_size: 0.20       # 20 percent of the rest = 16 percent overall
model:
  max_iter: 200
  learning_rate: 0.05
  max_leaf_nodes: 31
gate:
  # Chosen before any number was seen. Illustrative, not a business target.
  min_val_pr_auc: 0.20
  recall_target: 0.75

Step 3: Write the data module

If you finished Reproducible ML Training and Project Structure you already have mlops-lab with data/payments.csv. This project is self-contained instead: data.py regenerates a seeded 20,000-row synthetic payments file on every run, so nobody trains on a stale or hand-edited copy, and training records the file's SHA-256 in the metadata. The fraud signal is weak and noisy on purpose, like real fraud.

src/acme_fraud_api/data.py:

Python

"""Make the payments dataset from the seed and split it the same way every time."""
import hashlib
from pathlib import Path
 
import numpy as np
import pandas as pd
import yaml
from sklearn.model_selection import train_test_split
 
FEATURES = ["amount", "hour", "merchant_category", "country_mismatch",
            "card_age_days", "txns_last_24h"]
CATEGORIES = ["grocery", "electronics", "travel", "fuel", "fashion", "gaming"]
 
 
def load_params(path: str = "params.yaml") -> dict:
    with open(path) as f:
        return yaml.safe_load(f)
 
 
def make_payments(rows: int, fraud_rate: float, seed: int) -> pd.DataFrame:
    """Synthetic card payments with a weak, noisy fraud signal."""
    rng = np.random.default_rng(seed)
    df = pd.DataFrame({
        "txn_id": [f"txn_{i:06d}" for i in range(rows)],
        "amount": np.round(rng.lognormal(mean=3.5, sigma=1.0, size=rows), 2),
        "hour": rng.integers(0, 24, size=rows),
        "merchant_category": rng.choice(CATEGORIES, size=rows),
        "country_mismatch": rng.binomial(1, 0.08, size=rows),
        "card_age_days": rng.integers(1, 3650, size=rows),
        "txns_last_24h": rng.poisson(2.0, size=rows),
    })
    score = (
        0.9 * np.log1p(df["amount"]) / 5
        + 1.2 * df["country_mismatch"]
        + 0.6 * df["merchant_category"].isin(["electronics", "gaming"])
        + 0.5 * ((df["hour"] <= 4) | (df["hour"] >= 23))
        + 0.25 * np.minimum(df["txns_last_24h"], 8) / 2
        - 0.6 * (df["card_age_days"] > 365)
        + rng.normal(0, 1.0, size=rows)          # noise: the signal is weak on purpose
    )
    cutoff = np.quantile(score, 1 - fraud_rate)
    df["is_fraud"] = (score >= cutoff).astype(int)
    return df
 
 
def write_dataset(params: dict) -> str:
    """Regenerate the CSV from the seed every time, so nobody trains on a stale
    or hand-edited copy. Returns the file's sha256 (recorded in metadata)."""
    path = Path(params["data"]["path"])
    path.parent.mkdir(parents=True, exist_ok=True)
    make_payments(params["data"]["rows"], params["data"]["fraud_rate"],
                  params["seed"]).to_csv(path, index=False)
    return hashlib.sha256(path.read_bytes()).hexdigest()
 
 
def load_splits(params: dict):
    """Return train, val, test frames. Stratified, seeded: 64/16/20."""
    write_dataset(params)
    df = pd.read_csv(params["data"]["path"])
    seed = params["seed"]
    rest, test = train_test_split(df, test_size=params["split"]["test_size"],
                                  stratify=df["is_fraud"], random_state=seed)
    train, val = train_test_split(rest, test_size=params["split"]["val_size"],
                                  stratify=rest["is_fraud"], random_state=seed)
    return train, val, test

Step 4: Write the training script with the gate

src/acme_fraud_api/train.py:

Python

"""Train the model, pick the threshold on validation, gate it, save it.
 
Exit codes: 0 = model saved, 1 = gate failed (nothing saved).
"""
import hashlib
import json
import os
import sys
from datetime import datetime, timezone
from pathlib import Path
 
import joblib
import numpy as np
import sklearn
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.metrics import average_precision_score, precision_score, recall_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder
 
from acme_fraud_api.data import FEATURES, load_params, load_splits
 
MODEL_DIR = Path(os.environ.get("MODEL_DIR", "model"))
 
 
def sha256_of(path: Path) -> str:
    return hashlib.sha256(path.read_bytes()).hexdigest()
 
 
def build_model(params: dict, seed: int) -> Pipeline:
    pre = ColumnTransformer(
        [("cat", OneHotEncoder(handle_unknown="ignore"), ["merchant_category"])],
        remainder="passthrough",
    )
    clf = HistGradientBoostingClassifier(
        max_iter=params["model"]["max_iter"],
        learning_rate=params["model"]["learning_rate"],
        max_leaf_nodes=params["model"]["max_leaf_nodes"],
        class_weight="balanced",
        random_state=seed,
    )
    return Pipeline([("pre", pre), ("clf", clf)])
 
 
def pick_threshold(y_true, proba, recall_target: float) -> float:
    """Highest threshold that still reaches the recall target on validation.
    0.0 is the last candidate (flag everything); if even that misses, stop."""
    for t in np.round(np.arange(0.99, -0.001, -0.01), 2):
        if recall_score(y_true, proba >= t) >= recall_target:
            return float(t)
    raise ValueError(f"no threshold reaches recall {recall_target} on validation")
 
 
def main() -> int:
    params = load_params()
    seed = params["seed"]
    train, val, _test = load_splits(params)   # test is NOT touched here
 
    model = build_model(params, seed)
    model.fit(train[FEATURES], train["is_fraud"])
 
    val_proba = model.predict_proba(val[FEATURES])[:, 1]
    pr_auc = average_precision_score(val["is_fraud"], val_proba)
    threshold = pick_threshold(val["is_fraud"], val_proba, params["gate"]["recall_target"])
    pred = val_proba >= threshold
    recall = recall_score(val["is_fraud"], pred)
    precision = precision_score(val["is_fraud"], pred, zero_division=0)
    print(f"val_pr_auc={pr_auc:.4f} threshold={threshold:.2f} "
          f"val_recall={recall:.4f} val_precision={precision:.4f}")
 
    floor = params["gate"]["min_val_pr_auc"]
    if pr_auc < floor:
        print(f"GATE FAILED: val_pr_auc {pr_auc:.4f} < floor {floor}. Nothing saved.")
        return 1
 
    MODEL_DIR.mkdir(parents=True, exist_ok=True)
    tmp = MODEL_DIR / "model.joblib.tmp"
    joblib.dump(model, tmp)
    os.replace(tmp, MODEL_DIR / "model.joblib")        # atomic swap
    meta = {
        "model_sha256": sha256_of(MODEL_DIR / "model.joblib"),
        "params_sha256": sha256_of(Path("params.yaml")),
        "data_sha256": sha256_of(Path(params["data"]["path"])),
        "threshold": threshold,
        "val_pr_auc": round(pr_auc, 4),
        "val_recall": round(recall, 4),
        "val_precision": round(precision, 4),
        "features": FEATURES,
        "sklearn_version": sklearn.__version__,
        "trained_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
    }
    tmp_meta = MODEL_DIR / "metadata.json.tmp"
    tmp_meta.write_text(json.dumps(meta, indent=2))
    os.replace(tmp_meta, MODEL_DIR / "metadata.json")  # atomic swap, like the model
    print(f"saved {MODEL_DIR}/model.joblib sha256={meta['model_sha256'][:12]}")
    return 0
 
 
if __name__ == "__main__":
    sys.exit(main())

Four details worth saying out loud in a review:

pick_threshold uses only validation data. The test set is loaded by load_splits but never used here.
pick_threshold tries 0.0 last (flag everything). If even that misses the recall target, training stops with an error instead of saving a model that breaks its own promise.
The model and the metadata are each written to a temp file and swapped in with os.replace, so a crash mid-write never leaves a half-written file for the API to load.
metadata.json stores the SHA-256 of the model and of the data. The API compares the model hash before loading (Milestone 3).

Step 5: Train twice and prove it is reproducible

Bash

python -m acme_fraud_api.train
python -m acme_fraud_api.train
cat model/metadata.json

Expected output (real run):

TEXT

val_pr_auc=0.2322 threshold=0.47 val_recall=0.7500 val_precision=0.1057
saved model/model.joblib sha256=e70c4a9d2e04
val_pr_auc=0.2322 threshold=0.47 val_recall=0.7500 val_precision=0.1057
saved model/model.joblib sha256=e70c4a9d2e04

Same seed, same data, same library versions, same bytes: both runs produce the identical file hash. If yours differs from the hash above, that is fine (a different CPU or OS can change float rounding), but your two runs must match each other.

Acceptance check

python -m acme_fraud_api.train exits 0 and prints the same sha256 twice in a row.
model/model.joblib and model/metadata.json exist.

Common Mistake
Choosing the threshold on the test set "because it is the honest data". The moment you tune anything on test, it becomes a second validation set and you no longer have an unbiased final number.

Milestone 2: Score the Test Set Once and Keep the Record

This milestone produces your one honest quality claim: the test-set result, written once to a file you commit, with guards that refuse a second look but still let you retry if the evaluation itself crashed.

Why "once" needs code, not willpower

If you can rerun the test evaluation, you will, after "just one small tweak". Each rerun quietly turns the test set into a tuning set. The script enforces "once" in three moves:

Reserve: create reports/.test_report.lock with O_EXCL, so two evaluations cannot run at the same time.
Evaluate first: check the model hash against its metadata (the threshold must belong to this exact file), score the test set, and write the result to a temp file.
Publish atomically: os.link the temp file to reports/test_report.json. The link fails if the report already exists. The lock is always released, so a crash leaves no report and you can fix the problem and retry.

Step 1: Write the test-once script

src/acme_fraud_api/evaluate_test.py:

Python

"""Score the held-out test set ONCE and keep a durable record.
 
Exit codes: 0 = report written, 3 = already evaluated or in progress (refused),
            1 = evaluation failed (no report written, safe to fix and retry).
"""
import json
import os
import sys
from datetime import datetime, timezone
from pathlib import Path
 
import joblib
from sklearn.metrics import average_precision_score, precision_score, recall_score
 
from acme_fraud_api.data import FEATURES, load_params, load_splits
from acme_fraud_api.train import MODEL_DIR, sha256_of
 
REPORT = Path("reports/test_report.json")
LOCK = Path("reports/.test_report.lock")   # reservation while an evaluation runs
 
 
def evaluate() -> dict:
    meta = json.loads((MODEL_DIR / "metadata.json").read_text())
    model_hash = sha256_of(MODEL_DIR / "model.joblib")
    if model_hash != meta["model_sha256"]:
        # the threshold in metadata belongs to a different model file
        raise RuntimeError("model hash does not match training metadata")
    _train, _val, test = load_splits(load_params())
    model = joblib.load(MODEL_DIR / "model.joblib")
    proba = model.predict_proba(test[FEATURES])[:, 1]
    pred = proba >= meta["threshold"]
    return {
        "model_sha256": model_hash,
        "data_sha256": meta["data_sha256"],
        "threshold": meta["threshold"],
        "test_pr_auc": round(average_precision_score(test["is_fraud"], proba), 4),
        "test_recall": round(recall_score(test["is_fraud"], pred), 4),
        "test_precision": round(precision_score(test["is_fraud"], pred, zero_division=0), 4),
        "rows": int(len(test)),
        "positives": int(test["is_fraud"].sum()),
        "evaluated_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
    }
 
 
def main() -> int:
    REPORT.parent.mkdir(parents=True, exist_ok=True)
    if REPORT.exists():
        print(f"REFUSED: {REPORT} already exists. The test set is looked at once.")
        return 3
    try:
        # O_EXCL: only one evaluation can hold the reservation at a time
        os.close(os.open(LOCK, os.O_CREAT | os.O_EXCL | os.O_WRONLY))
    except FileExistsError:
        print(f"REFUSED: {LOCK} exists (another evaluation running, or a crashed one: check, then delete it).")
        return 3
    try:
        report = evaluate()                      # 1. evaluate first
        tmp = REPORT.with_suffix(".json.tmp")
        tmp.write_text(json.dumps(report, indent=2))
        os.link(tmp, REPORT)                     # 2. publish: fails if REPORT already exists
        tmp.unlink()
    except Exception as err:                     # nothing published, retry is allowed
        print(f"EVALUATION FAILED, no report written: {err}")
        return 1
    finally:
        LOCK.unlink(missing_ok=True)             # 3. always release the reservation
    print(json.dumps(report, indent=2))
    return 0
 
 
if __name__ == "__main__":
    sys.exit(main())

Step 2: Prove a crash does not use up your one look

Hide the model, run the script, and check that nothing was published and no lock was left:

Bash

mv model/model.joblib /tmp/model.joblib
python -m acme_fraud_api.evaluate_test; echo "exit=$?"
for f in reports/test_report.json reports/.test_report.lock; do [ -e "$f" ] && echo "$f: EXISTS" || echo "$f: none"; done
mv /tmp/model.joblib model/model.joblib

Expected output (real run):

TEXT

EVALUATION FAILED, no report written: [Errno 2] No such file or directory: 'model/model.joblib'
exit=1
reports/test_report.json: none
reports/.test_report.lock: none

Step 3: Run it once, then try again

Bash

python -m acme_fraud_api.evaluate_test; echo "exit=$?"
python -m acme_fraud_api.evaluate_test; echo "exit=$?"

Expected output (real run):

TEXT

{
  "model_sha256": "e70c4a9d2e0434dd995eb1414938f3a2f993f502f53ffeecc9e3e6b437223db2",
  "data_sha256": "7e1b05cb61997fbb5af9b386055d4c4f549e5d56b6cb2708bd73d8215773aaca",
  "threshold": 0.47,
  "test_pr_auc": 0.1804,
  "test_recall": 0.7583,
  "test_precision": 0.104,
  "rows": 4000,
  "positives": 120,
  "evaluated_at": "2026-11-04T19:47:58+00:00"
}
exit=0
REFUSED: reports/test_report.json already exists. The test set is looked at once.
exit=3

Step 4: Read the number honestly

Test PR-AUC (0.18) came in below validation (0.23). That is normal here, not a bug. The test set has only 120 fraud rows and validation has 96, so a handful of hard cases moves PR-AUC by several points. Recall held at the target (0.76 vs 0.75). Report both numbers and the row counts. Do not retrain to "fix" the test score: that is exactly the peeking this script exists to stop.

Bash

git add reports/test_report.json params.yaml
git commit -m "Record test-set result for model e70c4a9d2e04"

Acceptance check

reports/test_report.json is committed and contains the same model_sha256 and data_sha256 as model/metadata.json.
A second run exits 3 and leaves the file unchanged.
No reports/.test_report.lock is left behind.

Common Mistake
Deleting test_report.json to "rerun cleanly". If you change the model, that is a new model with a new hash and a new record. Keep the old record in Git history.

Milestone 3: Serve the Model with FastAPI

This milestone wraps the model in a small HTTP API with strict input validation, a liveness check, a readiness check, and a check that the model file matches the hash recorded in its metadata before it is loaded.

Why validation and a hash check

A model will happily score hour = 37 or a negative amount and return a confident, meaningless number. Pydantic rejects those with a 422 before the model sees them. The hash comparison is a consistency check: it catches a model file that was swapped, corrupted or retrained without updating its metadata, which matters because joblib.load runs pickle code. It is not proof of origin. Anyone who can replace both files can replace the recorded hash too. Proving a model came from a trusted training run needs signing or a registry, which later roadmap steps cover.

Step 1: Write the first version of the API

Write it the way most notebook-to-API ports start. Note the one line marked v1: it reloads the model inside the request handler. Predictions work, so it looks fine. Keep it for now. Milestone 5 measures what it costs.

src/acme_fraud_api/app.py (v1):

Python

"""FastAPI service for the acme-fraud model (v1: first working version)."""
import hashlib
import json
import os
import threading
from contextlib import asynccontextmanager
from pathlib import Path
from typing import Literal
 
import joblib
import pandas as pd
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
 
MODEL_DIR = Path(os.environ.get("MODEL_DIR", "model"))
STATE: dict = {}                 # holds the loaded model and its metadata
PREDICT_LOCK = threading.Lock()  # serialises predictions in this process (see Milestone 5)
 
Category = Literal["grocery", "electronics", "travel", "fuel", "fashion", "gaming"]
 
 
# ---- Request and response contracts: bad input gets a 422, never reaches the model ----
class Payment(BaseModel):
    txn_id: str = Field(min_length=1, max_length=64)
    amount: float = Field(gt=0, le=1_000_000)
    hour: int = Field(ge=0, le=23)
    merchant_category: Category
    country_mismatch: Literal[0, 1]
    card_age_days: int = Field(ge=0, le=36_500)
    txns_last_24h: int = Field(ge=0, le=1_000)
 
 
class Score(BaseModel):
    txn_id: str
    fraud_probability: float
    is_fraud: bool
    threshold: float
    model_version: str
 
 
def load_model() -> None:
    """Check the model file matches the hash in its metadata, then load it."""
    meta = json.loads((MODEL_DIR / "metadata.json").read_text())
    digest = hashlib.sha256((MODEL_DIR / "model.joblib").read_bytes()).hexdigest()
    if digest != meta["model_sha256"]:
        # A consistency check: catches a swapped or corrupted file. It is NOT proof of
        # origin: someone who can replace both files can replace the hash too.
        raise RuntimeError(f"model sha256 {digest[:12]} != metadata {meta['model_sha256'][:12]}")
    STATE["model"] = joblib.load(MODEL_DIR / "model.joblib")
    STATE["meta"] = meta
 
 
@asynccontextmanager
async def lifespan(app: FastAPI):
    yield
    STATE.clear()
 
 
app = FastAPI(title="acme-fraud-api", lifespan=lifespan)
 
 
@app.get("/health")
def health() -> dict:
    """Liveness: the process is up. Says nothing about the model."""
    return {"status": "ok"}
 
 
@app.get("/ready")
def ready() -> dict:
    """Readiness: the model is loaded and can answer."""
    if "model" not in STATE:
        raise HTTPException(status_code=503, detail="model not loaded")
    return {"status": "ready", "model_version": STATE["meta"]["model_sha256"][:12]}
 
 
@app.post("/predict", response_model=Score)
def predict(payment: Payment) -> Score:   # plain "def": FastAPI runs it in a thread pool
    load_model()   # v1: reload the model on every request (we measure this later)
    meta = STATE["meta"]
    row = pd.DataFrame([payment.model_dump()])[meta["features"]]  # same column order as training
    with PREDICT_LOCK:
        proba = float(STATE["model"].predict_proba(row)[0, 1])
    return Score(
        txn_id=payment.txn_id,
        fraud_probability=round(proba, 4),
        is_fraud=proba >= meta["threshold"],   # threshold was chosen on validation, not here
        threshold=meta["threshold"],
        model_version=meta["model_sha256"][:12],
    )

Tip
/health and /ready answer different questions. Liveness asks "should this process be restarted?" Readiness asks "should traffic be sent here yet?" In v1, /ready returns 503 until the first prediction loads the model, which is your first hint that something is off.

Step 2: Run it and call it

Bash

uvicorn acme_fraud_api.app:app --port 8000

In a second terminal:

Bash

curl -s -X POST localhost:8000/predict -H 'Content-Type: application/json' \
  -d '{"txn_id":"txn_demo_1","amount":129.99,"hour":2,"merchant_category":"electronics","country_mismatch":1,"card_age_days":40,"txns_last_24h":6}'
curl -s -o /dev/null -w "%{http_code}\n" -X POST localhost:8000/predict -H 'Content-Type: application/json' \
  -d '{"txn_id":"x","amount":-5,"hour":2,"merchant_category":"electronics","country_mismatch":1,"card_age_days":40,"txns_last_24h":6}'

Expected output (real run):

TEXT

{"txn_id":"txn_demo_1","fraud_probability":0.8637,"is_fraud":true,"threshold":0.47,"model_version":"e70c4a9d2e04"}
422

Step 3: Add tests

pytest.ini:

INI

[pytest]
pythonpath = src
testpaths = tests

tests/test_train.py:

Python

import json
from pathlib import Path
 
from acme_fraud_api.data import load_params, load_splits
 
 
def test_splits_are_disjoint_and_sized():
    train, val, test = load_splits(load_params())
    assert len(train) + len(val) + len(test) == 20_000
    assert len(test) == 4_000 and len(val) == 3_200
    ids = [set(df["txn_id"]) for df in (train, val, test)]
    assert not (ids[0] & ids[1]) and not (ids[0] & ids[2]) and not (ids[1] & ids[2])
 
 
def test_saved_model_passed_gate():
    meta = json.loads(Path("model/metadata.json").read_text())
    params = load_params()
    assert meta["val_pr_auc"] >= params["gate"]["min_val_pr_auc"]
    assert meta["val_recall"] >= params["gate"]["recall_target"]
 
 
def test_model_matches_its_metadata():
    from acme_fraud_api.train import sha256_of
    meta = json.loads(Path("model/metadata.json").read_text())
    assert sha256_of(Path("model/model.joblib")) == meta["model_sha256"]

tests/test_api.py:

Python

import json
from pathlib import Path
 
import pytest
from fastapi.testclient import TestClient
 
from acme_fraud_api.app import app
 
GOOD = {
    "txn_id": "txn_test_1", "amount": 129.99, "hour": 2,
    "merchant_category": "electronics", "country_mismatch": 1,
    "card_age_days": 40, "txns_last_24h": 6,
}
 
 
@pytest.fixture(scope="module")
def client():
    with TestClient(app) as c:   # "with" runs the lifespan, so the model loads
        yield c
 
 
def test_ready(client):
    r = client.get("/ready")
    assert r.status_code == 200
    assert r.json()["status"] == "ready"
 
 
def test_predict_shape(client):
    r = client.post("/predict", json=GOOD)
    assert r.status_code == 200
    body = r.json()
    assert 0.0 <= body["fraud_probability"] <= 1.0
    assert body["threshold"] == json.loads(Path("model/metadata.json").read_text())["threshold"]
 
 
@pytest.mark.parametrize("field,value", [
    ("amount", -5), ("hour", 24), ("merchant_category", "casino"), ("country_mismatch", 2),
])
def test_rejects_bad_input(client, field, value):
    r = client.post("/predict", json={**GOOD, field: value})
    assert r.status_code == 422
 
 
def test_missing_field(client):
    bad = dict(GOOD)
    bad.pop("amount")
    assert client.post("/predict", json=bad).status_code == 422

Bash

pytest -q

Expected output with v1 (real run, last lines):

TEXT

FAILED tests/test_api.py::test_ready - assert 503 == 200
1 failed, 9 passed in 1.55s

The test is right and the code is wrong. v1 has no model in memory until the first prediction, so a load balancer asking /ready would never send it traffic. Leave the failure in place: Milestone 5 shows the same line also costs you latency, and one fix clears both.

Acceptance check

A valid payment returns a probability and model_version. An invalid one returns 422.
pytest -q shows 9 passed and only test_ready failing (all 10 pass after Milestone 5).

Common Mistake
Writing async def predict around a scikit-learn call. Model inference is CPU work that blocks the event loop, so every other request (including /health) waits behind it. A plain def lets FastAPI run it in a thread pool.

Milestone 4: Containerise with Docker and Compose

This milestone packages code, pinned dependencies and the trained model into one image that runs as a non-root user, plus a Compose file that runs the same image on your laptop as the free local fallback for the AWS milestones.

Why bake the model into the image

At this stage the image tag (a git commit) then identifies code, dependencies and model together. One tag, one behaviour, which makes rollback "run the previous tag". The cost is a rebuild for every new model. Later roadmap steps move the model into a registry, when that trade starts to hurt.

Step 1: Write the Dockerfile and .dockerignore

Dockerfile:

Dockerfile

# Small official Python base. After your first pull, pin it by digest (see Milestone 4).
FROM python:3.12-slim
 
# No .pyc files, logs straight to stdout, one OpenMP thread per process,
# and tell the app where the model and the code live.
ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    OMP_NUM_THREADS=1 \
    MODEL_DIR=/app/model \
    PYTHONPATH=/app/src
 
WORKDIR /app
 
# Dependencies first: this layer is cached until requirements.txt changes.
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
 
# Code and the trained model (model/ must exist: run training before building).
COPY src/ src/
COPY model/ model/
 
# Never run as root inside the container.
RUN useradd --create-home --uid 10001 app
USER app
 
EXPOSE 8000
# Liveness only: is the process answering? Readiness (/ready) is checked by the deploy script.
HEALTHCHECK --interval=15s --timeout=3s --start-period=20s --retries=3 \
  CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=2)"
 
CMD ["uvicorn", "acme_fraud_api.app:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1"]

.dockerignore (keeps .venv, data and any .env out of the build context):

Bash

# Keep the build context small and free of secrets
.venv/
.git/
data/
reports/
tests/
loadtest/
deploy/
**/__pycache__/
*.env
.env*

OMP_NUM_THREADS=1 stops scikit-learn's histogram gradient boosting from starting one OpenMP thread per CPU for every single-row prediction. Under concurrent load those threads fight each other. Scale with processes instead (Milestone 5 has the experiment).

Step 2: Add the Compose file

compose.yaml:

YAML

# Local fallback: the same image you ship to AWS, on your laptop.
services:
  api:
    build: .
    image: acme-fraud-api:local
    ports:
      - "8000:8000"
    deploy:
      resources:
        limits:          # roughly the size of a small cloud VM
          cpus: "2"
          memory: 1g
    restart: unless-stopped

Step 3: Build, run and check

Bash

python -m acme_fraud_api.train          # model/ must exist before the build
docker compose up -d --build
docker compose ps                        # wait for "healthy"
curl -s localhost:8000/health

Expected output (the /health reply is from the sandbox run of the same app outside Docker; the Docker build itself was not run here):

JSON

{"status":"ok"}

Step 4: Pin the base image by digest

A tag like python:3.12-slim moves when a new patch ships. After your first successful build, look up the exact digest you got and pin it:

Bash

docker image inspect python:3.12-slim --format '{{index .RepoDigests 0}}'
# then in the Dockerfile:  FROM python:3.12-slim@sha256:<the digest printed above>

Acceptance check

docker compose ps shows the service as healthy.
docker compose exec api whoami prints app, not root.
docker image ls acme-fraud-api:local shows an image well under 1 GB.

Security
Never COPY . . without a .dockerignore. It copies .env, your .venv, and the raw training data into an image you are about to push to a registry.

Milestone 5: Load Test with Locust and Fix the Bottleneck

This milestone puts the v1 service under steady load three times, records p50, p95, error rate, CPU and memory, makes one fix, and repeats the same measurement with identical settings so the comparison is fair.

Why percentiles, and what kind of load this is

An average hides the slow tail. p50 is what a typical request feels. p95 is what 1 in 20 users feels, and it is where queueing shows up first. Error rate tells you whether "fast" is only fast because requests are failing.

This test is closed-loop: 20 simulated users each send a request, wait for the answer, pause 100 ms, and send the next. A faster server therefore receives more requests per second. That is fine for comparing two versions under identical settings, but it is not a fixed arrival rate. To ask "what happens at exactly 150 requests per second?" you need an open-loop, arrival-rate workload (k6's constant-arrival-rate executor, for example) and a load generator that can sustain it.

Step 1: Write the Locust file, a benchmark script and a summary script

loadtest/locustfile.py:

Python

"""Locust load test: each simulated user sends a random valid payment."""
import random
 
from locust import HttpUser, constant, task
 
CATEGORIES = ["grocery", "electronics", "travel", "fuel", "fashion", "gaming"]
 
 
class PaymentUser(HttpUser):
    wait_time = constant(0.1)   # each user pauses 100 ms between requests
 
    @task
    def predict(self):
        payload = {
            "txn_id": f"lt_{random.randint(0, 10**9)}",
            "amount": round(random.lognormvariate(3.5, 1.0), 2),
            "hour": random.randint(0, 23),
            "merchant_category": random.choice(CATEGORIES),
            "country_mismatch": random.choice([0, 0, 0, 1]),
            "card_age_days": random.randint(1, 3650),
            "txns_last_24h": random.randint(0, 10),
        }
        # name= groups every call under one row in the report
        with self.client.post("/predict", json=payload, name="/predict",
                              catch_response=True) as r:
            if r.status_code != 200:
                r.failure(f"status {r.status_code}")

loadtest/bench.sh starts a fresh server per run, throws away a 10-second warm-up, measures 45 seconds, records server CPU and memory, and kills exactly the server it started (by PID):

Bash

#!/usr/bin/env bash
# Benchmark the API 3 times with identical settings. Usage: loadtest/bench.sh <label> [app module]
# Each run: start a fresh server, 10 s warm-up (thrown away), 45 s measured run.
set -euo pipefail
LABEL="$1"; APP="${2:-acme_fraud_api.app:app}"
USERS=20; SPAWN=10; WARMUP=10s; DURATION=45s; RUNS=3
export PYTHONPATH=src OMP_NUM_THREADS=1
mkdir -p reports
for run in $(seq 1 "$RUNS"); do
  uvicorn "$APP" --port 8000 --workers 1 --log-level warning 2> "reports/${LABEL}_server_run${run}.log" &
  PID=$!                                                    # kill exactly this process later
  until curl -sf localhost:8000/health >/dev/null; do sleep 0.5; done
  locust -f loadtest/locustfile.py --headless -u $USERS -r $SPAWN -t $WARMUP \
    --host http://127.0.0.1:8000 --only-summary >/dev/null 2>&1          # warm-up, discarded
  LINUX=0; [ -r "/proc/$PID/stat" ] && LINUX=1      # CPU sampling needs Linux /proc
  if [ "$LINUX" = 1 ]; then CPU0=$(awk '{print $14+$15}' "/proc/$PID/stat"); T0=$(date +%s.%N); fi
  locust -f loadtest/locustfile.py --headless -u $USERS -r $SPAWN -t $DURATION \
    --host http://127.0.0.1:8000 --csv "reports/${LABEL}_run${run}" --only-summary >/dev/null 2>&1
  CPU="n/a (use docker stats or Activity Monitor)"
  if [ "$LINUX" = 1 ]; then
    CPU1=$(awk '{print $14+$15}' "/proc/$PID/stat"); T1=$(date +%s.%N)
    CPU=$(printf "%.0f%%" "$(echo "($CPU1-$CPU0)/$(getconf CLK_TCK)/($T1-$T0)*100" | bc -l)")
  fi
  RSS=$(( $(ps -o rss= -p "$PID") / 1024 ))                 # works on Linux and macOS
  WARN=$(grep -c UserWarning "reports/${LABEL}_server_run${run}.log" || true)
  echo "${LABEL} run${run}: server cpu=${CPU} rss=${RSS}MB workers=1 sklearn_warnings=${WARN}"
  kill "$PID"; wait "$PID" 2>/dev/null || true
done

loadtest/summary.py:

Python

"""Print p50, p95, error rate and throughput from Locust --csv runs,
then the median p95 (and range) for each label.
 
Usage: python loadtest/summary.py reports/before_run1 reports/before_run2 ...
"""
import csv
import statistics
import sys
from collections import defaultdict
 
by_label = defaultdict(list)
print(f"{'run':<14}{'requests':>10}{'fails':>7}{'p50 ms':>8}{'p95 ms':>8}{'errors':>9}{'req/s':>8}")
for prefix in sys.argv[1:]:
    with open(f"{prefix}_stats.csv") as f:
        row = next(r for r in csv.DictReader(f) if r["Name"] == "Aggregated")
    total, fails = int(row["Request Count"]), int(row["Failure Count"])
    err = 100 * fails / total if total else 0.0
    name = prefix.split("/")[-1]
    print(f"{name:<14}{total:>10}{fails:>7}{row['50%']:>8}{row['95%']:>8}"
          f"{err:>8.2f}%{float(row['Requests/s']):>8.1f}")
    by_label[name.split("_run")[0]].append(float(row["95%"]))
 
print()
for label, p95s in by_label.items():
    print(f"{label:<14} p95 median {statistics.median(p95s):.0f} ms "
          f"(range {min(p95s):.0f} to {max(p95s):.0f}, {len(p95s)} runs)")

Bash

chmod +x loadtest/bench.sh

Tip
Locust and the API share your CPU in this setup, which inflates latency for both versions. That is acceptable for a before and after comparison on the same machine. For absolute numbers, run Locust from a second machine.

Step 2: Measure v1 (before)

With app.py still at v1:

Bash

./loadtest/bench.sh before

Expected output (real run, 2 vCPU sandbox):

TEXT

before run1: server cpu=94% rss=192MB workers=1 sklearn_warnings=0
before run2: server cpu=95% rss=188MB workers=1 sklearn_warnings=0
before run3: server cpu=95% rss=187MB workers=1 sklearn_warnings=0

The server process is pinned near 100 percent of one core while serving about 72 requests per second. Something is expensive per request.

Step 3: Make the fix

The v1 handler calls load_model() on every request: it reads a 170 KB file, hashes it, and unpickles a whole scikit-learn pipeline, every time. The fix is two lines: load in the lifespan hook, once per process, before traffic arrives.

DIFF

 @asynccontextmanager
 async def lifespan(app: FastAPI):
+    load_model()   # v2 FIX: load once per worker process, before any traffic arrives
     yield
     STATE.clear()
@@
 def predict(payment: Payment) -> Score:   # plain "def": FastAPI runs it in a thread pool
-    load_model()   # v1: reload the model on every request (we measure this later)
     meta = STATE["meta"]

Also change the first line of the docstring to v2: load the model once. The full v2 file is:

Python

"""FastAPI service for the acme-fraud model (v2: load the model once)."""
import hashlib
import json
import os
import threading
from contextlib import asynccontextmanager
from pathlib import Path
from typing import Literal
 
import joblib
import pandas as pd
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
 
MODEL_DIR = Path(os.environ.get("MODEL_DIR", "model"))
STATE: dict = {}                 # holds the loaded model and its metadata
PREDICT_LOCK = threading.Lock()  # serialises predictions in this process (see Milestone 5)
 
Category = Literal["grocery", "electronics", "travel", "fuel", "fashion", "gaming"]
 
 
# ---- Request and response contracts: bad input gets a 422, never reaches the model ----
class Payment(BaseModel):
    txn_id: str = Field(min_length=1, max_length=64)
    amount: float = Field(gt=0, le=1_000_000)
    hour: int = Field(ge=0, le=23)
    merchant_category: Category
    country_mismatch: Literal[0, 1]
    card_age_days: int = Field(ge=0, le=36_500)
    txns_last_24h: int = Field(ge=0, le=1_000)
 
 
class Score(BaseModel):
    txn_id: str
    fraud_probability: float
    is_fraud: bool
    threshold: float
    model_version: str
 
 
def load_model() -> None:
    """Check the model file matches the hash in its metadata, then load it."""
    meta = json.loads((MODEL_DIR / "metadata.json").read_text())
    digest = hashlib.sha256((MODEL_DIR / "model.joblib").read_bytes()).hexdigest()
    if digest != meta["model_sha256"]:
        # A consistency check: catches a swapped or corrupted file. It is NOT proof of
        # origin: someone who can replace both files can replace the hash too.
        raise RuntimeError(f"model sha256 {digest[:12]} != metadata {meta['model_sha256'][:12]}")
    STATE["model"] = joblib.load(MODEL_DIR / "model.joblib")
    STATE["meta"] = meta
 
 
@asynccontextmanager
async def lifespan(app: FastAPI):
    load_model()   # v2 FIX: load once per worker process, before any traffic arrives
    yield
    STATE.clear()
 
 
app = FastAPI(title="acme-fraud-api", lifespan=lifespan)
 
 
@app.get("/health")
def health() -> dict:
    """Liveness: the process is up. Says nothing about the model."""
    return {"status": "ok"}
 
 
@app.get("/ready")
def ready() -> dict:
    """Readiness: the model is loaded and can answer."""
    if "model" not in STATE:
        raise HTTPException(status_code=503, detail="model not loaded")
    return {"status": "ready", "model_version": STATE["meta"]["model_sha256"][:12]}
 
 
@app.post("/predict", response_model=Score)
def predict(payment: Payment) -> Score:   # plain "def": FastAPI runs it in a thread pool
    meta = STATE["meta"]
    row = pd.DataFrame([payment.model_dump()])[meta["features"]]  # same column order as training
    with PREDICT_LOCK:
        proba = float(STATE["model"].predict_proba(row)[0, 1])
    return Score(
        txn_id=payment.txn_id,
        fraud_probability=round(proba, 4),
        is_fraud=proba >= meta["threshold"],   # threshold was chosen on validation, not here
        threshold=meta["threshold"],
        model_version=meta["model_sha256"][:12],
    )

Step 4: Measure v2 (after) and compare

Bash

./loadtest/bench.sh after
python loadtest/summary.py reports/before_run{1,2,3} reports/after_run{1,2,3}
pytest -q

Expected output (real run, same sandbox, same settings):

TEXT

after run1: server cpu=79% rss=177MB workers=1 sklearn_warnings=0
after run2: server cpu=79% rss=177MB workers=1 sklearn_warnings=0
after run3: server cpu=79% rss=177MB workers=1 sklearn_warnings=0
run             requests  fails  p50 ms  p95 ms   errors   req/s
before_run1         3136      0     190     290    0.00%    71.2
before_run2         3183      0     190     270    0.00%    72.3
before_run3         3220      0     180     280    0.00%    73.2
after_run1          6507      0      30      69    0.00%   147.7
after_run2          6479      0      31      66    0.00%   147.1
after_run3          6362      0      33      73    0.00%   144.4
 
before         p95 median 280 ms (range 270 to 290, 3 runs)
after          p95 median 69 ms (range 66 to 73, 3 runs)
10 passed in 1.51s

Across three runs each, median p95 fell from 280 ms to 69 ms, throughput doubled, failures stayed at zero, and the server used less CPU while doing twice the work. The run-to-run range is small compared with the change, so the improvement is not noise. Your numbers will differ by machine; the direction and rough size should not.

Step 5: Understand the lock (an experiment, not a rule)

app.py holds PREDICT_LOCK around predict_proba, so predictions inside one process run one at a time. It was added after an observation under 20 concurrent users, with the lock absent: the server log occasionally printed this warning, while predictions stayed correct.

TEXT

UserWarning: `sklearn.utils.parallel.delayed` should be used with `sklearn.utils.parallel.Parallel` to make it possible to propagate the scikit-learn configuration of the current thread to the joblib workers.

It appeared only with concurrent requests (0 times with 1 user, 19 times in one 15-second run with 20 users, before OMP_NUM_THREADS=1 and the lock were added). Our best explanation is that scikit-learn copies the calling thread's configuration and warning settings when it dispatches internal work, and under many simultaneous threads that copy is sometimes made at the wrong moment. We did not prove the mechanism, so treat it as a hypothesis.

Do not read this as "the GIL makes the lock free". The GIL does not force all inference to run one call at a time: NumPy and scikit-learn release it inside native code, so threads can overlap. A lock is a real serialisation point, and it can become the bottleneck. So measure it. Remove the two lines with PREDICT_LOCK: (de-indent the line below it), then:

Bash

./loadtest/bench.sh nolock
python loadtest/summary.py reports/after_run{1,2,3} reports/nolock_run{1,2,3}

Real run in the sandbox:

Variant (3 runs each)	p95 median	Throughput, CPU
v2 with lock	69 ms (66 to 73)	146 req/s, 79% CPU
v2 without lock	140 ms (140 to 140)	100 req/s, 59% CPU
Warnings without lock	2 in run 1, 0 in runs 2 and 3	predictions unaffected

On this machine, with this model, the lock made the service faster, most likely because 20 threads fighting over one interpreter cost more than queueing politely. On a different machine or a heavier model the result can flip. That is the lesson: keep the lock only because you measured it, and re-measure when anything changes. Put the lock back before you continue.

To use more CPU cores, compare --workers 2 under the same benchmark. Each worker is a separate process with its own copy of the model (about 180 MB of memory each here), so more workers are not automatically better on a small instance.

Acceptance check

reports/before_run*_stats.csv and reports/after_run*_stats.csv are committed.
summary.py shows a lower median p95 after the fix, with ranges that do not overlap, and 0 failures in every run.
/ready now returns 200 immediately after startup, and pytest -q shows 10 passed.

Common Mistake
Comparing a 30-second run with 10 users against a 45-second run with 20 users, or including the first seconds of a cold server. Change one thing at a time, warm up first, and keep users, spawn rate, duration and machine identical.

Milestone 6: Add CI with GitHub Actions

This milestone makes every pull request train the model, run the gate, run the tests and build the image, with a read-only token and no secrets.

Why CI trains instead of downloading a model

The gate only protects you if it runs on every change. Training here takes about 5 seconds, so CI can retrain from scratch and fail the pull request when the gate fails. Bigger models later move this to scheduled pipelines with tracked artifacts.

Step 1: Write the workflow

.github/workflows/ci.yml:

YAML

name: ci
on:
  pull_request:
  push:
    branches: [main]
 
permissions:
  contents: read        # read-only token; this workflow needs no secrets
 
jobs:
  test-and-build:
    runs-on: ubuntu-24.04
    steps:
      # Actions pinned by full commit SHA (checkout v7.0.1, setup-python v7.1.0)
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
      - uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0
        with:
          python-version: "3.12"
          cache: pip
      - run: pip install -r requirements-dev.txt
      - name: Train (fails the job if the validation gate fails)
        run: python -m acme_fraud_api.train
        env:
          PYTHONPATH: src
      - run: pytest -q
      - name: Build the image (not pushed)
        run: docker build -t acme-fraud-api:ci .

Step 2: Push and open a pull request

Bash

git add . && git commit -m "acme-fraud-api: train, gate, serve, containerise"
git remote add origin git@github.com:<you>/acme-fraud-api.git
git push -u origin main
git switch -c try-ci && git commit --allow-empty -m "trigger CI" && git push -u origin try-ci

Open the pull request and watch the ci check. (Not run for this guide: no GitHub access from the sandbox. The train and pytest steps ran locally and passed. The docker build step was not run.)

Acceptance check

The ci check is green on the pull request.
Actions are referenced by 40-character commit SHA, not @v7.

Security
This workflow gets contents: read and no secrets, which is what any workflow triggered by a pull request should get. Pushing to ECR from CI needs AWS credentials through OIDC on main only. That is covered in CI/CD and Continuous Training for ML, not here.

Milestone 7: Deploy to AWS EC2 from ECR

This milestone pushes the image to a private ECR repository tagged with the git commit, and starts one EC2 instance that pulls and runs it, with no SSH port, no stored keys, and port 8000 open to your IP only. Every resource carries a unique deployment ID, and teardown deletes only resources with that exact tag.

Cost and account check first

Security
This milestone creates billable AWS resources. Do it in one sitting and run the teardown in Milestone 8 the same day.

Accounts created on or after 15 July 2025 get USD 100 sign-up credit (up to USD 200 with onboarding tasks) on the Free plan for 6 months, or until the credits run out. AWS lists t3.small and t3.micro as free-tier eligible for these accounts, paid from the credits.
Accounts created before 15 July 2025 were on the legacy 12-month Free Tier, which has most likely expired by now. Check your account's Free Tier status and the Billing console before deploying. Do not assume t3.micro is free.
Rough on-demand cost if you pay: t3.small in ap-south-1 is about USD 0.0224 per hour, a public IPv4 address USD 0.005 per hour, plus cents for an 8 GB disk and a few hundred MB in ECR. A 3-hour session is roughly USD 0.10. Prices checked 5 Nov 2026; confirm in the AWS Pricing Calculator.
Set an AWS Budgets alert at USD 1 before you start.
No AWS account? Skip to Milestone 8 Step 1 and run the same load test against docker compose. That is the supported free fallback.

Security
This is a temporary learning deployment. Port 8000 is plain HTTP and the API has no authentication. Do not send real payment data or anything sensitive to it. Production needs TLS and an authentication and authorisation layer in front of the service.

How the deploy works

sequenceDiagram participant Dev as Your laptop participant ECR as ECR repo participant EC2 as EC2 instance Dev->>ECR: push image:git-sha Dev->>EC2: run-instances + user data EC2->>EC2: install Docker EC2->>ECR: pull via instance role EC2->>EC2: docker run :8000 loop every 10 s Dev->>EC2: GET /ready end EC2-->>Dev: 200 ready

The script does not trust "instance running". An instance can be running for minutes while Docker installs and the image pulls. It only reports success when /ready answers 200 from your laptop.

Step 1: Configure the AWS CLI and pick the profile

Bash

aws configure                         # access keys, or:
aws sso login --profile <yours>
export AWS_PROFILE=<yours>            # both scripts use whatever profile this names
export AWS_REGION=ap-south-1
aws sts get-caller-identity           # check this is the account you expect
aws ec2 describe-vpcs --filters Name=is-default,Values=true --query 'Vpcs[0].VpcId' --output text

The last command must print a VPC ID, not None. The script needs a default VPC and stops with a clear error if the region has none.

Step 2: Write the boot script

deploy/user_data.sh:

Bash

#!/bin/bash
# Runs once as root when the EC2 instance first boots.
# __IMAGE__ and __REGION__ are filled in by aws_up.sh.
set -euxo pipefail
dnf install -y docker                      # Amazon Linux 2023 ships the AWS CLI already
systemctl enable --now docker
IMAGE="__IMAGE__"
REGISTRY="${IMAGE%%/*}"                     # the part before the first "/"
# The instance role (not a stored key) lets us pull from ECR.
aws ecr get-login-password --region "__REGION__" | docker login --username AWS --password-stdin "$REGISTRY"
docker run -d --name acme-fraud-api --restart unless-stopped \
  -p 8000:8000 --memory 1g "$IMAGE"

Step 3: Write the deploy script

deploy/aws_up.sh:

Bash

#!/usr/bin/env bash
# Build, push and run acme-fraud-api on one EC2 instance. Run from the repo root.
# Every resource gets a unique deployment ID and is tagged with it.
# aws_down.sh deletes ONLY resources carrying that exact tag.
set -euo pipefail
 
REGION="${AWS_REGION:-ap-south-1}"
TYPE="${INSTANCE_TYPE:-t3.small}"
ID_FILE="deploy/.deploy-id"            # gitignored; keeps the ID between up and down
 
# 0. Who am I? (uses AWS_PROFILE if you exported one)
echo "aws identity: $(aws sts get-caller-identity --query Arn --output text)  region: ${REGION}"
 
# 1. Deploy only committed code, so the image tag points at a real source tree.
if [ -n "$(git status --porcelain)" ]; then
  echo "ERROR: commit or discard all changes (including untracked files) before deploying."
  git status --short; exit 1
fi
TAG=$(git rev-parse --short=12 HEAD)
 
# 2. Unique deployment ID, reused on reruns of this same deployment.
if [ -f "$ID_FILE" ]; then DEPLOY_ID=$(cat "$ID_FILE")
else DEPLOY_ID="acme-fraud-api-$(od -An -N3 -tx1 /dev/urandom | tr -d ' \n')"; echo "$DEPLOY_ID" > "$ID_FILE"; fi
ROLE="${DEPLOY_ID}-ec2"
TAGS_KV="Key=project,Value=acme-fraud-api Key=deploy-id,Value=${DEPLOY_ID}"
echo "deployment: ${DEPLOY_ID}"
 
ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
REGISTRY="${ACCOUNT}.dkr.ecr.${REGION}.amazonaws.com"
IMAGE="${REGISTRY}/${DEPLOY_ID}:${TAG}"
 
stop_if_not_ours() {   # $1 = what, $2 = deploy-id tag value found on it
  if [ "$2" != "$DEPLOY_ID" ]; then
    echo "ERROR: $1 exists but is not tagged deploy-id=${DEPLOY_ID}. Not touching it. Resolve the conflict first."
    exit 1
  fi
}
 
# 3. Private ECR repository: immutable tags, scan on push.
if aws ecr describe-repositories --repository-names "$DEPLOY_ID" --region "$REGION" >/dev/null 2>&1; then
  ARN="arn:aws:ecr:${REGION}:${ACCOUNT}:repository/${DEPLOY_ID}"
  stop_if_not_ours "ECR repo ${DEPLOY_ID}" "$(aws ecr list-tags-for-resource --resource-arn "$ARN" \
    --region "$REGION" --query "tags[?Key=='deploy-id'].Value | [0]" --output text)"
else
  # shellcheck disable=SC2086
  aws ecr create-repository --repository-name "$DEPLOY_ID" --region "$REGION" \
    --image-tag-mutability IMMUTABLE --image-scanning-configuration scanOnPush=true \
    --tags $TAGS_KV >/dev/null
fi
 
# 4. Build for x86_64 and push (skip if this exact tag is already there).
aws ecr get-login-password --region "$REGION" | docker login --username AWS --password-stdin "$REGISTRY"
if aws ecr describe-images --repository-name "$DEPLOY_ID" --image-ids imageTag="$TAG" --region "$REGION" >/dev/null 2>&1; then
  echo "tag ${TAG} already in ECR, not pushing again"
else
  docker build --platform linux/amd64 -t "$IMAGE" .
  docker push "$IMAGE"
fi
 
# 5. Instance role that can only pull from THIS repository. Every piece is checked
#    and repaired, so a half-finished earlier run is completed, not skipped.
if aws iam get-role --role-name "$ROLE" >/dev/null 2>&1; then
  stop_if_not_ours "IAM role ${ROLE}" "$(aws iam list-role-tags --role-name "$ROLE" \
    --query "Tags[?Key=='deploy-id'].Value | [0]" --output text)"
else
  # shellcheck disable=SC2086
  aws iam create-role --role-name "$ROLE" --tags $TAGS_KV \
    --assume-role-policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}' >/dev/null
fi
aws iam put-role-policy --role-name "$ROLE" --policy-name ecr-pull-one-repo --policy-document "{
  \"Version\":\"2012-10-17\",\"Statement\":[
    {\"Effect\":\"Allow\",\"Action\":\"ecr:GetAuthorizationToken\",\"Resource\":\"*\"},
    {\"Effect\":\"Allow\",\"Action\":[\"ecr:BatchGetImage\",\"ecr:GetDownloadUrlForLayer\"],
     \"Resource\":\"arn:aws:ecr:${REGION}:${ACCOUNT}:repository/${DEPLOY_ID}\"}]}"   # same call creates or updates
if aws iam get-instance-profile --instance-profile-name "$ROLE" >/dev/null 2>&1; then
  stop_if_not_ours "instance profile ${ROLE}" "$(aws iam list-instance-profile-tags --instance-profile-name "$ROLE" \
    --query "Tags[?Key=='deploy-id'].Value | [0]" --output text)"
else
  # shellcheck disable=SC2086
  aws iam create-instance-profile --instance-profile-name "$ROLE" --tags $TAGS_KV >/dev/null
fi
if [ "$(aws iam get-instance-profile --instance-profile-name "$ROLE" --query 'length(InstanceProfile.Roles)' --output text)" = "0" ]; then
  aws iam add-role-to-instance-profile --instance-profile-name "$ROLE" --role-name "$ROLE"
fi
 
# 6. Firewall: port 8000 open to your CURRENT IP only. No SSH port at all.
VPC=$(aws ec2 describe-vpcs --filters Name=is-default,Values=true --query 'Vpcs[0].VpcId' --output text --region "$REGION")
if [ "$VPC" = "None" ]; then
  echo "ERROR: no default VPC in ${REGION}. Create one (aws ec2 create-default-vpc) or pick another region."; exit 1
fi
MYIP=$(curl -s https://checkip.amazonaws.com)
SG=$(aws ec2 describe-security-groups --region "$REGION" --filters Name=group-name,Values="$DEPLOY_ID" \
      --query 'SecurityGroups[0].GroupId' --output text)
if [ "$SG" = "None" ]; then
  SG=$(aws ec2 create-security-group --group-name "$DEPLOY_ID" --description "acme-fraud-api port 8000" \
        --vpc-id "$VPC" --region "$REGION" --query GroupId --output text \
        --tag-specifications "ResourceType=security-group,Tags=[{Key=project,Value=acme-fraud-api},{Key=deploy-id,Value=${DEPLOY_ID}}]")
else
  stop_if_not_ours "security group ${DEPLOY_ID}" "$(aws ec2 describe-security-groups --group-ids "$SG" --region "$REGION" \
    --query "SecurityGroups[0].Tags[?Key=='deploy-id'].Value | [0]" --output text)"
fi
OLD_RULES=$(aws ec2 describe-security-groups --group-ids "$SG" --region "$REGION" --query 'SecurityGroups[0].IpPermissions' --output json)
if [ "$OLD_RULES" != "[]" ]; then   # drop rules for an old IP, then add the current one
  aws ec2 revoke-security-group-ingress --group-id "$SG" --ip-permissions "$OLD_RULES" --region "$REGION" >/dev/null
fi
aws ec2 authorize-security-group-ingress --group-id "$SG" --protocol tcp --port 8000 \
  --cidr "${MYIP}/32" --region "$REGION" >/dev/null
echo "port 8000 open to ${MYIP}/32 only"
 
# 7. One instance per deployment. If one is already up, stop here.
RUNNING=$(aws ec2 describe-instances --region "$REGION" \
  --filters Name=tag:deploy-id,Values="$DEPLOY_ID" Name=instance-state-name,Values=pending,running,stopping,stopped \
  --query 'Reservations[].Instances[].InstanceId' --output text)
if [ -n "$RUNNING" ]; then echo "instance already exists for ${DEPLOY_ID}: ${RUNNING}. Run deploy/aws_down.sh first."; exit 1; fi
 
# 8. Launch the latest Amazon Linux 2023. Retry while the new instance profile propagates.
AMI=$(aws ssm get-parameter --name /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
       --query Parameter.Value --output text --region "$REGION")
sed -e "s|__IMAGE__|${IMAGE}|" -e "s|__REGION__|${REGION}|" deploy/user_data.sh > /tmp/acme_user_data.sh
ID=""
for attempt in 1 2 3 4 5 6; do
  if ID=$(aws ec2 run-instances --region "$REGION" --image-id "$AMI" --instance-type "$TYPE" \
        --security-group-ids "$SG" --iam-instance-profile Name="$ROLE" \
        --metadata-options HttpTokens=required \
        --user-data file:///tmp/acme_user_data.sh \
        --tag-specifications "ResourceType=instance,Tags=[{Key=Name,Value=${DEPLOY_ID}},{Key=project,Value=acme-fraud-api},{Key=deploy-id,Value=${DEPLOY_ID}}]" \
                             "ResourceType=volume,Tags=[{Key=project,Value=acme-fraud-api},{Key=deploy-id,Value=${DEPLOY_ID}}]" \
        --query 'Instances[0].InstanceId' --output text 2>/tmp/acme_run_err); then break; fi
  echo "launch attempt ${attempt} failed: $(tail -1 /tmp/acme_run_err)"; ID=""; sleep 10
done
[ -n "$ID" ] || { echo "could not launch instance"; exit 1; }
aws ec2 wait instance-running --instance-ids "$ID" --region "$REGION"
IP=$(aws ec2 describe-instances --instance-ids "$ID" --region "$REGION" \
      --query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
echo "instance ${ID} at ${IP}"
 
# 9. Do not trust "running": wait until /ready answers (Docker install + pull takes minutes).
for _ in $(seq 1 40); do
  if curl -sf "http://${IP}:8000/ready"; then echo; echo "DEPLOYED: http://${IP}:8000"; exit 0; fi
  sleep 10
done
echo "not ready after ~7 minutes. Check: aws ec2 get-console-output --instance-id ${ID} --latest --output text --region ${REGION}"
exit 1

What each choice protects against:

Choice	Protects against	Where
Clean tree, tag = 12-char commit	Image from uncommitted files	step 1
Unique `deploy-id` tag on everything	Touching resources you did not create	steps 2 to 8
Stop if a name exists without our tag	Name clashes in a shared account	`stop_if_not_ours`
Policy, profile, rule re-applied every run	Half-finished earlier runs, changed IP	steps 5 and 6
Role scoped to one repository	Leaked role reading all images	`put-role-policy`
Port 8000 to your /32, no SSH	Internet scanners	step 6
`HttpTokens=required`	SSRF stealing role credentials	`--metadata-options`
Poll `/ready`, not state	False "deployed"	step 9

Step 4: Deploy

Commit first. The script refuses to run with any uncommitted or untracked file, because an image built from such a tree cannot be traced back to a commit.

Bash

echo "deploy/.deploy-id" >> .gitignore
git add .gitignore src/acme_fraud_api/app.py reports/
git commit -m "v2: load model once"
git status --short                     # must print nothing
chmod +x deploy/*.sh
./deploy/aws_up.sh

Expected output shape (not run for this guide; the IDs and IP will be yours):

Bash

aws identity: arn:aws:sts::123456789012:assumed-role/...  region: ap-south-1
deployment: acme-fraud-api-3fa9c1
port 8000 open to 203.0.113.7/32 only
instance i-0abc123def4567890 at 13.233.xx.xx
{"status":"ready","model_version":"e70c4a9d2e04"}
DEPLOYED: http://13.233.xx.xx:8000

Check that the cloud is serving the exact model you recorded in Milestone 2:

Bash

curl -s http://<IP>:8000/ready
python -c "import json; print(json.load(open('reports/test_report.json'))['model_sha256'][:12])"

Both must show the same 12-character prefix.

Acceptance check

/ready on the public IP returns the same model_version as reports/test_report.json.
curl from another network (your phone's hotspot) times out, because port 8000 is open only to your IP.
Every resource in the console carries the tag deploy-id with your ID.

Common Mistake
Building on an Apple Silicon Mac without --platform linux/amd64. The image is ARM, the instance is x86_64, and the container exits with exec format error. Read it with aws ec2 get-console-output --instance-id <id> --latest --output text.

Milestone 8: Load Test in the Cloud and Tear Down

This milestone records the same numbers against the live endpoint, then deletes every resource tagged with your deployment ID and proves nothing is left.

Step 1: Run the load test

Bash

locust -f loadtest/locustfile.py --headless -u 20 -r 10 -t 10s --host http://<IP>:8000 --only-summary   # warm-up, ignore
for run in 1 2 3; do
  locust -f loadtest/locustfile.py --headless -u 20 -r 10 -t 45s \
    --host http://<IP>:8000 --csv reports/aws_run$run --only-summary
done
python loadtest/summary.py reports/aws_run1 reports/aws_run2 reports/aws_run3

No AWS? Run docker compose up -d and point the same loop at http://127.0.0.1:8000, saving as reports/compose_run*.

Expect cloud p50 to be higher than your laptop run, because every request now crosses the internet. For a rough network baseline, time a few /health calls, which do almost no server work:

Bash

for i in 1 2 3 4 5; do curl -o /dev/null -s -w "%{time_connect} %{time_total}\n" http://<IP>:8000/health; done

This is an approximation, not a precise split of network time and server time: it is one tiny request at a time, with no load.

Step 2: Fill in your results table

Run (3 x 45 s each)	p95 ms, median (range)	Error rate
Before fix (sandbox)	280 (270 to 290)	0.00%
After fix (sandbox)	69 (66 to 73)	0.00%
After fix on EC2	yours	yours

Step 3: Tear everything down

deploy/aws_down.sh:

Bash

#!/usr/bin/env bash
# Delete ONLY the resources tagged with this deployment's ID. Safe to run more than once.
# Usage: deploy/aws_down.sh            (reads deploy/.deploy-id)
#        deploy/aws_down.sh <deploy-id>
set -uo pipefail
REGION="${AWS_REGION:-ap-south-1}"
ID_FILE="deploy/.deploy-id"
DEPLOY_ID="${1:-$(cat "$ID_FILE" 2>/dev/null || true)}"
case "$DEPLOY_ID" in
  acme-fraud-api-*) ;;                       # only IDs that aws_up.sh could have made
  *) echo "ERROR: no valid deployment ID (got '${DEPLOY_ID}')."; exit 1 ;;
esac
ROLE="${DEPLOY_ID}-ec2"
ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
echo "tearing down ${DEPLOY_ID} in ${REGION} (account ${ACCOUNT})"
 
# 1. Instances carrying our tag.
IDS=$(aws ec2 describe-instances --region "$REGION" \
  --filters Name=tag:deploy-id,Values="$DEPLOY_ID" Name=instance-state-name,Values=pending,running,stopping,stopped \
  --query 'Reservations[].Instances[].InstanceId' --output text)
if [ -n "$IDS" ]; then
  # shellcheck disable=SC2086
  aws ec2 terminate-instances --instance-ids $IDS --region "$REGION" >/dev/null
  # shellcheck disable=SC2086
  aws ec2 wait instance-terminated --instance-ids $IDS --region "$REGION"
  echo "terminated: $IDS"
fi
 
# 2. Security group carrying our tag (deletable only after the instance is gone).
for SG in $(aws ec2 describe-security-groups --region "$REGION" --filters Name=tag:deploy-id,Values="$DEPLOY_ID" \
             --query 'SecurityGroups[].GroupId' --output text); do
  aws ec2 delete-security-group --group-id "$SG" --region "$REGION" && echo "deleted security group $SG"
done
 
# 3. Instance profile and role, each only if tagged with our ID.
if [ "$(aws iam list-instance-profile-tags --instance-profile-name "$ROLE" \
        --query "Tags[?Key=='deploy-id'].Value | [0]" --output text 2>/dev/null)" = "$DEPLOY_ID" ]; then
  aws iam remove-role-from-instance-profile --instance-profile-name "$ROLE" --role-name "$ROLE" 2>/dev/null
  aws iam delete-instance-profile --instance-profile-name "$ROLE" && echo "deleted instance profile $ROLE"
fi
if [ "$(aws iam list-role-tags --role-name "$ROLE" \
        --query "Tags[?Key=='deploy-id'].Value | [0]" --output text 2>/dev/null)" = "$DEPLOY_ID" ]; then
  aws iam delete-role-policy --role-name "$ROLE" --policy-name ecr-pull-one-repo 2>/dev/null
  aws iam delete-role --role-name "$ROLE" && echo "deleted role $ROLE"
fi
 
# 4. ECR repository (and its images), only if tagged with our ID.
ARN="arn:aws:ecr:${REGION}:${ACCOUNT}:repository/${DEPLOY_ID}"
if [ "$(aws ecr list-tags-for-resource --resource-arn "$ARN" --region "$REGION" \
        --query "tags[?Key=='deploy-id'].Value | [0]" --output text 2>/dev/null)" = "$DEPLOY_ID" ]; then
  aws ecr delete-repository --repository-name "$DEPLOY_ID" --force --region "$REGION" >/dev/null && echo "deleted ecr repo $DEPLOY_ID"
fi
 
# 5. Prove it. Every line must say "none".
echo "--- leftovers check for ${DEPLOY_ID} ---"
LEFT=0
check() { if [ -n "$2" ] && [ "$2" != "None" ]; then echo "$1: STILL THERE ($2)"; LEFT=1; else echo "$1: none"; fi; }
check instances "$(aws ec2 describe-instances --region "$REGION" --filters Name=tag:deploy-id,Values="$DEPLOY_ID" \
  Name=instance-state-name,Values=pending,running,stopping,stopped --query 'Reservations[].Instances[].InstanceId' --output text)"
check security-groups "$(aws ec2 describe-security-groups --region "$REGION" --filters Name=tag:deploy-id,Values="$DEPLOY_ID" \
  --query 'SecurityGroups[].GroupId' --output text)"
check instance-profile "$(aws iam get-instance-profile --instance-profile-name "$ROLE" --query InstanceProfile.InstanceProfileName --output text 2>/dev/null)"
check role "$(aws iam get-role --role-name "$ROLE" --query Role.RoleName --output text 2>/dev/null)"
check ecr-repo "$(aws ecr describe-repositories --repository-names "$DEPLOY_ID" --region "$REGION" --query 'repositories[0].repositoryName' --output text 2>/dev/null)"
if [ "$LEFT" = "0" ]; then rm -f "$ID_FILE"; echo "all clean"; else echo "something is left: rerun, or delete it in the console"; exit 1; fi

Bash

./deploy/aws_down.sh

Expected output shape (not run for this guide):

TEXT

tearing down acme-fraud-api-3fa9c1 in ap-south-1 (account 123456789012)
terminated: i-0abc123def4567890
deleted security group sg-0123456789abcdef0
deleted instance profile acme-fraud-api-3fa9c1-ec2
deleted role acme-fraud-api-3fa9c1-ec2
deleted ecr repo acme-fraud-api-3fa9c1
--- leftovers check for acme-fraud-api-3fa9c1 ---
instances: none
security-groups: none
instance-profile: none
role: none
ecr-repo: none
all clean

The script finds resources by the deploy-id tag and checks the tag before deleting anything, so a repository or role that merely shares a name is never touched. The ID is read from deploy/.deploy-id, which is removed only when the leftovers check is clean. If you lost that file, pass the ID shown by aws_up.sh: ./deploy/aws_down.sh acme-fraud-api-3fa9c1. Running it a second time should only print the leftovers check (this was reasoned from the code, not observed against AWS: confirm it on your run).

Acceptance check

The leftovers check prints none five times and all clean.
The next day, the Billing console shows no running EC2 or ECR charges for this project.

Common Mistake
Stopping the instance instead of terminating it. A stopped instance still bills for its disk, and an allocated Elastic IP bills while unattached.

Break It on Purpose

These three exercises break the real files the pipeline reads, so you see each guard fire with your own eyes. Each one ran for this guide and the output is real.

Step 1: Raise the gate and watch training refuse to ship

Bash

sha256sum model/model.joblib | cut -c1-12
sed -i 's/min_val_pr_auc: 0.20/min_val_pr_auc: 0.40/' params.yaml
python -m acme_fraud_api.train; echo "exit=$?"
sha256sum model/model.joblib | cut -c1-12
sed -i 's/min_val_pr_auc: 0.40/min_val_pr_auc: 0.20/' params.yaml    # put it back

Expected output:

TEXT

e70c4a9d2e04
val_pr_auc=0.2322 threshold=0.47 val_recall=0.7500 val_precision=0.1057
GATE FAILED: val_pr_auc 0.2322 < floor 0.4. Nothing saved.
exit=1
e70c4a9d2e04

The old model is untouched. In CI, exit 1 fails the pull request. (On macOS use sed -i ''.)

Step 2: Tamper with the model file and watch the API refuse to start

Bash

cp model/model.joblib /tmp/good.joblib
printf 'x' >> model/model.joblib                 # one extra byte
uvicorn acme_fraud_api.app:app --port 8001

Expected output (last lines):

TEXT

RuntimeError: model sha256 9f49bafd0788 != metadata e70c4a9d2e04
ERROR:    Application startup failed. Exiting.

A service that will not start is loud and safe. A service that serves a file that does not match its metadata is quiet and dangerous. (Replace both files together and this check passes: it guards consistency, not origin.) Restore with cp /tmp/good.joblib model/model.joblib and confirm /ready returns e70c4a9d2e04 again.

Step 3: Try to peek at the test set a second time

Bash

python -m acme_fraud_api.evaluate_test; echo "exit=$?"

Expected output:

TEXT

REFUSED: reports/test_report.json already exists. The test set is looked at once.
exit=3

Clean up after breaking

Bash

git status            # params.yaml and model/ should show no changes
python -m acme_fraud_api.train && pytest -q

Validation & Testing

Run this from a fresh clone to prove the whole project works end to end. Everything here is free and local.

Step 1: Rebuild from nothing

Bash

rm -rf data model
python -m acme_fraud_api.train          # same sha256 as your committed test report
pytest -q                               # 10 passed
docker compose up -d --build && docker compose ps
python loadtest/summary.py reports/before_run{1,2,3} reports/after_run{1,2,3}
docker compose down

Definition of done

One command trains, and two runs give the same model hash.
The gate floor was set before training and exits 1 when not met.
reports/test_report.json is committed, and a second test run exits 3.
The API rejects bad input with 422 and refuses a model file that does not match its metadata.
The image runs as a non-root user, with the base image pinned by digest.
CI trains, tests and builds on every pull request with no secrets and SHA-pinned actions.
Three before and three after load test runs are committed, with median p95 (and range), error rate, CPU and memory in the README.
The EC2 /ready showed the same model hash as the test report (or the Compose fallback was used and the README says so).
aws_down.sh ran and the leftovers check printed all clean.

Write the README

Your README is what a hiring manager reads. Keep it to five parts: what it does (one sentence and the diagram), how to run it (three commands), the results table from Milestone 8 labelled with machine, run count and test type (closed-loop, 20 users), the one fix and why it worked, and what you would do next (model registry, autoscaling, HTTPS and authentication behind a load balancer). Say plainly what you did not run, if anything.

Present This Project in an Interview

The 60-second version

"I took a fraud model from a training script to a deployed API on AWS. Training is driven by one params file and gated: the threshold is picked on validation, and if validation PR-AUC misses a floor I set before training, nothing is saved. The test set is scored once by a script that refuses a second run, and that record is committed. The model is served with FastAPI, with strict input validation, and the service checks the model file against the hash in its metadata before loading it. I packaged it in Docker as a non-root image, pushed it to ECR tagged with the git commit, and ran it on EC2 using an instance role scoped to that one repository, with no SSH. Under a Locust load test, three runs each, the median p95 was 280 ms. The cause was reloading the model on every request. Loading it once at startup took the median p95 to 69 ms with zero errors under identical settings."

Questions to expect

"Why is your test PR-AUC lower than validation?" Small sample: 120 fraud rows in test. I report both with row counts and did not retrain to chase it.
"How would you roll back?" Every image is tagged by a commit from a clean tree and tags are immutable, so rollback is starting the previous tag. The model ships inside the image, so code and model roll back together.
"What breaks at 10 times the traffic?" One worker process tops out near one core. I would benchmark --workers 2 (each worker holds its own model copy, about 180 MB here), then run several instances behind a load balancer with /ready as the health check. I would also switch to an arrival-rate load test, because my closed-loop test cannot say what happens at a fixed request rate.
"Why the lock around predict?" I measured it: on my machine the version with the lock had half the p95 of the version without. It is a measured choice for this model and machine, not a rule, and I would re-measure on new hardware.
"What is missing for production?" HTTPS, authentication, a model registry instead of baking the model in, monitoring for latency and prediction drift, and a deploy pipeline instead of a laptop script. Those are the next steps of the roadmap.

What to show on screen

The summary.py table with three before and three after runs.
The gate failing with exit 1 and the model hash unchanged.
/ready on EC2 showing the same hash as test_report.json.

Found a mistake, an outdated step or a broken link?

Deploy an ML Model with FastAPI, Docker and AWS

Blueprint Walkthrough

Architecture Overview & Problem Statement

The problem

What you build

Repository layout

Prerequisites (by module title)

Versions used

Milestone 1: Set Up the Repository and Train a Gated Model

Why a gate before anything else

Step 1: Create the project and environment

Step 2: Put every knob in params.yaml

Step 3: Write the data module

Step 4: Write the training script with the gate

Step 5: Train twice and prove it is reproducible

Acceptance check

Milestone 2: Score the Test Set Once and Keep the Record

Why "once" needs code, not willpower

Step 1: Write the test-once script

Step 2: Prove a crash does not use up your one look

Step 3: Run it once, then try again

Step 4: Read the number honestly

Acceptance check

Milestone 3: Serve the Model with FastAPI

Why validation and a hash check

Step 1: Write the first version of the API

Step 2: Run it and call it

Step 3: Add tests

Acceptance check

Milestone 4: Containerise with Docker and Compose

Why bake the model into the image

Step 1: Write the Dockerfile and .dockerignore

Step 2: Add the Compose file

Step 3: Build, run and check

Step 4: Pin the base image by digest

Acceptance check

Milestone 5: Load Test with Locust and Fix the Bottleneck

Why percentiles, and what kind of load this is

Step 1: Write the Locust file, a benchmark script and a summary script

Step 2: Measure v1 (before)

Step 3: Make the fix

Step 4: Measure v2 (after) and compare

Step 5: Understand the lock (an experiment, not a rule)

Acceptance check

Milestone 6: Add CI with GitHub Actions

Why CI trains instead of downloading a model

Step 1: Write the workflow

Step 2: Push and open a pull request

Acceptance check

Milestone 7: Deploy to AWS EC2 from ECR

Cost and account check first

How the deploy works

Step 1: Configure the AWS CLI and pick the profile

Step 2: Write the boot script

Step 3: Write the deploy script

Step 4: Deploy

Acceptance check

Milestone 8: Load Test in the Cloud and Tear Down

Step 1: Run the load test

Step 2: Fill in your results table

Step 3: Tear everything down

Acceptance check

Break It on Purpose

Step 1: Raise the gate and watch training refuse to ship

Step 2: Tamper with the model file and watch the API refuse to start

Step 3: Try to peek at the test set a second time

Clean up after breaking

Validation & Testing

Step 1: Rebuild from nothing

Definition of done

Write the README

Present This Project in an Interview

The 60-second version

Questions to expect

What to show on screen