A Razorpay risk team ships a fraud model that flags 99.8% of transactions as "not fraud" and calls it 99.8% accurate. Leadership is thrilled for exactly one week, until the fraud losses keep climbing and someone finally checks how many actual fraud cases the model caught. The number is close to zero. The model was never wrong on paper. It was wrong on the metric. It learned that predicting "not fraud" for everyone is a nearly perfect strategy when 998 out of 1000 transactions really are legitimate. This is the first trap every AI engineer walks into when they assume a fancy model, or worse, an LLM, is the answer to a problem that a well-evaluated classical model would have solved in an afternoon. You are not being trained to become a machine learning researcher. You will not be deriving new algorithms or publishing papers. But every AI engineer eventually hits a task that is structured, tabular, and repetitive, like predicting churn, scoring credit risk, or flagging fraud, and reaching for an LLM there is slower, more expensive, and often less accurate than a properly evaluated XGBoost model. This module gives you the working knowledge to make that call correctly. ### Where this fits in the bigger system you are building Later in this path you will build RAG pipelines and autonomous agents. Those systems still need classical ML underneath them constantly: a reranker scoring retrieved chunks, a classifier routing a support ticket before an LLM ever sees it, an anomaly detector flagging unusual agent behavior. Classical ML is not a competing skill to LLM engineering, it is a tool sitting right next to it in the same toolbox.
Before picking an algorithm, you need to know what kind of question you are even asking. Most classical ML problems you will encounter fall into one of two broad buckets, and picking the wrong bucket wastes days. **Supervised learning** is learning from labeled examples. You show the model past house prices along with their square footage, location, and age, and it learns to predict the price of a new house. The model always has a "right answer" to check itself against during training. **Unsupervised learning** is finding structure with no labels at all. You hand the model 10,000 Meesho customer purchase histories with no categories attached, and it groups similar customers together on its own. There is no "right answer" to check against, only patterns. > 💡 **Tip:** If your dataset has a column you are trying to predict, it is supervised. If you are trying to discover groups or structure with no target column, it is unsupervised. This one question resolves almost every "which type of ML is this" confusion. ### Regression versus classification inside supervised learning Supervised learning splits again based on what you are predicting. **Regression** predicts a continuous number: a Bangalore property price, tomorrow's demand for an IRCTC train route, the expected delivery time for a Zomato order. **Classification** predicts a category: fraud or not fraud, churn or retain, spam or not spam. Even when there are more than two categories (multi-class classification, like routing a support ticket to one of five departments), the output is still a label, not a number. > 📌 **Engineering Decision:** If you are predicting a continuous numerical value, you are solving a regression problem. If you are predicting a discrete category or class, you are solving a classification problem. Get this right before touching any code, because the wrong choice here breaks every metric downstream.
Regression models learn a relationship between input features and a continuous output. The simplest version, **linear regression**, fits a straight line (or a flat plane in higher dimensions) through your data that minimizes the distance between the line and every actual data point. ```python from sklearn.linear_model import LinearRegression from sklearn.model_selection import train_test_split import pandas as pd ## Load Bangalore property listings with features already engineered df = pd.read_csv("bangalore_properties.csv") ## X holds the input features, y holds what we are trying to predict X = df[["sqft", "bedrooms", "age_years", "distance_to_metro_km"]] y = df["price_lakhs"] ## Hold back 20% of the data to test on data the model has never seen X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42 ) model = LinearRegression() model.fit(X_train, y_train) predictions = model.predict(X_test) ``` > **Note:** `random_state=42` fixes the randomness used to split the data, so every time you rerun this code you get the exact same split. This makes your results reproducible, which matters when you are comparing two models later. **Logistic regression**, despite the name, is a classification algorithm, not a regression one. It outputs a probability between 0 and 1, and you pick a threshold (usually 0.5) to convert that probability into a class label. It is a strong first baseline for many binary classification tasks because it is fast, interpretable, and easy to evaluate. ### Why you should always try the simple model first **Decision trees** split your data into branches based on feature thresholds, like "is square footage above 1200? If yes, is distance to metro below 2km?" A single tree is easy to interpret but prone to overfitting. **Random forests** train many decision trees on random subsets of the data and features, then average their predictions. This averaging cancels out each individual tree's tendency to overfit. **XGBoost** (gradient boosting) builds trees sequentially, where each new tree specifically corrects the mistakes of the trees before it. It is often a very strong choice for structured, tabular data and is an important baseline to know, alongside close relatives like LightGBM and CatBoost, though which one wins on a given dataset depends on the data itself. > 📌 **Engineering Decision:** Start with a simple, interpretable baseline such as linear or logistic regression when appropriate, then compare it against stronger nonlinear models like random forest or XGBoost. A model that is 2% more accurate but impossible to explain to a compliance team is often the wrong trade for a regulated use case like credit scoring.
Raw data almost never goes straight into a model. Three problems show up constantly and each has a standard fix. **Missing values.** A customer's age might be blank in 8% of rows. You can drop those rows (costly if 8% is a lot of your data), fill with the median or mean (simple, works for many cases), or use a model-based imputer that predicts the missing value from other columns (more accurate, more complex). **Categorical encoding.** Models only understand numbers. A `city` column with values like `Mumbai`, `Pune`, `Hyderabad` needs to become numeric. **One-hot encoding** creates a separate binary column per city. **Label encoding** replaces categories with numbers instead, such as `Mumbai=0`, `Pune=1`, `Hyderabad=2`. Those numbers can accidentally suggest an order that does not actually exist, so one-hot encoding is usually safer for linear and logistic regression. Tree-based models can sometimes tolerate label encoding better, since they split on thresholds rather than treating the values as a continuous scale, but one-hot or native categorical handling is still often preferable. **Feature scaling.** Some algorithms are sensitive to the size of feature values. For example, logistic regression, K-Means, and SVMs can behave poorly when one feature ranges from 0-1 while another ranges from 0-10,000,000. **Standardization** (mean 0, standard deviation 1) or **min-max scaling** (compress to a 0-1 range) fixes this. Tree-based models like random forest and XGBoost do not need scaling at all, since they only compare values within a single feature at a time. ```python from sklearn.preprocessing import StandardScaler from sklearn.compose import ColumnTransformer from sklearn.pipeline import Pipeline ## Scale numeric features, leave categorical ones for a separate encoder numeric_features = ["sqft", "age_years", "distance_to_metro_km"] preprocessor = ColumnTransformer(transformers=[ ("scale", StandardScaler(), numeric_features) ]) ## Bundling preprocessing and the model together prevents leakage, ## since fit() only ever sees training data, never the test set pipeline = Pipeline(steps=[ ("preprocessor", preprocessor), ("model", LinearRegression()) ]) pipeline.fit(X_train, y_train) ``` > 🔴 **Common Mistake:** Fitting a scaler on the entire dataset before splitting into train and test. This leaks information about the test set's distribution into training, quietly inflating your evaluation metrics. Always fit preprocessing steps only on the training set, then apply that same fit to transform the test set, exactly like the pipeline above does automatically.
A single train/test split can get lucky or unlucky depending on which rows happened to land in the test set. **Cross-validation** fixes this by splitting the data into K folds (commonly 5), training on K-1 folds, testing on the remaining fold, and repeating K times so every row gets used for testing exactly once. You then average the K scores for a much more reliable estimate of real-world performance. Fold 1: [ TEST ][ train ][ train ][ train ][ train ] Fold 2: [ train ][ TEST ][ train ][ train ][ train ] Fold 3: [ train ][ train ][ TEST ][ train ][ train ] Fold 4: [ train ][ train ][ train ][ TEST ][ train ] Fold 5: [ train ][ train ][ train ][ train ][ TEST ] Final score = average of all 5 fold scores ```python from sklearn.model_selection import cross_val_score scores = cross_val_score( pipeline, X_train, y_train, cv=5, scoring="neg_mean_absolute_error" ) ## Convert negative MAE back to positive, sklearn stores errors as ## negative so that a higher score is always "better" internally mae_per_fold = -scores print(f"Average MAE across folds: {mae_per_fold.mean():.2f} lakhs") ``` > 💡 **Tip:** For imbalanced classification, like the fraud scenario later in this module, use `StratifiedKFold` instead of plain K-Fold. It keeps roughly the same class ratio in every fold, so a fold does not accidentally end up with zero fraud cases and produce a misleading score.
**Overfitting** happens when a model memorizes the training data, including its noise, instead of learning the actual pattern. It scores very well on training data and noticeably worse on new data. **Underfitting** happens when a model is too simple to capture the real pattern at all, scoring poorly on both training and test data. | Signal | Training score | Test score | Likely cause | |:---|:---|:---|:---| | Overfitting | Very high | Much lower | Model too complex, too little data, or too many features | | Underfitting | Low | Also low | Model too simple, or missing important features | | Good fit | High | Close to training score | Model is capturing the real pattern | The gap between training and test performance is the tell, assuming your evaluation setup itself is leakage-free and the test set is genuinely representative of real data. A random forest with unlimited tree depth trained on a small Cars24 dataset will often hit 99% accuracy on training data and a noticeably lower score on test data, a gap that points to overfitting. > 🔴 **Common Mistake:** Reaching for a more complex model the moment accuracy looks disappointing, without first checking whether the current model is overfitting or underfitting. Adding complexity to an already-overfitting model makes the problem worse, not better. Check the train-versus-test gap first, every time.
A Razorpay risk team ships a fraud model that flags 99.8% of transactions as "not fraud" and calls it 99.8% accurate. Le...
Before picking an algorithm, you need to know what kind of question you are even asking. Most classical ML problems you ...
Regression models learn a relationship between input features and a continuous output. The simplest version, linear regr...
Raw data almost never goes straight into a model. Three problems show up constantly and each has a standard fix. Missing...
A single train/test split can get lucky or unlucky depending on which rows happened to land in the test set. Cross-valid...
Overfitting happens when a model memorizes the training data, including its noise, instead of learning the actual patter...
Regression metrics measure how far your predictions land from the actual values. MAE (Mean Absolute Error) is the averag...
This is the section that would have saved the Razorpay fraud model at the start of this module. Accuracy alone is danger...
Logistic regression, and most classifiers, do not output a label directly, they output a probability, and a threshold (0...
You already saw one form of leakage: fitting a scaler on the full dataset before splitting. But leakage takes other form...
Hyperparameters are settings you choose before training, like how many trees a random forest builds or how deep each tre...
Some techniques are worth recognizing on sight even though mastering them is not the goal at this stage of your path. K-...
Install the required libraries. Create inspectdataset.py to load a transaction dataset and inspect the class balance. > ...
Concept When to use it Linear/logistic regression First interpretable baseline for regression or classification Random f...
Aligns directly with DevOps, Site Reliability (SRE), and Platform Engineering job descriptions.