MLOps Projects
Projects that build production ML infrastructure — from experiment tracking to automated retraining pipelines and monitoring systems.
Beginner Projects
1. Experiment Tracker
Add MLflow tracking to an existing sklearn model: log hyperparameters, metrics (per epoch and final), and the fitted model artifact. Compare 5 runs in the MLflow UI and identify the best configuration.
What you’ll practice: mlflow.log_param, mlflow.log_metric, mlflow.sklearn.log_model, run comparison
2. Model Registry
Train 3 versions of a model (different hyperparameters), register each in MLflow Model Registry, promote the best to “Staging,” and write a script that loads the staging model for inference.
What you’ll practice: mlflow.register_model, transition_model_version_stage, MlflowClient
3. Reproducible Training Script
Convert a notebook into a reproducible CLI training script: accept hyperparameters via argparse, log the git commit hash and random seed, save the model, and produce a training summary JSON.
What you’ll practice: argparse, reproducibility patterns, git integration, artifact saving
4. Data Versioning with DVC
Version a dataset with DVC: init a DVC repo, add the dataset, push to remote storage, modify the data, create a new version, and show how to switch between versions with git checkout + dvc checkout.
What you’ll practice: dvc init, dvc add, dvc push/pull, version switching
5. Dockerized Model Server
Containerize a trained model as a Docker image: install dependencies, copy the model artifact, build a FastAPI prediction endpoint, and verify it works with docker run and a curl request.
What you’ll practice: Dockerfile, FastAPI, model loading on startup, container testing
6. Feature Store (Lightweight)
Build a simple feature store: a class that computes features from raw data, caches them to Parquet, serves them by entity ID, and tracks feature versions. Use it for both training and serving.
What you’ll practice: Feature reuse, offline/online consistency, versioned feature sets
7. Model Performance Dashboard
Build a Streamlit dashboard that: loads prediction logs, shows accuracy over time, displays a confusion matrix, visualizes prediction confidence distribution, and flags days where performance drops.
What you’ll practice: Prediction logging, Streamlit, time-series performance monitoring
8. Automated Retraining Trigger
Build a script that: checks today’s model performance vs. a baseline, triggers retraining if performance drops below a threshold, trains a new model, and only promotes it if it beats the current production model.
What you’ll practice: Automated evaluation, conditional retraining, champion/challenger logic
9. Environment Reproducibility
Create a fully reproducible ML environment: requirements.txt with pinned versions, a Makefile with train/test/serve targets, a setup.sh that creates the venv, and GitHub Actions that verifies the env builds cleanly.
What you’ll practice: Dependency management, Makefile, CI environment validation
10. Batch Inference Pipeline
Build a batch inference pipeline: load a trained model, process input CSV in chunks (handle files too large for memory), write predictions with confidence scores to output CSV, and log statistics.
What you’ll practice: Chunked processing, model serving without HTTP, batch throughput optimization
Intermediate Projects
1. Full MLflow Pipeline
Build an end-to-end MLflow project: data preprocessing stage, feature engineering stage, training stage, evaluation stage — each as a separate tracked run within a parent run. Include param inheritance and metric comparison.
What you’ll practice: Nested runs, MLproject file, multi-stage pipelines, artifact lineage
2. DVC Pipeline with Experiments
Build a DVC pipeline (dvc.yaml) with 4 stages: download, preprocess, train, evaluate. Use params.yaml for all hyperparameters. Run dvc exp run to compare 10 configurations. Use dvc metrics diff to report changes.
What you’ll practice: dvc.yaml, params.yaml, dvc exp run, metric comparison
3. Blue-Green Deployment
Implement a blue-green deployment for a model API: run two versions simultaneously, route 10% of traffic to green, compare metrics for 1 hour, and automate the cutover decision based on performance.
What you’ll practice: Traffic splitting, parallel serving, automated promotion criteria
4. CI/CD for ML with GitHub Actions
Build a GitHub Actions workflow that: on every PR, retrains the model on a sample, validates metrics meet thresholds, checks for data schema drift, builds and tests the Docker image, and posts a report as a PR comment.
What you’ll practice: GitHub Actions, CI/CD for ML, automated quality gates, PR automation
5. Model Drift Monitor
Build a drift monitoring service: compare input feature distributions between training data and recent production requests using KS test and PSI, send alerts when drift exceeds thresholds, and log drift scores to a dashboard.
What you’ll practice: KS test, PSI calculation, statistical process control, alerting
6. Shadow Mode Testing
Deploy a new model in shadow mode: route all production requests to both models, compare predictions (agreement rate, score distribution), but only serve the old model’s predictions to users. Promote when quality is verified.
What you’ll practice: Shadow deployment, dual inference, comparison metrics, safe rollout
7. Feature Pipeline with Great Expectations
Build a data validation pipeline using Great Expectations: define expectations for each feature (range, null rate, cardinality), run them as a dvc pipeline stage, and block training if any expectation fails.
What you’ll practice: Great Expectations, data contracts, pipeline integration, data quality gates
8. Model Explanations Service
Build a model explanations microservice: accepts a prediction request, computes SHAP values, returns both the prediction and a ranked list of feature contributions. Cache explanations for repeated inputs.
What you’ll practice: SHAP integration in serving, explanation caching, API design
9. A/B Testing Infrastructure for Models
Build an A/B testing system: assign users to model variants by ID hash, serve the correct model per variant, log variant + outcome for each request, and compute statistical significance of the difference.
What you’ll practice: User assignment, variant logging, statistical testing, outcome tracking
10. Distributed Hyperparameter Tuning
Set up distributed hyperparameter tuning with Ray Tune or Optuna: define a search space, run 50 trials in parallel, log all trials to MLflow, select the best, and retrain a final model on full data.
What you’ll practice: Distributed search, Ray Tune or Optuna, MLflow integration, multi-objective optimization
Advanced Projects
1. Kubeflow Pipelines End-to-End
Build a complete Kubeflow pipeline: containerized components for data prep, training, evaluation, and deployment. Trigger on a schedule, track with MLflow, and deploy to a Kubernetes serving endpoint.
What you’ll practice: Kubeflow Pipelines SDK, containerized components, Kubernetes, scheduled pipelines
2. Real-Time Feature Store
Build a production feature store with an offline store (Parquet/data warehouse) and online store (Redis): compute features from raw events, write to both stores, serve low-latency features (<10ms) at prediction time.
What you’ll practice: Online/offline consistency, Redis serving, feature computation pipeline
3. LLMOps Platform
Build an LLMOps platform: track prompts and their versions, log every API call with cost and latency, run evaluation suites on every prompt change, and alert when evaluation score drops below threshold.
What you’ll practice: Prompt versioning, cost tracking, LLM evaluation, regression detection
4. Auto-Scaling Model Server
Build a model serving system with auto-scaling: TorchServe or Triton Inference Server, Kubernetes HPA based on request queue length, load test to verify scaling behavior, and benchmark latency under load.
What you’ll practice: TorchServe/Triton, Kubernetes HPA, load testing, latency profiling
5. Multi-Armed Bandit for Model Selection
Implement a Thompson Sampling bandit to route traffic between 3 model versions: accumulate performance feedback, update Beta distributions per model, and converge to the best model while minimizing regret.
What you’ll practice: Bandit algorithms, online learning for model selection, Bayesian updating
Portfolio Projects
1. End-to-End MLOps Platform
Build a complete ML platform from scratch: data versioning (DVC), experiment tracking (MLflow), automated training pipeline (GitHub Actions + DVC), model registry, REST API serving (FastAPI + Docker), drift monitoring, and a Grafana dashboard.
Tech stack: DVC, MLflow, GitHub Actions, FastAPI, Docker, Prometheus, Grafana
Demonstrates: Full MLOps lifecycle, infrastructure thinking, production operations
2. Self-Healing ML System
Build a system that monitors its own performance, detects drift, automatically retrains on recent data, validates the new model passes quality gates, and deploys it — with a full audit trail and rollback capability.
Tech stack: MLflow, FastAPI, Airflow/Prefect, statistical monitoring, Docker
Demonstrates: Autonomous operations, safety controls, audit trail design
3. ML Cost Optimizer
Build a cost optimization layer: profile training jobs for compute waste, implement spot instance training with checkpointing, compare batch vs. real-time serving costs, and produce a cost-per-prediction dashboard.
Tech stack: AWS/GCP SDK, MLflow, Docker, cost monitoring APIs
Demonstrates: Cloud cost awareness, production efficiency, infrastructure optimization
4. Feature Platform
Build an enterprise-grade feature platform: feature definitions as code, point-in-time correct training data generation, online serving with Redis, feature monitoring for drift, and a feature catalog UI.
Tech stack: Feast or custom, Redis, Parquet, FastAPI, Streamlit
Demonstrates: Feature engineering at scale, data engineering, platform design
5. ML Governance System
Build an ML governance platform: model cards for each registered model, bias evaluation reports (fairlearn), data lineage tracking, approval workflow for production promotion, and audit log for all model decisions.
Tech stack: MLflow, fairlearn, FastAPI, PostgreSQL, React
Demonstrates: Responsible AI, compliance awareness, enterprise ML maturity