LLM Engineering Projects
Projects that build real LLM-powered applications — from prompt engineering basics to production RAG systems and fine-tuned models.
Beginner Projects
1. Smart Summarizer
Build a document summarizer that handles different lengths intelligently: short docs get a 2-sentence summary, long docs get a structured summary with key points and action items. Use system prompts to control format.
What you’ll practice: System prompts, output formatting, handling variable-length inputs
2. Code Review Bot
Build a code review assistant that accepts code snippets and returns: bugs found, security issues, style violations, and suggested improvements — each in a separate section with line references.
What you’ll practice: Structured output prompting, technical prompts, few-shot examples
3. Email Draft Generator
Given bullet points and tone (formal/casual/assertive), generate a polished email. Support follow-up editing in conversation. Ensure the output never exceeds a user-specified word count.
What you’ll practice: Conversation history, length constraints, tone control
4. FAQ Answer Bot
Build a chatbot that answers questions from a static FAQ document. Use string matching to find relevant FAQ entries, inject them into context, and answer only from that context (refuse to hallucinate).
What you’ll practice: Context injection, grounded answering, refusal when uncertain
5. Recipe Generator with Constraints
Generate recipes given ingredients + dietary restrictions + cuisine type. Validate that the output respects all constraints. If an ingredient conflicts with a restriction, refuse and explain why.
What you’ll practice: Constraint satisfaction prompting, output validation, refusal handling
6. Interview Question Generator
Given a job description, generate role-specific interview questions at three difficulty levels (screening, technical, behavioral). Ensure no duplicates and good coverage across required skills.
What you’ll practice: Structured generation, deduplication prompting, multi-turn refinement
7. Text Simplifier
Rewrite complex text (legal, medical, academic) at a specified reading grade level (6th, 8th, 12th). Verify complexity reduction by checking average sentence length and syllable count.
What you’ll practice: Style transfer prompts, output verification, reading level targeting
8. Meeting Notes Formatter
Convert raw meeting transcript text into structured notes: attendees, key decisions, action items with owners, open questions. Output as both markdown and JSON.
What you’ll practice: Dual-format output, information extraction, structured prompting
9. Language Tutor
Build a language learning assistant that corrects grammar, explains mistakes, provides the corrected version, and gives three practice sentences using the same structure.
What you’ll practice: Multi-part responses, educational prompting, iterative conversation
10. Changelog Generator
Given a git diff or list of commit messages, generate a human-readable changelog grouped by: features, bug fixes, performance improvements, and breaking changes.
What you’ll practice: Classification + grouping prompts, technical domain, structured output
Intermediate Projects
1. RAG-Powered Documentation Assistant
Build a documentation chatbot: chunk a large codebase or docs site, embed with Claude, store in FAISS, retrieve relevant chunks, and generate grounded answers with source citations.
What you’ll practice: RAG pipeline, chunking strategies, source attribution, groundedness
2. Multi-Step Research Agent
Build an agent that: breaks a research question into sub-questions, searches (web or local docs) for each, synthesizes findings, identifies contradictions, and produces a structured report.
What you’ll practice: Tool use, multi-step planning, synthesis across sources
3. Automated Code Generator with Tests
Given a function signature and docstring, generate the implementation AND a pytest test suite. Run the tests programmatically and iterate if they fail — up to 3 self-correction rounds.
What you’ll practice: Code generation, self-correction loop, test execution feedback
4. Data Extraction Pipeline
Extract structured data from unstructured text at scale: build a prompt that extracts 10+ fields from business documents (invoices, contracts, emails), validate JSON output, handle edge cases.
What you’ll practice: JSON extraction prompts, schema validation, error handling at scale
5. Prompt Optimization System
Build a system that automatically improves prompts: generate 5 prompt variants, test each on 20 examples, score with LLM-as-judge, select the winner, and repeat for 3 generations.
What you’ll practice: Prompt evaluation, LLM-as-judge, optimization loop
6. Conversational SQL Generator
Build a text-to-SQL system: given a database schema, generate SQL queries from natural language questions. Validate the SQL syntax, handle ambiguous queries by asking for clarification.
What you’ll practice: Schema injection, SQL generation, ambiguity handling, validation
7. Content Moderation Pipeline
Build a multi-stage content moderation system: fast Haiku classifier for obvious violations, Claude for nuanced cases, human escalation for borderline content. Track false positive/negative rates.
What you’ll practice: Model routing, multi-stage pipelines, cost optimization, metrics tracking
8. Personalized Learning Path Generator
Given a learner’s background and goal, generate a 12-week learning plan with weekly topics, recommended resources, and mini-projects. Adapt the plan based on learner feedback.
What you’ll practice: Personalization prompts, long-form structured output, adaptive conversation
9. Automated Report Writer
Build a system that: fetches data from an API, analyzes it with code execution, generates a narrative explanation with Claude, and combines them into a formatted PDF report.
What you’ll practice: Data + LLM integration, narrative generation, report templating
10. Semantic Search Engine
Build a semantic search system over a document corpus: embed documents with Claude embeddings, implement cosine similarity ranking, support hybrid search (semantic + keyword), and evaluate with nDCG.
What you’ll practice: Embeddings API, vector similarity, hybrid search, IR evaluation
Advanced Projects
1. Multi-Agent Research System
Build a multi-agent pipeline: orchestrator agent coordinates a researcher, critic, fact-checker, and writer agent. Each agent has tools and memory. Produce a verifiable research report.
What you’ll practice: Multi-agent coordination, agent specialization, tool use, verification
2. Fine-Tuned Domain Expert
Fine-tune Claude on a proprietary domain corpus (legal, medical, financial). Build an evaluation suite with 100 domain-specific questions. Compare fine-tuned vs. base model with LLM-as-judge.
What you’ll practice: Fine-tuning API, evaluation design, domain adaptation, comparison methodology
3. Production RAG with Evaluation
Build a production RAG system with: query rewriting, HyDE (hypothetical document embeddings), re-ranking, faithfulness evaluation, and a full evaluation dashboard tracking retrieval and generation quality.
What you’ll practice: Advanced RAG patterns, evaluation metrics, dashboard design
4. Streaming Chat Application
Build a full-stack chat application: FastAPI backend with streaming responses, React frontend, conversation history, system prompt management, token usage display, and cost estimation.
What you’ll practice: Streaming API, full-stack integration, conversation management, cost tracking
5. LLM Test Automation Framework
Build a framework that auto-generates test cases from natural language descriptions, executes them against your LLM application, and produces a CI-ready report with regression tracking.
What you’ll practice: Test generation, LLM-as-judge evaluation, regression detection, CI integration
Portfolio Projects
1. Enterprise RAG Platform
Build a complete RAG platform: document ingestion pipeline (PDF, DOCX, HTML), chunking with overlap, hybrid retrieval (BM25 + embeddings), re-ranking with cross-encoder, streaming answers with citations, user feedback collection, and an analytics dashboard.
Tech stack: Anthropic API, FAISS/Weaviate, FastAPI, React, PostgreSQL
Demonstrates: Production RAG, system design, full-stack development
2. Automated Code Review Service
Build a GitHub App that auto-reviews PRs: security analysis, performance issues, test coverage gaps, style violations, and architecture feedback — each with severity ratings and fix suggestions. Track false positive rate over time.
Tech stack: Anthropic API, GitHub API, FastAPI, PostgreSQL
Demonstrates: Developer tooling, API integration, CI/CD, production ops
3. Intelligent Document Processing System
Build a document processing pipeline for business documents: classify document type, extract structured data, validate against business rules, flag exceptions for human review, and integrate with downstream systems.
Tech stack: Anthropic API, FastAPI, PostgreSQL, React
Demonstrates: Enterprise AI, extraction accuracy, human-in-the-loop design
4. AI Writing Assistant with Brand Voice
Build a writing assistant that learns a company’s brand voice from examples, generates on-brand content for different formats (blog, social, email), and scores drafts for brand alignment.
Tech stack: Anthropic API, fine-tuning, FastAPI, Next.js
Demonstrates: Fine-tuning for style, evaluation systems, product thinking
5. Conversational Analytics Agent
Build an agent that connects to a database, translates natural language questions into SQL, executes queries, generates insights from results, and produces shareable reports — handling follow-up questions in context.
Tech stack: Anthropic API, tool use, SQLAlchemy, FastAPI, Streamlit
Demonstrates: Agent architecture, data integration, business intelligence