Gradient Field Guide 0 / 8 levels done
A living syllabus for CS grads

Gradient Field Guide

Learning AI looks like a loss curve: confusion drops fast in the foundations, then flattens into the long tail where the frontier moves every week. Work down the curve, build the five projects, and read the feed.

Trends updated Refresh cadence every 5 h Your progress 0 / 8
L0 math L1 ML L2 deep L3 attn L4 LLMs L5 apps L6 agents L7 frontier study time → confusion
Two minutes, then go

Start here

You don't have to begin at L0. Pick the description closest to you and the path below highlights where to start and how long it should take at 8–10 hours a week. Your choice is remembered in this browser.

Pick a background above to get a suggested route.
Eight levels, in order

Learning path

Each level builds on the one before. Tick the checkpoint only when you can do it without notes; your ticks stay in this browser. Budget is a rough guide for 8–10 hours a week.

  1. L0~3 weeks

    Math and Python you actually need

    Vectors and matrices are how models store knowledge; derivatives are how they learn; probability is how they express uncertainty. You don't need proofs, you need intuition and NumPy fluency.

    vectors · dot productmatrix multiplygradients · chain ruleprobability · Bayesmean · varianceNumPy broadcasting
    My notes
  2. L1~4 weeks

    Classical machine learning

    The vocabulary every AI conversation assumes: training vs. test data, overfitting, loss functions, metrics. Most real-world tabular problems are still solved here, not with LLMs.

    supervised / unsupervisedbias–variancecross-validationprecision · recall · F1decision trees · XGBoostk-means · PCA
    My notes
  3. L2~5 weeks

    Neural networks and deep learning

    Build backpropagation by hand once, then let PyTorch do it. Understanding why training diverges or stalls is what separates engineers from API callers.

    backpropSGD · Adamactivationsbatch norm · dropoutCNNsPyTorch autogradGPU basics
    My notes
  4. L3~4 weeks

    Tokens, embeddings and attention

    The transformer is the engine inside every modern LLM. Once you've written self-attention yourself, papers and release notes stop being magic.

    tokenization (BPE)embeddingsself-attention · Q/K/Vmulti-headpositional encoding · RoPEdecoder-only GPT
    My notes
  5. L4~4 weeks

    How LLMs are trained, served and run locally

    Pretraining gives knowledge; fine-tuning and RLHF give behavior. On the serving side, quantization and the KV cache decide whether a model fits on your GPU. Rule of thumb: a 4-bit GGUF needs about 0.6 GB of memory per billion parameters, plus room for context.

    pretraining · scaling lawsSFT · RLHF · DPOsampling · temperaturecontext windowKV cachequantization (Q4, GGUF)LoRAprompting
    My notes
  6. L5~4 weeks

    Building LLM applications

    Real products combine a model with your data (RAG), structured outputs, tool calls and, most importantly, evaluations. Without an eval set you can't tell whether a change helped.

    embeddings · vector searchchunking · rerankingRAGtool use · JSON schemaevals · LLM-as-judgelatency & costprompt caching
    My notes
  7. L6~4 weeks

    Agents, protocols and guardrails

    An agent is a model in a loop that chooses tools until a goal is met. MCP standardizes how tools plug in; A2A how agents talk to each other. The hard part is containment: least-privilege tools, sandboxes, approvals and defenses against prompt injection.

    agent loopworkflows vs. agentsMCP serversA2Aorchestrator + sub-agentsprompt injectionsandboxing · approvalscomputer use
    My notes
  8. L7ongoing

    The frontier

    Where research is moving now. Pick one thread, read its key papers monthly, and reproduce one small result. That's the difference between following trends and understanding them.

    reasoning · test-time computeRL with verifiable rewardsdistillation · small modelsmixture of expertsmultimodal · voiceworld modelsinterpretabilityalignment & evals
    My notes
Learn by poking at it

Playground

Two fixed toy mechanisms plus a third demo that changes with every refresh, tied to whatever is in the news. None is a trained model: the gradient descent demo optimizes a real function, and the attention demo runs the real Q·K softmax math on embeddings generated from a word hash rather than learned weights. Enough to build intuition for L0 and L3.

Gradient descent on a double well

f(x) = x⁴⁄4 − x². Click the curve to place the ball, or drag the start slider, then step through gradient descent. Too high a learning rate overshoots the minimum; starting on the wrong side finds the other one.

Self-attention weights

Type a short sentence. Each word becomes an 8-dimensional vector from a deterministic hash, and the grid shows softmax(QKᵀ/√d) — the formula from Attention Is All You Need, just without training. Darker cells mean a word attends more to that column.

New this refresh · 2026-10-19 · connects to L7

Scaling laws: what more compute buys you

Frontier labs justify ever-bigger training runs with "scaling laws": loss falls as a power of compute, but with sharply diminishing returns. Drag the slider to pick a training-compute budget (FLOPs, log scale) and watch a toy loss curve respond. The curve shape and numbers here are invented to show the pattern — not a measured scaling law for any real model like Microsoft's MAI-Base-2 or Meta's Muse Spark.

Toy loss

Hybrid builds: Claude + a local model

Top 5 projects

Each project pairs a local open model (private, free per call, fast for simple work) with Claude (strong reasoning, long context, tool use). Learning when to use each is the skill employers want. Ranked by how much they teach per hour.

runs locallycalls Claude

Not started

    Building

      Shipped

        #1Not started

        Private second brain (hybrid RAG)

        Chat with your notes, PDFs and lecture slides. Embeddings and search never leave your machine; Claude only sees the few passages it needs.
        Status
        your files→chunk + embed (nomic-embed-text)→sqlite-vec / Chroma→Qwen3 8B: quick answers→Claude: synthesis & hard questions
        You learn
        embeddings, chunking, retrieval quality, citations, privacy boundaries
        Hardware
        8 GB GPU or 16 GB Apple Silicon; CPU works, slower
        Levels
        L4, L5
        1. Ingest a folder of PDFs/Markdown; chunk at ~500 tokens with overlap.
        2. Embed locally via Ollama; store vectors in sqlite-vec.
        3. Answer with the local model, citing chunk sources.
        4. Add an "escalate" button that sends top passages to Claude.
        5. Build a 30-question eval set; measure hit rate before and after adding a reranker.

        Stretch Add hybrid BM25 + vector search and show the eval gain.

        #2Not started

        Model router with an eval harness

        A local model handles each request first; a checker decides whether to escalate to Claude. Measure the accuracy, latency and cost of every routing policy.
        Status
        request→Gemma 4 / Qwen3 draft→confidence check→ low →Claude answer→log + dashboard
        You learn
        evals, LLM-as-judge, routing, cost/latency trade-offs, observability
        Hardware
        12–24 GB GPU for a 14–30B local model
        Levels
        L5, L7
        1. Collect 200 tasks with known answers (math, SQL, summarization).
        2. Run local-only and Claude-only baselines; record accuracy and time.
        3. Try routers: self-reported confidence, a judge model, a small trained classifier.
        4. Plot the accuracy-vs-cost curve for each policy.
        5. Write up which policy wins and why.

        Stretch Publish the harness as an open-source CLI others can point at their own tasks.

        #3Not started

        MCP dev-tools server with guardrails

        Expose your repo's tests, logs and issue tracker to Claude through an MCP server. A local model pre-filters noisy logs, and every risky action needs your approval.
        Status
        Claude Code / Claude API↔your MCP server→local model: summarize logs→tools: run_tests · search · open_issue (approval)
        You learn
        MCP, tool design, agent loops, least privilege, prompt-injection defense
        Hardware
        any laptop; 8 GB GPU for the log summarizer
        Levels
        L6
        1. Build an MCP server (Python or TypeScript SDK) with three read-only tools.
        2. Connect it to Claude Code and fix a real bug with it.
        3. Add a local-model tool that condenses 10k log lines into the 20 that matter.
        4. Add one write tool gated by human approval and an audit log.
        5. Plant a malicious instruction in a log file; show it's blocked.

        Stretch Run the tools in a container with no network egress.

        #4Not started

        Distill Claude into a tiny specialist

        Use Claude to generate and grade training data for one narrow task, fine-tune a 1.7–4B local model with LoRA, and see how close it gets at a fraction of the cost.
        Status
        Claude: generate + label data→clean & dedupe→LoRA fine-tune Qwen3 4B (Unsloth)→quantize to GGUF → Ollama→Claude: grade outputs
        You learn
        synthetic data, fine-tuning, LoRA, quantization, distillation, eval design
        Hardware
        16 GB GPU, or a free Colab/Kaggle GPU for training
        Levels
        L4, L7
        1. Pick a task: text-to-SQL for one schema, or support-ticket triage.
        2. Have Claude create ~2,000 diverse examples; hold out 200 for testing.
        3. Fine-tune with Unsloth LoRA; track loss.
        4. Export to GGUF, serve with Ollama, and compare base vs. tuned vs. Claude.
        5. Check the terms of service for the model outputs you train on.

        Stretch Add RL with a verifiable reward (does the SQL execute and return the right rows?).

        #5Not started

        Offline lecture and meeting assistant

        Transcribe audio on-device with Whisper, summarize locally, then let Claude turn a week of transcripts into study notes, flashcards and open questions.
        Status
        audio→whisper.cpp transcript→local model: per-session summary→Claude: weekly notes + flashcards
        You learn
        speech models, long-context handling, map-reduce summarization, multimodal pipelines
        Hardware
        any modern laptop; GPU speeds up transcription
        Levels
        L4, L5, L7
        1. Transcribe with whisper.cpp, keeping timestamps.
        2. Summarize each session locally in chunks (map), then merge (reduce).
        3. Send only the summaries to Claude for weekly synthesis.
        4. Export flashcards as CSV for Anki.
        5. Compare against Claude summarizing the full transcript: quality vs. cost.

        Stretch Add slide screenshots with a local vision model and link notes to slides.

        Your own record, kept in this browser

        Journal

        Log papers, videos and courses as you go. Finishing a checkpoint above or shipping a project logs itself. Nothing here leaves your machine — export it before clearing your browser data.

        0day streak
        0longest streak
        0entries logged
          Plain-English definitions

          Glossary

          The vocabulary from the levels above. Each term links to the level that teaches it.

          Gradient descentL0
          Repeatedly nudging parameters in the direction that lowers the loss.
          Loss functionL1
          A single number saying how wrong the model currently is; training tries to shrink it.
          OverfittingL1
          Memorising the training data so well that new data is handled badly.
          Cross-validationL1
          Rotating which slice of data is held out, to estimate how a model will do on unseen data.
          BackpropagationL2
          Applying the chain rule backwards through a network to get every parameter's gradient.
          Epoch / batchL2
          One full pass over the data / the small chunk processed per update.
          TokenL3
          A chunk of text (word piece) that a language model reads and writes.
          EmbeddingL3
          A vector of numbers representing a token or document so similar meanings sit close together.
          Self-attentionL3
          Each token computes weights over other tokens to decide what to draw information from.
          TransformerL3
          The attention-based architecture behind nearly all modern LLMs.
          Context windowL4
          The maximum number of tokens a model can consider at once.
          TemperatureL4
          A sampling knob: low is predictable, high is more varied.
          Fine-tuning (SFT)L4
          Further training a pretrained model on examples of the behaviour you want.
          RLHF / DPOL4
          Training a model to prefer answers humans rated higher.
          LoRAL4
          Fine-tuning a small set of added weights instead of the whole model.
          QuantizationL4
          Storing weights in fewer bits (e.g. 4-bit) so a model fits in less memory.
          KV cacheL4
          Saved attention keys/values so each new token doesn't recompute the past.
          RAGL5
          Retrieval-augmented generation: fetch relevant documents, then have the model answer from them.
          Vector searchL5
          Finding the stored embeddings nearest to a query embedding.
          Eval setL5
          A fixed set of test questions with known good answers, used to check whether a change helped.
          LLM-as-judgeL5
          Using a model to grade another model's outputs against a rubric.
          Tool useL6
          The model emits a structured request (e.g. JSON) that your code runs, then returns the result.
          AgentL6
          A model in a loop that chooses tools and actions until a goal is met.
          MCPL6
          Model Context Protocol: an open standard for plugging tools and data into AI apps.
          Prompt injectionL6
          Malicious text in the input or a tool result that tries to hijack the model's instructions.
          DistillationL7
          Training a small model to imitate a larger one.
          Mixture of expertsL7
          A model where each token is routed to only a few specialist sub-networks.
          InterpretabilityL7
          Research into what is happening inside a model and why.
          Weekly routine

          Staying sharp

          Read two newsletters

          Import AI for research and policy, The Batch for a practical weekly digest.

          Follow one builder

          Simon Willison tests nearly every release hands-on; Latent Space covers AI engineering in depth.

          Check the leaderboards, then doubt them

          LMArena for human preference rankings; confirm on your own eval set before switching models.

          Skim papers daily, read one weekly

          HF Daily Papers: read titles in 5 minutes, then one paper deeply on the weekend.

          Ship something monthly

          A project, a blog post or a pull request. Public work beats certificates in interviews.

          Revisit this page

          The feed above refreshes every 5 hours. Skim the "why it matters" lines and jump to the linked level when something is unfamiliar.