Laboratory track

Introduction to the Hugging Face / LLM ecosystem

Goal. Load a tokenizer and a small model, generate text, read a model card, load a dataset.

Needs. Unit 3. Files. Lab 1 notebook (new)

1. Open the Lab 1 notebook in Colab (CPU is enough).
2. Predict the tokens of 'unbelievable' before running the tokenizer cell.
3. Generate at two temperatures and compare with Unit 3.

Lab extras (.zip) Lab 1 notebook

Tokenization and embeddings

Goal. Train a small BPE tokenizer and turn text into vectors you can compare.

Needs. Unit 3. Files. Unit 3 code zip (tokens_decoding), Unit 5 code zip (rag_np)

1. tokens_decoding.train_bpe(text, 50) then encode(...): watch merges grow.
2. rag_np.Tfidf and rag_np.LSA: embed sentences and compare by cosine similarity.
3. Open the Unit 3 tokenizer widget in the playground.

Implement self-attention

Goal. Compute scaled dot-product attention by hand and check it against the module.

Needs. Unit 2. Files. Unit 2 code zip (transformer_np)

1. Read transformer_np.attention(Q, K, V, mask) (6 lines).
2. Apply causal_mask(n) and confirm the upper triangle of the weights is zero.
3. Without positions, permuting the input tokens only permutes the output rows; add sinusoidal_pe(n, d) and show that this is no longer true.

Use a pretrained LLM

Goal. Prompt a real small model and watch decoding settings change the output.

Needs. Unit 4. Files. Unit 4 Colab notebook

1. Open the Unit 4 Colab notebook.
2. Run one prompt at three temperatures; record the outputs.
3. Compare with tokens_decoding.sample on the toy distribution from Unit 3.

Prompt engineering experiments

Goal. Compare prompt variants with a measured, not guessed, result.

Needs. Unit 4. Files. Unit 4 code zip (prompting_tools)

1. Build prompts with render and few_shot_messages.
2. Score variants over the same questions; report with paired_bootstrap_ci.
3. Make the output machine-readable with structured_call and a schema.

Build an LLM chatbot

Goal. A multi-turn chatbot that trims history to the context window and pins what must stay.

Needs. Unit 4. Files. Lab extras zip (chatbot_np, new)

1. python codes/chatbot/chatbot_np/example_basic.py (offline, mock model).
2. Shrink window until old turns drop; check last_dropped.
3. Wrap a real API client in a class with complete(messages) and swap it in.

Lab extras (.zip)

Build a PDF / document Q&A system

Goal. Chunk documents, retrieve the best chunks, and answer from them with citations.

Needs. Unit 5. Files. Unit 5 code zip (rag_np)

1. load_corpus, then chunk_corpus(corpus, 200, overlap=40).
2. Rank with BM25; build the answer with build_context and extractive_answer.
3. To use your own PDF, extract the text first (for example with pypdf) and put it in data/.

Implement RAG with a vector database

Goal. Replace brute-force search with an index and fuse two rankers.

Needs. Unit 5. Files. Unit 5 code zip and Colab notebook

1. Build an IVF index; search also returns the number of distance computations. Compare recall and that count with exhaustive search for several nprobe.
2. Fuse BM25 and LSA rankings with rrf.
3. In the Unit 5 Colab notebook, repeat with a real embedding model and FAISS.

Fine-tune a small LLM with LoRA / QLoRA

Goal. See why a low-rank update is enough, then tune a real small model.

Needs. Unit 6. Files. Unit 6 code zip and Colab notebook

1. adapt_np.lora_fraction(d_out, d_in, r) for several r.
2. fit_low_rank_gd versus truncated_svd_error: how close does gradient descent get?
3. Unit 6 Colab notebook: LoRA on a small model with peft.

Build an image / text multimodal application

Goal. Embed images and text in one space and sample from a diffusion model.

Needs. Unit 7. Files. Unit 7 code zip and Colab notebook

1. Train the CLIP-style toy with train_clip, project with embed, and measure image-to-text retrieval.
2. Sample with sample_ddim at 5, 10 and 50 steps.
3. Unit 7 Colab notebook: real vision-language models.

Build an LLM agent with tools

Goal. Run a reason-act loop with a tool registry and see where it fails.

Needs. Unit 8. Files. Unit 8 code zip and Colab notebook

1. scripted_run(kind, params), then inject a failure with fail_at.
2. Add make_verifier and compare success rates with wilson intervals.
3. Test defend against the injected text in attack_outcome.

Evaluate hallucination and RAG quality

Goal. Measure retrieval quality and answer groundedness on the same questions.

Needs. Unit 9. Files. Unit 9 code zip and Colab notebook

1. rag_np.evaluate for recall at k.
2. evaluation_np.grounded_score per answer sentence; flag unsupported ones.
3. Compare two judges with cohen_kappa.

Deploy an LLM application

Goal. Serve the RAG retriever behind an HTTP API and measure it.

Needs. Unit 10. Files. Unit 10 code zip and Colab notebook

1. Install fastapi uvicorn requests pydantic.
2. uvicorn app_fastapi:app then POST to /ask; GET /health.
3. python bench_service.py for latency under concurrent load.

Capstone project

Goal. Combine retrieval, an agent loop, evaluation and a service into one application.

Needs. Unit 11. Files. Unit 11 (capstone) notes and code

1. Read Unit 11, pick a brief, build the minimum version first.
2. Evaluate before you tune.
3. Write the report using the template.