Capstone projects
Write and execute a capstone proposal whose evidence milestones and stop conditions fit the available hardware and schedule
Before you begin
Follow a linked prerequisite when you cannot explain it from memory. Background skills without links are stated as abilities rather than hidden terminology.
- A reusable benchmark and correctness harness
- A completed causal performance report
- The technical prerequisites for the selected project
A learner has already implemented and benchmarked a correct row-wise softmax CUDA kernel. The next project proposal says only `implement FlashAttention`, which would require online softmax, tiled data reuse, mask handling, several head dimensions, framework integration, forward and backward computation, and comparison with mature libraries. A six-week project cannot investigate every item with equal depth, so the proposal has no clear completion condition.
The learner narrows the question to forward-only causal attention for BF16 tensors with head dimension 128 and sequence lengths from 512 through 4,096 on one GPU. The project will calculate expected high-bandwidth-memory traffic, implement online softmax in a tiled kernel, use NVIDIA Nsight Compute to measure transferred bytes, verify output against a PyTorch reference, and compare ordinary CUDA-event timings with a mature attention library. Framework integration becomes one later milestone instead of an unstated requirement.
A capstone adds one new integration task to a measurement workflow that the learner has already completed. A verified kernel benchmark can add PyTorch operator registration; a measured single-GPU training step can add one collective on the recurring eight-GPU server; a scheduler simulation can add observations from one real serving engine. The project finishes when the central question has a supported answer, the declared tests pass, and another engineer can reproduce one result table from the archived artifacts.
Add one new integration task to verified work
List the work you can already perform with evidence. If you can implement and benchmark one GPU kernel, the next project can add framework registration and compiled execution. If you can measure a single-node training step, the next project can add one collective or one failure mode. Adding several unfamiliar layers at once makes a failed result difficult to diagnose.
The project needs a real uncertainty. Reimplementing a tutorial exactly can develop mechanics, but the result is already known. Add a bounded question: which row-width region favors one reduction mapping, how a page size changes decode traffic, or when a compiler chooses a generated GEMM instead of a library. The question should admit a result even when the new implementation is slower.
Confirm that the required hardware, software, and data are available. A project that requires a 64-GPU topology cannot be completed with one GPU unless the proposal explicitly limits itself to simulation and small-scale mechanism tests. Rewrite the claim to match the resources before implementation begins.
Write a proposal with stop conditions
State the problem, user or engineering decision, baseline, input family, metric, correctness contract, quantitative model, proposed intervention, and expected result. List prerequisites and identify which ones are already demonstrated. Estimate compute time, storage, cluster access, and implementation time.
Write the measurement sequence in advance: reference validation, naive baseline, isolated transformation, profiler test, shape sweep, and end-to-end evaluation. Define what evidence is decisive at each stage. This prevents the project from collecting large traces before basic correctness or timing boundaries are stable.
Stop conditions protect the schedule. Define material gain numerically before implementation, such as at least 5 milliseconds of p99 request margin, 2% of step time, or a declared fleet-cost reduction. Stop performance implementation if the analytical upper bound cannot reach that threshold. Stop a private kernel path if the mature library remains faster throughout the required shapes and the remaining value is educational. Stop scope expansion after the central question is answered. A stopped project can still produce a useful negative report.
Organize milestones around evidence
Milestone 1 produces the semantic reference, correctness tests, workload generator, environment record, and quantitative model. No optimized implementation is required. Review this milestone for a fair baseline and a falsifiable prediction.
Milestone 2 produces a measured baseline and confirms that the expected resource is relevant. A roofline, byte count, timeline, or utilization trace can reject the original opportunity. If the workload is not limited by the targeted resource, revise or stop before building a complex optimization.
Milestone 3 implements one change and supplies a controlled ablation. Milestone 4 covers the declared shape or load distribution, losing regions, integration, and end-to-end impact. Milestone 5 publishes the report and artifacts. This order produces a coherent result even if later milestones do not improve performance.
Control scope during implementation
Maintain a scope table with included, excluded, and deferred items. A forward-only attention capstone can exclude backward, dropout, arbitrary masks, and every head dimension while declaring these limits. Each added feature needs a reason tied to the main question. Do not add a feature only because a nearby library supports it.
Keep a known-good path available. Use feature flags or explicit dispatch so each transformation can be disabled. This supports ablations and rollback. Record interface changes separately from performance changes. A framework integration failure should not be confused with a kernel scheduling result.
Review the critical path weekly. If most time is spent on build systems, dataset access, or unsupported hardware, reduce the project to the smallest artifact that tests the mechanism. A narrower complete report demonstrates more technical control than a broad repository with an unverified central claim.
Publish a result even when the optimization loses
A mature library can remain faster. The capstone can still explain why. Show the theoretical opportunity, implementation decisions, profiler evidence, and the remaining gap. Examples include instruction eligibility, register spills, dispatch overhead, or library autotuning. State what additional control or hardware would be required to test the next hypothesis.
Negative results require the same correctness and measurement quality as positive results. Confirm that the contender implemented the intended mechanism and that the baseline was appropriate. Do not infer that the mechanism is invalid everywhere from one unsuccessful implementation.
Publish source, environment, commands, raw data, tests, report, and limitations. If the result depends on inaccessible hardware, include a small functional case and the measured reports permitted by policy. A future reader should be able to continue from a defined question rather than repeat the setup and failed hypotheses.
Worked example: Scoping an attention capstone
The initial idea is ‘implement FlashAttention,’ which is too broad and invites comparison with highly mature libraries.
- Narrow to a fixed forward-only causal attention subset with declared head dimensions, dtypes, and sequence range.
- Set a primary question: does an online-softmax tiled implementation reduce measured HBM traffic in agreement with its byte model for sequence lengths 512–4096? Beating a mature library is not the primary success condition.
- Derive online softmax and an IO model. Implement a trusted reference, then a simple tiled kernel in Triton or CUDA.
- Define correctness for random, extreme, all-masked, and tail cases. Record the allowed error by datatype before timing.
- Compare with the framework and mature library across the supported shape range. The objective is model–measurement agreement and understanding, not necessarily winning.
- Add one boundary: integrate as a custom op or study decode paging. Publish correctness, roofline, profiler evidence, losses, and limitations.
- Stop adding features after the selected boundary works. Complete the artifact instructions and ask another person to reproduce one table from a clean environment.
Conclusion: The scoped project remains technically rich while producing a defensible artifact even if the mature baseline is faster.
Common errors
- Choosing a project three layers beyond current practice. Too many unknowns make failure uninterpretable.
- Defining success only as faster execution than a vendor library. A project can demonstrate complete causal analysis without exceeding a mature library's peak performance.
- Adding features after the core question is answered. Finish and write before expanding scope.
- Leaving the baseline until the end. A mature implementation can reject the performance opportunity before most custom code is written.
- Treating a simulator as a substitute for every physical measurement. Calibrate the mechanism and state the scale that remains modeled.
Reference summary
- Kernel capstone: implement online softmax or fused normalization in CUDA and Triton, with a roofline, numerical tests, disassembly, and shape sweep.
- Compiler capstone: integrate the kernel as a framework operator, preserve compilation and autograd, and explain graph, guard, and fallback behavior.
- Distributed capstone: design a topology-aware parallel plan, validate collectives on a small cluster, and diagnose a synthetic straggler.
- Serving capstone: implement a continuous-batching simulator with paged KV allocation and optimize goodput under TTFT and TPOT constraints.
- End-to-end capstone: profile a real small-model workload, change one layer of the stack, and report mechanism, useful value, risks, and rollback.
Knowledge check
Starting from a verified CUDA softmax kernel, which single new interface—online attention tiling, framework registration, or serving-cache access—would create a finishable six-week capstone, and what evidence would define completion?
Show the answer guide
- Framework registration is the most bounded single interface for a verified CUDA softmax kernel because the kernel's numerical and device behavior already exists and the new work is limited to making that behavior visible to PyTorch.
- The scope should define one operator schema, supported shape and datatype region, device dispatch, fake or meta shape behavior, compilation behavior, and a correct fallback. Add autograd only when the capstone explicitly targets training; otherwise declare the operator inference-only.
- Completion requires eager and compiled execution without a graph break, exact dispatch evidence showing that the CUDA kernel runs for eligible inputs, and fallback evidence for inputs immediately outside the supported region.
- Correctness evidence includes the existing softmax reference tests plus odd widths, fully masked rows, non-contiguous inputs if supported, operator registration checks, and gradient checks only when differentiation is in scope.
- Performance evidence includes raw kernel and enclosing-graph timing distributions over the declared shape family, launch or graph inspection, compilation and first-use cost, and comparison with the original framework path.
- The six-week stop condition is a tested, documented integration for the bounded region. Online attention tiling and serving-cache access remain separate later projects rather than being added when time remains.
Run a one-day registration spike
Before committing to the capstone, register a minimal softmax-shaped custom operator that calls the trusted framework reference or an existing verified kernel. Exercise eager execution, fake or meta execution, torch.compile capture, one eligible contiguous shape, and one deliberately unsupported shape that must use the fallback.
Evidence to keep: Keep operator-check and correctness output, the captured graph or graph-break log, dispatch logs for fast and fallback paths, a short timing table, and a scope decision listing the interfaces proven feasible and those excluded from the six-week project.
Completed capstone projects provide reference measurements and methods for evaluating new hardware and software.
Sources and further reading
The lesson explanations synthesize primary papers and official references. Follow the originals for proofs, exact APIs, architecture details, and version-sensitive behavior.
- The Rust Programming Language↗
Structural inspiration: progressive chapters, concrete examples, and explicit learning flow.
- CUDA C++ Best Practices Guide↗
- Nsight Compute Profiling Guide↗
- MLPerf Inference Rules↗
- torch.compile Programming Model↗
- PagedAttention / vLLM↗