Developer guide

Testing and CI

Running the tests#

python -m pytest                  # the full suite, about 40 s
python -m pytest -m "not slow"    # skip the tests that run real kernels
python -m pytest -m docker        # grade a notebook in the sandbox image (needs it built)
ruff check .                      # syntax errors and undefined names

Marker	Meaning
`slow`	Runs a real notebook kernel (a few seconds each)
`docker`	Runs in the sandbox image. Skipped unless `musa-grader:latest` is built.

Ruff is limited to the checks that mean the grader would crash (E9, F63, F7, F82). assignment_template/ and notebooks are excluded because the templates are deliberately unfinished code.

What the suite covers#

File	Covers
`test_discovery.py`	Folder and Canvas layouts, notebook choice, ambiguous submissions
`test_data_supply.py`	Placing instructor data at the paths each notebook reads, path safety
`test_execution.py`, `test_docker_execution.py`	Local and sandboxed execution, timeouts, probe stripping
`test_hw1.py`, `test_hw2.py`, `test_hw3.py`	Per-assignment checks and named near-misses
`test_hw2_integration.py`, `test_integration.py`	End-to-end grading of demo classes
`test_answers.py`, `test_charts.py`	Reading answers and chart specs from saved outputs
`test_real_submission_patterns.py`	Failure patterns seen in real classes, reproduced synthetically
`test_results.py`	Session persistence, overrides, exports

Continuous integration#

.github/workflows/ci.yml runs on every push to main and every pull request:

Tests on Python 3.10, 3.11, 3.12 and 3.13: install, ruff, python cli.py rubrics, then the full suite with real kernels in local mode.
Docker images: build the sandbox, import every package the assignments use, build the app image, grade a real notebook inside the sandbox, grade the demo class from inside the app container (and assert student_001 scores 100), and check that the app answers its health check.

Dependabot opens weekly update PRs for the grader's packages, the sandbox's pinned packages, the base images and the Actions. Each one goes through the same CI.

Building these docs#

The site is static HTML in docs/. Guide pages are written by hand as fragments in docs/_pages/. The API reference is generated from the docstrings in grader/ and assignments/. The build reads the source with ast instead of importing it, so it needs only the standard library:

python docs/build.py

Rebuild after changing a docstring or a page, and commit the output. To preview locally:

python -m http.server -d docs 8000

To publish, enable GitHub Pages for the repository with Deploy from a branch → main → /docs.

To change	Edit
A guide page	`docs/_pages/<page>.html`, then rebuild
Page order or the API module list	`GUIDE` and `API_MODULES` in `docs/build.py`
An API description	The docstring in the source, then rebuild
Styling or behaviour	`docs/assets/style.css`, `docs/assets/main.js`