MUSA 5500 · Geospatial Data Science in Python

MUSA Grader

Semi-automatic grading for Jupyter Notebook assignments. It automates what can be tested objectively, flags anything ambiguous, and keeps human review quick. Every deduction traces back to a deterministic test, a structural check, a notebook result or a person's decision.

Get started → How grading works API reference

The grader screens submissions. It does not set marks on its own.

The grader can clear a submission, but it never marks one down by itself. Any submission that comes back below full marks goes to a person, and the deduction stands only if that person agrees. The time saved comes from the submissions the grader clears. Where the grader is unsure, nobody relies on it.

What it does#

Finds the notebook

Reads folders of student work or flat Canvas export ZIPs. When two notebooks look equally likely, it flags the submission instead of guessing.

Runs it in a sandbox

Each notebook runs in its own Docker container with no network, capped memory and CPU, and a read-only filesystem.

Inspects results, not names

A probe describes every DataFrame, GeoDataFrame, array and chart in the kernel, so students can name their variables however they like.

Rubric as configuration

Expected values, tolerances and credit ratios live in YAML. To fix the grading, edit the file and press Regrade. No Python changes needed.

Review queue

Lists only the submissions that need a decision, each with the reason it was flagged, the evidence and a manual override.

Exports

grades.csv, the review queue, per-student markdown feedback and the full session JSON, all stored on disk under results/.

Three assignments, three ways to grade#

What the grader can check depends on the dataset. Each of the three assignments gives it a different situation:

	Assignment 1: The Donut Effect	Assignment 2: Exploratory Visualization	Assignment 3: Evictions and the NDVI
Dataset	One Zillow file, downloaded by each student	The student's own choice, never handed in	One zip, distributed with the assignment
Answer key?	No. Zillow revises the file monthly	No. Every student uses a different dataset	Yes. Everyone runs the same five files
What is graded	Whether each step worked, checked against the student's own data	Whether the student can produce the required elements	Whether each result is correct
Evidence from	The grading run's own execution	The outputs saved in the notebook	Both, combined

See Assignments for how each one is graded and where its 100 points go.

Student data#

Grading runs entirely on the instructor's or TA's machine. Submissions are never uploaded anywhere.
The Streamlit UI binds to localhost, and notebooks run in a Docker container with no network access.
Failure patterns are reproduced in tests/ with synthetic fixtures. No student file, name or identifier appears in the repository.
Optional LLM grading of written answers is off by default. When it is on, names, emails and ID numbers are redacted before any text leaves the machine.

Architecture at a glance#

Streamlit UI→GradingService→executor→assignment grader→rubric engine→results store

The UI contains no grading logic. It calls grader.service.GradingService and displays what comes back, and the CLI works the same way. You can drive the grader from your own Python code in a few lines:

from grader import GraderSettings, GradingService

service = GradingService.from_rubric_path(
    "rubrics/hw1.yaml",
    GraderSettings(shared_data_paths=["examples/data/zillow_zhvi.csv"]),
)
session = service.run(service.load_submissions("examples/submissions").candidates)
print(session.summary())