API reference · grader/service.py · 460 lines

`grader.service`

The grading service the UI talks to.

Streamlit never calls the executor or the rubric engine directly (design.md §4): it hands a source and a config to this service and receives progress snapshots and a GradingSession back.

Summary

Name	Kind	Description
`GraderSettings`	class	Everything the Settings page can change (design.md §33).
`LoadedSubmissions`	class	What `GradingService.load_submissions` found: one `SubmissionCandidate` per student, plus where a Canvas ZIP was extracted to.
`GradingService`	class	Orchestrates a grading run: find submissions, place the data, execute each notebook, run the assignment's checks and apply the review policy.

Constants

Name	Value	Notes
`STAGE_LABELS`	`[('execution', 'Notebook execution'), ('structural', 'Structural checks'), ('hidden', 'Hidden tests'), ('feedback', 'Feedback generation')]`

Classes

class GraderSettings #[source]

dataclass

Everything the Settings page can change (design.md §33).

Fields

Name	Type	Default	Notes
`execution_mode`	`str`	`MODE_DOCKER`
`docker_image`	`str`	`'musa-grader:latest'`
`timeout_seconds`	`int`	`600`
`cell_timeout_seconds`	`int`	`300`
`memory_limit`	`str`	`'2g'`
`cpu_limit`	`str`	`'1.0'`
`shared_data_paths`	`list[str]`	`field(default_factory=list)`	Data files the instructor supplies for every submission (design note: from HW1 onwards students hand in a notebook and nothing else).
`confidence_threshold`	`float`	`DEFAULT_CONFIDENCE_THRESHOLD`
`review_below_full_marks`	`bool`	`True`	Any deduction goes to the review queue, not just uncertain ones.
`llm_enabled`	`bool`	`False`
`llm_provider`	`str`	`'anthropic'`
`llm_model`	`str`	`'claude-sonnet-5'`
`keep_workdirs`	`bool`	`False`

Methods

execution_config(rubric: 'Rubric | None' = None) -> ExecutionConfig #[source]

Execution limits, raised if the assignment declares it needs longer.

A rubric's execution_timeout_seconds is assignment knowledge, not a preference: Assignment 3 reads a 24 MB GeoJSON, joins 34,108 points and masks a ten-band raster twice, and cutting it short times out the whole class at once. So it acts as a floor on the configured timeout, never as a cap — a TA who raises the limit keeps their value.

to_dict() -> dict[str, Any] #[source]: The settings as stored in grading_session.json.

class LoadedSubmissions #[source]

dataclass

What GradingService.load_submissions found: one SubmissionCandidate per student, plus where a Canvas ZIP was extracted to.

Fields

Name	Type	Default
`source`	`str`	required
`candidates`	`list[SubmissionCandidate]`	`field(default_factory=list)`
`extracted_root`	`Path \| None`	`None`

Properties

property ok_count: int #[source]: No description.

property warning_count: int #[source]: No description.

class GradingService(rubric: Rubric, settings: GraderSettings | None = None, qualitative_grader: Any | None = None) #[source]

Orchestrates a grading run: find submissions, place the data, execute each notebook, run the assignment's checks and apply the review policy.

This is the only entry point the Streamlit UI and cli.py use. Construct it from a loaded Rubric (or with from_rubric_path); the assignment grader registered under rubric.id is selected automatically.

Methods

classmethod from_rubric_path(path: str | Path, settings: GraderSettings | None = None, qualitative_grader: Any | None = None) -> 'GradingService' #[source]: Load a rubric YAML and build a service for it.

preflight() -> list[str] #[source]

Problems that would make a grading run misleading rather than wrong.

The big one: this assignment's students submit a notebook and nothing else, so without instructor-supplied data every notebook fails at read_csv and the whole class looks broken.

load_submissions(source: str | Path, extract_to: str | Path | None = None) -> LoadedSubmissions #[source]: Accept a folder or a Canvas ZIP (design.md §16).

new_session(source: str) -> GradingSession #[source]: An empty GradingSession for this rubric and these settings, with a fresh timestamped session id.

Grade every candidate and return the saved session.

Submissions are graded one at a time. progress_callback receives a GradingProgress snapshot at every stage (a callback that raises never stops grading); setting cancel_event stops after the current submission. preserve_overrides is a snapshot from GradingSession.collect_overrides to re-apply on top of the fresh automatic scores. Working directories are deleted afterwards unless GraderSettings.keep_workdirs is set.

regrade(session: GradingSession, candidates: Iterable[SubmissionCandidate], preserve_overrides: bool = True, progress_callback: ProgressCallback | None = None, cancel_event: threading.Event | None = None) -> GradingSession #[source]

Grade the same candidates again into the same session id and directory.

With preserve_overrides (the default) every manual override is snapshotted first and restored on top of the new automatic scores.