API reference · grader/rubric.py · 164 lines

`grader.rubric`

YAML rubric loading (design.md section 20).

A rubric file declares the assignment, its point breakdown, and per-item config blocks. Expected values (Center City ZIPs, tolerances, the final answer) live here rather than in code so a TA can correct them against the official solution without touching Python.

Summary

Name	Kind	Description
`RubricItem`	class	One graded line of a rubric, as declared in the YAML `rubric:` list.
`Rubric`	class	A parsed rubric file. Every top-level YAML block is kept as-is, so assignment graders can read their own configuration from `settings`, `probe`, `discovery` and `data`.
`RubricError`	class	Raised when a rubric file is missing or structurally invalid.
`load_rubric()`	function	Parse and validate a rubric YAML file.
`available_rubrics()`	function	Map assignment id -> rubric path for every parsable rubric on disk.

Constants

Name	Value	Notes
`RUBRIC_DIR`	`Path(__file__).resolve().parent.parent / 'rubrics'`
`VALID_TYPES`	`{'execution', 'structural', 'deterministic', 'hidden_test', 'qualitative'}`

Classes

class RubricItem #[source]

dataclass

One graded line of a rubric, as declared in the YAML rubric: list.

Fields

Name	Type	Default
`id`	`str`	required
`name`	`str`	required
`points`	`float`	required
`type`	`str`	`'deterministic'`
`description`	`str`	`''`
`config`	`dict[str, Any]`	`field(default_factory=dict)`

Properties

property is_qualitative: bool #[source]: No description.

class Rubric #[source]

dataclass

A parsed rubric file. Every top-level YAML block is kept as-is, so assignment graders can read their own configuration from settings, probe, discovery and data.

Fields

Name	Type	Default	Notes
`id`	`str`	required
`name`	`str`	required
`version`	`str`	required
`total_points`	`float`	required
`items`	`list[RubricItem]`	required
`settings`	`dict[str, Any]`	`field(default_factory=dict)`
`probe`	`dict[str, Any]`	`field(default_factory=dict)`
`discovery`	`dict[str, Any]`	`field(default_factory=dict)`
`data`	`dict[str, Any]`	`field(default_factory=dict)`	Data the instructor supplies for every submission (design.md §16-17).
`source_path`	`str \| None`	`None`

Properties

property requires_data: bool #[source]: No description.

property offers_data: bool #[source]

Whether the sidebar should let a TA supply data for this assignment.

Assignment 3 does not require its zip — it is graded from the outputs each student saved — but supplying it lets the grader re-run every notebook and confirm them, so the option has to be there.

property declared_points: float #[source]: No description.

Methods

item(rubric_id: str) -> RubricItem | None #[source]: No description.

label() -> str #[source]: No description.

class RubricError(ValueError) #[source]

Raised when a rubric file is missing or structurally invalid.

Functions

load_rubric(path: str | Path) -> Rubric #[source]

Parse and validate a rubric YAML file.

Raises RubricError when the file is missing or unparsable, assignment.id is absent, the rubric is empty, an item id is missing or duplicated, an item type is unknown, or the item points do not add up to total_points.

available_rubrics(directory: str | Path = RUBRIC_DIR) -> dict[str, Path] #[source]: Map assignment id -> rubric path for every parsable rubric on disk.