API reference · grader/llm.py · 217 lines

`grader.llm`

Optional LLM-assisted qualitative grading (design.md §26).

Nothing in the engine requires this module to do anything: the default grader is disabled, and a disabled grader means qualitative items fall back to structural scoring plus human review.

Two rules are enforced here rather than left to the caller:

* student names never reach the provider — only the response text,
* a low-confidence judgement is always routed to a human.

Summary

Name	Kind	Description
`QualitativeGrader`	class	Interface the assignment graders depend on.
`DisabledQualitativeGrader`	class	The default. Qualitative items stay with the human.
`LLMQualitativeGrader`	class	Provider-independent grader.
`anthropic_grader()`	function	Anthropic-backed grader, or a disabled one when it cannot be set up.
`build_grader()`	function	Construct a grader from UI settings.

Constants

Name	Value	Notes
`DEFAULT_REVIEW_THRESHOLD`	`0.8`
`SYSTEM_PROMPT`	`'You are helping a teaching assistant grade short written answers in a graduate geospatial data science course. Grade only the response tex…`
`USER_TEMPLATE`	`'Question / rubric item:\n{question}\n\nMaximum score: {max_score}\n\nRubric criteria:\n{criteria}\n\nStudent response:\n"""\n{response}\n"…`

Classes

class QualitativeGrader(Protocol) #[source]

Interface the assignment graders depend on.

Class attributes

Name	Type	Default	Notes
`enabled`	`bool`	required

Methods

grade(question: str, student_response: str, rubric: dict[str, Any]) -> dict[str, Any] #[source]: No description.

class DisabledQualitativeGrader #[source]

dataclass

The default. Qualitative items stay with the human.

Fields

Name	Type	Default	Notes
`enabled`	`bool`	`False`

Methods

grade(question: str, student_response: str, rubric: dict[str, Any]) -> dict[str, Any] #[source]: No description.

class LLMQualitativeGrader(complete: Callable[[str, str], str], review_threshold: float = DEFAULT_REVIEW_THRESHOLD, model_name: str = 'unspecified') #[source]

Provider-independent grader.

complete is any callable taking (system_prompt, user_prompt) and returning the model's text. Swapping providers means passing a different callable — the engine never imports a vendor SDK.

Methods

grade(question: str, student_response: str, rubric: dict[str, Any]) -> dict[str, Any] #[source]: No description.

Functions

anthropic_grader(model: str = 'claude-sonnet-5', api_key: str | None = None, review_threshold: float = DEFAULT_REVIEW_THRESHOLD) -> QualitativeGrader #[source]

Anthropic-backed grader, or a disabled one when it cannot be set up.

This is a convenience wrapper, not a dependency: anthropic is imported lazily and its absence simply leaves qualitative grading manual.

build_grader(settings: dict[str, Any]) -> QualitativeGrader #[source]: Construct a grader from UI settings.