API reference · grader/charts.py · 741 lines

`grader.charts`

Chart evidence for visualization assignments (design.md §23, §25).

Assignment 2 grades charts, so the gradeable facts are not variable values but what was drawn: which library drew it, whether it actually rendered, and — for Altair — what the chart specification says it does.

Where the evidence comes from

An Altair chart compiles to a Vega-Lite specification, and that specification is in the notebook's own output. So a transformation, a brush selection and a cross-filter are not guessed from source text: they are read out of the spec the student's code actually produced. Three output shapes are handled, because the mime type depends on the Altair version the student happens to have:

application/vnd.vegalite.v4+json … v6+json — the spec, directly,
text/html carrying a vegaEmbed(...) call — the spec is the first argument of the embedded script (Altair 6's default renderer),
text/plain reading alt.Chart(...) — the chart rendered, but this notebook was saved without the rich output, so only its presence is known.

Matplotlib and seaborn have no specification, only a picture. For those the evidence is the rendered image plus what the source says about labelling and colour — enough to tell a TA where to look, never enough to score aesthetics automatically.

Which notebook the evidence comes from

Two notebooks may hold the same chart: the one this grading run executed, and the one the student handed in with its outputs already saved. Neither is reliably the better source.

For Assignment 2 the students choose their own dataset and do not hand it in, so the grading run has nothing to read and every cell after read_csv fails — the submitted outputs are usually the only place a chart exists. But when an instructor does supply data, a freshly executed chart is the stronger evidence, because a saved output can be stale or hand-edited.

So the two are merged cell by cell rather than ranked: for each chart, whichever notebook actually produced it wins, and the record says which one that was. ChartRecord.output_source and ChartEvidence.source carry that through to the rubric evidence, and the checks lower their confidence for a chart only seen in the student's own file.

Summary

Name	Kind	Description
`SpecFacts`	class	What one Vega-Lite specification says the chart does.
`ChartRecord`	class	One chart, as drawn by one cell.
`ChartEvidence`	class	Every chart found in a submission, in the order the cells appear.
`libraries_in_source()`	function	Which plotting libraries this cell draws with, most specific first.
`extract_specs()`	function	Vega-Lite specifications carried by one notebook output.
`strip_spec_data()`	function	Drop inlined rows so a spec can be stored as rubric evidence.
`analyze_spec()`	function	Read one Vega-Lite spec, at any level of composition.
`merge_facts()`	function	One view of a cell that displayed more than one chart.
`aesthetic_signals()`	function	Presence of the labelling and colour choices a TA would look for.
`collect_charts()`	function	Chart evidence, taking each chart from whichever notebook drew it.
`prose_word_count()`	function	Words of actual discussion — headings and placeholders do not count.

Constants

Name	Value	Notes
`MATPLOTLIB`	`'matplotlib'`
`SEABORN`	`'seaborn'`
`ALTAIR`	`'altair'`
`SOURCE_EXECUTED`	`'executed'`
`SOURCE_SUBMITTED`	`'submitted'`
`SOURCE_MERGED`	`'merged'`	Some charts from each: the usual case when students do not hand in their data.
`SOURCE_NONE`	`'none'`
`SEABORN_NON_PLOTTING`	`('set', 'set_theme', 'set_style', 'set_context', 'set_palette', 'color_palette', 'axes_style', 'plotting_context', 'despine', 'load_dataset…`	Seaborn helpers that style a figure without drawing one; a cell holding only these is setup, not a chart.
`SEABORN_PLOT_RE`	`re.compile('\\b(?:sns\|seaborn)\\.(?!(?:' + '\|'.join(SEABORN_NON_PLOTTING) + ')\\b)(\\w+)\\s*\\(')`
`MATPLOTLIB_PLOT_METHODS`	`('plot', 'scatter', 'bar', 'barh', 'hist', 'hist2d', 'boxplot', 'pie', 'imshow', 'contour', 'contourf', 'fill_between', 'fill_betweenx', 's…`	Matplotlib Axes/pyplot methods that put marks on a canvas.
`MATPLOTLIB_PLOT_RE`	`re.compile('\\b(?:plt\|pyplot\|ax\|axs\|axes\|axis\|ax\\d+\|ax_\\w+\|axs\\[[^\\]]\\]\|axes\\[[^\\]]\\])\\.(?:' + '\|'.join(MATPLOTLIB_PLOT_METHODS)…`
`PANDAS_PLOT_RE`	`re.compile('\\.plot\\s\\(\|\\.plot\\.\\w+\\s\\(')`	`df.plot(...)`, `df.plot.bar(...)`, `gdf.plot(column=...)` — pandas and geopandas draw through matplotlib, and students count these as matplotlib.
`ALTAIR_RE`	`re.compile('\\balt\\.\\w+\|\\baltair\\.\\w+\|\\.mark_\\w+\\s*\\(')`
`ALTAIR_CHART_RE`	`re.compile('\\b(?:alt\|altair)\\.(?:Chart\|layer\|hconcat\|vconcat\|concat)\\s\\(\|\\.mark_\\w+\\s\\(')`	Altair calls that only exist to draw, so a cell holding one is a chart cell even when the chart is assigned to a name and shown later.
`VEGALITE_MIME_RE`	`re.compile('^application/vnd\\.vega(?:lite)?\\.v\\d+\\+json$')`
`VEGA_EMBED_MARKER`	`'})('`	Altair's HTML renderer ends with `})(<spec>, <embedOpt>);` inside a <script>.
`COMPOSITION_KEYS`	`('hconcat', 'vconcat', 'concat', 'layer', 'facet', 'repeat', 'spec')`
`TRANSFORM_OPS`	`{'aggregate', 'bin', 'calculate', 'density', 'flatten', 'fold', 'impute', 'joinaggregate', 'loess', 'lookup', 'pivot', 'quantile', 'regress…`	"A transformation (mean, count, binning, etc)". A bare filter is a subset, not a transformation of the values, so it is deliberately not in this set.
`ENCODING_TRANSFORM_KEYS`	`('aggregate', 'bin', 'timeUnit')`
`AESTHETIC_PATTERNS`	`{'title': re.compile('\\.set_title\\s\\(\|\\bplt\\.title\\s\\(\|\\.suptitle\\s\\(\|\\btitle\\s='), 'x_label': re.compile('\\.set_xlabel\\s…`
`IMAGE_MIMES`	`('image/png', 'image/jpeg', 'image/svg+xml')`
`ALTAIR_REPR_RE`	`re.compile('^\\s*alt\\.(?:Chart\|LayerChart\|HConcatChart\|VConcatChart\|ConcatChart\|FacetChart\|RepeatChart)')`	Altair's plain-text fallback when a notebook was saved without rich output.
`HEADING_RE`	`re.compile('^\\s{0,3}#{1,6}\\s')`
`PLACEHOLDER_RE`	`re.compile('^\\s(?:\\{0,2})(?:your (?:answer\|discussion\|conclusion\|response) here\|todo\|tbd\|write your\|conclusion:?\|discussion:?)(?:\\*{0,…`

Classes

class SpecFacts #[source]

dataclass

What one Vega-Lite specification says the chart does.

Fields

Name	Type	Default
`marks`	`list[str]`	`field(default_factory=list)`
`transform_ops`	`list[str]`	`field(default_factory=list)`
`encoding_transforms`	`list[str]`	`field(default_factory=list)`
`selections`	`list[dict[str, Any]]`	`field(default_factory=list)`
`filter_params`	`list[str]`	`field(default_factory=list)`
`condition_params`	`list[str]`	`field(default_factory=list)`
`composition`	`list[str]`	`field(default_factory=list)`
`view_count`	`int`	`0`
`channels`	`list[str]`	`field(default_factory=list)`
`fields`	`list[str]`	`field(default_factory=list)`
`titles`	`list[str]`	`field(default_factory=list)`
`color_schemes`	`list[str]`	`field(default_factory=list)`
`has_tooltip`	`bool`	`False`

Properties

property brushes: list[dict[str, Any]] #[source]

Interval selections a viewer can drag — pan/zoom bindings excluded.

.interactive() also compiles to an interval selection, but with bind: "scales": it zooms the axes rather than selecting data. Counting it as a brush would give the mark to every student who added pan/zoom.

property has_brush: bool #[source]: No description.

property has_transformation: bool #[source]: No description.

property is_multi_view: bool #[source]: No description.

property cross_filter_params: list[str] #[source]: Selections that a different view filters on — a real cross-filter.

property has_cross_filter: bool #[source]: No description.

Methods

to_dict() -> dict[str, Any] #[source]: No description.

class ChartRecord #[source]

dataclass

One chart, as drawn by one cell.

Fields

Name	Type	Default	Notes
`library`	`str`	required
`code_cell`	`int`	required
`cell_index`	`int`	required
`source`	`str`	required
`rendered`	`bool`	`False`
`errored`	`bool`	`False`
`error`	`str`	`''`
`output_mimes`	`list[str]`	`field(default_factory=list)`
`specs`	`list[dict[str, Any]]`	`field(default_factory=list)`
`facts`	`SpecFacts`	`field(default_factory=SpecFacts)`
`discussion`	`str`	`''`
`discussion_words`	`int`	`0`
`preamble`	`str`	`''`
`preamble_words`	`int`	`0`
`aesthetics`	`dict[str, bool]`	`field(default_factory=dict)`
`output_source`	`str`	`SOURCE_NONE`	Which notebook these outputs were read from.

Properties

property ok: bool #[source]

Drew something, and the cell that drew it did not fail.

These are separate facts on purpose. The inline backend flushes any figure a cell created even when that cell then raised, so a seaborn call that died on a NameError still leaves an empty PNG in the notebook — rendered records that an image exists, ok records that it is a chart rather than the wreckage of one.

property has_discussion: bool #[source]: No description.

Methods

summary(max_source: int = 700) -> dict[str, Any] #[source]: Compact, TA-readable evidence (design.md §12).

class ChartEvidence #[source]

dataclass

Every chart found in a submission, in the order the cells appear.

Fields

Name	Type	Default
`source`	`str`	`SOURCE_NONE`
`notebook_path`	`str \| None`	`None`
`charts`	`list[ChartRecord]`	`field(default_factory=list)`
`n_code_cells`	`int`	`0`
`error`	`str \| None`	`None`

Properties

property from_execution: bool #[source]: Every chart that came out was produced by this grading run.

Methods

verified(records: list['ChartRecord']) -> bool #[source]: Were these particular charts produced here, rather than by the student?

by_library(library: str) -> list[ChartRecord] #[source]: No description.

rendered(library: str) -> list[ChartRecord] #[source]: No description.

summary() -> dict[str, Any] #[source]: No description.

Functions

libraries_in_source(source: str) -> list[str] #[source]

Which plotting libraries this cell draws with, most specific first.

Seaborn shadows matplotlib deliberately: sns.boxplot(...) followed by plt.title(...) is one seaborn chart, not a seaborn chart plus a matplotlib chart. Grading the two requirements separately depends on it.

extract_specs(output: dict[str, Any]) -> list[dict[str, Any]] #[source]: Vega-Lite specifications carried by one notebook output.

strip_spec_data(node: Any, depth: int = 0) -> Any #[source]

Drop inlined rows so a spec can be stored as rubric evidence.

A student's chart embeds their whole dataset in the spec; keeping it would put megabytes of their data into every results file.

analyze_spec(spec: dict[str, Any]) -> SpecFacts #[source]: Read one Vega-Lite spec, at any level of composition.

merge_facts(specs: Iterable[dict[str, Any]]) -> SpecFacts #[source]: One view of a cell that displayed more than one chart.

aesthetic_signals(source: str) -> dict[str, bool] #[source]

Presence of the labelling and colour choices a TA would look for.

These are signals, never a score: a chart can set every label and still be unreadable, and the assignment says a person grades the aesthetics.

collect_charts(executed_path: str | Path | None, submitted_path: str | Path | None = None) -> ChartEvidence #[source]

Chart evidence, taking each chart from whichever notebook drew it.

The executed notebook is the instrumented copy of the submitted one, so their code cells line up one to one and a chart can be matched across the two by its cell number and library. Where both drew it, the executed run wins; where only the student's saved output has it, that is used and recorded as such.

prose_word_count(text: str) -> int #[source]: Words of actual discussion — headings and placeholders do not count.