API reference · assignments/hw3.py · 2574 lines

`assignments.hw3`

Assignment 3 autograder — evictions, code violations and the NDVI.

Graded against assignment_template/assignment-3.ipynb.

Why this grader looks nothing like the other two

Assignments 1 and 2 have no answer key: one class downloads its own Zillow extract, the other picks its own dataset. Assignment 3 hands out one zip file and everyone uses it, so for the first time the right answers are known. Philadelphia has 384 census tracts in PA-tracts.geojson; the fifteen listed violation types cover 34,108 rows of li_violations.csv; the median NDVI inside the city limits is 0.2025. Those are facts about the shipped data, so the checks below compare against them rather than inferring intent.

That also changes what confidence means here. In Assignment 1 several frames may plausibly be "the Philadelphia subset", and the ambiguity is the finding. Here a frame either carries 384 Philadelphia tracts or it does not; if one does, the step demonstrably happened, and it does not matter which variable holds it. So a matched expected value scores at confidence 1.0, and confidence drops only where the grader is choosing between readings of a partial answer.

Where the evidence comes from

Two places, and every item is graded on both.

The first is the notebook the student handed in. Re-running is the fragile part here — five data files read from whatever relative path each student used — and when the placement misses, the notebook dies at the first read_file and a correct submission grades as a column of zeros. Their own run did not have that problem, and the file carries what it printed: 384, 34108, Median NDVI, city: 0.2025, and :HoloMap [year] for the chart in 1.1.5. grader/answers.py harvests those, and the evaluators at the bottom of this module read them.

The second is the probe payload, extended for this assignment (see grader/probe_runtime.py): GeoDataFrames now carry their CRS, geometry types and total area; numpy arrays carry NaN-aware statistics, which is the only way to describe an NDVI array that is mostly NaN; shapely geometries and open rasterio datasets are described in place.

The plots are the interesting part. An hvplot call returns a holoviews object that knows what it is — a HoloMap carries the dimension its widget selects and one frame per value of it, a Layout carries its row/column shape — and those objects are read out of the kernel, Out included, because df.hvplot(...) on the last line of a cell is never bound to a name. So groupby= and dynamic=False are graded by their effect rather than by searching the source for the keywords.

The matplotlib figures (the hex bin map, the two tree plots) have no such structure, only a picture, so those reuse grader/charts.py exactly as Assignment 2 does — which already merges the executed and submitted notebooks — and the one item the assignment marks on styling comes back for a person to confirm.

The two readings are combined in grade() by taking the better of them: a step counts as done if either shows the right answer. Neither is reliably better, so neither is ranked above the other; what differs is the confidence attached, and whether a TA is told which one carried the item.

Summary

Name	Kind	Description
`HW3Grader`	class

Constants

Name	Value	Notes
`WIDGET_CONTAINERS`	`('HoloMap', 'GridSpace', 'NdLayout')`	holoviews container types that mean "one chart per value of a dimension".
`LAZY_CONTAINERS`	`('DynamicMap',)`	The same thing, but never evaluated: hvplot returns this when `dynamic=False` is left out, and it holds no frames at all.
`LAYOUT_CONTAINERS`	`('Layout', 'NdLayout')`

Classes

class HW3Grader(_SavedAnswerChecks, AssignmentGrader) #[source]

registered grader

No description.

Class attributes

Name	Default	Notes
`assignment_id`	`'hw3'`
`SUBMITTED_CONFIDENCE`	`0.9`	Confidence for a step seen only in the student's own saved output: above the review threshold, so it does not flag a correct submission on its own, but below the 1.0 of a result this grading run produced itself. A saved output is the student's claim about their run — it can be stale, and it can be edited by hand.
`NARROW_FRAME_COLUMNS`	`12`	An Eviction Lab frame carries 399 numeric columns, and across 3,217 tracts one of them will sum to almost any total by coincidence. A result of a group-by or a merge is narrow, so width is the guard: it keeps the raw file out of searches that look for a frame by what its numbers add up to.

Methods

answers(ctx: GradingContext) -> answerlib.NotebookAnswers #[source]: What the notebooks show, read once per submission and reused.

grade(ctx: GradingContext) -> list[RubricItemResult] #[source]

Grade on the objects this run produced and on what the file shows.

The two readings are independent and neither is reliably better, so they are combined by taking the better of them rather than by ranking them: a step counts as done if the executed run shows the right answer or the submitted notebook does. That is what stops a data path the grader could not resolve from turning a correct submission into a column of zeros.

charts(ctx: GradingContext) -> chartlib.ChartEvidence #[source]: matplotlib figures, read once per submission and reused.

Rubric checks 22

Each check_<rubric_id> method grades the rubric item with that id and returns one RubricItemResult.

check_eviction_data_loading(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

3 ptsstructural1.1.1 Eviction data loaded with geopandasrubrics/hw3.yaml

PA-tracts.geojson is read with geopandas from a relative path.

Config keys: absolute_path_penalty expected_rows missing_read_penalty not_geodataframe_penalty required_imports row_tolerance wrong_size_penalty

check_philadelphia_tracts(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

5 ptsdeterministic1.1.2 Data trimmed to Philadelphiarubrics/hw3.yaml

The Pennsylvania tracts are trimmed to the 384 tracts of Philadelphia County. The file has no city column — the county lives in pl ("Philadelphia County, Pennsylvania") and in the 42101 prefix of GEOID.

Config keys: expected_rows geoid_prefix impure_credit_ratio incomplete_credit_ratio min_purity

check_tidy_evictions(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

6 ptsdeterministic1.1.3 Melted to tidy formatrubrics/hw3.yaml

pd.melt() over the fourteen eviction columns e-03 to e-16, keeping GEOID and geometry as id_vars — 384 tracts x 14 years = 5,376 rows.

Config keys: expected_rows expected_value_sum expected_year_labels missing_geometry_credit_ratio required_id_columns value_column_hints value_sum_rel_tol wrong_year_span_credit_ratio

check_evictions_by_year_plot(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

5 ptsdeterministic1.1.4 Yearly eviction trend plotted with hvplotrubrics/hw3.yaml

A groupby over the year column, summed, plotted with hvplot — any chart type. Graded on the numbers actually plotted.

Config keys: accepted_elements expected_max expected_points expected_total total_rel_tol wrong_library_credit_ratio wrong_values_credit_ratio

check_evictions_choropleth(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

5 ptsdeterministic1.1.5 Year-by-year eviction choropleth with a widgetrubrics/hw3.yaml

An hvplot choropleth of the tidy frame with groupby= on the year column and dynamic=False, producing a HoloMap with one frame per year.

Config keys: dynamic_map_credit_ratio expected_frames expected_keys key_dimension_hints no_widget_credit_ratio polygon_elements wrong_dimension_credit_ratio

check_violations_choropleth(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

5 ptsdeterministic1.2.7 Choropleth per violation type with a widgetrubrics/hw3.yaml

An hvplot choropleth with groupby= on the violation type and dynamic=False — a HoloMap with one frame per type.

Config keys: dynamic_map_credit_ratio expected_frames key_dimension_hints no_widget_credit_ratio polygon_elements wrong_dimension_credit_ratio

check_violations_loading(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

4 ptsdeterministic1.2.1 Violations loaded and converted to a GeoDataFramerubrics/hw3.yaml

li_violations.csv read with pandas and converted to a GeoDataFrame of 434,052 points built from the lng/lat columns, with a CRS set.

Config keys: bounds_tolerance expected_bounds expected_crs_epsg expected_geometry expected_rows missing_crs_credit_ratio not_geodataframe_credit_ratio row_tolerance swapped_coordinates_credit_ratio

check_violations_subset(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

4 ptsdeterministic1.2.2 Trimmed to the fifteen maintenance violation typesrubrics/hw3.yaml

The listed fifteen violation types, which cover 34,108 of the 434,052 rows.

Config keys: expected_rows expected_type_count partial_types_credit_ratio violation_types

check_hexbin_map(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

4 ptsdeterministic1.2.3 Hex bin map with census tracts overlaidrubrics/hw3.yaml

matplotlib's hexbin() over the violation points with the tract boundaries drawn on top, in a projected CRS rather than EPSG:4326.

Config keys: hexbin_patterns missing_overlay_credit_ratio not_rendered_credit_ratio overlay_patterns reprojection_patterns unprojected_credit_ratio

check_spatial_join(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

5 ptsdeterministic1.2.4 Violations spatially joined to census tractsrubrics/hw3.yaml

geopandas.sjoin() attaches a GEOID to each of the 34,108 violations. Every one of them falls inside a Philadelphia tract, so a correct join loses no rows and leaves no GEOID missing.

Config keys: expected_rows geoid_prefix lossy_join_credit_ratio requires_columns row_tolerance unfiltered_expected_rows unfiltered_join_credit_ratio

check_violations_by_tract(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

5 ptsdeterministic1.2.5 Violations counted per type and tractrubrics/hw3.yaml

A groupby over violation type and GEOID giving a three-column frame (violationdescription, GEOID, N) whose counts sum to 34,108.

Config keys: allowed_row_counts count_column_hints expected_count_sum expected_type_count partial_grouping_credit_ratio required_columns

check_merge_geometries(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

3 ptsdeterministic1.2.6 Counts merged back onto tract geometriesrubrics/hw3.yaml

pandas.merge() with the tract geometries first, so the result is a GeoDataFrame and not a plain DataFrame.

Config keys: allowed_row_counts not_geodataframe_credit_ratio require_geodataframe

check_side_by_side(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

4 ptsdeterministic1.3 Evictions and violations side by siderubrics/hw3.yaml

2016 evictions and a single violation type, drawn as two choropleths and composed into one row of two columns.

Config keys: eviction_total_rel_tol expected_columns expected_eviction_year_total not_composed_credit_ratio polygon_elements still_grouped_credit_ratio wrong_orientation_credit_ratio wrong_subset_credit_ratio

check_landsat_loading(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

3 ptsdeterministic2.1.1 Landsat scene opened with rasteriorubrics/hw3.yaml

The ten-band Landsat 8 scene is opened with rasterio.

Config keys: expected_array_shape expected_band_count expected_crs_epsg expected_height expected_width wrong_raster_credit_ratio

check_city_suburb_polygons(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

5 ptsdeterministic2.1.2 City and suburb polygons builtrubrics/hw3.yaml

The city limits polygon, and a suburbs polygon made by subtracting it from its own envelope. Both must be in the raster's CRS for the mask to work.

Config keys: area_rel_tol envelope_not_differenced_credit_ratio expected_city_area expected_suburb_area expected_suburb_geom_types missing_suburbs_credit_ratio unprojected_credit_ratio

check_ndvi_masking(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

6 ptsdeterministic2.1.3 Raster masked and NDVI computed for each arearubrics/hw3.yaml

rasterio.mask.mask() applied with each polygon, and NDVI computed from the red and near-infrared bands of the masked array.

Config keys: expected_city_median expected_city_pixels expected_suburb_median expected_suburb_pixels mask_patterns median_abs_tol one_area_credit_ratio pixel_rel_tol valid_range wrong_bands_credit_ratio

check_ndvi_medians(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

4 ptsdeterministic2.1.4 Median NDVI compared between city and suburbsrubrics/hw3.yaml

np.nanmedian() over each NDVI array, printed, with the suburbs coming out greener than the city.

Config keys: expect_suburbs_greater expected_city_median expected_suburb_median median_abs_tol nan_result_credit_ratio print_patterns right_direction_credit_ratio

check_tree_data_loading(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

3 ptsdeterministic2.2.1 Street tree data loadedrubrics/hw3.yaml

The 2,480 street tree points are loaded with geopandas.

Config keys: expected_geometry expected_rows raster_crs_epsg row_tolerance unprojected_credit_ratio

check_tree_ndvi(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

4 ptsdeterministic2.2.2 NDVI sampled at the tree locationsrubrics/hw3.yaml

rasterstats used to read the NDVI at each of the 2,480 points, giving one value per tree.

Config keys: alternative_patterns count_tolerance expected_count expected_mean expected_median max_nan_fraction median_abs_tol rasterstats_patterns valid_range wrong_values_credit_ratio

check_tree_plots(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

4 ptsdeterministic2.2.3 Histogram and mapped tree pointsrubrics/hw3.yaml

A matplotlib histogram of the NDVI values with a vertical line at zero, and a geopandas plot of the tree points coloured by NDVI with the city limits drawn on it.

Config keys: boundary_patterns detail_share histogram_patterns histogram_share not_rendered_credit_ratio point_plot_patterns zero_line_patterns

check_tree_plot_styling(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

3 ptsqualitative2.2.3 Figure styling (labels, legend, colour choices)rubrics/hw3.yaml

The one item on this assignment that a person has to look at.

The assignment asks for figures that are "clear and well-styled", which is a statement about a picture. The signals below say where to look; they cannot say whether the picture reads, so this always comes back as manual review with a provisional score rather than a silent mark.

Config keys: provisional_ceiling_ratio weighted_signals

check_common_violations_extra_credit(ctx: GradingContext, item: RubricItem) -> RubricItemResult #[source]

0 ptsdeterministicExtra credit: the twenty most common violation typesrubrics/hw3.yaml

The twenty most common violation types over 2012-2016 mapped as a second set of interactive choropleths, with three types named as having little spatial overlap with evictions. Worth no points in the rubric total; when it is found, the item is flagged so a TA can award the bonus by hand.

Config keys: expected_count_sum expected_frames min_discussion_words suggested_points