# civic-data

> Cleaned, normalized, and documented Louisville public open data. This document is the as-built data reality — what we actually hold, what it honestly measures, how much to trust it, and (in the lineage half) exactly what we did to it and why. It is regenerated from the live database + ingest code; the numbers cannot drift from it.

## About this data

- Geography unit: ZIP code (41-ZIP Louisville boundary set)
- As of: 2026-07-12 (data_version: 2026-07-12+630e3609)
- 20 datasets, 3,974,314 rows total
- Format: DuckDB over GeoParquet (no server). Geometry is WKB, EPSG:4326.
- Read any dataset as read_parquet('<path>/**/*.parquet').
- Full data dictionary SHIPS IN THIS LAKE: `data-dictionary.md` (human) / `data-reality.json` (machine) - dataset-level caveats, transformation lineage, loss accounting, null rates.

## Trust tiers

- **trendable**: Clean, full multi-year coverage — safe for time series and trend claims.
- **viewable_caveated**: Real data with gaps/cliffs/discontinuities — viewable within a stated window, must not be presented as an uncaveated trend.
- **snapshot_only**: No honest time series — current-state snapshot only.
- **unsupported**: Claims the data cannot back — named so nobody proposes vaporware on top.

## Datasets

- **crime** - Crime incidents (LMPD / NIBRS) (1,242,777 rows; trendable; domain; partitioned by year; geometry)
- **service_requests** - 311 service requests (Open311) (2,209,988 rows; viewable_caveated; domain; partitioned by year; geometry)
- **landbank_sales** - Land Bank Authority sales (historical dispositions) (399 rows; viewable_caveated; domain; temporal via sale_date)
- **lien_orders** - Code-enforcement lien final orders (blight signal) (298 rows; snapshot_only; domain; temporal via notification_date; geometry)
- **property_foreclosures** - Property foreclosure filings (1,734 rows; viewable_caveated; domain; temporal via action_filed)
- **food_service_establishments** - Permitted food service establishments (3,366 rows; snapshot_only; domain)
- **abc_licenses** - ABC (alcohol) licenses — local + state (7,149 rows; snapshot_only; domain; geometry)
- **restaurant_inspections** - Restaurant inspection scores (11,273 rows; viewable_caveated; domain; temporal via inspection_date)
- **construction_permits** - Active construction permits (23,308 rows; viewable_caveated; domain; temporal via issue_date; geometry)
- **str_licenses** - Short-Term Rental registrations (1,248 rows; snapshot_only; domain; temporal via date_opened; geometry)
- **zip_codes** - Canonical 41-ZIP Louisville boundary set (ArcGIS portal) (41 rows; reference; reference; geometry)
- **address_points** - LOJIC Jefferson County address points (~450K) (450,308 rows; reference; reference; geometry)
- **street_intersections** - LOJIC Jefferson County street intersections (~21K) (21,048 rows; reference; reference; geometry)
- **zip_to_district** - DERIVED crosswalk: ZIP ↔ Council district area overlap (bidirectional coverage fractions, ::geography, sliver-filtered >0 (133 rows; reference; reference)
- **zip_to_neighborhood** - DERIVED crosswalk: ZIP ↔ neighborhood area overlap (318 rows; reference; reference)
- **zip_to_hud_zone** - DERIVED crosswalk: ZIP ↔ HUD 1979 grant-zone area overlap (159 rows; reference; reference)
- **council_districts** - Metro Council district boundaries (versioned) (26 rows; reference; mdm; geometry)
- **neighborhoods** - Neighborhood boundaries (150 rows; reference; mdm; geometry)
- **hud_zones** - HUD 1979 grant-zone boundaries (71 rows; reference; mdm; geometry)
- **zip_population** - Census ACS population by ZIP (per-capita denominator) (520 rows; reference; mdm)

## Getting the CLI and this lake

```
pip install civic-data                    # consumer install - no Postgres needed
civic-data cities                         # list published city lakes
civic-data get louisville                  # download + verify this lake into ./export
```

## Querying

With the `civic-data` CLI (every verb takes `--json` and `--lake <dir>`):

```
civic-data catalog --json --lake .        # the machine-readable contract (start here)
civic-data datasets --lake .              # list datasets by key
civic-data schema crime --lake .          # one dataset's columns + shape
civic-data query -d crime --zip 40202 --year 2024 --limit 5 --json --lake .
civic-data place 40202 --lake .           # summary bundle for one place
civic-data sql "SELECT zip_code, count(*) FROM crime GROUP BY 1" --lake .
```

Or raw DuckDB (no CLI needed):

```sql
-- illustrative --
SELECT zip_code, count(*) FROM read_parquet('crime/**/*.parquet') WHERE zip_code='40202' GROUP BY 1;
```

## Honesty notes

- Trust tiers are load-bearing: `snapshot_only` datasets are current rosters (no history); `viewable_caveated` datasets have documented capture gaps. Read `trust_tiers` above before trending anything.
- BEFORE interpreting unusual values (odd dates, empty columns, short coverage windows), read the dataset's `honesty` block in the catalog and its full dictionary entry (`civic-data schema <key> --full`). Most anomalies are DOCUMENTED source shape - report them as documented, not as discoveries.
- District filtering on point datasets (crime, service_requests, lien_orders) is a query-time point-in-polygon over geocoded rows only - rows without geometry are excluded and the CLI says so on stderr.
- `place` reports "latest year" capped at the catalog's as_of year: the source data contains future-dated typos (kept as-is - source shape is documented, not fixed).

Generated from catalog.json (data_version 2026-07-12+630e3609) - do not hand-edit; re-run `civic-data export` (or `civic-data place <place> --export` for a bundle).