Civic Data
Cleaned, normalized, and documented Louisville public data you can actually build with. Two commands away.
Civic Data takes messy public datasets and turns them into a documented, queryable data layer. The point is not to mirror an open data portal. The point is to show what changed, what was dropped, what is trustworthy, and what claims the data cannot honestly support.
The current release covers Louisville: crime, 311 requests, property distress, permits, inspections, licenses, short-term rentals, ZIP boundaries, council districts, neighborhoods, HUD zones, population, and geography crosswalks — twenty datasets, about four million rows, every one carrying its lineage and license.
Get started in 60 seconds
python -m venv venv && venv\Scripts\activate # a venv keeps the CLI on your PATH pip install civic-data # consumer install — no database required civic-data get louisville # download the lake (sha256-verified) into ./export civic-data query -d crime --zip 40202 --limit 5 civic-data schema crime # columns + the dataset's honesty block (--full for everything)
Requires Python 3.11+. The lake is a ~200 MB download (~260 MB unpacked) of GeoParquet served from data.newguyai.com, and it ships its own full data dictionary. Every command takes --json; every dataset answers --sources with its provenance, license, and a how-to-cite line. Using an AI coding assistant? Point it at this page — or at llms.txt — and ask it to get you set up. (Outside a venv, python -m civic_data ... works even when the script is not on PATH.)
What it provides
A CLI
Query by dataset, ZIP, council district, or year; summarize a place; or drop to raw DuckDB SQL. The producer pipeline ships too, as an optional extra.
A data lake
GeoParquet files designed for DuckDB, local analysis, and AI-assisted workflows. Versioned, immutable releases; download any published version.
Lineage
A generated data dictionary that explains schema, caveats, transformations, and loss.
Why it is different
- It is built from source, not hand-maintained spreadsheets or one-off downloads.
- The data dictionary is generated from the live database and authored lineage config.
- Trust tiers distinguish trendable data from snapshots and caveated views.
- Every download is sha256-verified; every dataset cites its sources and license.
- Private-individual PII is stripped at the ingest boundary.
For AI agents
The lake describes itself. An agent can bootstrap from llms.txt, read the full machine contract at catalog.json, study the complete generated data dictionary (human / machine — both also ship inside the lake), and discover published releases at data.newguyai.com/registry.json. Every document is generated by the pipeline on each release — never hand-edited. One rule the docs teach agents: unusual values are usually documented source shape — read the dataset's honesty block before reporting discoveries.
Data licensing and attribution
Louisville Metro open data is published under the Open Data Commons Public Domain Dedication and License (PDDL). Geographic reference layers: Mapping Data Source: LOJIC. Population figures: this product uses the Census Bureau Data API but is not endorsed or certified by the Census Bureau.
The underlying data is public record, provided as-is. Civic Data licenses only its added value — the normalization and documentation — under CC BY 4.0 (code: MIT). Each dataset's exact terms, source pulls, and citation line ship in its catalog provenance: civic-data query -d <dataset> --sources.