World Libraries Directory

A Python pipeline that pulls library records from Wikidata, classifies them by country, region, library type, and subject specialty, and loads them into a Notion database for browsing, review, and mapping.

What it does

1. Extract: queries the Wikidata SPARQL endpoint for everything that is a library (or a subclass, such as a national, academic, law, or medical library) in the requested countries. Large countries are handled in batches, with retries for rate limits and truncated responses.
2. Transform: maps Wikidata type labels to a controlled vocabulary:
   • Library Type: National, Academic, Public, Research / Special, Government, Digital, Religious, School
   • Specialties: Law, Medicine & Health, Music, Art & Architecture, Maps & Geography, Science & Technology, Religion & Theology, Archives, and more
   • Region: continent-based, with Latin America & Caribbean and the Middle East split out
3. Load: writes rows to Notion through the public API (version 2025-09-03), skipping anything already present by Wikidata ID or by name + country, so runs are safely repeatable. An optional flag also fills Notion's place property so libraries appear on a map view.

Each run can also save a CSV copy of the fetched data, including coordinates, for use in QGIS or other GIS tools.

Notion database schema

Property	Type	Notes
Library Name	Title
Country, Region	Select
City	Text	From Wikidata's "located in the administrative territorial entity"
Library Type	Select	Rule-based from Wikidata types
Specialties	Multi-select	Rule-based from Wikidata types
Founded	Number	Year
Website	URL
Latitude, Longitude	Number
Location	Place	Powers the map view (--place / --backfill-place)
Wikidata ID	Text	Used for de-duplication
Source	Select	Wikidata or Manual
Verified	Checkbox	For manual review

Setup

python3 -m venv .venv
.venv/bin/python -m pip install -r requirements.txt

1. Create an internal integration at https://www.notion.so/profile/integrations and copy its secret.
2. In Notion, open the database, go to ••• → Connections, and add the integration.
3. Set the secret as an environment variable. Never commit it.

   export NOTION_TOKEN="ntn_..."

4. To target a different database, set NOTION_DATA_SOURCE_ID or pass --data-source-id.

Usage

# Preview without writing to Notion
.venv/bin/python import_libraries.py --countries IE NZ --dry-run --csv preview.csv

# Import with map pins
.venv/bin/python import_libraries.py --countries CA AU --place --csv batch.csv

# Cap a very large country
.venv/bin/python import_libraries.py --countries US --limit 1000 --place

# Add map pins to rows that are already in Notion
.venv/bin/python import_libraries.py --backfill-place

Countries are ISO 3166-1 alpha-2 codes. Notion writes run at about two rows per second, so large countries take a while. Progress is printed every 50 rows, and pressing Ctrl+C stops cleanly; rerunning resumes where it left off.

Limitations

• Coverage reflects Wikidata, which is strongest for national, academic, and notable libraries and patchy for small public and school libraries.
• Library Type and Specialties are inferred from keyword rules (TYPE_RULES and SPECIALTY_RULES in the script), so entries should be spot-checked. The Verified checkbox supports that review.
• Notion's public API format for place properties isn't clearly documented. The script tries known formats and turns the feature off for the run if Notion rejects them.
• Notion slows down past roughly 10,000 rows, so the full global dataset is better suited to a database such as PostgreSQL/PostGIS.

Data and license

Library data comes from Wikidata and is available under CC0. The code is released under the MIT License (see LICENSE).