Loc Events
From Ticketmaster to the city.
The starting point.
An events app needs dependable external data before people can discover events, save favourites, or say they are going. Ticketmaster responses need validation, bounded pagination, and a reliable path into the application database.
What I built.
I implemented Ticketmaster ingestion with Python and Pydantic, expanded date-window pagination, and integrated PostgreSQL publishing into Airflow. I changed the refresh to preserve backend dependencies and contributed a Streamlit health page for event coverage and freshness.
This is a HYF team project. My work built on the provided operational template and focused on ingestion, publishing, orchestration integration and monitoring. Pavel led dbt transformations and Databricks. The frontend, wider backend and baseline infrastructure were shared team and mentor work.
Follow the flow.
Request Netherlands events with countryCode=NL, five consecutive 30-day windows, and bounded pagination to respect the API’s deep-paging limit.
Five consecutive 30-day windows expand ingestion coverage. One development run landed 1,914 events with 0 rejected records. Publishing preserves existing backend views, permissions, and indexes.
Decisions that matter.
- 01
Keep the source boundary explicit.
Filter for the Netherlands at the API. Pydantic validation and event-ID deduplication happen before downstream publishing.
- 02
Publish without dropping dependencies.
A full refresh must not destroy views or permissions. I replaced table-dropping behaviour with transactional publishing into the existing PostgreSQL table.
- 03
Expand coverage with bounded windows.
A single query runs into Ticketmaster’s deep-paging limit. Five consecutive 30-day windows expand coverage while keeping pagination and duplicates under control.
The work is open.
Implementation, decisions, and review history.
Built with
Python / Pydantic / Ticketmaster API / Azure / Airflow / PostgreSQL / Docker / GitHub Actions / pytest