Company Analytics API
Athena KPIs and Glue-loaded network data behind one Python API
An analytics platform with two data paths behind one Python API. KPIs are computed straight from raw JSON files in S3 with Amazon Athena — no warehouse to load — and a cache in front keeps repeat requests fast and cheap. Alongside it, an AWS Glue ETL job reads CSV files of company network information and loads them into Postgres, which the same API serves to a dashboard that shows each company as a list of status flags.
- My role
- Data architecture, ETL & query design, Python API
- Stack
- PythonAmazon AthenaAmazon S3AWS Glue (ETL + Data Catalog)PostgreSQLJSONCSVboto3SQLResult caching
Architecture
overview
Highlights
- Right engine for each access pattern: Athena for heavy aggregate KPIs over raw files, Postgres for fast lookups of per-company records.
- No ingestion pipeline for KPIs: JSON lands in S3 and is queryable immediately — Athena reads it in place with schema-on-read.
- A cache in front of Athena turns repeat KPI requests from seconds into milliseconds and keeps the pay-per-data-scanned bill predictable.
- Glue ETL turns CSV network data into clean relational tables, so the dashboard's company flags come from simple, indexed queries.
- One API for every consumer: dashboards never touch SQL, S3, Athena or Postgres directly.
Curious how this would fit your team?
Ask me anything about Company Analytics API — I reply personally.