Selected Work
Legacy Data Migration Platform
A relational data platform consolidating historically grown legacy records into a clean target structure, with a Dash front end and REST API for exploration.
← All projectsHighlights
What Shipped
- Migrated over 100,000 structured records from inconsistent Excel-based legacy sources into a new relational schema.
- Automated transformation, cleansing, and validation with Python, removing manual reconciliation work.
- Enriched part data using external APIs and similarity analysis to map outdated naming conventions onto current ones.
- Delivered a Dash web application and REST API so the migrated data could be explored without SQL access.
The Story
Behind the Build
Overview
Built at BMW Group. The source data had grown historically rather than by design: part records spread across inconsistent Excel files, with naming conventions that had drifted over time and no single structure holding them together. The work was to move all of it into a clean relational target structure without losing anything in the process — a little over 100,000 structured records in the end.
Approach
Extraction, transformation, cleansing and validation were automated in Python rather than reconciled by hand, so the migration could be re-run rather than performed once and trusted. Validation ran as part of the pipeline, not as a review step afterwards.
Outdated part names were the harder problem: they could not simply be mapped, because the correct current name was not always recorded anywhere. That was resolved by integrating external APIs and using similarity analysis to enrich the part data and match old naming conventions onto current ones.
Outcome
The manual reconciliation work the process previously required was removed. A Dash web application and a REST API were delivered on top of the migrated database, so the data could be explored and interacted with directly rather than through SQL.
Keep Exploring