What you'll do

• Lead our Databricks Unity Catalog migration. Own the transition from legacy Databricks workspaces to Unity Catalog and Terraformenabled workspaces — ensuring governance, lineage, and access control are built into the platform from the start.
• Own our Kubernetesbased compute platform. Manage the Kubernetes infrastructure that provides compute for Spark and Airflow jobs, keeping it reliable, efficient, and easy to build on.
• Build and maintain infrastructureascode. Own our Terraform modules and GitHub Actions pipelines, and set the standard for how infrastructure is provisioned and managed across the platform.
• Ensure platform reliability and observability. Keep critical data paths highly available, improve operational maturity, and invest in automation so the platform runs with minimal manual intervention.
• Collaborate across the data org. Partner with data engineering and analytics teams to ensure clean handover points and that platform changes enable rather than block downstream work.

What you'll bring

• Strong Python and Terraform skills, with a track record of building and operating production data infrastructure SQL knowledge is a plus
• Hands-on experience with Databricks — Unity Catalog experience is a strong plus.
• Deep familiarity with Spark and distributed compute.
• Experience orchestrating workflows with Airflow in production environments.
• Proficiency with Terraform and AWS infrastructure.
• Experience managing Kubernetes clusters for data workloads.
• A strong ownership mindset — you care about reliability, efficiency, and the developer experience of the teams you serve.

Nice to have

• Experience with Snowflake and large-scale data migrations.
• Familiarity with data lakehouse architectures.
• Prior experience contributing to internal developer platforms.
•