Key Responsibilities

• Lead the end-to-end architecture and deployment of large-scale data platforms using Databricks, Apache Spark, Delta Lake and Unity Catalog.
• Design scalable batch and real-time data processing solutions and ETL/ELT pipelines.
• Develop medallion and lakehouse architecture patterns for enterprise data workloads.
• Architect secure and resilient Databricks environments across Azure, AWS or GCP.
• Establish data governance, access controls, lineage and compliance standards through Unity Catalog.
• Design integrations between Databricks and cloud storage, databases, APIs, BI platforms and enterprise applications.
• Define platform security standards covering identity, networking, encryption, secrets management and role-based access.