JOB DESCRIPTION

• Design and implement scalable data pipelines and transformation primarily using Databricks. Nice to have experience on Microsoft Fabric tools such as Azure Data Factory, PySpark notebooks, Spark SQL, and Python.
• Develop ETL processes to extract, transform, and load data from diverse sources into data warehouses or analytical systems within Databricks.
• Optimize big data processing workflows to enhance performance and efficiency using Databricks capabilities.
• Ensure data security and compliance with data privacy regulations throughout the data engineering process.
• Collaborate with stakeholders to gather requirements and create technical solutions using Microsoft Fabric.

KEY SKILLS

• Proficiency in SQL and Python for data modeling and querying within Microsoft Fabric.
• Experience with Databricks and Microsoft Fabric and its components like Azure Synapse, Data Lake, and Databricks.
• Knowledge of data warehousing and data governance within the Fabric ecosystem.
• Understanding of ETL development and data transformation processes using Fabric tools.
• Familiarity with cloud-based data engineering solutions, particularly Azure

Please refer the Job description for details