DATA ENGINEER - AZURE & FABRIC

We are seeking a highly skilled Data Engineer for a contract role focused on a critical data migration project. This is a hybrid position based in Melbourne, requiring 2 days in the office. The ideal candidate will have strong expertise in building and optimizing data pipelines in the Azure ecosystem.

REQUIRED QUALIFICATIONS & SKILLS:

• 5-6 years of hands-on experience in data engineering, data migration, and ETL development.
• Strong expertise in Azure Data Factory (ADF), including pipeline design, performance tuning, and exception handling.
• Proficiency in SQL and experience with SQL Server database design and optimization.
• Experience with Azure Storage solutions (Blob, ADLS Gen2).
• Knowledge of Python and PySpark for data transformation and cleansing.
• Familiarity with Microsoft Fabric components (Dataflows, Pipelines, Lakehouse, and Warehouse) is a strong plus.
  This is an excellent opportunity to play a key role in a high-impact data migration initiative

• Design, develop, and manage robust data pipelines using Azure Data Factory (ADF) and Microsoft Fabric components.
• Implement and optimize full and incremental data load strategies for a large-scale data migration.
• Ensure data integrity and quality through meticulous transformation, cleansing, and validation using Python and PySpark.
• Develop and manage complex SQL queries, stored procedures, and scripts for data extraction and loading.
• Work extensively with Azure Storage (Blob & ADLS Gen2) and Azure SQL databases.
• Collaborate with cross-functional teams to align data migration activities with business objectives and ensure high data quality.