Join our team as a Data Engineering / Data Migration Lead and take the helm in an exciting migration of data workloads from SAS to Databricks for a major brand client. If you are passionate about Databricks, Spark, and cloud-based data engineering, this is your opportunity to drive a high-impact, transformative project. Immediate start available!
Job Type: Contract (with potential for full-time permanent position).
Apply for a position via our career portal
https://ingrity.com/careers/?jobId=Wj-MLfcyg9ie

ABOUT THE JOB:

We are seeking a skilled and experienced Data Engineering/ Data Migration Lead with extensive Databricks expertise to lead the migration of data workloads from SAS to Databricks. The ideal candidate will possess a deep understanding of data engineering principles and a strong grasp of data processing, cloud-based data platforms, and ETL optimisation within Databricks. This role requires collaboration with cross-functional teams to ensure a seamless transition, optimise data pipelines, and enhance our data infrastructure. This position is available to start immediately.

REQUIREMENTS:

• 5+ years of experience in leading Data Engineering/Data Migration projects, with a strong emphasis on Databricks.
• Proven expertise in Databricks, including cluster management, Delta Lake, and performance optimization.
• Strong hands-on experience in Spark (PySpark, Scala, or SQL) for data processing.
• Deep knowledge of ETL processes, data modelling, and data warehousing concepts.
• Experience with cloud platforms (Azure, AWS) and their data services, particularly in relation to Databricks.
• Excellent problem-solving and troubleshooting skills.
• Strong communication skills, with the ability to explain technical concepts to non-technical stakeholders.
  Apply for a position through our company career portal using the link below for a quick and easy application process.
  https://ingrity.com/careers/?jobId=Wj-MLfcyg9ie

• Lead the migration of data processing and analytics workloads from SAS to Databricks, ensuring a seamless transition with minimal business disruption.
• Re-engineer SAS-based transformations and business logic into Databricks using Spark SQL, PySpark, or Scala.
• Design, develop, and optimise data pipelines and ETL workflows in Databricks for scalability and performance.
• Work closely with data scientists, analysts, and business stakeholders to align data infrastructure with business needs.
• Implement best practices for data governance, security, and quality assurance within Databricks.
• Troubleshoot and resolve migration challenges, ensuring data integrity and operational efficiency.
• Provide technical leadership and mentorship to the team.
• Document migration strategies, workflows, and processes for future reference.