London, Leicestershire - England, United Kingdom
-
Full Time
Start Date
Immediate
Expiry Date
07 Jan, 27
Salary
35000.0
Posted On
09 Oct, 26
Experience
2 year(s) or above
Remote Job
Yes
Telecommute
Yes
Sponsor Visa
No
Skills
Industry
Information Services
Description
Key ResponsibilitiesDesign, build, and maintain scalable and robust data pipelines using SQL and Python, and leveraging technologies such as: Databricks, Snowflake, Azure Data Factory, AWS Glue, Apache Airflow and Pyspark.
Lead the integration of complex data systems and ensure consistency and accuracy of data across multiple platforms.
Implement continuous integration and continuous deployment (CI/CD) practices for data pipelines to improve efficiency and quality of data processing.
Work closely with data architects, analysts, and other stakeholders to understand business requirements and translate them into technical implementations.
Oversee and manage a team of data engineers, providing guidance and mentorship to ensure high-quality project deliverables.
Knowledge of best practices in data governance, security, and compliance within the organisation.
Continuously evaluate new technologies, AI trends and tools to enhance the capabilities of the data engineering function.
QualificationsBachelor's or Master’s degree in Computer Science, Engineering, or a related field.
3+ years of industry experience in data engineering with a strong technical proficiency in SQL, Python, and big data technologies.
Expertise in building scalable data workflows using cloud-native orchestration tools and distributed data processing frameworks.
Solid understanding of CI/CD principles and DevOps/DevSecOps practices.
Experience leading on project workstreams.
Excellent problem-solving skills and the ability to work with ambiguity.
Proficient in leveraging AI-assisted workflows to optimise task efficiency.
Strong communication and interpersonal skills.
Responsibilities
Key ResponsibilitiesDesign, build, and maintain scalable and robust data pipelines using SQL and Python, and leveraging technologies such as: Databricks, Snowflake, Azure Data Factory, AWS Glue, Apache Airflow and Pyspark.
Lead the integration of complex data systems and ensure consistency and accuracy of data across multiple platforms.
Implement continuous integration and continuous deployment (CI/CD) practices for data pipelines to improve efficiency and quality of data processing.
Work closely with data architects, analysts, and other stakeholders to understand business requirements and translate them into technical implementations.
Oversee and manage a team of data engineers, providing guidance and mentorship to ensure high-quality project deliverables.
Knowledge of best practices in data governance, security, and compliance within the organisation.
Continuously evaluate new technologies, AI trends and tools to enhance the capabilities of the data engineering function.
QualificationsBachelor's or Master’s degree in Computer Science, Engineering, or a related field.
3+ years of industry experience in data engineering with a strong technical proficiency in SQL, Python, and big data technologies.
Expertise in building scalable data workflows using cloud-native orchestration tools and distributed data processing frameworks.
Solid understanding of CI/CD principles and DevOps/DevSecOps practices.
Experience leading on project workstreams.
Excellent problem-solving skills and the ability to work with ambiguity.
Proficient in leveraging AI-assisted workflows to optimise task efficiency.