About the job

Overview

Ref: 118584

Senior Data Engineer

Exciting opportunity for a Senior Data Engineer to own the maintenance, expansion, and performance of Databricks-based data platform. This is a hands-on role, you'll be writing pipeline code, building jobs, and modelling as the main part of your day to day.

Role Responsibilities

Responsibilities will include:

• Designing, building, and operating Databricks Pipelines and Jobs
• Designing and building Delta Lake pipelines
• Modelling data
• Owning Databricks deployment bundles, packaging, versioning, and promoting pipeline code and job definitions
• Building and maintaining environment-variable-driven Azure Data Factory pipelines
• Building the data layer that feeds Microsoft Fabric/OneLake

Person Specification

Essential skills will include:

• Hands-on experience building and running Databricks Pipeline/Job structure in a production
• Strong data modelling ability, able to look at raw source data and design sensible, well-structured silver/gold tables
• Strong understanding of medallion (bronze/silver/gold) design patterns, and data architecture right at table level
• Solid Delta Lake knowledge, deletion vectors, time travel, optimization, compatibility considerations
• Experience with Databricks Asset Bundles or equivalent deployment/CI practices for data pipelines
• Practical experience building environment-variable/parameter-driven Azure Data Factory pipelines
• Proven experience of managing large data architectures
• Strong SQL, and comfort in at least one of Python/Scala for pipeline logic

You will be the technical expert for the data solution, to work alongside the engineering lead to manage the data aspects of the digital solution.

Next Steps

Apply by contacting Ciaran Ahern, cahern@fpsg.com

Equal Opportunities

FPSG is committed to equal opportunities regardless of gender, race, disability, sexual orientation, religion or belief and age.