About the job

KEY INFORMATION

Position: Senior Data Engineer

Reporting to: Data Engineering Manager

Location: Manchester

Overview of the Role:

As a Senior Data Engineer, you will be an integral part of a cross-functional feature development squad You will play a key role in the evolution of our payments intelligence platform towards a Databricks-native and increasingly agentic operating model. You will design, build and operate secure data ingestion, orchestration and platform capabilities, using Databricks as the first-choice execution platform and Azure services where appropriate. The focus remains on production data engineering rather than AI model development: creating reliable, observable and well-governed foundations that can be operated by both engineers and automated/agentic workflows. You will work across client onboarding, data retrieval, ELT, CI/CD and platform reliability to deliver continuous business value.

JOB ROLE

Key accountability of this role:

Delivery, support and continuous improvement of secure, scalable, Azure/Databricks data capabilities and infrastructure on our payments intelligence platform.

Key responsibilities will include:

• Collaborate within a cross-functional squad to define and deliver scalable, secure data and platform capabilities aligned with business outcomes
• Design Databricks-native solutions as the default, using Spark/SQL, Lakeflow Jobs, Unity Catalog, Volumes and Databricks Asset Bundles, with Azure services used where they add value
• Build and operate data retrieval and ingestion across APIs, SFTP/FTPS and browser automation, migrating suitable orchestration from external services into Databricks-native workflows
• Own client data onboarding flows, including configuration, historical backfills, validation, data quality, monitoring and operational handover
• Build reusable platform capabilities and guardrails that enable agentic workflows to safely automate onboarding, operational diagnosis and routine engineering tasks, with human approval where required
• Engineer for production reliability through observability, alerting, retries/idempotency, failure recovery, performance and cost optimisation; investigate complex failures and drive root-cause fixes
• Deliver changes through version control, automated testing, CI/CD and Infrastructure as Code, maintaining secure coding and deployment standards through code review
• Apply strong security and governance practices across Unity Catalog, Entra/RBAC, secrets, credentials and data access
• Use advanced Python with strong SQL/Spark skills to develop maintainable, reusable libraries, frameworks and data pipelines
• Mentor data engineers and drive continuous improvement, evaluating new Databricks and automation capabilities against clear technical and business value
• 
  

Requirements

WHAT WE'RE LOOKING FOR

You are a great match if:

• 3+ years professional experience as a Data Engineer, DevOps Engineer or Systems Engineer delivering production data platforms
• Strong hands-on experience with Azure Databricks, including Spark/SQL, workflow orchestration and production lakehouse data engineering
• Advanced proficiency in Python for data retrieval, processing, automation, integration and reusable engineering libraries
• Experience designing and operating ETL/ELT pipelines with data quality controls, schema handling, monitoring and failure recovery
• Expertise in CI/CD, version-controlled deployment and automated testing for data or platform workloads
• Experience integrating data through APIs and secure file transfer (SFTP/FTPS), including secure handling of credentials and secrets
• Strong Microsoft Azure knowledge, including ADLS, Azure DevOps and Entra/RBAC, with experience designing and deploying infrastructure
• Strong production troubleshooting, problem-solving and operational support skills
• Degree in Computer Science, Mathematics, Physics or other STEM related subject
• 
  

Benefits

What we offer...

• Comprehensive, payments industry training by in-house and industry experts
• Excellent performance-based earning opportunity, including OKR-driven bonuses
• Future opportunity for equity, rewarded to high performers
• Personal and professional learning opportunities and growth experiences aligned with your career aspirations
• Quarterly values award for all employees - with a financial prize
• Monthly Reimbursement of commuting costs
• Regular company wide socials and team building events
• 22 vacation days + UK public holidays and discretionary office closure during Christmas
• Competitive Pension plan, Vitality healthcare cover (after 6-months) and Cycle to work scheme

Benefits found i