Senior Data Engineer at Hanson Wade Group

City of Westminster, England, United Kingdom -

Full Time

Start Date

Immediate

Expiry Date

14 Sep, 26

Salary

0.0

Posted On

16 Jun, 26

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Databricks, Python, SQL Server, PostgreSQL, Apache Spark, PySpark, Git, CI/CD, GitHub Actions, Linux, Airbyte, Terraform, Azure, Pandas, Polars, SQLAlchemy

Industry

Information Services

Description

At Beacon Intelligence, we are a SaaS data and analytics business focused on delivering high-value insights through our proprietary platform. Data sits at the core of everything we do — powering our products, driving customer value, and enabling scalable growth. We have built a modern data platform over the last 24 months, consisting of Airbyte as our ETL tool and Databricks as our scalable data processing layer, all hosted on Azure Cloud Services. This platform underpins our product offering and is central to our continued evolution as a data-driven SaaS business. Role Description The Senior Data Engineer position combines hands-on technical expertise, stakeholder engagement, and leadership responsibilities. You will play a critical role in evolving and scaling the data platform that powers our SaaS product — driving new feature development, improving data reliability and performance, and enabling the delivery of customer-facing insights. Leadership & Stakeholder Management Develop a deep understanding of the Beacon Intelligence product, customers, and key data-driven use cases Own the data platform capabilities and lead architectural decision-making aligned to product strategy Partner with senior stakeholders across Product, Engineering, and Commercial teams to deliver data-driven features and insights Play a key role in shaping the data roadmap, with a focus on scalability, performance, and data quality Mentor and support junior data engineers, helping to raise technical capability across the team Identify opportunities to enhance our data offering and contribute to the long-term data platform strategy Required Skills Strong expertise in the Databricks platform, with proven commercial experience Advanced Python development skills Strong experience with SQL Server and PostgreSQL, including complex query optimisation Experience with Python data libraries such as Pandas / Polars, Requests, SQLAlchemy, Beautiful Soup Solid experience working with Apache Spark (PySpark) for batch and streaming pipelines Experience with Git-based version control (GitHub) Hands-on experience building CI/CD pipelines (GitHub Actions, Bash, Python) Strong understanding of data quality practices (testing, monitoring, validation) Experience working in Linux environments (Ubuntu, WSL) Understanding of security best practices within data platforms Desirable Skills Experience with Airbyte, including developing custom connectors Infrastructure-as-code experience using Terraform (Databricks, Azure, Airbyte) Experience working in product-led or SaaS environments Familiarity with Agile delivery methodologies (Scrum, Kanban) Knowledge of Azure services (Functions, Storage Accounts, Key Vaults) Abilities Strong problem-solving mindset with a focus on scalable solutions Clear and effective communicator, able to translate technical concepts to non-technical stakeholders Builds trust and credibility quickly across teams Collaborative approach, with experience working cross-functionally (Product, Engineering, Commercial) Self-motivated and comfortable operating in a fast-paced, product-driven environment Highly organised with strong attention to detail

Responsibilities

Lead the evolution and scaling of the SaaS data platform, focusing on architectural decision-making and feature development. Partner with cross-functional stakeholders to deliver data-driven insights and mentor junior data engineers.