Senior Data Engineer

at  Department for Business and Trade Digital Data and Technology

ABOUT US

The Department for Business and Trade (DBT) has a clear mission - to grow the economy. Our role is to help businesses invest, grow and export to create jobs and opportunities right across the country. We do this in three ways.
Firstly, we help to build a strong, competitive business environment, where consumers are protected and companies rewarded for treating their employees properly.
Secondly, we open international markets and ensure resilient supply chains. This can be through Free Trade Agreements, trade facilitation and multilateral agreements.
Finally, we work in partnership with businesses every day, providing advance, finance and deal-making support to those looking to start up, invest, export and grow.
The Digital, Data and Technology (DDaT) directorate develops and operates tools and services to support us in this mission.

SKILLS AND EXPERIENCE

It is essential that you have:

• Agile methodologies. You can lead work following agile methodologies on data projects.
• Data development process. You can lead design, build and test of data products based on feeds from multiple systems using a range of different storage technologies and/or access methods. You create repeatable, reusable products.
• Data integration design. You know how to select and implement the appropriate technologies to deliver resilient, scalable and future-proofed data solutions.
• Data modelling. You understand the concepts and principles of data modelling and can produce relevant data models across multiple subject areas. You know how to reverse-engineer data models from a live system. You understand industry-recognised data modelling patterns and standards and when to apply them. You can compare and align different data models.
• Problem resolution (data). You know how to respond to problems in databases, data processes, data products and services as they occur. You can initiate actions, monitor services and identify trends to resolve problems. You can determine the appropriate remedy and assist with implementation of it as well as preventative measures.
• Programming and build (data engineering). You know how to use agreed standards and tools to design, code, test, correct and document moderate-to-complex programs and scripts from agreed specifications and subsequent iterations. You can collaborate with others to review specifications where appropriate.
• Technical understanding (data engineering). You understand core technical concepts like data normalisation, modelling, performance of data intensive solutions. You understand basic software development concepts.

It is desirable that you have:

• Communication skills (data). You know about the need to translate technical concepts into non-technical language and understand what communication is required for internal and external stakeholders.
• Cloud infrastructure. Experience with cloud infrastructure, especially AWS, and infrastructure as code tools, such as Terraform.

TECHNICAL SKILLS

• Problem resolution (data)
• Programming and build (data engineering)
• Technical understanding
• Testing

MORE ABOUT US

This role can only be worked from within the UK, not overseas. If you are based in London, you will receive London weighting. DBT employees work in a hybrid pattern, spending 2-3 days a week (pro rata) in the office on average. Travel to your primary office location will not be paid for by DBT, but costs for travel to an office which is not your main location will be covered.
You can find out more about our office locations, how we calculate salaries, our diversity statement and reasonable adjustments, the Recruitment Principles, the Civil Service code and our complaints procedure on our website.
Find out more about life at DBT, our benefits and meet the team by watching our video or reading our blog!

ABOUT THE ROLE

As a Senior Data Engineer, you will design, implement, test and operate our data products, pipelines and services. You will work in a cross-functional, agile, team to integrate them into wider systems and business processes.
While you will have opportunities to contribute to the wider solution to satisfy our user needs, the team will rely on your ability to lead development of data products, to optimise the code, and help with database management. You will ensure internal and industry standards are followed in delivery of reusable solutions. You will help more junior team members to gain your level of expertise, and ensure the teams are working to the standards set by the lead data engineer.
You will be confident in using languages like SQL and Python, and have deeper understanding of how they work.

MAIN RESPONSIBILITIES

You will be:

• Lead the design, build and test of data products and services using standard techniques and tools.
• Lead delivery of data solutions which are scalable, resilient and future proof.
• Have thorough understanding of common problems in databases, data processes, data products and services. You will understand and be able to apply typical solutions to those problems.
• Follow internal and industry standards to design, code, test and document complex data solutions.
• Undertake data profiling and source system analysis including techniques like reverse engineering.
• Be a key contributor in an Agile and cross-functional team