Senior Data Operations Engineer

at  Ofgem

JOB SUMMARY

Use your data expertise to help us change the future of how we use data.
It is important that we become a fully digital regulator so that we can transform the energy sector. We are therefore embarking on a high-profile data and digitalisation programme, and it is within this scope of work that you could find a career-defining challenge.
Our team is working with the most cutting-edge cloud and data technologies, so this is a great opportunity to further your knowledge with the support of experienced colleagues. As such, you will help deliver our new Strategic Common Data Platform, which will connect internal and external data sources. Taking responsibility for designing and developing Azure pipelines (which will include data transformation and cleansing), you will create accessible data sets that can be easily distributed and analysed. You will also engineer manual data flows that are both scalable and repeatable, and you will document source-to-target mappings. Across all that you do, you will align your projects and datasets with Purview.
This is an exciting opportunity to network extensively with both internal colleagues and external partners. You will therefore be a natural fit in our team if you have good communication skills and you can bring a proactive mindset to identifying and resolving issues in databases, and data processes, products and services. With a background of working with Azure Data Lakes, data warehousing, and storage accounts, you will be capable of building all pipeline facets including data interfaces. Proficient in in T-SQL and Python, you will be able to transform data into appropriate storage formats, engineer data for analytics and devise error handling.
Ofgem relies on having a workforce that reflects the society we serve, so we welcome candidates from all backgrounds, and especially those from underrepresented groups. While this is a full-time role, flexible working patterns and job shares are welcome, and we offer a fully hybrid environment. We’ll also support you with excellent training and development opportunities, plus a competitive pension and generous annual leave.

JOB DESCRIPTION

We play a key role in supporting Ofgem to be a dynamic, inclusive and high performing organisation, both striving to continuously improve our efficiency and effectiveness in meeting the needs of the organisation, and also setting clear corporate standards and controls in order to ensure the organisation operates within our statutory and legal requirements, and good practice standards.
The Data Operations Engineer is an expert role to help accelerate data driven decision making, digital transformation and innovation across Ofgem through the provision of high quality, timely and valuable data and analytics in a resilient and consistent manner.

The role will be key to achieving Ofgem’s vision of Digitisation and supporting the innovation and growth agenda of:

• Making OFGEM a recognised digitalisation leader with innovative approaches in the exploitation of data using emerging technologies
• Establishing the capabilities to focus on specialist areas including data analytics/insights, data innovation, artificial intelligence, data sourcing and automated data pipelines
• Training our workforce to be comfortable and skilled in understanding, using and asking questions of data, visualisations and analytics output
• Achieving cost efficiencies in targeted areas e.g. automate data preparation and processing, automate production of commonly used reporting, share data live online

This role oversees the service provided by our modern cloud data platform. This platform includes; data ingestion, data processing, data quality improvement, master data management, data modelling, data science and data visualisation*.
Items with a * have peer team leads responsible for specifying and leading development of new content but this role could help deliver and will be responsible for ongoing running of our end to end data analytics pipelines.
This role will work with the Head of Data and AI Enablement to monitor our end to end service, establishing trust in the data and analytics provided and will aim to optimise the usefulness and value it provides.

Your initiative and ideas on how to measure, grow and improve the following will be required:

• Levels of usage of data/data products e.g. user session count, user session duration
• Levels of value provided by each data product e.g. dashboard
• Levels of value provided by each underlying data set e.g. source file
• User trust and satisfaction e.g. net promoter scores or review feedback

This role works closely with the Head of Data Management and Operations to deliver priority improvements, oversee current work and improve data platform service levels. This along with the proposed benefit of each proposed action will help this role holder demonstrate service value ongoing as well as the planned functional improvements each work package will deliver.

TECHNICAL SKILLS

We’ll assess you against these technical skills during the selection process:

• Please refer to the Candidate Pack and Role Profile attached for full details.

Benefits
Alongside your salary of £45,614, OFGEM contributes £13,214 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.
Ofgem can offer you a comprehensive and competitive benefits package which includes; 30 days annual leave after 2 years; Excellent training and development opportunities; The opportunity to join the generous Civil Service pension which also includes a valuable range of benefits; Hybrid working, flexible working hours and family friendly policies. Plus lots of other benefits including clean and bright offices based centrally, engaged networks and teams and an opportunity to contribute to our ambitious and important targets of establishing a Net Zero energy system by 2050. This exciting blend of professional challenge and personal reward identifies career opportunities at Ofgem as something to get excited about.
Things you need to know

NATIONALITY REQUIREMENTS

This job is broadly open to the following groups:

• UK nationals
• nationals of the Republic of Ireland
• nationals of Commonwealth countries who have the right to work in the UK
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
• Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements (opens in a new window)

The role will be key to achieving Ofgem’s vision of Digitisation and supporting the innovation and growth agenda of:

• Making OFGEM a recognised digitalisation leader with innovative approaches in the exploitation of data using emerging technologies
• Establishing the capabilities to focus on specialist areas including data analytics/insights, data innovation, artificial intelligence, data sourcing and automated data pipelines
• Training our workforce to be comfortable and skilled in understanding, using and asking questions of data, visualisations and analytics output
• Achieving cost efficiencies in targeted areas e.g. automate data preparation and processing, automate production of commonly used reporting, share data live onlin