Data Engineer

at  Meddbase

SKILLS & QUALIFICATIONS

• Willingness and ability to engage with advanced background checks including police vetting.
• Expertise in ETL.
• Expertise in data investigation, rationalising, and manipulation.
• Expertise in version control.
• Expertise in Azure tool and services.
• Expertise in PySpark, Python, SQL 5.

DESIRED SKILLS & QUALIFICATIONS:

• Client-facing experience.
• Healthcare experience.
• Understanding of data governance and compliance standards.
• Data Engineering Certificates, such as Azure Data Engineer Associate (DP-203)

PURPOSE OF THE ROLE:

The majority of new Meddbase customers have pre-existing data that needs to be migrated into our platform. These can be in-house bespoke systems created by the customer or off the shelf solutions.
We have recently updated our approach to data migrations to take advantage of advances in industry-standard data tools and technologies. You will be working on a daily basis with Python, PySpark, Synapse, SQL and Azure tools and services, as well as developing a deep understanding of our proprietary data schemas and product features.
Beyond the technology, a successful data migration depends on regular, high-quality communication with clients, transparency and rigour in our processes, documentation and auditing. In a growing team you’ll be expected to contribute positive ideas and feedback on the process to help us constantly get better.
Each data migration will follow a standard process, as we aim to provide predictable delivery to project managers and clients. You’ll work with clients and the project manager to analyse, map, define and sometimes clean their data and propose solutions to issues that arise. You’ll write defensive, repeatable transformation notebooks that ensure integrity and lineage of the transformed data. Together with the Project Manager and clients, you’ll help to estimate and break down the work into manageable chunks. Finally, you’ll see the data through testing and sign off to its destination in a live system supporting health professionals in the delivery of excellent care.

PROJECT RESPONSIBILITIES

• Liaise with clients to take receipt of their data.
• Work with client to define their data.
• Provide data integrity reports and propose solutions for issues.
• Estimation and time management.
• Regular updates to all stakeholders on progress and blockers.
• Assuring the quality of a data migration.
• Providing guidance to clients on performing successful UAT.
• Establish robust data pipelines from source data to curated data.
• Post-import hyper-care.
• Ensuring the security and privacy of their data throughout.

TEAM RESPONSIBILITIES

• Maintain standard schemas for the Meddbase product and transformers for commonly migrated source systems.
• Building and maintaining common tools.
• Support QA with test suites, data sets, automation and the raising of data import related defects in the application.
• Building and maintaining documentation and user guides to streamline the data migration process.