SENIOR DATA ENGINEER

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by diversity and inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health equity on a global scale. Join us to start Caring. Connecting. Growing together.

REQUIRED QUALIFICATIONS OF THE SENIOR DATA ENGINEER:

• Extensive experience designing data solutions including data modeling. Data Architecture experience is a plus.
• Extensive hands-on experience developing data processing jobs (PySpark / SQL) that demonstrate a strong understanding of software engineering principles.
• Experience orchestrating data pipelines using technology like ADF, Airflow etc
• Experience working with Hive /HBase / Presto
• Fluent in SQL (any flavor), with experience using Window functions and more advanced features.
• Experience of DevOps tools, Git workflow and building CI/CD pipelines
• Experience applying data governance controls within a highly regulated environment.
• The ability to communicate effectively with users. There will be a requirement to be able to understand new required changes.

PREFERRED QUALIFICATIONS OF THE SENIOR DATA ENGINEER:

• Bachelor’s Degree or higher in Database Management, Information Technology, Computer Science or similar
• Proven Data Engineering experience
• A motivated self-starter who excels at managing their own tasks and takes ownership.
• Experience working in projects with agile/scrum methodologies.
• Experience with Azure Databricks and Snowflake
• Well versed in Python regarding data manipulation, cleaning, transforming, and analyzing structured data to support our data-driven initiatives.
• Familiarity with production quality ML and/or AI model development and deployment.

ABOUT THE ROLE:

As a Senior Data Engineer you will be working on developing and maintaining data pipelines that extract, transform, and load (ETL) data from various sources into a centralized data storage system, such as a data warehouse or data lake. Ensure the smooth flow of data from source systems to destination systems while adhering to data quality and integrity standards. In addition to having impact on a great team, you’ll also discover the career opportunities you’d expect from an Industry Leader.

PRIMARY RESPONSIBILITIES OF THE SENIOR DATA ENGINEER:

• Data Integration: Integrate data from multiple sources and systems, including databases, APIs, log files, streaming platforms, and external data providers. Handle data ingestion, transformation, and consolidation to create a unified and reliable data foundation for analysis and reporting.
• Data Transformation and Processing: Develop data transformation routines to clean, normalize, and aggregate data. Apply data processing techniques to handle complex data structures, handle missing or inconsistent data, and prepare the data for analysis, reporting, or machine learning tasks.
• Contribute to common frameworks and best practices in code development, deployment, and automation/orchestration of data pipelines.
• Implement data governance in line with company standards.
• Partner with Data Analytics and Product leaders to design best practices and standards for developing and productionalizing analytic pipelines.
• Partner with Infrastructure leaders on architecture approaches to advance the data and analytics platform, including exploring new tools and techniques that leverage the cloud environment (Azure, Snowflake, others)
• Monitoring and Support: Monitor data pipelines and data systems to detect and resolve issues promptly. Develop monitoring tools, alerts, and automated error handling mechanisms to ensure data integrity and system reliability.