Data Engineer at Optum
Dublin, Dublin County, Ireland -
Full Time


Start Date

Immediate

Expiry Date

01 Feb, 27

Salary

75000.0

Posted On

03 Nov, 26

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Yes

Skills

Industry

Information Technology & Services

Description

What you will do:

  • Data Pipeline Development: Develop and maintain data pipelines that extract, transform, and load (ETL) data from various sources into a centralized data storage system, such as a data warehouse or data lake. Ensure the smooth flow of data from source systems to destination systems while adhering to data quality and integrity standards
  • Data Integration: Integrate data from multiple sources and systems, including databases, APIs, log files, streaming platforms, and external data providers. Handle data ingestion, transformation, and consolidation to create a unified and reliable data foundation for analysis and reporting
  • Data Transformation and Processing: Develop data transformation routines to clean, normalize, and aggregate data. Apply data processing techniques to handle complex data structures, handle missing or inconsistent data, and prepare the data for analysis, reporting, or machine learning tasks
  • Contribute to common frameworks and best practices in code development, deployment, and automation/orchestration of data pipelines
  • Implement data governance in line with company standards
  • Monitoring and Support: Monitor data pipelines and data systems to detect and resolve issues promptly. Develop monitoring tools, alerts, and automated error handling mechanisms to ensure data integrity and system reliability

You will be rewarded and recognized for your performance in an environment that will challenge you and give you clear direction on what it takes to succeed in your role, as well as providing development for other roles you may be interested in.

What you will bring:

  • Proven hands-on experience developing data pipelines, with a demonstrated solid understanding of software engineering principles
  • Proven ability to troubleshoot and resolve complex data issues across large-scale datasets containing billions of records
  • Demonstrated advanced proficiency in SQL, including experience utilizing window functions and other advanced SQL capabilities
  • Demonstrated understanding of DevOps tools, Git workflows, and CI/CD pipeline development and deployment practices
  • Proven ability to collaborate with business and technical stakeholders during requirements gathering sessions, technical design workshops, and SQL development and debugging activities
  • Demonstrated proficiency in Python, with the ability to develop solutions across a range of general-purpose use cases, including data pipelines, APIs, automation, and analytical applications

Preferred Qualifications of Data Engineer:

  • Experience with Snowflake or Databricks
  • Experience with relational data models and OLAP processing
  • Experience building data pipelines on either AWS, Azure or GCP, following best practices in Cloud deployments
  • Familiar with Airflow or similar orchestration tool
  • Experience working in projects with agile/scrum methodologies
  • Experience with shell scripting languages
  • Experience working with both real-time and batch data, knowing the strengths and weaknesses of both and when to apply one over another


How To Apply:

Incase you would like to apply to this job directly from the source, please click here

Responsibilities
Loading...