Piper Companies is seeking an Data Engineer to join an established Healthcare organization located in Maryland. This position is HYBRID and REQUIRES you to be onsite in Columbia, MD once a week. The Data Engineer will design, develop, and maintain a data analytics platform solution, focusing on creating new use cases through Databricks programming within the healthcare data domain.

REQUIREMENTS OF THE DATA ENGINEER:

• Design, build, and maintain large, complex data processing pipelines that meet business requirements.
• Ensure the quality of deliverables by developing automated controls and performing unit, integration, and user acceptance testing.
• Develop scalable and reusable frameworks for ingestion and transformation of large data sets.
• Troubleshoot and perform root cause analysis on data pipeline issues.
• Collaborate with source system owners and business owners to incorporate changes into data pipelines.
• Create comprehensive documentation of data workflows, processes, and infrastructure.

QUALIFICATION OF THE DATA ENGINEER:

• 4+ years of experience in data integration.
• Advanced programming skills in PySpark, Python, Spark SQL.
• Thorough understanding of Data Lakes, raw/enriched/curated layer concepts, and ETL within the Azure framework.
• Proven experience in building complex data pipelines for lakehouse/data warehouses using Agile methodology.
• Experience in architecture, design, and implementation using Databricks and Azure Data Factory.
• Experience working with Databricks (preferred).
• 4+ year college degree (required).

Please refer the Job description for details