Position Overview
The Senior Data Engineer works closely with local City Year data experts, school district IT professionals and a multitude of partners to manage ingestion of data from many sources. Our ideal candidate is professionally experienced in all things Azure and familiar with dev ops. They will use their Azure experience, especially with Azure Data Factories and Databricks, to lead the end-to-end development and implementation of modern ETL/ELT data pipelines for our team. This candidate will be excited to promote the effective use of timely and accurate k-12 education data and empower front-line practitioners with the information needed to have greater impact on the students we serve. The Senior Data Engineer reports to the Director of Data Management and Reporting.
Job Description

As a Senior Data Engineer at City Year, you will:

• Be our resident Azure expert and trusted advisor for Azure
• Own the design, development, and implementation of modern data pipelines, data factories and data streams. This is a hands-on role.
• Lead the planning and then implement the data platform services including sizing, configuration, and needs assessment
• Own the management and development of various third-party data integrations
• Lead development of frameworks, and data architectures for large-scale data processing that ensure timely and accurate processing of data into and among City Year’s systems and help implement them.
• Influence and make recommendations on the technical direction for the team by leveraging your prior experiences and your knowledge of emerging technologies and approaches
• Lead our team in identifying and promoting data management best practices
• Conduct full technical discovery, identify pain points, gather business and technical requirements, and explain the “as is” and “to be” scenarios to fully understand user stories from stakeholders and users
• Lead and participate in deep architectural discussions to build confidence and ensure customer success when building new solutions and migrating existing data applications on the Azure platform
• Serve as a leader in bridging the gap between technical and non-technical staff in understanding our data and its processing at City Year.
• Own the implementation and use of ETL and data management best practices, devops, ci/cd, and deployment strategies to ensure product quality, agility, robustness, and recoverability
• Partner with districts and internal customers to understand their requirements and implement data solutions.
• Create integrations between internal systems and support them operationally.
• Teach others in the DMAR team about Azure, databricks and best practices to ensure all services and technology implemented can be supported by others on the team.

You must have:

• At least 3+ years of professional experience (not simply coursework or capstone projects) in the following:
• Working in an Azure environment on ETL/ELT projects
• Azure DevOps
• Azure Databricks
• Azure Data Factories

Please refer the Job description for details