Senior Data Engineer

at  Levy Professionals

Job Summary:
We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. The ideal candidate will have a strong background in developing, and delivering scalable data solutions using Azure Databricks, Azure Data Factory, and Power BI. This role will involve heavy hands on in programming languages like python, SQL & on topics like Azure databricks, optimizing dataworkflows, and collaborating with internal teams on various aspects &extensive hands on experience in Azure Data engineering Tech stack
Responsibilities:
1. Develop, and implement scalable data pipelines using Azure Databricks andAzure Data Factory.

1. Create and maintain ETL workflows to ensure data quality, integrity, and availability.

• Demonstrated ability to manage and analyze data at scale within the Azure ecosystem.
• Implement CI/CD pipelines using Azure DevOps tostreamline deployment processes.5. Hands on programming languages like Python, SQL and experience in optimizing the code to meet performance targets5. Conduct code reviews, provide mentorship, and facilitate knowledge sharingsessions within the team.6. Integrate data from various sources into a centralized data lake, enablingunified access and analysis.7. Perform data cleansing, transformation, and migration tasks as required.8. Monitor and troubleshoot data pipelines and resolve any issues promptly.

REQUIREMENTS:

1. Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
2. 7+ years of experience in data engineering or related roles.
3. 5+ years of experience in Azure Databricks, Azure Data Factory, and Power BI & proven track record of building data engineering solutions
4. Proficient in programming languages such as Python, SQL, and Scala.
5. Experience with big data technologies including Apache Spark, Hadoop, and Hive.

6. Hands-on experience with cloud platforms, particularly Microsoft Azure.

7. Proficiency in ETL processes, data warehousing, and data modeling.

• Knowledge of tools and technologies such as AzureSynapse Analytics, and Apache Kafka is an added advantage9. Solid understanding of CI/CD pipelines and hands-on experience with DevOpstools such as Git for continuous integration and deployment.
• Excellent problem-solving skills and the ability to troubleshoot complex data issues.11. Strong coaching abilities, Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams.

PREFERRED QUALIFICATIONS:

Microsoft Certified: Azure Data Engineer Associate.
Databricks Certified Associate Developer for Apache Spark.
Experience with data governance and data security practices.
Knowledge of Docker and Kubernetes for containerization and orchestration.

Please refer the Job description for details