DESCRIPCIÓN BREVE

We are seeking a skilled and motivated Data Engineer to join our team. The ideal candidate will be proficient in PySpark, Azure Data Bricks, Azure Data Factory, and SQL. As a Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support our data-driven initiatives.

Responsibilities:

• Develop and maintain data pipelines using PySpark to ingest, process, and transform large volumes of data.
• Design and implement ETL processes using Azure Data Factory to move data between various data sources and destinations.
• Collaborate with data scientists and analysts to understand data requirements and translate them into technical solutions.
• Optimize and tune data pipelines for performance and scalability.
• Ensure data quality and reliability by implementing data validation and monitoring processes.
• Troubleshoot and resolve issues related to data pipelines and infrastructure.
• Develop and maintain documentation for data pipelines, workflows, and data sources.
• Stay up-to-date with emerging technologies and best practices in data engineering and cloud computing.

Qualifications:

• Bachelor’s degree in Computer Science, Engineering, or related field.
• Strong programming skills in Python and experience with PySpark for big data processing.
• Proficiency in SQL for querying and manipulating data in relational databases.
• Hands-on experience with cloud platforms such as Azure, particularly Azure Data Bricks and Azure Data Factory.
• Experience designing and building scalable and reliable data pipelines.
• Familiarity with data modeling concepts and techniques.
• Excellent problem-solving and troubleshooting skills.
• Strong communication and collaboration skills, with the ability to work effectively in a team environment.

Preferred Qualifications:

• Experience with other big data technologies such as Hadoop, Kafka, or Apache Spark.
• Knowledge of containerization and orchestration tools such as Docker and Kubernetes.
• Experience with version control systems such as Git.
• Familiarity with machine learning concepts and frameworks.
• Certification in cloud computing or big data technologies is a plus.

If you’re passionate about leveraging data to drive insights and decision-making, and you thrive in a fast-paced, collaborative environment, we encourage you to apply for this exciting opportunity!

LI-AU1

• Develop and maintain data pipelines using PySpark to ingest, process, and transform large volumes of data.
• Design and implement ETL processes using Azure Data Factory to move data between various data sources and destinations.
• Collaborate with data scientists and analysts to understand data requirements and translate them into technical solutions.
• Optimize and tune data pipelines for performance and scalability.
• Ensure data quality and reliability by implementing data validation and monitoring processes.
• Troubleshoot and resolve issues related to data pipelines and infrastructure.
• Develop and maintain documentation for data pipelines, workflows, and data sources.
• Stay up-to-date with emerging technologies and best practices in data engineering and cloud computing