Key Responsibilities

Data Management:

• Design, implement, and manage high-performing data pipelines, focusing on scalability, efficiency, and stability.
• Continuously optimize data processing performance through proactive issue identification, query tuning, and best practice implementation.
• Automate routine tasks like ETL processes, data integration, and reporting to improve efficiency and minimize errors.
• Troubleshoot and resolve data pipeline issues swiftly and effectively, ensuring minimal downtime.

Data Expertise:

• Collaborate with stakeholders to understand data needs and translate them into efficient data models and solutions.
• Develop and execute data extraction, transformation, and loading (ETL) pipelines for various data sources and destinations.
• Perform data analysis tasks using SQL and Python to identify trends, patterns, and insights.
• Participate in data governance initiatives, ensuring data quality, consistency, and accessibility.
• Stay current with the latest trends and technologies in data engineering and analysis.

Data Migrations:

• Lead or participate in complex data migrations from one platform to another.
• Develop and execute well-defined migration plans with minimal downtime and data loss.
• Utilize specialized tools and techniques for seamless data transfer and schema conversion.
• Perform extensive testing and validation to ensure data integrity and functionality after migration.

Preferred Qualifications:

• 5+ years of experience as a Data Engineer, with a proven track record of success in complex data environments.
• Expertise in a major data processing framework like Apache Spark, Flink, or similar.
• Strong understanding of data engineering principles and best practices.
• Experience with data warehousing concepts and tools is a plus (e.g., Redshift, Snowflake).
• Proficiency in SQL and Python for data manipulation and analysis is beneficial.
• Experience leading or participating in successful data migrations is highly desirable.
• Excellent problem-solving, analytical, and communication skills.
• Ability to work independently and manage multiple priorities effectively.
• Self-motivated and a continuous learner with a passion for data.