ABOUT THE JOB YOU’RE CONSIDERING

We are looking for an AWS Data Engineer to join our team

YOUR SKILLS AND EXPERIENCE

• Bachelor’s degree in computer science, management information systems, or related discipline, or equivalent work experience
• MUST HAVE Technology skills (7/10 or higher):
• Strong/expert Spark (PySpark) Using Jupyter Notebooks, Colab or DataBricks (preferred)
• Hands-on data pipeline development, ingest patterns in Azure
• Orchestration tools, ADF or Airflow

• Development of robust ETL/ELT data pipelines, ensuring efficient data ingestion, processing, and transformation from diverse sources into AWS data warehouses and data lakes
• Lead the design and development of robust ETL/ELT data pipelines, ensuring efficient data ingestion, processing, and transformation from diverse sources into AWS data warehouses and data lakes. This includes designing and implementing solutions for batch and streaming data, handling various data formats like JSON, CSV, Parquet, and Avro.
• Architect, build, and optimize scalable data architectures, including data lakes (e.g., S3, Delta Lake, Iceberg) and data warehouses (e.g., Redshift, Snowflake) on AWS, ensuring optimal performance and data accessibility.
• Collaborate closely with data scientists, analysts, and other stakeholders to understand data requirements, design appropriate data models and schemas, and deliver tailored data solutions that enable data-driven decision-making.
• Implement advanced data quality and governance practices, ensuring data accuracy, consistency, and compliance with relevant regulations.
• Optimize data retrieval and develop dashboards and reports using various tools, leveraging deep understanding of data warehousing and analytics concepts.
• Proactively identify and resolve operational issues, troubleshoot complex data pipeline failures, and implement evolutionary recommendations for system improvements.