Intermediate/Senior AWS Data Engineer

at  PBT Group

The role involves working with advanced AWS technologies to design, build, and maintain scalable data pipelines and architectures. The ideal candidate will have strong expertise in SQL, Python/PySpark, and key AWS services including Glue, Step Functions, EMR (EC2), S3, Lambda, and Athena. The position is focused on leveraging DevOps practices for smooth delivery and deployment.

Key Responsibilities:

• Develop and manage data pipelines using Python/PySpark and AWS Glue.
• Implement AWS Step Functions and EMR for orchestrating and processing data workflows.
• Work with S3 for efficient data storage solutions.
• Leverage Lambda and Athena for real-time data queries and analytics.
• Apply DevOps principles to optimise data engineering processes and streamline delivery.
• Collaborate with cross-functional teams to align data engineering efforts with business objectives.

Skills & Experience Required:

• Proficiency in SQL and Python/PySpark for data processing and analysis.
• Hands-on experience with key AWS services (Glue, Step Functions, EMR, S3, Lambda, Athena).
• Knowledge of DevOps practices and tools for CI/CD in data engineering projects.
• Strong problem-solving skills and ability to work within an agile development environment.
• Prior experience in data engineering roles with a focus on cloud-based architectures.

Additional Criteria:

• Experience within the insurance industry is advantageous.
• Ability to work independently as well as part of a collaborative team.
• Strong communication skills for stakeholder interaction and requirement gathering.

Qualifications/Certification:

• A degree in Computer Science, Information Systems, Business Administration, Commerce, or an equivalent field.
• Relevant certifications in AWS, data engineering, or DevOps are highly desirable.
• In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

• Develop and manage data pipelines using Python/PySpark and AWS Glue.
• Implement AWS Step Functions and EMR for orchestrating and processing data workflows.
• Work with S3 for efficient data storage solutions.
• Leverage Lambda and Athena for real-time data queries and analytics.
• Apply DevOps principles to optimise data engineering processes and streamline delivery.
• Collaborate with cross-functional teams to align data engineering efforts with business objectives