Accountabilities

• Design and develop scalable streaming data pipelines using Kafka/MSK, Flink or Spark Streaming, and reliable routing into curated data storage.
• Build batch transformation pipelines within a medallion lakehouse architecture, supporting bronze, silver, and gold data layers on Amazon S3 and using formats such as Iceberg and Parquet.
• Design and manage data storage solutions across relational databases such as Aurora/RDS PostgreSQL, NoSQL technologies including DynamoDB, and object-based lakehouse storage.
• Design, deploy, and maintain AWS infrastructure supporting data and machine learning workloads, using services such as EKS, Fargate, Lambda, S3, VPC, Aurora/RDS, and DynamoDB.
• Take data solutions from prototype through to production by refactoring, optimizing, and applying software engineering, CI/CD, containerization, and deployment best practices.
• Collaborate with international teams to gather requirements, contribute to solution design, and define effective data loading and ingestion processes.
• Document technical developments, perform integration testing, and manage deployments within AWS environments.
• Support the transition of completed solutions to the relevant support and governance teams, ensuring reliable operation and clear ownership.
• Monitor production pipelines and ensure strong observability, data quality, performance, reliability, and maintainability.
• Participate in an application maintenance and support environment, investigating user-reported issues and implementing improvements to data ingestion processes.
•