Start Date
Immediate
Expiry Date
12 Feb, 27
Salary
50000.0
Posted On
14 Nov, 26
Experience
2 year(s) or above
Remote Job
Yes
Telecommute
Yes
Sponsor Visa
No
Skills
Industry
Information Technology & Services
Key Responsibilities
Design, develop, and maintain scalable ETL pipelines using AWS Glue, EMR (PySpark), and Lambda functions
Implement real-time data streaming solutions using Confluent Kafka with source and sink connectors
Develop and optimize PySpark jobs on EMR clusters for large-scale data transformations
Build serverless data processing workflows using AWS Lambda
Design and optimize queries in Amazon Athena for analytical workloads
Manage data warehouse solutions in Amazon Redshift including data modeling and performance tuning
Implement monitoring and alerting using CloudWatch for data pipeline health and performance
Provision and manage AWS infrastructure using Terraform
Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements
Implement data governance and security best practices using Lake Formation, IAM, and KMS
Additional Technical Experience
Knowledge of AWS EventBridge and Step Functions for workflow orchestration
Familiarity with EMR runtime roles and execution roles for secure data access
Experience with DynamoDB for NoSQL data storage
Knowledge of API Gateway for building data APIs
Preferred Qualifications
Bachelor's degree in Computer Science, Data Engineering, Information Technology, or related field
Strong understanding of distributed computing and big data processing frameworks
Experience with Agile development methodologies
Excellent problem-solving and analytical skills
Strong communication and collaboration abilities
How To Apply:
Incase you would like to apply to this job directly from the source, please click here
Key Responsibilities
Design, develop, and maintain scalable ETL pipelines using AWS Glue, EMR (PySpark), and Lambda functions
Implement real-time data streaming solutions using Confluent Kafka with source and sink connectors
Develop and optimize PySpark jobs on EMR clusters for large-scale data transformations
Build serverless data processing workflows using AWS Lambda
Design and optimize queries in Amazon Athena for analytical workloads
Manage data warehouse solutions in Amazon Redshift including data modeling and performance tuning
Implement monitoring and alerting using CloudWatch for data pipeline health and performance
Provision and manage AWS infrastructure using Terraform
Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements
Implement data governance and security best practices using Lake Formation, IAM, and KMS
Additional Technical Experience
Knowledge of AWS EventBridge and Step Functions for workflow orchestration
Familiarity with EMR runtime roles and execution roles for secure data access
Experience with DynamoDB for NoSQL data storage
Knowledge of API Gateway for building data APIs
Preferred Qualifications
Bachelor's degree in Computer Science, Data Engineering, Information Technology, or related field
Strong understanding of distributed computing and big data processing frameworks
Experience with Agile development methodologies
Excellent problem-solving and analytical skills
Strong communication and collaboration abilities