Senior Data Engineer at Qode

Ramagundam, Telangana, India -

Full Time

Start Date

Immediate

Expiry Date

13 Apr, 26

Salary

0.0

Posted On

13 Jan, 26

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Data Engineering, AWS Glue, Lambda, S3, SQL, Python, Pyspark, Numpy, ETL Pipelines, Data Warehousing, Data Governance, Automation, Troubleshooting, Collaboration, Problem-Solving, Communication

Industry

Software Development

Description

Senior Data Engineer with AWS Key Responsibilities:· Design and implement robust ETL pipelines using AWS Glue, Lambda, and S3.· Monitor and optimize the performance of data workflows and batch processing jobs.· Troubleshoot and resolve issues related to data pipeline failures, inconsistencies, and performance bottlenecks.· Collaborate with cross-functional teams to define data requirements and ensure data quality and accuracy.· Develop and maintain automated solutions for data transformation, migration, and integration tasks.· Implement best practices for data security, data governance, and compliance within AWS environments.· Continuously improve and optimize AWS Glue jobs, Lambda functions, and S3 storage management.· Maintain comprehensive documentation for data pipeline architecture, job schedules, and issue resolution processes. Required Skills and Experience:· Strong experience with Data Engineering practices.· Experience in AWS services, particularly AWS Glue, Lambda, S3, and other AWS data tools.· Proficiency in SQL, python , Pyspark, numpy etc and experience in working with large-scale data sets.· Experience in designing and implementing ETL pipelines in cloud environments.· Expertise in troubleshooting and optimizing data processing workflows.· Familiarity with data warehousing concepts and cloud-native data architecture.· Knowledge of automation and orchestration tools in a cloud-based environment.· Strong problem-solving skills and the ability to debug and improve the performance of data jobs.· Excellent communication skills and the ability to work collaboratively with cross-functional teams.· Good to have knowledge of DBT & SnowflakePreferred Qualifications: · Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or a related field.· Experience with other AWS data services like Redshift, Athena, or Kinesis.· Familiarity with Python or other scripting languages for data engineering tasks.· Experience with containerization and orchestration tools like Docker or Kubernetes.· Location: Candidates should be based in Gurgaon/Hyderabad

Responsibilities

The Senior Data Engineer will design and implement robust ETL pipelines using AWS services and monitor the performance of data workflows. They will also troubleshoot issues related to data pipeline failures and collaborate with cross-functional teams to ensure data quality.