Senior Data Engineer at Quantiphi Inc

Bengaluru, karnataka, India -

Full Time

Start Date

Immediate

Expiry Date

22 May, 26

Salary

0.0

Posted On

21 Feb, 26

Experience

2 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Spark Streaming, Aws Glue, Emr, Pyspark, Python, Kafka, S3, Redshift, CloudWatch, Secrets Manager, Medallion Architecture, Sql, Datadog, Airflow, Terraform, Ci/Cd

Industry

IT Services and IT Consulting

Description

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi! Must have skills: 3+ years of hands-on experience in building large-scale data pipelines. Strong experience with Spark Streaming, AWS Glue, and EMR for processing both real-time and batch workloads Proficiency in Pyspark/Python, including building Kafka producers for data ingestion Experience working with Confluent Kafka/spark streaming for streaming ingestion from on-premise sources. Solid understanding of AWS services including S3, Redshift, Glue, CloudWatch, and Secrets Manager Experience working with Medallion architecture and hybrid data destinations (e.g., Redshift and on-premise Oracle) Ability to implement monitoring dashboards and observability for pipelines using tools like CloudWatch or Datadog Strong SQL skills for validating data accuracy and developing job-level metrics Experience with building alerting mechanisms for pipeline failures and performance issues Strong collaboration and communication skills Proven ability to take ownership of technical deliverables, driving execution from design to deployment Experience in guiding junior developers or data engineers, providing technical direction and code reviews to uphold best practices AWS Data Engineering - Associate Certified. Good to have skills : Experience with orchestration tools like Apache Airflow or AWS Step Functions Exposure to Bigdata stacks like Sqoop,HDFS,Hive,Nifi Exposure to Terraform for infrastructure automation Familiarity with CI/CD pipelines for data workflows If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us! Thank you for your interest in Quantiphi! We are a team that dreams together and collaborates to ensure success. We are currently looking for exceptional individuals who can join us and contribute to a fun, diverse and hybrid work culture. As a part of the Quantiphi family, you will get ample opportunities to learn, grow and interact with colleagues from varied experience and backgrounds around the globe. If this excites you, explore our current openings and apply to any job roles that suit and interest you. Quantiphi is an award-winning AI-first digital engineering company driven by the desire to reimagine and realize transformational opportunities at the heart of business. We solve the toughest and most complex business problems with the latest and cutting-edge technologies and to make this happen, we have a vibrant, diverse and talented set of professionals we proudly refer to as the Quantiphi family. Our culture is built on transparency, diversity, integrity, learning and growth. We are committed to provide our teams with an environment that helps them to learn, grow and flourish in their professional as well as personal lives.

How To Apply:

Incase you would like to apply to this job directly from the source, please click here

Responsibilities

The role involves building large-scale data pipelines utilizing technologies like Spark Streaming, AWS Glue, and EMR for both real-time and batch processing. Responsibilities include implementing monitoring, validating data accuracy with SQL, and guiding junior developers.