Senior Data Engineer at HG INSIGHTS INC

Pune, maharashtra, India -

Full Time

Start Date

Immediate

Expiry Date

10 May, 26

Salary

0.0

Posted On

09 Feb, 26

Experience

10 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Data Engineering, Big Data, ETL, ELT, SQL, Python, Airflow, Data Lakes, Data Quality, CI/CD, Docker, Kubernetes, Terraform, Data Governance, Analytics Engineering, Monitoring Tools

Industry

Software Development

Description

About Us HG Insights is the global leader in technology intelligence, delivering actionable AI driven insights through advanced data science and scalable big data solutions. Our platform informs go-to-market decisions, and we influence how businesses spend millions of marketing and sales budgets. What You’ll Do: Design, build, and optimize large-scale distributed data pipelines for processing billions of events from and to multiple sources of data. Architect and scale enterprise-grade big-data systems, including data lakes, ETL/ELT workflows, and syndication platforms for customer-facing products. Orchestrate pipelines and workflows with Airflow. Write optimized data into analytical databases such as ClickHouse, DuckDB, and Redshift Ensure data quality, consistency, and reliability across the pipeline. Monitor pipeline performance and troubleshoot data issues. Collaborate with product teams to develop features across databases, and backend services. Implement cutting-edge solutions for data ingestion, transformation, and analytics. Drive system reliability through automation, CI/CD pipelines (Docker, Kubernetes, Terraform), and infrastructure-as-code practices. What You’ll Be Responsible For Development of the Data side of our Platform, ensuring scalability, performance, and cost-efficiency across distributed systems. Collaborating in agile workflows (daily stand-ups, sprint planning) to deliver features rapidly while maintaining system stability. Ensuring security and compliance across data workflows, including access controls, encryption, and governance policies. What You’ll Need BS/MS/Ph.D. in Computer Science or related field, with 7+ years of experience building production-grade big data systems. Strong SQL skills and solid data modeling fundamentals with hands-on experience building ETL / ELT pipelines. Proficiency in Python for data processing and integrations. Familiarity with dbt, Airflow, Databricks and modern analytics engineering practices Nice-to-Haves Knowledge of data governance frameworks and compliance standards (GDPR, CCPA). Contributions to open-source big data projects or published technical blogs/papers. DevOps proficiency in monitoring tools (Prometheus, Grafana) and serverless architectures.

How To Apply:

Incase you would like to apply to this job directly from the source, please click here

Responsibilities

Design and optimize large-scale distributed data pipelines and architect enterprise-grade big-data systems. Collaborate with product teams to ensure system reliability and compliance across data workflows.