Y-Axis Jobs Logo
Send us feedback
telephone  +91 7670 800 001
Log in Sign up Try Premium
Big Data Lead at HEXAWARE
, , India -
Full Time

Start Date

Immediate

Expiry Date

18 Oct, 26

Salary

0.0

Posted On

20 Jul, 26

Experience

10 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

No

Skills

Python, Data Pipelines, Airflow, SQL, Snowflake, DBT, GitHub, CI/CD, AWS S3, AWS ECS, AWS IAM, Jinja Templates, Unit Testing, Integration Testing, System Testing, ETL

Industry

IT Services and IT Consulting

Description
Job Description Job Descriptions Data Engineer Must have below: 10+ Years Experience Great Communicator/Client Facing/ Attention to detail Individual Contributor and ability to work as a team. 100% Hands on in the mentioned skills Programming Skills: Python and Data Pipelines: Advanced Proficiency in Python concepts like Code Structures, Modules, Packages, Class, SubClass, Inheritance, Multi-Threading and Functional Programming. Experience in developing reusable Python packages for internal or public usage Ability to write automating ETL processes and scheduling jobs like Airflow DAG. Ability to track job pipeline runs to reprocess error records Ability to orchestration different pipelines to run in sequence or parallel. Troubleshoot data pipeline errors and fix issues Export or Import data to/from various formats like CSV, JSON, XML etc preferably from S3 or other cloud storage. Experience in using AI IDE tool SQL: Advanced SQL skills, including complex joins, CTE's and subqueries Experience in optimizing SQL queries for performance and optimization in data warehouse technologies preferably Snowflake Testing and documentation: Proficiency in Python unit, integration and system test. Proficiency in implementing DBT tests for data validation and quality checks Code Generation: Experience in generating code using configurations using python and jinja templates Version control: Experience in GitHub, including implementing CI/CD process from scratch AWS Expertise: Data Storage solutions: In depth understanding of AWS S3 for data storage, ECS, IAM including best practices for organization and security
Responsibilities
Lead the development of automated ETL processes and data pipelines using Python and Airflow. Orchestrate complex data flows and optimize SQL queries within Snowflake for high performance.