Sign up with

Already have an account? Log in here

Need some help?
Talk to us at +91 7670800001

Big Data Engineer at HEXAWARE

, , India -

Full Time

Start Date

Immediate

Expiry Date

07 Sep, 26

Salary

0.0

Posted On

09 Jun, 26

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Databricks, PySpark, Python, HDFS, Spark, GitHub, Data Warehousing, Data Modelling, Azure Data Factory, ETL, SparkSQL, SQL

Industry

IT Services and IT Consulting

Description

Job Description: • Databricks, PySpark and Python project development work experience is a must • Develop, design, tune and maintain PySpark scripts using Databricks Notebook. • Expertise in Bigdata Eco systems like HDFS and Spark • Experience in GitHuB repository • Data Warehouse/Data Marts/Data Modelling/Analytics experience is a must • Able to convert the SQL stored procedures to Python code in Pyspark frame work using Dataframes. • Implementing data ingestion pipelines from multiple data sources using Azure Data Factory, Azure Databricks and other ETL tools. • Developing Big Data and non-Big Data cloud-based enterprise solutions in PySpark and SparkSQL and related frameworks/libraries. • Developing scalable and re-usable, self-service frameworks for data ingestion and processing. • Integrating end to end data pipelines to take data from data source to target data repositories ensuring the quality and consistency of data. • Processing performance analysis and optimization. • Collaborate with business users, support team members, and other developers throughout the organization to help everyone understand issues that affect the data warehouse • Good experience on customer interaction is required. • Possesses good interpersonal and communication skills.

Responsibilities

Design, develop, and maintain scalable data ingestion pipelines and PySpark scripts using Databricks. Collaborate with business users and support teams to optimize data warehouse performance and ensure data consistency.