Sign up with

Already have an account? Log in here

Need some help?
Talk to us at +91 7670800001

Databricks Engineer at NTT DATA

Hyderabad, Telangana, India -

Full Time

Start Date

Immediate

Expiry Date

30 Jan, 26

Salary

0.0

Posted On

01 Nov, 25

Experience

2 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Databricks, Apache Spark, ETL, ELT, Delta Lake, Data Lakehouse, CI/CD, Azure DevOps, GitHub Actions, PySpark, SQL, Python, Databricks SQL, Unity Catalog, Data Warehousing, Streaming Data

Industry

IT Services and IT Consulting

Description

Design, build, and maintain robust ETL/ELT data pipelines using Apache Spark on Databricks. Implement data Lakehouse architecture using Delta Lake for cost-effective data storage and analytics. Use Databricks Workflows for orchestrating batch and streaming pipelines. Develop and maintain CI/CD pipelines for data applications using tools such as Azure DevOps, GitHub Actions or Databricks Repos. Monitor pipeline performance and troubleshoot data issues in real-time and batch environments. Document solutions, workflows, and technical standards. Experience in data engineering with a strong focus on Databricks and Apache Spark. Proficiency in PySpark, SQL, and Python. Experience with Delta Lake, Databricks SQL, and Unity Catalog. Familiarity with data lakehouse architecture, data warehousing and streaming data Strong understanding of ETL best practices, data partitioning, and performance tuning.

Responsibilities

Design, build, and maintain robust ETL/ELT data pipelines using Apache Spark on Databricks. Implement data Lakehouse architecture using Delta Lake for cost-effective data storage and analytics.