Data Bricks - Data Engineer

at  Wipro Limited

JOB DESCRIPTION:

• Minimum 5+ years of IT experience and minimum 3+ years of hands-on experience in Databricks and pyspark with validated knowledge in designing database models to store structured & unstructured data efficiently and in creating effective data tools for analytics experts along with following:
• Familiarity with Lakehouse Architecture with Delta Tables – Delta tables, Schema Evolution, ACID compatibility, History capture etc.
• Integrating Databricks with various ETL/Orchestration tools and perform required operations.
• Python is Must
• CI/CD Pipeline with AWS
• SQL and SPARK with Date Engineering background
• Expertise in Spark ecosystems, Data frame API, Data set API, RDD APIs & Spark SQL.
• Expertise in workspace and cluster configuration, performance tuning and log monitoring.
• Experience with structured streaming along with streaming frameworks.
• Hands on experience with Python, Scala and SQL.
• Create and maintain optimal data pipeline architecture with knowledge in best practices from Databricks around Cluster setup/Development/Deployment
• Strong understanding of Data Modelling and defining conceptual logical and physical data models.
• Experience in building Data Engineering solutions using AWS native stack
• Experience in managing various client relationships to get consensus on solutions.
• Nice to have AI/ML skill and any architect level certification from Databricks