Responsibilities

• Design, build, and optimize scalable data pipelines using Scala and Hadoop frameworks.
• Develop and implement techniques and analytics applications to transform raw data into meaningful information using data-oriented programming languages and visualisation software.
• Write and optimize complex SQL queries for efficient data retrieval and transformation.
• Troubleshoot and resolve performance issues in queries and data workflows.
• Manage Hadoop-based data infrastructure, including HDFS, Hive, and related components.
• Monitor system performance and optimize resource utilization in a distributed environment.

Skills/Requirement

• Proficiency in Scala programming and functional programming principles.
• Strong SQL skills, including query optimization and database design.
• Hands-on experience with Hadoop ecosystem tools (HDFS, Hive, YARN, MapReduce).
• Familiarity with other big data tools like Spark, Kafka, or Flink is a plus.
• Strong analytical skills with the ability to troubleshoot and resolve complex data issues.
• 6+ years of experience in data engineering, big data technologies, or a similar role.

• Design, build, and optimize scalable data pipelines using Scala and Hadoop frameworks.
• Develop and implement techniques and analytics applications to transform raw data into meaningful information using data-oriented programming languages and visualisation software.
• Write and optimize complex SQL queries for efficient data retrieval and transformation.
• Troubleshoot and resolve performance issues in queries and data workflows.
• Manage Hadoop-based data infrastructure, including HDFS, Hive, and related components.
• Monitor system performance and optimize resource utilization in a distributed environment