Lead Data Engineer at CLP Group

Hong Kong, Hong Kong Island, Hong Kong S.A.R. -

Full Time

Start Date

Immediate

Expiry Date

03 Aug, 26

Salary

0.0

Posted On

05 May, 26

Experience

10 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Azure, Databricks, PySpark, SQL, Python, Delta Lake, Azure Data Factory, Azure Synapse, CI/CD, Terraform, Lakehouse Architecture, Data Pipeline Design, NoSQL, Azure DevOps, GitHub, DataOps

Industry

Utilities

Description

Location: Kai Tak Office Employment period: 1-year contract (For internal application, only short-term employment contract’s terms and condition will be offered for this position regardless of job applicant’s current job category) You will work closely with Data Architects, Platform Engineers, AI Engineers, and Business Stakeholders to design, build, and operate scalable, secure, and intelligent data platforms on Azure. You will build pipelines to gather data from across the enterprise and integrate into databases and other platforms from where it can be used by other end-users to generate meaningful insights into our business. Reporting to the Senior Manager – Business Delivery, Data Services, the appointee will be responsible for the following duties: Data Engineering & Platform Development * Design, build, and maintain end‑to‑end data pipelines (batch, streaming, and CDC) using Azure‑native and Databricks technologies * Develop scalable Lakehouse architectures leveraging Databricks, Delta Lake, and medallion design patterns (Bronze/Silver/Gold) * Integrate data from enterprise systems, SaaS platforms, IoT sources, APIs, and external providers * Optimize data pipelines for performance, reliability, cost, and security * Implement data quality checks, reconciliation, and observability using automated frameworks * Build solutions using Azure Data Factory, Azure Databricks, Azure Synapse, Azure SQL, Azure Data Lake Storage Gen2 * Implement CI/CD pipelines for data solutions using Azure DevOps / GitHub, Infrastructure as Code (Terraform) * Apply DevOps and DataOps best practices for release management, deployment, and monitoring * Partner with Platform and Security teams to align with cloud governance, IAM, and enterprise security standards * Leverage AI coding agents and copilots (e.g. for Spark, SQL, Python) to: * Accelerate pipeline development and debugging * Auto‑generate transformation logic and documentation * Improve test coverage and code quality * Metadata‑driven pipelines Requirements Academic Qualification: * Bachelor or Masters degree in a related field (e.g. computer science, information technology, etc.) Professional Experience: * At least 8-year experience in data engineering or analytics engineering * Experience with Azure Cloud Services, Databricks & Spark (PySpark / Python) , SQL scripting languages, relational databases (e.g. SQL DB/DW), NoSQL platforms (e.g. HBase, MongoDB, Cassandra) * Proven experience building large‑scale, production‑grade data pipelines using on-premises or cloud-based data platforms * Experience with structured, semi‑structured, and unstructured data * Experience with Data lakes, Lakehouse, or modern data warehouse platforms * Experience with BI and analytics integration (Power BI, Tableau, or equivalent) * Experience in coding in data management, data warehousing or unstructured data environments * Less experience with good attitude will be considered as Senior Data Engineer Competencies Technical (Functional): * Programming: Python, PySpark, SQL, Python * Data Platforms: Databricks Lakehouse, Delta Lake * Cloud: Azure (mandatory); AWS experience is a plus * DevOps/DataOps: CI/CD, Git, Infrastructure as Code * Databases: Relational (SQL DW), NoSQL (Cosmos DB, MongoDB, Cassandra) * Streaming (nice to have): Kafka, Event Hubs, Spark Structured Streaming ABOUT CLP CLP was founded in Hong Kong in 1901, at a time when electricity was still a novelty worldwide. Today we power millions of homes and businesses across the Asia-Pacific region with over 8,000 employees. In Hong Kong, we operate a vertically integrated electricity business providing a highly reliable supply of electricity to over 80% of the city’s population. Outside Hong Kong, we invest in the energy sector on the Chinese Mainland, in Australia, India, Taiwan Region and Thailand. Our business spans the electricity value chain ranging from power generation, transmission and distribution to retail and smart energy services. We have a diversified portfolio of generating assets that uses a wide range of fuels sources including nuclear, renewables, gas and coal.  To meet the evolving needs of energy users in a world being reshaped by decarbonisation and digitalisation, we strive to embrace new opportunities and expand our horizons as we fulfil our purpose to Power Brighter Tomorrows. 

Responsibilities

Design, build, and maintain scalable end-to-end data pipelines and Lakehouse architectures on Azure using Databricks. Collaborate with stakeholders to integrate enterprise data and implement CI/CD and DataOps best practices.