Senior Data Engineer (Pipelines & Data Quality) - Data Platform at Makro PRO

Bangkok, , Thailand -

Full Time

Start Date

Immediate

Expiry Date

30 Sep, 26

Salary

0.0

Posted On

02 Jul, 26

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

PySpark, SQL, Databricks, Data Quality Frameworks, Lakehouse Architecture, CI/CD, Azure, Spark Performance Tuning, Data Modeling, Git, Great Expectations, Unity Catalog, Streaming, Power BI, Data Lineage, Retail Data

Industry

Software Development

Description

We are seeking an experienced Senior Data Engineer to own the pipeline-standardization and data-quality program for the enterprise lakehouse. This role ships compliance gates that block non-compliant deployments, stands up the data-quality framework, builds the dashboards business users trust, and drives measurable reductions in data incidents across the retail data estate. Key Responsibilities: Design, ship, and operate a pipeline-compliance checker that validates naming, metadata, config schema, DQ-rule declarations, and cluster-policy reference on every new deployment. Deploy a data-quality framework (Great Expectations, Databricks DQ Rules, or equivalent) across new production pipelines; build a domain onboarding template; configure alert routing by severity. Build and publish the Data Quality Dashboard — quality health by domain, source, table; near-real-time refresh; freshness, completeness, accuracy. Establish Source Change Management agreements with key source systems (SLA contracts, change-request process, automated schema-change alerting); map source lineage end-to-end. Lead the migration playbook to bring the legacy pipeline estate to standard; mentor engineers executing migration; own the playbook, not every migration. Drive data-incident reduction through prevention (compliance gate, DQ framework, DCM, lineage), not reactive firefighting; lead incident response and post-mortems for major DQ failures. Partner with platform engineering on Event Stream domain-event schemas and data-product contracts. Author runbooks, code review at senior level, and contribute to engineering culture. Bachelor's degree in Computer Science, Data Engineering, or a related discipline. 5+ years designing, building, and operating production data pipelines on a major lakehouse or warehouse (Databricks, Snowflake, BigQuery). Strong PySpark and SQL; understands Spark performance tuning at production scale. Deep experience with data-quality frameworks (Great Expectations, dbt tests, Soda, Monte Carlo) — has defined SLAs, set thresholds, tuned alert noise. Built and operated medallion / multi-layer lakehouse architectures with explicit transformation layers. Solid Git / CI experience for data code; opinions on testing data transformations. Comfortable defining and enforcing standards (naming, partitioning, retention, PII tagging) and reviewing PRs against them. Cloud platform experience (Azure preferred; AWS / GCP transferable). Preferred Qualifications Streaming experience (Spark Structured Streaming, Delta Live Tables, Flink, Kafka Streams). Data modeling discipline (Kimball, Data Vault 2.0) with clear rationale; Unity Catalog production experience (lineage, tags, RLS). Retail data exposure — POS, inventory, replenishment, loyalty — and BI optimization for Power BI consumption. Vendor certifications such as Databricks Data Engineer Professional or Azure Data Engineer Associate.

Responsibilities

Own the pipeline standardization and data quality program for the enterprise lakehouse, including the creation of compliance gates and quality dashboards. Lead the migration of legacy pipelines to current standards and drive the reduction of data incidents through prevention and robust monitoring.