Senior Data Support Engineer at Thoughtworks

, , Singapore -

Full Time

Start Date

Immediate

Expiry Date

20 Apr, 26

Salary

0.0

Posted On

20 Jan, 26

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Python, PySpark, Scala, Databricks, Apache Airflow, Terraform, PowerShell, Azure, Data Modeling, Spark, Monitoring, Debugging, CI/CD, Data Governance, Security, Compliance

Industry

Information Technology & Services

Description

Due to the project requirement, candidates must be Singaporean citizens or already hold Singaporean Permanent Residency (PR) at the time of application. As a Senior Data Support Engineer, you will independently own the reliability, performance, and operational health of critical data pipelines and platforms. You’ll collaborate closely with clients and internal teams to diagnose complex issues, implement scalable solutions, and drive continuous improvement in our DataOps practices. You’ll guide junior engineers, contribute to architectural decisions, and ensure that data systems meet high standards for availability, security, and efficiency. Job responsibilities Lead end-to-end support and operations of data pipelines in production. Design, implement, and maintain robust Apache Airflow DAGs to orchestrate complex data workflows. Provision, manage, and evolve cloud infrastructure using Terraform (IaC) across cloud environments. Proactively monitor, troubleshoot, and optimize data pipelines to ensure SLA compliance and high data quality. Develop automation scripts (Python, PowerShell) to streamline operational tasks, incident response, and recovery. Enforce data quality standards through validation rules, anomaly detection, and alerting mechanisms. Drive root cause analysis for production incidents and implement preventive measures. Collaborate with clients and engineering teams on feature deployments, impact assessments, and smooth rollouts. Communicate clearly with stakeholders on operational performance, incident status, and resolution timelines. Mentor less experienced peers through code reviews, pair troubleshooting, and knowledge sharing while participating in 24x7 on-call rotations and leading incident response during critical outages. Job qualifications Technical Skills Strong hands-on experience with Python and PySpark; solid working knowledge of Scala. Proven experience building and operating data pipelines on Databricks in Azure. Expertise in orchestrating workflows using Apache Airflow (including custom operators, sensors, and error handling). Proficiency in Terraform for managing Azure resources (e.g., Databricks workspaces, storage accounts, networking). Experience writing operational and automation scripts in PowerShell and Python. Deep understanding of Azure data services (e.g., ADLS Gen2, Synapse, Key Vault, Managed Identities). Solid grasp of data modeling, partitioning strategies, and performance tuning in Spark. Experience with monitoring and alerting tools integrated with Azure (e.g., Azure Monitor, Log Analytics, Grafana). Strong debugging skills across distributed systems and familiarity with logging/observability best practices. Good to have: experience with API development or integration, CI/CD pipelines for data infrastructure (Azure DevOps, GitHub Actions), and familiarity with data governance, security, and compliance in cloud environments. Professional Skills Ability to communicate technical issues clearly to both technical and non-technical stakeholders. Self-driven problem solver who performs well in ambiguous and high-pressure environments. Experience mentoring junior engineers through code reviews, pairing, and knowledge sharing. Comfortable working in Agile environments, including sprint planning, retrospectives, and demos. Strong sense of ownership for production systems and operational excellence. Willingness and ability to participate in need-based 24x7 on-call support. Other things to know Learning & Development There is no one-size-fits-all career path at Thoughtworks: however you want to develop your career is entirely up to you. But we also balance autonomy with the strength of our cultivation culture. This means your career is supported by interactive tools, numerous development programs and teammates who want to help you grow. We see value in helping each other be our best and that extends to empowering our employees in their career journeys. About DAMO At DAMO™ Managed Services, we go beyond routine maintenance - we focus on continuous evolution to help organizations achieve extraordinary impact. Here, you’ll work on proactive improvements rather than reactive fixes. We're at the forefront of cost optimization, automation, and scalable solutions. Your expertise will play a key role in streamlining operations, boosting efficiency, and ensuring our systems grow with our clients’ needs. Join and be part of a team that thrives on curiosity, innovation, and purpose. About Thoughtworks Thoughtworks is a dynamic and inclusive community of bright and supportive colleagues who are revolutionizing tech. As a leading technology consultancy, we’re pushing boundaries through our purposeful and impactful work. For 30+ years, we’ve delivered extraordinary impact together with our clients by helping them solve complex business problems with technology as the differentiator. Bring your brilliant expertise and commitment for continuous learning to Thoughtworks. Together, let’s be extraordinary. #LI-Onsite See here our AI policy.

Responsibilities

The Senior Data Support Engineer will lead the support and operations of data pipelines in production, ensuring reliability and performance. They will also mentor junior engineers and drive improvements in DataOps practices.