Consultant - Data Engineer at AstraZeneca

Chennai, tamil nadu, India -

Full Time

Start Date

Immediate

Expiry Date

09 Sep, 26

Salary

0.0

Posted On

11 Jun, 26

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Databricks, Redshift Serverless, PySpark, Spark SQL, AWS, Unity Catalog, Linux System Administration, Bash Scripting, Python, CI/CD, GitHub, IAM/RBAC, SRE, ETL/ELT, Data Governance, Performance Tuning

Industry

Pharmaceutical Manufacturing

Description

Job Title: Consultant - Data Engineer Career Level: C3 Introduction to role: Are you ready to run high-impact data platforms at scale and see your decisions translate into faster, smarter outcomes for patients and the business? Join a team where the reliability, security, and efficiency of our data stack directly accelerates how insights flow from idea to action. Based in Chennai, you will administer and evolve Databricks and Redshift Serverless on AWS, keeping production stable while unlocking performance and cost efficiency for analytics and machine learning. You will coordinate ETL/ELT end to end, maintain strong governance, and enable colleagues to move from proof of concept to enterprise-grade solutions with confidence. Do you thrive where platform administration meets SRE and data engineering? This role places you at the nexus of operations and delivery, partnering closely with engineers, analysts, and vendors to set standards, solve complex issues, and improve benchmarks for how we use data to drive impact. Accountabilities: Platform Operations: Provision and lead Databricks workspaces, cluster policies, and Workflows; administer Redshift Serverless workgroups and name spaces; lead all aspects of Linux runtimes. Your ownership ensures availability, reliability, and a consistent operating model from day one. User and Access Governance: Implement IAM/RBAC and Unity Catalog permissions, handle Redshift roles and data-level controls, and implement least-privilege. You will protect critical data while enabling teams to move quickly. Performance and Cost Optimization: Monitor and tune PySpark/Spark SQL and Redshift Serverless; apply right-sizing and cost controls to deliver faster pipelines and predictable spend that scales with demand. Security and Compliance: Implement encryption, key and secret management, patching, and hardening; maintain audit and lineage logging. Your efforts underpin trust in our data and alignment to regulatory expectations. Automation and SRE: Use Bash/Python and CI/CD to codify infrastructure and deployments; set up monitoring, alerting, SLIs/SLOs, and runbooks. You will reduce toil, increase repeatability, and improve time to recovery. Data and ETL Operations: Orchestrate and support ingestion and transformation pipelines across S3, Databricks, and Redshift; implement data quality checks and resilient recovery. Your work keeps insights flowing to the people who need them. Incident and Change Leadership: Lead incident response and root-cause analysis; plan and implement changes, releases, and maintenance windows; communicate clearly with customers to minimize risk and downtime. Release Management: Own GitHub strategy, tagging, and protection rules; run pull requests and code reviews; maintain CI/CD workflows for Databricks jobs, notebooks, and IaC with robust versioning and rollback plans. Communication and Collaboration: Produce clear documentation and handovers; partner with engineers, DBAs, and vendors. You will align teams with varied strengths around shared standards and outcomes. Essential Skills/Experience: Databricks: PySpark and Spark SQL, Delta Lake guidelines, Workflows/Jobs, cluster policies, Unity Catalog governance, and cost controls. Redshift Serverless: Workgroup administration, RPU management, query monitoring rules, performance tuning, Spectrum/external tables, COPY/UNLOAD to S3, snapshots/restore, security and permissions. Unix/Linux: System administration (users/groups, SSH, systems, networking, filesystems), Bash scripting, log management, and OS hardening. AWS: S3, IAM, CloudWatch/Cloud Trail, basic VPC concepts, budgeting and tagging. DevOps/IaC: Git and GitHub (branching, PRs, Actions), CI/CD pipelines and deployments. Support: Strong troubleshooting experience across Spark/SQL/OS; on-call readiness and clear incident communications. Educational Background: B.E/B.Tech/MCA/MSc (or equivalent experience) Overall Experience: 5 - 6 years Shifts and On-Call: Rotational Shift: 06:00-15:00 (First) or 09:00–18:00 (General) or 14:00–23:00 (Second); Weekend: On-call support once per month (Saturday and Sunday) Desirable Skills/Experience: Cloud Infrastructure: Amazon EC2, S3, and exposure to PostgreSQL. Tooling and Process: Jira and ServiceNow for ITSM. Orchestration and Workflow: Airflow (or similar). Fast learner, proactive problem solver, able to grasp new technologies and concepts. Preferable Industry: IT, Manufacturing, Pharma, or similar enterprise environments. When we put unexpected teams in the same room, we ignite bold thinking with the power to inspire life-changing medicines. In-person working gives us the platform we need to connect, work at pace and challenge perceptions. That's why we work, on average, a minimum of three days per week from the office. But that doesn't mean we're not flexible. We balance the expectation of being in the office while respecting individual flexibility. Join us in our unique and ambitious world. Why AstraZeneca: Join a global, mission-led company where data and digital capabilities power every function—from freeing scientists to do more discovery to reimagining enterprise processes. You will work with curious, kind, and ambitious colleagues who put unexpected teams in the same room to spark bold thinking, using predictive analytics, smart automation, and modern data platforms to solve real problems that matter to patients. Here, you will be trusted to act, supported to learn, and encouraged to challenge how things have always been done—so your governance expertise translates directly into safer, faster impact at scale. If you are ready to turn thorough governance into an impetus for responsible innovation, take the lead and help us set the standard for credible data and AI today! Date Posted 11-Jun-2026 Closing Date 13-Jun-2026 AstraZeneca embraces diversity and equality of opportunity. We are committed to building an inclusive and diverse team representing all backgrounds, with as wide a range of perspectives as possible, and harnessing industry-leading skills. We believe that the more inclusive we are, the better our work will be. We welcome and consider applications to join our team from all qualified candidates, regardless of their characteristics. We comply with all applicable laws and regulations on non-discrimination in employment (and recruitment), as well as work authorization and employment eligibility verification requirements. AstraZeneca is a global, science-led, patient-focused biopharmaceutical company. We focus on discovering, developing and commercialising prescription medicines for some of the world’s most serious diseases. But we are more than one of the world’s leading pharmaceutical companies. At AstraZeneca, we’re dedicated to being a Great Place to Work. Where you are empowered to push the boundaries of science, challenge convention and unleash your entrepreneurial spirit. To embrace differences and take bold actions to drive the change needed to meet global healthcare and sustainability challenges. There is no better place to make a difference in medicine, patients, and society. An inclusive culture where you will connect different thinking to generate new and valuable opportunities. Where you will find a commitment to lifelong learning, growth and development for all. Our Inclusion & Diversity (I&D) mission is to create an inclusive and equitable environment where people belong, using the power of our diversity to push the boundaries of science to deliver life-changing medicines to patients. Inclusion and diversity are fundamental to the success of our company, because innovation requires breakthrough ideas that only come from a diverse workforce empowered to challenge conventional thinking. We’re curious about science and the advancement of knowledge. We find creative ways to approach new challenges. We’re driven to make the right choices and be accountable for our actions. As an organisation centred around what makes us human, we put a big focus on people. Across our business, we want colleagues to wake up excited about their day at the office, in the field, or in the lab. Along with our purpose to bring life-changing medicines to people across the globe, we have a promise to you: to help you realise the full breadth of your potential. Here, you’ll do work that has the potential to change your life and improve countless others. And, together with your team, you’ll shape a culture that unites and inspires us every day. This is your life at AstraZeneca.

Responsibilities

Administer and evolve Databricks and Redshift Serverless platforms on AWS to ensure stability, security, and cost efficiency. Lead platform operations, user governance, and CI/CD workflows to support enterprise-grade data and machine learning solutions.