Data and AI Engineer (Manager) at Visa

Bengaluru, karnataka, India -

Full Time

Start Date

Immediate

Expiry Date

11 May, 26

Salary

0.0

Posted On

10 Feb, 26

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Python, Scala/Java, Apache Spark, Hadoop Ecosystem, Kafka, SQL, ETL/ELT Pipelines, AWS/Azure, CI/CD Pipelines, Docker, Kubernetes, Airflow, MLOps, Feature Stores, GenAI Applications, Technical Leadership

Industry

IT Services and IT Consulting

Description

Company Description Visa is a world leader in payments technology, facilitating transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories, dedicated to uplifting everyone, everywhere by being the best way to pay and be paid. At Visa, you'll have the opportunity to create impact at scale — tackling meaningful challenges, growing your skills and seeing your contributions impact lives around the world. Join Visa and do work that matters — to you, to your community, and to the world. Progress starts with you. Job Description We are seeking an experienced Manager - Data & AI Engineer to join our CEMEA team. This is a Staff-level individual contributor role for someone who thrives on solving complex technical challenges, architecting scalable data platforms, and driving engineering excellence. You'll lead critical data engineering initiatives, mentor talented engineers, and build the data infrastructure that powers insights and AI-driven solutions for Visa's global clients. What You'll Do: Data Platform Architecture & Development: Design and build enterprise-scale data platforms using modern big data technologies including Spark, Hadoop, Kafka, and cloud-native services. Architect robust, scalable data pipelines that process petabytes of data for batch, streaming, and real-time analytics Drive technical decisions on architecture, tooling, and engineering practices that impact multiple projects and teams Establish and enforce engineering standards, best practices, and code quality across data engineering initiatives Build Production-Grade Data Pipelines: Develop and optimize large-scale ETL/ELT pipelines for data ingestion, transformation, quality assurance, and feature engineering Implement streaming data pipelines using Kafka and Spark Streaming for real-time analytics and decision-making Design data models, partitioning strategies, and optimization techniques for distributed systems Ensure data quality, reliability, and observability across all data workflows Enable AI/ML & Advanced Analytics: Build data infrastructure that supports AI/ML workloads including feature stores, training pipelines, and model serving infrastructure Collaborate with data scientists to productionize machine learning models through robust MLOps practices Design and implement data pipelines for GenAI applications including embeddings generation, vector storage, and retrieval systems Support deployment of AI/ML models with scalable inference pipelines and monitoring Drive Cloud Infrastructure & DevOps Excellence: Manage and optimize AWS/Azure cloud infrastructure (S3, EMR, EC2, Lambda, Glue, Redshift, SageMaker) Build CI/CD pipelines and automate deployments using Jenkins, Git, Docker, and Kubernetes Implement workflow orchestration using Airflow, Prefect, or Control-M Design for high availability, disaster recovery, and system reliability Technical Leadership & Collaboration: Mentor junior data engineers, fostering a culture of continuous learning and innovation Code reviews and technical discussions to elevate team capabilities Partner with product managers, data scientists, and business stakeholders to translate requirements into technical solutions Stay current with emerging technologies and drive adoption of best practices in data engineering and AI/ML infrastructure This is a hybrid position. Expectation of days in office will be confirmed by your hiring manager. Qualifications 7+ years of hands-on data engineering experience with a Bachelor's degree, or 6+ years with a Master's degree in Computer Science, Engineering, Statistics, or related technical field Proven track record of building and leading complex data engineering projects at scale Must-Have Technical Skills: Core Data Engineering Expertise: Expert proficiency in Python and Scala/Java for building production data systems Deep hands-on experience with Apache Spark (Spark SQL, DataFrames, Streaming) including performance tuning and optimization Strong expertise in Hadoop ecosystem: HDFS, Hive, HBase, YARN Production experience with Kafka for building event-driven and streaming architectures Advanced SQL skills with experience in both RDBMS and NoSQL databases (Cassandra, MongoDB, Redis) Proven experience designing and deploying large-scale ETL/ELT pipelines processing terabytes of data Strong AWS/Azure experience: S3, EMR, EC2, Lambda, Glue, Redshift, SageMaker Solid understanding of data modeling, partitioning strategies, and distributed systems optimization DevOps & Infrastructure: Experience with CI/CD pipelines (Jenkins, GitLab CI, GitHub Actions) Hands-on experience with Docker and Kubernetes for containerization and orchestration Proficiency with workflow orchestration tools like Airflow, Prefect, or Control-M Experience with infrastructure as code and automation MLOps & AI Infrastructure: Experience building feature engineering pipelines and feature stores Understanding of MLOps workflows: model deployment, versioning, monitoring, and automation Experience building data pipelines that support ML/AI workloads Familiarity with model lifecycle management and productionization Preferred Skills: Advanced Data Engineering: Experience with real-time processing frameworks (Flink, Spark Streaming, Kafka Streams) Familiarity with modern data platforms (Databricks, Snowflake) Experience with data quality frameworks and observability tools (Great Expectations, Datadog, Prometheus) Knowledge of DR/HA architectures and reliability engineering Multi-cloud experience (Azure, GCP) Understanding of data governance, security, and compliance AI/GenAI Infrastructure: Experience with vector databases (Pinecone, Weaviate, Milvus, ChromaDB, FAISS) Understanding of RAG (Retrieval-Augmented Generation) system architectures Familiarity with Model Context Protocol (MCP) for LLM integrations Experience with embeddings generation and semantic search pipelines Exposure to model serving frameworks (TensorFlow Serving, Triton, SageMaker endpoints) Knowledge of cloud AI services (AWS Bedrock, Azure OpenAI, Vertex AI) Experience with LLM orchestration frameworks (LangChain, LlamaIndex) What Makes You Stand Out: Strong architectural thinking with ability to design systems for scale, reliability, and maintainability Proven ability to drive technical initiatives independently with minimal supervision Deep problem-solving skills and comfort navigating ambiguity in complex technical environments Excellent communication and stakeholder management abilities Passion for mentoring and elevating engineering teams Curiosity and adaptability to stay ahead of emerging technologies in data engineering and AI/ML Additional Information Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law. Job Family Group: Sales and Business Development

Responsibilities

The role involves designing and building enterprise-scale data platforms using modern big data technologies, architecting robust, scalable data pipelines, and establishing engineering standards. Responsibilities also include developing production-grade ETL/ELT pipelines, enabling AI/ML workloads, and driving cloud infrastructure excellence.