ML Infrastructure Engineer at Jobgether

, , United States -

Full Time

Start Date

Immediate

Expiry Date

02 Oct, 26

Salary

150000.0

Posted On

04 Jul, 26

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Yes

Skills

GPU Cluster Infrastructure, Distributed Training, Python, Go, C++, Kubernetes, Slurm, Ray, PyTorch, JAX, RDMA, InfiniBand, NCCL, Linux Internals, CI/CD, High-Performance Computing

Industry

Internet Marketplace Platforms

Description

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a ML Infrastructure Engineer based in United States. This role focuses on building and operating the core platform that powers large-scale machine learning training and inference workloads. You will work on GPU cluster infrastructure spanning cloud, on-prem, and hybrid environments. The position plays a critical role in enabling efficient, reliable, and scalable AI development across multiple teams. You will design systems for scheduling, distributed training, storage throughput, and high-performance networking. The environment is highly technical, combining systems engineering, ML frameworks, and platform reliability at scale. You will collaborate closely with ML researchers and engineers to optimize performance, cost, and developer experience. This is a hands-on engineering role where impact is measured by infrastructure efficiency and production readiness of AI workloads. \n Accountabilities: Design, build, and operate GPU and accelerator infrastructure for large-scale training and inference workloads across cloud, on-prem, and hybrid environments. Develop scheduling, queueing, and resource management systems to maximize utilization of compute clusters. Integrate and support ML frameworks such as PyTorch, JAX, DeepSpeed, FSDP, Megatron-LM, and Ray-based training workflows. Build and maintain high-performance storage and data pipelines ensuring consistent GPU throughput. Design and optimize networking layers including RDMA, InfiniBand, and NCCL-based communication. Implement observability, monitoring, and failure analysis tools for distributed ML workloads. Drive automation for provisioning, lifecycle management, and infrastructure configuration. Partner with ML teams to forecast capacity needs and improve developer workflows and tooling. Ensure security, isolation, and multi-tenant access control across AI infrastructure systems. Optimize cost efficiency across compute, storage, and networking through intelligent resource management. Requirements: Bachelor’s or Master’s degree in Computer Science or related field. 6+ years of experience in infrastructure, platform engineering, or high-performance computing environments. Hands-on experience operating GPU clusters or large-scale ML training systems in production. Strong proficiency in Python and at least one systems programming language (Go or C++ preferred). Deep understanding of distributed systems, accelerator architectures, and ML training workflows. Experience with Kubernetes, Slurm, Ray, or similar orchestration/scheduling systems. Strong knowledge of Linux internals, networking concepts, and high-performance storage systems. Familiarity with at least one major cloud provider’s ML infrastructure stack. Solid software engineering practices including testing, CI/CD, and code review workflows. Strong communication skills and ability to collaborate across research and engineering teams. Experience with RDMA/InfiniBand, FinOps for ML workloads, or open-source ML infrastructure is a plus. Benefits: Competitive salary range: $100,000 – $150,000 100% remote (within the United States) Full-time W2 employment structure H1B transfer support for eligible candidates Opportunity to work on large-scale AI infrastructure systems Exposure to cutting-edge ML frameworks and distributed training technologies Strong engineering culture focused on performance, reliability, and scalability Direct impact on production AI systems and research acceleration Comprehensive career growth opportunities in advanced ML infrastructure. \n How Jobgether works: We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best! Why Apply Through Jobgether? Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time. #LI-CL1

Responsibilities

Design, build, and operate GPU and accelerator infrastructure for large-scale machine learning training and inference workloads. Develop resource management systems and optimize networking and storage layers to maximize compute cluster utilization.