Lead Software Engineer - Data Platform ML (ASE)

at  Apple

SUMMARY

Posted: Aug 29, 2024
Role Number:200560390
The Data Platform team within the services organization is responsible for enabling analytics, experimentation, and ML feature engineering to support Siri, Search, iCloud, Apple Pay, and other beloved ML features on Apple devices. The mission of the Data Platform organization is to provide engineers and data scientists with an innovative, reliable, secure, and user-friendly infrastructure for ingesting, storing, processing, and interacting with data. This infrastructure ultimately supports teams in building successful data-intensive applications including machine learning, deep learning and high-performance computing. You will collaborate with numerous cross-functional teams to lead the planning, execution, and success of technical projects aimed at improving the Siri and Search experience for Apple customers. We are looking for a Staff Data Infrastructure Engineer passionate about advancing our data platform by building frameworks and architectures using state-of-the-art technology across the technical stack. You will collaborate with product and infrastructure teams to ensure operational efficiency is integral to every feature we launch. Are you passionate about developing an ML compute-enabled data platform and addressing large-scale data challenges? Join us and be part of the Data Platform journey.

DESCRIPTION

You will be responsible for defining and driving the infrastructure roadmap for our data platform, offering the best automation, tooling and data security control across our stack at Apple scale. You will collaborate with cross functional teams of innovative software engineers, product managers, and engineering managers to ensure that GPU infrastructure is reliable, scalable and optimized for performance. We embrace the use of open source technologies including Kubernetes, Spark, Flink, Trino, Iceberg for data processing and Ray, Feature Platform for ML compute usecases. RESPONSIBILITIES INCLUDE: Define and drive technical vision, roadmap, and strategy to manage GPU infrastructure of our platform for analytical and ML usecases. Participate in product design reviews to ensure performance optimization and monitoring is a core component of design Collaborate with stakeholders and cross-functional leaders in engineering, product, and operations across Apple to ensure the adoption of our data platform is done in a security compliant manner Liaison and coordinate with Corporate Information Security group for reviews, risk assessment, vulnerability treatment, security patches, etc Lead and mentor new hires or junior engineers Provide guidance and establish processes to ensure engineering excellence and operational sustainability with security compliance Foster a healthy, inclusive, collaborative, and technology-driven culture

• 10+ years of software development experience
• Experience with commercial and/or open source large scale data processing, storage frameworks and platforms
• Strong experience with infrastructure automation and provisioning including Kubernetes/Terraform.
• Experience architecting, building and operating large scale data processing systems in the public cloud
• Proficient in best practices and enforcement for data security, automation-driven, proactive monitoring
• Excellent verbal and written communication skills, able to collaborate cross-functionally with program managers and engineering partners
• Experience in influencing and driving key product innovations and opportunities across diverse collaborators
• BS, MS, or PhD degree in Computer Science or equivalent experience

PREFERRED QUALIFICATIONS

• Familiarity with GPU computing or ML modeling frameworks strongly preferred.
• Experience working with or developing Large-language models (LLMs)
• Experience developing and optimizing algorithms that run efficiently on resource constrained platforms
• Design, implementation and benchmarking/fine-tuning of ML/deep learning algorithms
• Experience with observability tools like Prometheus and Grafana.

• 10+ years of software development experience
• Experience with commercial and/or open source large scale data processing, storage frameworks and platforms
• Strong experience with infrastructure automation and provisioning including Kubernetes/Terraform.
• Experience architecting, building and operating large scale data processing systems in the public cloud
• Proficient in best practices and enforcement for data security, automation-driven, proactive monitoring
• Excellent verbal and written communication skills, able to collaborate cross-functionally with program managers and engineering partners
• Experience in influencing and driving key product innovations and opportunities across diverse collaborators
• BS, MS, or PhD degree in Computer Science or equivalent experienc