The AIML – On-Device Machine Learning group is responsible for the creation of amazing on-device ML experiences. The team builds foundational machine learning frameworks and tools to optimize large language/vision/multi-modal models that power on-device ML features across Apple products and services. The group is looking for a senior software engineer to help define and implement features that accelerate and compress large state of the art (SoTA) models (e.g., LLMs) in our on-device inference stack. This is a unique opportunity to work on exciting new technologies and contribute to Apple’s ecosystem, with a commitment to privacy and user experience impacting millions of users worldwide.
Are you someone who can write high-quality, well-tested code and collaborate cross-functionally with partner Hardware, Software, Machine Learning, and Research teams across the company? Do you have any experience building Machine Learning compilers/runtimes/kernels/optimization tools? If so, come join us and be a part of the team that is helping Machine Learning developers innovate and ship enriching experiences on Apple devices!

This role sits at the intersection of software engineering and ML engineering. As a member of this team, the successful candidate will:

– Build features for our on-device inference stack to support the most relevant accuracy preserving, general purpose techniques that empower model developers to compress and accelerate SoTA models (e.g., LLMs) in apps

– This includes building Machine Learning compilers, runtimes, execution kernels, optimizations on ML models, tooling for debugging/visualization of ML models, etc.

– Convert models from a high-level ML framework to a target device (CPU, GPU, Neural Engine) for optimal functional accuracy and performance

– Write unit and system integration tests to ensure functional correctness and avoid performance regressions

– Diagnose performance bottlenecks and work with HW & SW Arch teams to co-design solutions that further improve latency, power, and memory footprint of neural network workloads

– Analyze impact of model optimization (compression/quantization etc) on model quality by partnering with modeling and adaptation teams across diverse product use cases.

Bachelors/Masters/PhD in Computer Science or related fields.At least 4 years of experience in Machine Learning (ML) Engineering, System Software Engineering, or related fields.Strong proficiency in C/C++ and Python.Familiarity with ML fundamentals.Familiarity with developing or using ML Frameworks.