Member of Technical Staff - Training Infrastructure Engineer

About the Role

Join our Research & Engineering team as a Training Infrastructure Engineer to architect and optimize the systems that power the next generation of efficient AI models, operating at the edge and under real-time constraints. This is a full-time, remote opportunity available across the United States, Austria, Canada, France, Germany, Netherlands, Switzerland, and the UK, offering a chance to directly shape the frontier of intelligent systems by solving complex, large-scale model training challenges.

Responsibilities

• Design, build, and maintain robust distributed training infrastructure for advanced language and multimodal AI models.
• Implement and optimize training frameworks such as PyTorch Distributed, DeepSpeed, or Megatron-LM.
• Tackle complex systems challenges in large-scale model training, including efficient multimodal data loading, sophisticated sharding strategies, and resilient checkpointing mechanisms.
• Optimize communication patterns for various parallelism strategies, leveraging a deep understanding of hardware accelerators and networking topologies.

Requirements

• Extensive professional experience in building distributed training infrastructure for language and multimodal models.
• Hands-on expertise with leading distributed training frameworks like PyTorch Distributed, DeepSpeed, or Megatron-LM.
• Demonstrated ability to solve complex systems challenges related to large-scale model training.
• Deep understanding of hardware accelerators and networking topologies, with a proven track record of optimizing communication patterns.