JOB SUMMARY

We are looking for an innovative LLM Architect to lead the design and development of custom language models at AllCloud. This role will be responsible for architecting, training, and optimizing large language models based on modified transformer architectures. The ideal candidate will have deep expertise in NLP, transformer model design, and efficient training methodologies. You’ll work alongside GPU Engineers and ML Engineers to create state-of-the-art language models that meet our customers’ specific requirements, pushing the boundaries of what’s possible with generative AI.

SUMMARY OF KEY REQUIREMENTS

• 4+ years of experience in deep learning research or development with a focus on NLP and transformer models
• Strong understanding of transformer architecture and its variants (GPT, BERT, T5, etc.)
• Experience designing and training large language models from scratch
• Expertise in PyTorch or TensorFlow for implementing custom model architectures
• Knowledge of distributed training approaches for large models (DeepSpeed, Megatron, etc.)
• Experience with model compression techniques (quantization, pruning, knowledge distillation)
• Strong background in mathematics, particularly linear algebra, differential equations, probability, and statistics
• Familiarity with current research in LLM development, including attention mechanisms, mixture of experts, and efficient training methods
• Master’s or PhD in Computer Science, Machine Learning, or related field
• Publication record in NLP, LLMs, or transformer architecture (strongly preferred)

• Design custom transformer-based language model architectures tailored to specific use cases
• Develop and implement modifications to transformer architectures to enhance performance, efficiency, or capabilities
• Create and execute model pre-training, fine-tuning, and evaluation strategies
• Implement techniques like quantization, pruning, and knowledge distillation to optimize model size and performance
• Design and implement training data pipelines, including data selection, cleaning, and augmentation
• Establish rigorous evaluation frameworks to assess model performance, fairness, and safety
• Research and implement state-of-the-art techniques in LLM development
• Create detailed documentation on model architectures, training methodologies, and performance characteristics
• Collaborate with GPU Engineers to implement efficient training strategies across distributed systems
• Work with customers to understand their unique requirements and translate them into model design decisions
  Requirements: