Senior AI Model Evaluation Engineer - LLMs & Generative AI

at  Truveta

SENIOR AI MODEL EVALUATION ENGINEER - LLMS & GENERATIVE AI

Truveta is the world’s first health provider led data platform with a vision of Saving Lives with Data. Our mission is to enable researchers to find cures faster, empower every clinician to be an expert, and help families make the most informed decisions about their care. Achieving Truveta’ s ambitious vision requires an incredible team of talented and inspired people with a special combination of health, software and big data experience who share our company values.

WHO WE NEED

Truveta is rapidly building a talented and diverse team to tackle complex health and technical challenges. Beyond core capabilities, we are seeking problem solvers, passionate and collaborative teammates, and those willing to roll up their sleeves while making a difference. If you are interested in the opportunity to pursue purposeful work, join a mission-driven team, and build a rewarding career while having fun, Truveta may be the perfect fit for you.

THIS OPPORTUNITY

We are seeking a highly skilled and experienced Machine Learning Expert to join our team and evaluate the effectiveness of our AI models utilizing Large Language Models (LLMs). The ideal candidate will have a strong background in machine learning, natural language processing (NLP), and a proven track record of working with LLMs. You will be responsible for assessing the performance, accuracy, and efficiency of our AI models, identifying areas for improvement, and providing actionable insights to enhance their effectiveness. You will work in an exciting and fast-paced environment, collaborating closely with multiple teams across the company. You will work as part of an organization that brings together talent from diverse backgrounds including software engineering, big data, machine learning and AI, and clinical experts, making our team an exciting place to work. We value and encourage diversity in the belief that our differences make us and our products better.

KEY QUALIFICATIONS

• Master’s or Ph.D. in Computer Science, Machine Learning, Artificial Intelligence, or a related field.
• 3+ years of experience in machine learning, with a focus on natural language processing and large language models.
• Proven track record of evaluating and improving AI models, particularly those involving LLMs.
• Strong programming skills in Python and experience with machine learning frameworks such as TensorFlow, PyTorch, or similar.
• Familiarity with performance evaluation metrics and techniques for LLMs.
• Excellent analytical and problem-solving skills.
• Ability to communicate complex technical concepts to non-technical stakeholders.

PREFERRED QUALIFICATIONS

• Experience with popular LLMs such as GPT, BERT, T5, or similar.
• Knowledge of deployment and scaling of AI models in a production environment.
• Previous experience working in a fast-paced, agile development environment.
• Contributions to the AI/ML community through research papers, conference presentations, or open-source projects.
  This position offers a unique chance to work on state-of-the-art language models and contribute to transformative research with the vision of Saving Lives with Data. You will be part of a diverse and team who are passionate about pushing the boundaries of machine learning and natural language understanding in the healthcare domain. Join us and make a significant impact on the future of healthcare and patient well-being.
  Why Truveta?
  Be a part of building something special. Now is the perfect time to join Truveta. We have strong, established leadership with decades of success. We are well-funded. We are building a culture that prioritizes people and their passions across personal, professional, and everything in between. Join us as we build an amazing company together.

• Evaluate the performance of AI models that leverage Large Language Models (LLMs) in various applications.
• Develop and implement metrics to measure the accuracy, efficiency, and overall effectiveness of LLM-based AI models.
• Conduct thorough analyses to identify strengths and weaknesses in model performance.
• Provide detailed reports and recommendations to improve model accuracy, efficiency, and scalability.
• Collaborate with the development team to refine and enhance AI models based on evaluation results.
• Stay up-to-date with the latest advancements in machine learning, NLP, and LLMs to ensure our models remain at the forefront of the industry.
• Ensure that models meet business requirements and deliver tangible value to our clients.