RESEARCH INTERN - LARGE LANGUAGE MODELS AS A JUDGE (M/W/D)

at  fortiss GmbH

WHO ARE WE?

fortiss is the state research institute of the Free State of Bavaria for the development of software-intensive systems, based in Munich. The scientists at the institute work in research, development and transfer projects with universities, research institutions and technology leaders in Bavaria, Germany and Europe. They research and develop methods, techniques and tools for reliable, secure and comprehensible software solutions and artificial intelligence applications. fortiss is organised in the legal form of a non-profit limited liability company. The shareholders are the Free State of Bavaria (majority shareholder) and the Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. (Fraunhofer Society for the Promotion of Applied Research).
To strengthen our Machine Learning team, we are looking for a
Research intern / Forschungspraxis
Large Language Models (LLM) as a judge (M/W/D)

Who We Are and How We Work:

• We work with e-commerce data analysis and prediction.
• We exchange ideas on projects and new tasks in weekly meetings.
• We always keep our common goal in mind and support each other in achieving it.
• Our cooperation is characterized by flat hierarchies and teamwork.
• We always have an open ear for new ideas, and we tackle new challenges together.
• Enthusiasm for scientific work and research projects invites us to exchange ideas.

• LLM-as-a-judge is an approach that uses LLM to replace manual evaluation or feedback on different tasks.
• Conduct a comprehensive literature review and explore methodologies for using LLMs to evaluate the performance of fine-tuned models.
• Establish criteria and benchmarks for LLM-based evaluation.