Software Engineer, LLM Evaluation – English

Description

Our client’s mission is to empower people, build community, and bring the world closer together. Through their apps and services, they are building a different kind of company that connects people worldwide and provides meaningful ways to share what matters most.

We are looking for a Software Engineer, LLM Evaluation, to join their team in the Netherlands remotely.

The successful candidate will work closely with researchers and engineers to evaluate model performance, conduct data-driven analyses, and contribute to research initiatives related to LLM pretraining and evaluation.

Job Profile for Software Engineer, LLM Evaluation
Responsibilities will include, but not be limited to:

• Analyse and evaluate large language models and their performance across various tasks and benchmarks
• Execute benchmark evaluations and generate performance metrics and insights
• Conduct quantitative and qualitative data analysis to support research objectives
• Contribute to the design, implementation, and validation of new evaluation methodologies
• Support research initiatives related to LLM pretraining and model evaluation
• Develop and maintain machine learning and deep learning systems and tooling
• Build research workflows and experimental frameworks using Python and PyTorch
• Enable rapid experimentation and support the execution of research initiatives
• Collaborate with researchers and engineers within a multidisciplinary team environment

Candidate Profile for Software Engineer, LLM Evaluation

• Must be fluent in English, both written and spoken
• Bachelor’s degree in Computer Science, Artificial Intelligence, Machine Learning, or a related technical discipline. Master’s degree in Computer Science, Machine Learning, AI, or a related discipline is desirable; a PhD in a relevant field would be considered a strong advantage
• 3–5 years of experience working with large language models, including pretraining and evaluation
• Strong hands-on programming experience in Python
• Proven experience building and maintaining ML/DL systems and research infrastructure
• Experience developing machine learning and deep learning solutions using PyTorch
• Publications in machine learning, natural language processing, artificial intelligence, or related fields are a plus
• Experience working with transformer architectures, large language models, and/or multimodal models is an advantage
• Ability to write clean, efficient, and production-quality code
• Strong interest in model evaluation, experimentation, and data analysis
• Demonstrated scientific curiosity and problem-solving skills

What Our Client Offers

• 25 holidays per annum
• Pension plan
• Opportunity to work alongside experienced researchers and engineers on cutting-edge AI initiatives
• Exposure to state-of-the-art methodologies and technologies within the LLM ecosystem
• Collaborative and research-driven environment in a technologically advanced office

Already imagine yourself in this position? To make the first step, click Apply or email us your CV at cv@bluelynx.com

Please note

As part of the Blue Lynx recruitment process, you may be asked to provide references and copies of your diplomas. Please note that a background check may also be conducted.

Due to the high volume of applications, we are unable to respond to every candidate individually. If you have not received a call or email from a Blue Lynx recruiter within five working days, your application has not been shortlisted on this occasion.

To ensure you are considered for job opportunities, we encourage you to register an account by selecting Register Online

For any queries, please email us at: cv@bluelynx.com

To learn more about Blue Lynx and view all current vacancies, please visit our website: www.bluelynx.com

INTERESTED IN THIS JOB? – CLICK HERE