This role supports a Globalisation Evaluation Programme by delivering high-quality data analysis and measurement within defined frameworks and methodologies. You will transform complex evaluation and annotation data into clear, structured findings that highlight trends, vendor and evaluator performance, dataset integrity, and quality issues. Using SQL or Python and modern data visualisation tools, you will build dashboards and reports that inform operational decisions, while working systematically within established standards and escalating key insights promptly and clearly.

Responsibilities

• Execute statistical analyses and inter-rater reliability metrics within defined frameworks, translating outputs into clear, structured findings.
• Operate within established evaluation frameworks, QA methodologies, and quality metrics to ensure consistent and reliable measurement.
• Curate, validate, and maintain golden datasets to defined specifications, documenting data integrity findings for review.
• Perform data quality control on large, complex datasets, identifying patterns, inconsistencies, and root causes.
• Analyse annotation and human evaluation data, including annotator feedback, to identify operational improvement opportunities.
• Assess vendor quality through structured evaluations of external rating quality, documenting accuracy and reliability findings.
• Produce recurring quality trend reports that surface patterns, loss trends, and inconsistencies across datasets, and escalate findings for strategic assessment.
• Deliver pre-delivery quality check reports using established QA methodologies and sampling frameworks.
• Monitor bias and fairness in evaluation datasets, producing documented reports and escalating issues according to defined thresholds.
• Design, develop, and maintain operational dashboards and KPI tracking systems in tools such as Tableau or Power BI to support quality visibility and decision-making.
• Query and analyse large datasets using SQL or Python to support both recurring reporting and ad hoc analytical requests.
• Create and maintain process and reporting documentation, ensuring evaluation quality processes and standards remain current and accessible.
• Collaborate cross-functionally with engineering, evaluation, and vendor teams to investigate evaluation issues and provide documented findings and improvement recommendations.
• Communicate insights clearly to stakeholders, including timely escalation of systemic issues and risks.
• Present analysis and findings in a structured, visually compelling format suitable for both technical and non-technical audiences.

Essential Skills

• Strong proficiency in executing statistical analyses and inter-rater reliability metrics within defined frameworks.
• Ability to work effectively within established evaluation frameworks, QA methodologies, and quality metrics.
• Experience with golden dataset curation, validation, and data quality control.
• Proven Experience working with annotation or human evaluation data, including annotator feedback analysis and vendor quality assessment.
• Strong analytical thinking with the ability to identify patterns, inconsistencies, and root causes across large datasets.
• Demonstrated expertise in data visualisation and dashboard development using Tableau, Power BI, or similar tools.
• Hands-on Experience querying and analysing large datasets using SQL or Python.
• Ability to work cross-functionally with clear communication and escalation skills.
• Experience producing dashboards, reports, and KPI tracking systems to support operational decision-making.
• Capability to present complex analytical findings in a clear, structured, and accessible manner.

Additional Skills & Qualifications

• Relevant degree in a quantitative, technical, or data-focused discipline.
• Experience with data engineering concepts and working with large-scale data environments.
• Familiarity with analytics platforms and reporting tools beyond Tableau and Power BI.
• Experience working with world-class AI tools and technologies such as Claude.
• Strong documentation skills, including the ability to define and maintain reporting standards and process guidelines.
• Background in bias and fairness analysis within datasets or evaluation frameworks.
• Experience in remote collaboration with cross-functional and vendor teams.
• Ability to manage multiple concurrent analyses and deliverables within defined timelines.

100% Remote Opportunity

Location

Cork, Ireland

Trading as TEKsystems. Allegis Group Limited. Level 1, The Chase, Carmanhall Road, Sandyford, Dublin 18, Ireland. No. 909257. TEKsystems is a company within the Allegis Group network of companies (collectively referred to as "Allegis Group"). Aerotek, Aston Carter, EASi, Talentis Solutions, TEKsystems, Stamford Consultants and The Stamford Group are Allegis Group brands. If you apply, your personal data will be processed as described in the Allegis Group Online Privacy Notice available at https://www.allegisgroup.com/en-gb/privacy-notices.

To access our Online Privacy Notice, which explains what information we may collect, use, share, and store about you, and describes your rights and choices about this, please go to https://www.allegisgroup.com/en-gb/privacy-notices.

We are part of a global network of companies and as a result, the personal data you provide will be shared within Allegis Group and transferred and processed outside the UK, Switzerland and European Economic Area subject to the protections described in the Allegis Group Online Privacy Notice. We store personal data in the UK, EEA, Switzerland and the USA. If you would like to exercise your privacy rights, please visit the "Contacting Us" section of our Online Privacy Notice at https://www.allegisgroup.com/en-gb/privacy-notices for details on how to contact us. To protect your privacy and security, we may take steps to verify your identity, such as a password and user ID if there is an account associated with your request, or identifying information such as your address or date of birth, before proceeding with your request. If you are resident in the UK, EEA or Switzerland, we will process any access request you make in accordance with our commitments under the UK Data Protection Act, EU-U.S. Privacy Shield or the Swiss-U.S. Privacy Shield.