Are you passionate about advancing healthcare through responsible AI evaluation, pushing forward methods for prospective evaluation of AI tools, and building open-source software to support others to do the same? Do you have a strong background in data science, statistical analysis, and software development? If so, Stanford University invites you to apply for the role of Data Scientist within the Biomedical Informatics Research (BMIR) division GUIDE-AI Lab—a leading initiative focused on the responsible deployment and evaluation of AI in healthcare. This role will collaborate closely with other data scientists and clinicians within the Stanford Health Care Data Science Team, trainees, and students. Stanford University’s BMIR division is at the forefront of integrating AI and informatics into healthcare. Our team is committed to improving patient outcomes, enhancing healthcare delivery, and contributing to the ethical, safe, and impactful use of AI in clinical settings.
As a Data Scientist for the GUIDE-AI lab, you will develop and implement Python, R, and SQL-based tools for AI evaluation, working closely with clinicians, data scientists, and external partners to create scalable, reproducible assessment frameworks. You will contribute to the design and development of new software tools for evaluating AI models in healthcare, building on examples like APLUS-ML, a Python library for simulating machine learning model usefulness in clinical workflows, and Seismometer, an open-source monitoring system for tracking AI performance shifts in deployment. Through this work, you will help establish structured evaluation methodologies that enable healthcare organizations to rigorously assess AI models for impact, fairness, and reliability before clinical integration. Beyond software development, you will contribute to statistical analyses, prospective AI evaluation designs, and grant proposals, shaping study methodologies, constructing statistical plans, and supporting funding applications to advance GUIDE-AI’s mission. Additionally, you will play a key role in mentoring trainees and students, maintaining well-documented code repositories, and driving long-term strategic initiatives for AI evaluation in healthcare.

DESIRED QUALIFICATIONS:

• Proficiency in R, Python, and SQL, with hands-on experience in data analysis, statistical modeling, and AI evaluation.
• Expertise in full-cycle application development, including design, testing, deployment, and optimization.
• Strong analytical and problem-solving skills, with the ability to define and address complex technical challenges.
• Effective communicator, able to collaborate with both technical and non-technical stakeholders.
• Proven ability to lead structured team development projects, ensuring efficient workflows and high-quality outcomes.
• Solid foundation in probability and statistics, including power analysis, hypothesis testing, and uncertainty quantification (e.g., constructing confidence intervals, bootstrapping).
• Experience in AI model evaluation, including performance metric selection, fairness and bias assessment, and data visualization.
• (Preferred) Experience with prospective AI evaluations, including clinical trial design, real-world validation, and sample size estimation.

EDUCATION & EXPERIENCE (REQUIRED):

• Bachelor’s degree and five years of relevant experience, or a combination of education and relevant experience.

KNOWLEDGE, SKILLS AND ABILITIES (REQUIRED):

• Expertise in designing, developing, testing, and deploying applications.
• Proficiency with application design and data modeling.
• Ability to define and solve logical problems for highly technical applications.
• Effective communication skills with both technical and non-technical clients.
• Ability to lead activities on structured team development projects.
• Ability to select, adapt, and effectively use a variety of programming methods.
• Knowledge of application domain.

PHYSICAL REQUIREMENTS:

• Constantly perform desk-based computer tasks.
• Frequently sit, grasp lightly/fine manipulation.
• Occasionally stand/walk, writing by hand.
• Rarely use a telephone, lift/carry/push/pull objects that weigh up to 10 pounds.

DUTIES INCLUDE:

• Conceptualize design, implement, and develop solutions for complex system/programs independently, such as developing and implementing Python, R, and SQL-based tools for AI evaluation.
• Engage in long-term strategic planning in collaboration with staff and project leadership.
• Work with a variety of users to gain information, and develop intra-system tradeoffs between different users, as necessary; interact with a diverse client base and outside vendor contacts.
• Document system builds and application configurations; maintain and update documentation as needed.
• Provide technical analysis, design, development, conversion, and implementation work.
• Work as a project leader, as needed, for projects of moderate complexity.
• Serve as a technical resource for grant applications.
• Contribute to statistical analyses, prospective AI evaluation designs, and grant proposals, shaping study methodologies, constructing statistical plans, and supporting funding applications to advance GUIDE-AI’s mission.
• Compare, evaluate, and implement new features and technologies, and integrate them into the computing environment.
• Follow team software development methodology.
• Mentor lower-level software developers, including trainees and students.

STANFORD UNIVERSITY PROVIDES PAY RANGES REPRESENTING ITS GOOD FAITH ESTIMATE OF WHAT THE UNIVERSITY REASONABLY EXPECTS TO PAY FOR A POSITION. THE PAY OFFERED TO A SELECTED CANDIDATE WILL BE DETERMINED BASED ON FACTORS SUCH AS (BUT NOT LIMITED TO) THE SCOPE AND RESPONSIBILITIES OF THE POSITION, THE QUALIFICATIONS OF THE SELECTED CANDIDATE, DEPARTMENTAL BUDGET AVAILABILITY, INTERNAL EQUITY, GEOGRAPHIC LOCATION AND EXTERNAL MARKET PAY FOR COMPARABLE JOBS.

At Stanford University, base pay represents only one aspect of the comprehensive rewards package. The Cardinal at Work website (https://cardinalatwork.stanford.edu/benefits-rewards) provides detailed information on Stanford’s extensive range of benefits and rewards offered to employees. Specifics about the rewards package for this position may be discussed during the hiring process.