Senior Research Scientist for Foundational AI Safety

About The Job

Job summary

The AI Safety Institute is the first state-backed organisation focused on advanced AI safety for the public interest. We launched at the AI Safety Summit because we believe taking responsible action on this extraordinary technology requires a capable and empowered group of technical experts within government. Our staff includes senior alumni from OpenAI, Google DeepMind, start-ups and the UK government, and ML professors from Oxford and Cambridge. We are now calling on the world’s top technical talent to build the institute from the ground up. This is a truly unique opportunity to help shape AI safety at an international level.

We are intent on pulling in the best technical talent from academia and industry, and we move at a pace and urgency befitting the need to stay on top of cutting-edge AI development. Our top priorities are to:

• Drive foundational AI safety research. We will launch research projects in foundational AI safety to support new forms of governance and enable fundamentally safer AI development, both internally and by supporting world-class external researchers. This is the topic of this specific call.
• Develop and conduct evaluations on advanced AI systems. We will characterise safety-relevant capabilities, understand the safety and security of systems, and assess their societal impacts.
• Facilitate information exchange. We will establish clear information-sharing channels between the Institute and other national and international actors. These include policymakers, international partners, private companies, academia, civil society, and the broader public.

Job Description

As a Senior Research Scientist at AISI, you will lead Research Scientists and Research Engineers in planning and executing on research directions in foundational AI safety. You’ll lead and contribute to projects aimed at improving the fundamental safety of advanced AI systems, making novel research contributions that are aimed at informing governance with technical tools on safe AI.

We draw on a wide range of disciplines, and value a diversity of research expertise across our four workstreams. You’ll be primarily associated with one of our research workstreams (please specify in your application which you’re most interested in, details below), but sometimes your work will intersect multiple workstreams:

• Capabilities elicitation & jailbreaking. This workstream consists of research to ensure that AISI can get closer to the “capabilities ceiling” of the models we work with, including development of novel jailbreaks and methods such as “next-gen chain-of-thought" to increase model performance on our evals.
• AI model explainability & accountability. This workstream focuses on techniques for improving the transparency of model behaviour via e.g. casual reasoning traces.
• Interventions on model behaviour. This workstream will investigate ways of intervening on model behaviour pre- and post-deployment to solve safety issues beyond anecdotal patching, i.e. in ways that will be robust to improved prompting techniques. E.g. via data attribution techniques.
• Rethinking alignment. This workstream consists of developing methods for alignment that go beyond existing state-of-the-art, for example by reliance on formal languages or probabilistic approaches.

You will work under Prof. Yarin Gal, and work closely with your team comprising of multiple Research Scientists, Research Engineers, and Software Engineers, as well as benefit from support from our cross-functional Platform Engineering team. You’ll also collaborate with external topic-level experts, contractors, partner organisations and policy makers to coordinate and build on external research.

Day-to-day, you may:

• Conduct novel research
• Supervise Research Scientists in your team and mentor them
• Design experiments to validate hypotheses about models’ behaviour
• Work with experts to assemble relevant literature or datasets
• Prepare papers for internal or external publication
• Manage external research collaborations
• Prepare a write-up of your research to brief policy makers

The ideal candidate will have the skills and experience that allow them to additionally:

• Design experiments to conduct rigorous and impactful machine learning research
• Analyse and interpret data in ways that meaningfully contribute to research at AISI
• Understand the capabilities and potential impact of advanced AI systems, including large language models
• Keep track of and respond to the fast-moving literature, ensuring that our research is additional to that being conducted elsewhere
• Coordinate closely with research and engineering partners in your team and stakeholders across AISI to design new methods
• Clearly communicate insights from studies to influence research decisions across stakeholders throughout AISI and DSIT

There will be significant scope to contribute to the strategy of your workstream team and to design experiments with set-ups of increasing complexity.

Person specification

For the senior research scientist role, you’ll have conducted ML research, research in a domain relevant to your primary workstream, or research at the intersection of your domain and frontier AI systems.

We expect experts in both ML and a specific domain relevant to one of our workstreams to be rare, so we encourage you to apply no matter which research expertise you’re excited to bring to the institute.

We look for the following skills, experience and attitudes:

• Significant track record of conducting and disseminating rigorous and impactful research
• Significant experience leading work in AI/ML research
• PhD or equivalent research experience in a field related to your workstream, or in machine learning. If you have more work experience, you might be assigned a more senior role in the team
• Experience with large language models, potentially related to prompt engineering, tooling, or fine tuning
• Possess a strong curiosity in technical understanding of AI systems and studying their limitations
• Motivated to conduct research that is not only curiosity driven but also solves concrete open questions in AI safety and has a strong grounding for how research may impact governance and policy making
• Experience of mentoring others in a research setting
• Statistics expertise (e.g., data analysis)
• Good technical and coding skills, e.g. Python
• Strong project management skills and demonstrable experience of teamwork, including building rapport with people from diverse backgrounds

Given the changing nature of the field, it’s most of all important to us to build a team with strong problem-solving skills and a preparedness to acquire any missing knowledge necessary to get the job done.

Owing to the rapid development of advanced AI systems, you will likely be up to date with the latest advancements.