AI Safety and Testing: Who Checks the Systems That Are Changing Britain?

1. What is being funded?

The central programme in this investigation is the UK Government's investment in the AI Security Institute (AISI), the government research organisation responsible for evaluating advanced AI systems and investigating the risks they may pose. The institute was formerly known as the AI Safety Institute and was renamed in February 2025. It operates within the Department for Science, Innovation and Technology (DSIT).

The latest funding announcement came in the Government's Budget 2026, published on 28 October 2026. The Budget commits an additional £120 million to the institute over the four financial years from 2026–27 to 2029–30. The stated purpose is to strengthen frontier AI evaluation and testing, improve understanding of advanced AI capabilities and develop technical standards that help mitigate risks.

This is additional to the £240 million allocated to AISI at Spending Review 2025, which the Government said would support frontier model testing, foundational safety research and societal resilience. These are separate funding announcements covering different periods. They should be reported as distinct commitments, with the Government's financial documents used to establish the full funding profile, rather than assuming that every announced pound has already been spent.

The new £120 million commitment averages £30 million per financial year if spread evenly over its four-year period. That is an illustrative calculation, not a published annual spending schedule. Actual spending may vary according to recruitment, research contracts, computing requirements and the timing of individual projects.

Sources: HM Treasury, Budget 2026; AI Opportunities Action Plan: One Year On, January 2026.

2. What does AI safety testing actually mean?

AI safety testing is the process of investigating how an AI system behaves under specified conditions, particularly where errors, misuse or unexpected capabilities could cause harm. It is not a single examination with a simple pass-or-fail result. Different tests examine different abilities, failure modes and deployment contexts.

For example, a research team might test whether an advanced AI model can assist with cyber attacks, whether its safeguards prevent prohibited assistance, whether it follows instructions reliably or whether it can complete a sequence of actions with limited human supervision. Other evaluations examine manipulation, the ability to evade monitoring and potential effects on society as AI becomes more widely deployed.

These tests matter because a system that performs well on everyday tasks may behave differently when given complex instructions, access to external tools or opportunities to carry out actions over an extended period. A model's ability to describe a dangerous activity is also different from its ability to carry it out successfully in a real environment.

AISI's published approach makes an important qualification: evaluations measure particular capabilities and risks, rather than comprehensively certifying an entire AI system as safe. The institute is not a regulator and does not make the final decision about whether a company releases its product.

This distinction is fundamental. A favourable test result means that a system performed within specified limits under the conditions examined. It does not prove that every risk has been identified or that the system will behave safely in every real-world situation.

Source: AISI, Approach to Evaluations.

3. Who benefits, and who bears the risks?

The immediate beneficiaries of public investment include researchers, specialist testing teams, universities and organisations developing methods for evaluating advanced AI. Companies may also benefit from clearer evidence about how their systems behave, while policymakers gain information that can inform decisions about regulation and public-sector adoption.

The wider public has a stake because AI is increasingly used in areas that affect employment, education, healthcare, financial services, communications and public administration. If testing identifies a serious weakness before a system is widely deployed, it may help reduce the chance of harm. If testing is inadequate, the consequences may fall on people who had no involvement in developing or purchasing the technology.

There is a potential conflict of interest to manage. AI companies have commercial incentives to release useful products quickly, attract customers and maintain investor confidence. They also have legitimate reasons to protect confidential research and security-sensitive information. Government evaluators need enough independence and access to conduct credible tests without allowing commercial pressure to determine their conclusions.

There is a second distributional question: whose risks receive attention? Advanced technical risks deserve serious investigation, but safety work must also address harms that people already experience, including discrimination, inaccurate automated decisions, privacy breaches, scams, impersonation and inaccessible services. A credible programme should explain how it prioritises different risks and how those priorities relate to the systems people actually encounter.

4. What evidence is emerging from testing?

AISI's Frontier AI Trends Report presents findings from evaluations of advanced AI systems conducted since November 2023. The institute reports that capabilities have improved rapidly across the domains it tests. In cyber-related tasks, for example, it found that models could complete apprentice-level tasks around 50% of the time on average, compared with just over 10% in early 2024.

These findings suggest that the capabilities relevant to safety and security cannot be treated as static. A testing programme that was adequate for an earlier generation of models may need to change as systems become more capable, gain access to tools or operate with greater autonomy.

However, the figures need context. Performance on controlled tasks does not automatically translate into equivalent success in real-world situations. Real environments introduce additional obstacles, including access restrictions, unreliable information, cost, latency, security controls and the need to coordinate multiple actions. The institute also warns that its findings are a snapshot and may not cover every recently released model.

Some technical details are withheld because publishing them could enable misuse. This creates a genuine tension: researchers need enough openness for independent scrutiny, but disclosing every test procedure or vulnerability may help people circumvent safeguards. The answer is not necessarily total disclosure. It is a system in which sensitive details can be reviewed by appropriately qualified independent experts, while the public receives meaningful summaries of the methods, limitations and results.

Source: AI Security Institute, Frontier AI Trends Report.

5. How are grants supporting AI safety research?

Alongside its in-house testing, AISI has supported external research through grant programmes. Its published grants information identifies more than £15 million in funding through the Alignment Fund, alongside other programmes addressing different AI safety and security questions.

The Alignment Project, launched in 2025, is an international, cross-sector coalition supporting research into whether advanced AI systems can reliably behave as intended. Its first funding round awarded more than £27 million to over 60 projects, with funding from government, industry and philanthropic participants. Awards were reported at between £50,000 and £1 million per project.

The total value of the Alignment Project should not be mistaken for a grant allocation funded entirely by the UK taxpayer. The programme brings together multiple funders, including the UK institute, international government partners, research organisations and private companies. A proper account of UK public spending must identify the contribution from each funding source and avoid counting the same commitment more than once.

These grants can support work that would otherwise struggle to attract funding, including research into monitoring, oversight, interpretability and techniques for keeping systems aligned with intended objectives. External research can also challenge assumptions made by the organisations building and evaluating AI.

But grants awarded are outputs, not proof of successful outcomes. The number of projects funded, papers published or researchers recruited provides useful information about activity. It does not by itself demonstrate that the resulting methods work reliably or reduce real-world harm.

According to AISI's current grants page, its listed programmes are not accepting applications. Its Alignment Project website also states that applications are closed and that a similar programme is unlikely to run in 2026, although further academic grant programmes are expected. This status should be checked again before any prospective applicant relies on it.

Sources: AISI Grants; The Alignment Project.

6. Where will the money be spent?

The institute is a UK government organisation, but its work can have international implications because leading AI models are developed, evaluated and deployed across national borders. Research partnerships, access to advanced models and shared testing methods can allow findings from one jurisdiction to inform the work of others.

The Government has also reported that AISI has grown to more than 100 technical staff. Building this capability requires more than paying researchers. It can involve access to advanced computing, specialist infrastructure, secure working environments, technical tools, recruitment, collaboration agreements and the ongoing maintenance of evaluation systems.

The publicly announced funding figures do not, on their own, provide a complete breakdown of how the additional £120 million will be distributed across salaries, computing, grants, procurement, international collaboration and other costs. Nor do the headline commitments establish how much of the money has been spent to date.

To judge geographical fairness, the Government should also explain where specialist roles, external contracts and research partnerships are located. A national security capability may reasonably require concentration in certain locations, but opportunities for universities, smaller research organisations and qualified specialists outside established technology centres should not be overlooked without justification.

7. What are the main weaknesses of current testing?

Testing cannot cover every possible situation

AI systems can be used in an enormous range of contexts. A test suite can examine selected behaviours, but no realistic programme can reproduce every possible user, instruction, data source, tool interaction or deployment environment. Passing known tests is therefore not a guarantee of safe behaviour elsewhere.

Testing should combine controlled benchmarks, adversarial exercises, independent research, deployment monitoring and repeated assessment after significant system changes. Results should be revisited when models are updated or connected to new tools, rather than treated as permanent certificates.

Companies may know more than evaluators

Developers generally have access to detailed information about the design, training and internal performance of their systems. External evaluators may have to work with restricted interfaces or limited access. That can make independent assessment harder and may prevent tests from identifying important weaknesses.

Clear agreements on access, confidentiality and disclosure can help. For the most consequential systems, government should have sufficient technical access to investigate relevant risks, while protecting legitimate trade secrets and sensitive information.

Tests can create false reassurance

A score can look precise even when the underlying measurement is incomplete. Different test designs may produce different results, and the relationship between performance on a benchmark and real-world harm may be uncertain. A company might also improve its score on a known test without addressing the underlying problem that the test was intended to measure.

Independent replication, varied testing methods and transparent reporting of uncertainty can reduce these risks. Reports should explain what was tested, what was not tested and what assumptions limit the conclusions.

Safety extends beyond model behaviour

A model may behave as expected in a laboratory but still be deployed in a poorly designed service. Harm can arise from inadequate staff training, weak security, inappropriate access permissions, misleading interfaces or decisions that give people no meaningful opportunity to challenge an outcome.

For that reason, technical model testing must be complemented by system-level evaluation. The relevant unit of safety is often the complete service, including its users, organisational processes, data, tools and oversight arrangements.

8. What if the programme succeeds or fails?

Best-case scenario: the additional investment gives the UK a stronger, independent evidence base on advanced AI risks. Testing identifies important weaknesses earlier, researchers develop more reliable safeguards and public authorities make better-informed decisions about deployment and regulation. Findings are shared appropriately with international partners.

Middle scenario: the institute produces valuable research and improves technical understanding, but results remain uneven. Some tests become useful tools for policy and industry, while other risk areas remain difficult to measure. Companies continue to make release decisions, and safety outcomes depend on a wider combination of voluntary practices, procurement requirements and regulation.

Worst-case scenario: capabilities develop faster than evaluation methods, important risks remain undetected, and public spending produces technical reports without sufficient influence on deployment decisions. Alternatively, governance becomes so opaque that outsiders cannot assess the institute's performance, or the programme becomes overly dependent on a small number of suppliers and research partners.

None of these scenarios is inevitable. The programme's value depends on the quality of its research, access to the systems being tested, the independence of evaluation and whether findings lead to practical changes.

9. Alternatives and complementary approaches

The first alternative is to rely more heavily on developers' own safety testing and public disclosures. Companies possess detailed knowledge of their systems and can test frequently during development. This approach may be faster and less expensive for routine evaluation, but it does not replace independent scrutiny where commercial incentives and public risks may differ.

The second is to prioritise independent research distributed across universities, non-profit organisations and civil society. A broader research ecosystem can encourage different methods, improve external challenge and reduce dependence on one institution. Its limitations include fragmented funding, uneven access to advanced models and difficulty coordinating results into consistent government advice.

The third is to combine independent technical evaluation with clear legal duties and proportionate regulatory oversight. Regulation can establish minimum expectations for documentation, incident reporting, risk management and accountability. Its effectiveness depends on well-defined obligations, competent enforcement and the ability to adapt to changing technology.

These approaches are complementary rather than mutually exclusive. An effective system needs developer testing, independent research, government technical expertise, appropriate regulation and monitoring after deployment. No single laboratory or testing method can provide all the assurance required.

10. A practical action plan

Immediate priorities

• DSIT and AISI: publish a delivery plan for the additional £120 million, including annual budget profiles, priorities, milestones and the intended outcomes of each major area of expenditure.
• AISI leadership: publish an evaluation framework that distinguishes controlled test results from evidence of real-world performance, and explains the principal limitations of each method.
• Independent scientific reviewers: assess whether the institute's methods are sufficiently robust, reproducible and appropriate for the risks being examined.
• Government procurement teams: establish proportionate requirements for independent testing, incident reporting and post-deployment monitoring in high-risk public-sector AI contracts.

Medium-term priorities

• Publish regular reports showing how evaluation findings have informed policy, procurement, safeguards or decisions to delay deployment.
• Expand collaboration with universities, civil society and specialist researchers, including organisations able to examine the effects of AI on vulnerable groups and people with disabilities.
• Develop methods for testing complete AI-enabled services, not just standalone models, including their human oversight, security and complaint processes.
• Commission independent evaluations of the grants programme, including whether funded research produces usable safeguards and whether funding reaches a sufficiently diverse research community.

Long-term priorities

• Maintain evaluation capacity as systems and deployment methods change, with reassessment following significant model or tool updates.
• Support proportionate international cooperation on testing methods, incident reporting and emerging risk assessment.
• Review funding against independently assessed outcomes, and redirect resources where the evidence shows that alternative approaches would deliver greater public benefit.

11. How should success be measured?

The Government should publish a clear set of performance indicators and report against them annually. These should include the number and range of systems evaluated, the time taken to assess priority risks, the proportion of findings independently reviewed, and the extent to which identified weaknesses lead to meaningful mitigation.

For external grants, indicators should distinguish money awarded, projects completed and research findings from practical outcomes. Measures could include independent replication, successful testing of proposed safeguards, adoption by relevant organisations and evidence that a mitigation reduces a defined risk under specified conditions.

For public accountability, reporting should show expenditure against budget, major procurement commitments, staffing and capability development, and significant changes in programme priorities. Independent evaluators should examine both successful and unsuccessful projects, rather than relying solely on the institute's own account of its achievements.

Some details will need to remain confidential for legitimate security or commercial reasons. Nevertheless, aggregate expenditure, evaluation coverage, methodology limitations and independently assessed progress can often be published without revealing sensitive test instructions or exploitable vulnerabilities.

12. Accountability: who checks the checkers?

AISI's position within DSIT gives it access to government expertise and policy discussions, but its research role should not be confused with independent statutory regulatory authority. The institute evaluates systems and provides evidence; responsibility for release decisions remains with the relevant developers and deployers, while legal accountability depends on the applicable laws and circumstances.

DSIT should remain accountable for the programme's objectives and public funding. AISI's leadership should explain its methods, limitations, results and use of resources. Independent researchers should be able to challenge its conclusions, and appropriate audit and parliamentary processes should examine spending and delivery.

The Government should also explain how conflicts of interest are handled when staff, advisers or research partners have current or previous relationships with AI developers. Such relationships do not automatically invalidate research, but transparent declarations, appropriate safeguards and independent review help maintain confidence.

Public trust requires more than assurances that experts are doing their best. It requires evidence that the work is rigorous, that limitations are acknowledged and that findings can influence decisions. An institute should be judged not only by its technical sophistication but also by the quality of the decisions its evidence enables.

13. What remains unanswered?

The Budget establishes an additional £120 million commitment for 2026–27 to 2029–30, but the public needs a detailed spending profile to understand how the money will be allocated. It also needs evidence of how the new funding will build on the existing £240 million Spending Review allocation without duplication.

Further questions concern the extent of model access, the proportion of evaluations that can be independently reproduced, the criteria for selecting systems to test and how the institute decides which risks deserve priority. It is also important to understand how evaluation findings influence companies that retain control over their own release decisions.

Finally, the public should be able to see how testing benefits people outside the specialist AI sector. That means evidence of improved safeguards in real services, stronger procurement decisions, better incident response and meaningful protection for people who might otherwise bear the costs of AI failure.

Conclusion: funding tests is necessary, but it is not the same as proving safety

The UK's investment in the AI Security Institute recognises a real challenge: advanced AI capabilities are changing, and governments need independent technical expertise to understand the risks. The additional £120 million announced in Budget 2026 builds on the £240 million Spending Review allocation and provides an opportunity to strengthen evaluation, research and technical standards.

The evidence from AISI's own reports shows why ongoing assessment matters, while its published limitations make clear that testing cannot certify every system as safe. A favourable result is useful evidence, not a guarantee. Nor can technical testing replace sound organisational practices, effective regulation or human accountability.

The investment will represent value for money only if the UK can demonstrate that the research improves decisions and reduces identifiable risks. The priority now should be transparent financial reporting, independent scrutiny, clear performance measures and a documented link between testing results and real-world safeguards.

The central question is not simply whether Britain can fund more AI safety testing. It is whether that testing is sufficiently independent, rigorous and influential to make advanced AI safer for the people who will live and work with it.