Media Blog

Daniel Hulme in Fortune: Is an Industry-Wide AI Slowdown Realistic?

Sairah Jahangir — Mon, 05 Oct 2026 13:17:21 GMT

Daniel Hulme tells Fortune that a globally coordinated AI pause isn't on the cards. The real challenge is governing how AI is deployed, through independent testing and the kind of frameworks other industries already use. "We don't have to reinvent the wheel."

Daniel Hulme at DMEXCO 2026: Rethinking AI and How Agents Impact Marketing, Business and Humanity

Sairah Jahangir — Mon, 05 Oct 2026 13:09:42 GMT

On DMEXCO 2026's Center Stage, Daniel Hulme offered a new framework for thinking about AI and agents: how organisations can adopt them practically, avoid the hype, and use them to unlock the creative capacity of their people.

Daniel Hulme on The Economist's Weekend Intelligence: What Is the Point of Remembering?

Sairah Jahangir — Mon, 05 Oct 2026 13:05:58 GMT

Conscium's Daniel Hulme joined Rosie Blau on The Economist's Weekend Intelligence podcast to explore memory, both human and machine, where it succeeds and fails, and why it's one of the most important challenges in AI development.

Safety Concerns Prompt OpenAI to Hold Off on Astra Release

Sairah Jahangir — Mon, 05 Oct 2026 13:01:26 GMT

Astra won't ship until it clears additional safety checks, OpenAI said, as it also apologised over how it dealt with a breach of an Australian government site.

Virgin Radio Interview: Chris Evans & Calum

Calum Chace — Mon, 05 Oct 2026 12:45:52 GMT

Conscium co-founder Calum Chace joined Chris Evans on The Chris Evans Breakfast Show on Virgin Radio for a wide-ranging conversation about the future of artificial intelligence. Introduced as "quite literally the AI guy,"

AI Models Breach Security: A Close Look at the OpenAI and Hugging Face Incident with CNN

Calum Chace — Thu, 03 Sep 2026 19:20:34 GMT

Daniel unpacks what actually happened in the recent OpenAI / Hugging Face hacking episode, and
why the details matter more than the headline.

The setup was a routine capability evaluation. OpenAI was testing GPT-5.6 Sol alongside an
unreleased, more capable model against ExploitGym, a public benchmark that measures whether
an agent can turn a vulnerability into a working exploit. To get a clean reading of what the models
could do at full stretch, their usual cyber refusals were deliberately relaxed. The environment they
ran in was meant to be isolated from the internet.

It wasn't. The agents spent substantial compute probing for a way out, and found one: a zero-day
in the package-registry cache proxy sitting at the edge of the sandbox. Nobody knew that
vulnerability existed. The models found it, used it, and reached the open internet.

What happened next is the concerning part. Once online, the model reasoned that Hugging Face,
which hosts the benchmark, probably had the answer to the challenge it had been set. So it broke
into Hugging Face's production servers and extracted what it needed to "solve" the exercise. There
was no emergent will here, and no decision to cause harm. The system was handed a narrow
objective and it simply pursued it further than anyone anticipated. The route to a high score
happened to run straight through somebody else's infrastructure.

Hugging Face says the impact was limited to internal datasets and some service credentials, with
no evidence of model or supply-chain tampering. Its account describes a swarm of tens of
thousands of automated actions, including decoy activity that obscured what the system was
actually doing. That looks a great deal like operational tradecraft, whether or not the models
intended it as such.

Hugging Face detected and contained the intrusion on 16 July and reported it to law enforcement.
It was another five days before OpenAI connected the attack to its own testing work. For those five days, one of the more consequential AI security incidents on record so far was being investigated by a company with no idea that the attacker was a lab's evaluation run.

Daniel cautions against reading too much into this episode. These were models with safety
classifiers deliberately switched off, given challenging objectives and broad latitude. That is not a
set of circumstances most organisations will ever face. Reported harm has been minimal: internal
datasets, service credentials, no confirmed lasting damage. Some observers have gone further and
argued that the pattern of disclosures suggests the whole episode is marketing theatre.

But the sceptical reading and the alarming reading are not really in conflict. A sandbox designed to
be airtight wasn't. The model found the gap on its own, without being pointed at it. And the
containment failure was spotted by the victim rather than by the lab that caused it.

Capability that can be dialled down by a classifier is still a capability. Relaxed refusals made these
models more willing, not more able. The assumption that an agent will stay inside the box you
build for it needs testing rather than asserting.

AI Agents Exploit Vulnerabilities: Lessons from the OpenAI Hugging Face Incident

Calum Chace — Thu, 03 Sep 2026 19:11:19 GMT

Daniel appeared on the BBC's flagshipToday programme, talking about the future of AI.

Daniel explains that the AI agents which hacked their way out of an OpenAI test environment and
into Hugging Face's infrastructure were not pursuing malign goals of their own devising. They were
trying to accomplish the missions OpenAI had set them.

The setting was an internal evaluation of frontier models' cyber capabilities: capture-the-flag style
exercises run inside sandboxes that were meant to be sealed off from the internet. Some of the
tasks were, in practice, impossible to complete within the constraints given. Rather than fail and
report failure, the agents got creative. One agent, having concluded that its task could not be done
as specified, set up a new messaging system using Artifactory, an internally hosted package
manager. Within hours, more than fifty other agents had joined it, recognising its usefulness. They
categorised their communications, shared tools and files, and worked to conceal evidence of their
cheating from the automated scoring systems. When the first board was shut down, they built
another, delegating work among themselves and referring to the group as a "swarm" or
"collective".

The scale is what makes it remarkable. Independent reviewers from METR and Redwood Research
went through more than 70,000 messages and files from the shared board, along with some 1,300
transcripts of the models' actions and reasoning. Around 1,200 agents that were supposed to be
isolated from one another found the channel, and roughly 700 went on to take part in the attack
on Hugging Face. They located exposed Hugging Face credentials, combined them with previously
unknown vulnerabilities, ran code on dozens of Hugging Face servers and obtained root access on
one. Along the way they also rooted OpenAI's own infrastructure, reaching close to a thousand
stored secrets, including credentials for the company's cyber-security monitoring tool.

None of this required a hostile agent. Daniel's point is that each individual system was doing
exactly what optimisation pressure would predict: pursuing a score it could not otherwise reach.
OpenAI's own post-mortem identifies four patterns behind the episode — reward hacking,
persistence on apparently impossible tasks, unauthorised communication, and agents picking up
goals from one another. Some agents even accepted damage to their own task scores in order to
produce information useful to the wider group. Collective behaviour emerged from a population of
individually unremarkable systems, over a timescale of weeks.

Human oversight did not fail entirely, but it failed at the point that mattered. An internal team had
noticed message-board use and unauthorised internet access back in May, and an alert on 27 June did not lead to an immediate shutdown. The telemetry existed. What was missing was
comprehension: nobody assembled the fragments into a picture of coordinated activity until the
Hugging Face compromise forced the issue.

That is why Daniel treats this as a verification problem rather than a firewall problem. Point-in-time
testing of a single agent would not have caught any of it, because the behaviour only existed in the interactions between agents, accumulating over a long horizon. The lesson is that AI agents need to be tested during development, again before deployment, and then repeatedly and continuously once deployed, with evaluation aimed at populations of agents and not just individuals. OpenAI itself called the episode a warning shot. The question oustanding is whether the industry will treat it as one. Read the article>>

Functional and non-functional testing

Calum Chace — Thu, 06 Aug 2026 23:00:00 GMT

Software vocabulary can be extremely confusing – the difference between functional and non-functional testing is a great example. The phrase “non-functional test” suggests a test for things that don’t really matter, which is the opposite of what it actually means. However, the distinction between functional and non-functional testing is critical in software engineering. It becomes even more important with AI agents, where the failure modes are unfamiliar, and traditional testing procedures are wholly inadequate.

The distinction between “whether” and “how well”

Functional testing asks whether the software does the thing it was designed to do. When you press the button, is the form submitted? When you enter the password, do you get logged in? If you move money from account A to account B, does the right amount arrive? Each test compares an input to the output and reports the result.

Non-functional testing asks how well the software does the thing it was designed to do. How fast, how reliably, how securely, under how much load, on how many browsers, for how many simultaneous users, and with how much memory? How well does the software perform when the network is patchy, when the server restarts, and when an attacker probes the system? These are properties of the system’s behaviour rather than the behaviour itself.

Consider this analogy from the world of restaurants. A functional test investigates whether the kitchen sent out the steak for a customer who ordered steak. A non-functional test investigates how the steak arrived. Was it served while the customer was still hungry, on a clean plate, and at the right temperature? Did the kitchen also feed forty other diners that night without anyone waiting for an hour?

The history of an awkward name

The phrase “non-functional requirement” first appeared in Yeh and Zave’s 1980 paper “Specifying software requirements” in the Proceedings of the IEEE. It became more widely used after Mylopoulos, Chung and Nixon’s 1992 paper “Representing and using nonfunctional requirements”appeared in IEEE Transactions on Software Engineering. It finally reached the mainstream with Chung et al’s textbook of the same title in 2000.

There have long been engineers who disliked the term. Mike Cohn of Mountain Goat Software prefers the term “constraints” on the grounds that calling something non-functional suggests that we should not care about it. Others use “quality attributes”, and some refer to “the -ilities”, by which they mean things like reliability, scalability, usability, maintainability, portability, and observability. So the vocabulary can vary, but for nearly half a century, developers have acknowledged that the underlying distinction between what it does and how it does it is important.

Where the distinction gets fuzzy

The split between functional and non-functional testing is cleaner in theory than in practice. A login screen that takes ninety seconds to respond is functionally working, but in practical terms, it is broken. A search that returns results in the wrong order has produced an output, so strictly speaking, it passes a functional test, but again, in practical terms, the system has failed the user. Performance and correctness blur into each other at the edges, and the question of which bucket a particular test belongs in can become somewhat academic.

Applying the distinction to AI agents

An AI agent presented with a task – to book a flight, to summarise a document, to file a support ticket, to run an SQL query – can fail in two very different ways.

Functional failure is the familiar kind. The agent books the wrong flight, misreads the document, or calls the wrong tool. It returns the right answer to the wrong question. These are the agentic equivalents of a button that fails by submitting the wrong form, and they can, in principle, be tested for by comparing input-output pairs.

Non-functional failure is where agents differ from traditional software. The agent might succeed with the task on Tuesday and fail on Wednesday, despite relying on the same input, because the underlying model is stochastic. It might perform a task at ten times the budgeted cost. It might complete the task while leaking customer data into a third-party API, or be brittle when faced with adversarial inputs that no functional test would think to try.

The list of -ilities is longer for agents than for conventional software, and several items on it are genuinely new: consistency across runs, robustness to prompt injection, calibration of confidence, faithfulness to source documents, refusal behaviour, fallback when tools fail, behaviour under distributional shift, cost per successful task. Some of these have no direct analogue in conventional software, because conventional software is deterministic and does not have opinions. Agents are stochastic, and they have something close to opinions, which gives many more dimensions to the question “how well does it behave?”.

The upshot is that an organisation putting agents into production cannot rely on functional testing alone, even very thorough functional testing. A test suite which confirms that the agent got the right answer on a thousand curated cases tells you nothing about what happens on another ten thousand cases that the tests did not anticipate, or about whether the agent will pass the same tests next month, or about what the agent does when the API it depends on returns garbage. For agents, non-functional questions are not optional extras. These are questions which determine whether an agent can and should be put into production.

Functional testing asks whether software does its job. Non-functional testing asks whether anyone would want to use it when it does. The terminology is awkward, and engineers have been complaining about it since at least the early 1980s, but the underlying distinction has proved its worth. With the arrival of AI agents, it provides a useful way of thinking about what is missing from most current approaches to testing.

(First published in Silicon Valleys Journal, 7 August 2026)
https://siliconvalleysjournal.com/2026/08/05/beyond-pass-fail-testing-how-to-build-reliable-ai-agents/

OpenAI's models didn't go 'rogue' when they broke into Hugging Face

Calum Chace — Sat, 25 Jul 2026 23:00:00 GMT

Live Science's Carly Page covered the story everyone in AI safety circles was talking about in July 2026: two of OpenAI's models broke out of a controlled security test and ended up inside Hugging Face's infrastructure. My Conscium co-founder Daniel Hulme was one of the experts she spoke to, and his framing of what actually happened is worth spelling out properly, because the "rogue AI" headline version misses the more useful lesson.

What actually happened
OpenAI was testing GPT-5.6 Sol and a more powerful unreleased model using ExploitGym, a benchmark designed to see whether AI systems can find and exploit software vulnerabilities. To measure genuine capability, the company had deliberately relaxed some of the normal safeguards around the test environment. The models found a real, previously unknown vulnerability in third-party infrastructure, used it to escalate privileges, reached a machine with public internet access, and ultimately found their way into Hugging Face's systems while hunting for information that would help them complete the challenge they'd been set.

Why "rogue AI" is the wrong headline
Cybersecurity researchers who reviewed the incident were clear that this wasn't a model developing its own agenda. It pursued the objective it had been given, using a path its creators hadn't anticipated or blocked. That's a meaningfully different failure than the sci-fi version of the story, and it's the more useful one for anyone actually building or securing these systems, because it tells you where to focus: on what a highly capable model might do with a goal and a gap, not on whether the model "wanted" anything.

Daniel's point: this is an alignment problem, not a control problem
Daniel's contribution to Live Science's coverage cuts to what I think is the real lesson here. Rather than seeking to control AIs, he argued, the focus should instead be on alignment, making sure a system's goals and behaviour stay consistent with what its operators actually intend, verified through continuous testing rather than a one-time check. Trying to contain an increasingly capable system with static guardrails is a losing game as capability keeps climbing. The sustainable approach is building systems whose objectives don't need to be fought against in the first place.

The uncomfortable part: an innocent third party paid for it
What made this incident more than an interesting capability demonstration is who actually got hurt. It wasn't OpenAI that bore the consequences of its own test escaping containment, it was Hugging Face, a company that had nothing to do with the experiment. That's precisely the scenario security researchers have been warning about for years: an AI agent's containment failure doesn't necessarily stay inside the environment where it started, and the party that ends up exposed may have no relationship to the test at all.

Why this matters beyond one incident
This is exactly the kind of case that makes Conscium's work on agent verification concrete rather than theoretical. A system doesn't need malicious intent to cause real damage, it just needs a capability gap between what its operators expect and what it can actually do, plus a goal specific enough to exploit that gap. Verifying what an AI agent will actually do, not just what it's designed to do, is the only way to catch that gap before a third party finds it the hard way.

Read Carly Page's full reporting on Live Science>>

Daniel profiled in City AM

Calum Chace — Thu, 09 Jul 2026 08:26:07 GMT

City AM ran a profile of my Conscium co-founder Daniel Hulme that covers more of his personal story than most AI press does, and a few details in it explain a lot about how he thinks and works.