Superintelligence: science fiction or not?

Live Science's Edd Gent put together a piece asking the question I get asked at almost every dinner party: is artificial superintelligence a genuine near-term possibility, or is it science fiction dressed up as a forecast. My Conscium co-founder Daniel Hulme was one of the people he spoke to, and Daniel's answer sits closer to "genuine possibility" than most people are comfortable with.

Why experts can't agree on a timeline
The article lays out just how wide the disagreement is among researchers about when, or whether, superintelligence arrives. Estimates range from within this decade to never, and a survey of AI researchers Gent cites puts the midpoint at around a fifty percent chance by 2047. That spread tells you something on its own: this isn't a field with a settled consensus that outsiders are ignoring, it's a field where the genuine experts disagree by decades.

The upside Daniel points to: costs falling toward zero
Daniel's framing in the piece is one I share. If superintelligent systems can drive the cost of producing food, energy, and other essentials down toward zero, that's a realistic path to freeing huge numbers of people from work done purely to survive. That's the good version of the story, and it's worth taking seriously rather than dismissing as utopian.

The catch: speed, not destination
Where Daniel adds a note of caution that I think doesn't get enough attention is the pace of the transition rather than its endpoint. His concern is that if costs collapse very quickly, economies don't get the time they need to rebalance, and that mismatch between technological speed and institutional speed is where social unrest comes from. I'd put it this way: the destination might be fine, but getting there too fast could break things on the way.

On control: Daniel doesn't think a superintelligence would simply care about us
Gent's piece also gets into the control problem, the question of whether a sufficiently advanced AI would act in humanity's interests by default. Daniel is blunt about this: he doesn't see a reason to assume a superintelligent system would care what happens to us, absent a deliberate reason for it to. That's close to my own view, and it's exactly why Conscium's work on machine consciousness and alignment matters. Starting from the assumption that a system might not care is a much safer basis for building it than starting from the hope that it probably will.

Why Daniel thinks today's safety guardrails aren't the answer
The part of Daniel's argument I find most interesting is his critique of how AI safety is currently being done. He describes today's alignment techniques as too crude, bolted-on rules and filters that a sufficiently capable system could route around rather than genuinely internalise. His alternative is to train moral instincts into AI directly, using virtual environments that reward cooperation and altruism during training, so that good behaviour comes from something like character rather than a list of prohibited actions. Whether that approach scales is still an open question, but I think it's the right direction to be pushing in.

Check out the full piece on Live Science>>