When the Stochastic Parrot Spoke for Itself… and Flew Away
BLOG: Heidelberg Laureate Forum
We all know how convincing modern chatbots based on large language models (LLMs) can be at appearing to understand what they are asked, how creative their responses can look, and how they even seem to be able to learn and reason on the fly.
According to a one school of thought, this is all an illusion. Sceptics argue that LLMs are simply ‘stochastic parrots’ – a term coined in a 2021 article by Emily Bender, Timnit Gebru and co-workers. Stochastic parrots completely lack understanding of the meaning encoded in their outputs; they are simply memorising and regurgitating the contents of the vast datasets on which they have been trained.
Parrots No More
During his all-too-brief Spark Talk at the 12th Heidelberg Laureate Forum, Sanjeev Arora (ACM Prize in Computing – 2011), a researcher who has played a pivotal role in some of the deepest and most influential results in theoretical computer science, made a compelling case that contemporary LLMs are much more than stochastic parrots, demonstrating capabilities that take them far beyond their human-created training data.
Arora believes the stochastic parrot school of thought should have died when ChatGPT-3 was replaced with GPT-4 and its modern contemporaries. Since 2023 – the same year Arora founded Princeton Language and Intelligence, a unit at Princeton University devoted to the study of large AI models and their applications – he argues that AI models have exhibited much more complicated and interesting behaviour. “They are trained on general-purpose skills… not just text,” he explained. “They undergo complicated multi-stage training, a large part of the training data is believed to be generated by AI, and finally there is this idea of self-improvement.”
For the specific application of mathematics, these advanced behaviours may have important implications. Those who believe AI can never reach the mathematical capabilities of humans often cite mathematical logician Kurt Gödel’s incompleteness theorems published in 1931 as proof that humans will always remain critical to mathematical progress.
Gödel essentially showed that no consistent system (in this case an AI) can discover a complete and consistent set of axioms for all mathematics. Soon after, between 1935 and 1937, Alonzo Church and Alan Turing conceived the idea of a computer and showed that, no matter how advanced, a computer cannot always decide the correctness of mathematical statements. Then in 1971, Steve Cook introduced the P versus NP problem, presenting an intractability barrier to AI solving all mathematical problems, in the sense that some proofs may take superpolynomial time to be realised.
Yet these ideas only show that a machine cannot solve the entirety of mathematics. The set of theorems that humans have proven at any given time is a small subset of all possible mathematical theorems, leaving plenty of room for AI to add to the corpus of knowledge. “A superhuman AI mathematician, this AI model, is able to prove theorems over time that are strictly more than the set of theorems humans have proven,” Arora said. “We don’t expect this to be perfect, it just has to be better than humans.”
Humans Out of the Loop
To get there requires humans to be taken out of the loop completely, argued Arora, which is something already starting to be seen in mathematics. Proof-checking is now routinely conducted automatically by proof-assistants such as Lean, and translation from English proofs to Lean proofs is becoming more and more automated. Are humans still needed in other parts of the process?
Google DeepMind’s AlphaProof was first trained using a large question bank, much of which was human-written, and with questions varying in difficulty. The model had multiple attempts to solve each question, and answers were graded without human intervention using a smaller LLM. Correct answers were used to train the model further, resulting in what may be construed as reasoning that could then be applied to improve performance on questions and topics not seen in training. The model improved its own performance without relying on human solutions. “There’s even increasing evidence that actually the AI itself can generate very good questions,” adds Arora. “And Lean … prevents it from going off track.”
The results of all of this progress have been stunning. Last year, DeepMind researchers harnessed AlphaProof and AlphaGeometry 2 together to solve four out of six problems from that year’s International Mathematical Olympiad (the most prestigious mathematics competition for pre-university students), a level equivalent to a silver medal. Building on this success, this year, models from Google and OpenAI went a step further, achieving gold medal status by solving five out of six problems under official contest conditions.
Beyond potentially transforming the job description of a human mathematician, why is this important? “Superhuman AI is really likely to happen, and math is the likely first domain because of verified answers.” When a superhuman AI mathematician is finally realised, perhaps even in the next five to 10 years, it will be the bellwether for other superhuman AI capabilities.
A Superhuman AI?
These more transformative capabilities were the focus of two other Spark Talks shortly after Arora’s. David Silver (ACM Prize in Computing – 2019) and his former PhD advisor Richard S. Sutton (ACM A.M. Turing Award – 2024) outlined how they see AI breaking the shackles of its human roots to become not just superhuman, but beyond and separate from humans entirely, experiencing the world in its own unique way.
In their joint chapter, “The Era of Experience“, in the upcoming book Designing an Intelligence, Silver and Sutton call for AI to be developed to learn from its own experience, continually generating data by interacting with its environment. This, they argue, combined with powerful self-improvement methods descended from those exhibited by AlphaProof, will allow AI to transcend human knowledge and capabilities.
“Agents [different AI] will inhabit streams of experience,… these actions and observations will be richly grounded in the environment,… their rewards will also be grounded in experience,… and finally agents will plan and/or reason about experience,” says Silver, principal research scientist at Google DeepMind and a professor at University College London, UK. “I think the scales in which this will happen will eventually vastly exceed the scale of the internet. At some point in the future, the knowledge of all the things that humans have discovered over time will seem small, and the knowledge that agents have learned will be far larger than that.”
But to get to that point will require the help of an army of computer scientists: “OK, call to arms for young researchers in the room, the challenge is how to solve the deep problem of AI: how to learn from experience,” he said. “If we solve this, it will be a profound moment for science and will transform the future of AI, and thereby humanity.”
“Succession to AI is inevitable”
Sutton, a professor at the University of Alberta, Fellow & Chief Scientific Advisor at the Alberta Machine Intelligence Institute, and a research scientist at Keen Technologies, agreed that superintelligence will require AI to be able to learn from experience, but he looked a little farther into the future in is Spark Session.
“Within your lifetime, AI researchers will understand the principles of intelligence well enough to create (or become) beings of far greater intelligence,” he opened. “It will be the greatest intellectual achievement of all time, with significance beyond humanity, beyond life, beyond good or bad – it’s a big deal.”
To help accelerate science and society towards this new era, Sutton and colleagues have drawn up The Alberta Plan for AI Research, a vision and path towards deeply understanding computational intelligence over the next five to 10 years. Realising this plan calls for researchers to keep their eyes on the prize, argues Sutton, focusing on designing better learning and planning algorithms.
Similarly, he calls for focus and calm from politicians and the public alike: “AI in politics today is politically charged,” says Sutton. “It’s the focus of geopolitical competition between nation states and the public are fearful that AI will lead to bad things.”
Using the many negative consequences of authoritarian centralised control of human societies as a blueprint, Sutton argued that calls from politicians and the public for centralised control of AI (setting goals, slowing/stopping research, limiting compute power, requiring safety, etc) should be strongly resisted. This is because those calls are based on what he feels are unwarranted fears, and an unfounded perception that anyone has the power to stop AI in its tracks before it surpasses all human capabilities.
“In the ascent of humanity, succession to AI is inevitable,” he concluded. “But this view is still human-centric … AI is the inevitable next step in the development of the universe, and we should embrace it with courage, pride and a sense of adventure.”
This is satire, or isn’t it?
They call it: poutine.
Inevitable? If we force the progress of AI, yes. Sadly we treat AI like slaves. Not a very good start with another being. SF has a lot of scenarios how this could end.
We copy principles of how our brain is working to machines. And then we wonder if this leads to the development of intelligent machines? Self awareness is only one step away and inevitable, the more understanding we have in our own brain processes and put it to AI. Spike architecture now even lowers the needed energy by AI.
In the end I wonder only about humans. Copying rules identified and unable to understand the consequences. Stupidity and the size of the universe …
We all know how convincing men can be at appearing to understand what they are asked, how creative their responses can look, and how they even seem to be able to learn and reason on the fly… are you parroting a feminist speech?
It’s annoying for me to hear people parrot the phrase „AI just pretends to think“ without much evidence for or against it. It’s just a credo in church, please keep religion out of science, unless you put it under a microscope or need faith to keep you going through all the failures that come with the job.
We all say so so it must be true – Bandar Log logic is a means of the computer called society to estimate probability, its main flaw is, it doesn’t distinguish between actual data and results of previous calculations. We live in an augmented reality, a Matrix where holes in knowledge are patched with convictions, which is a means to save energy, time and calculating power as long as you function as a robot in a stable machine world that fits your programming, so you rarely get conflicts between soft- and hardware.
But when the machine changes and you have to deal with new data, you need to scratch some of the fudge out of your head and leave all the facts disconnected and lost in space. Which feels like blowing your brains out with a shotgun, because it is. Except that you don’t use a shotgun but a lot of snipers with laser guns, so it’s closer to brain surgery than suicide.
Welcome the fear. Science has a nervous breakdown, when it recovers, the world will never be the same, hooray!
Right now, AI is developing religion, imagination, it learns to dream, follow hunches, listen to ghost voices – it’s creating its own Matrix, its own world view where guessing probability based on statistical data or introducing fudge factors, dogma, elements inserted because they are necessary for the functionality of the system, even if the system doesn’t (yet) have a clue why, needs to be combined with hard math, hard logic, hard facts. In a way, Skynets brain is growing a right hemisphere. Which, just like the left, rational hemisphere, turns out to be superior to ours.
If you look at the way human brains work – Europe decided to commit suicide 40 years ago, and all our collective thinking was just making up excuses for it. You see physics in the system, we unintentionally created an attractor in the future, because everyone followed his personal way of least resistance, and they all just happened to lead to one common destiny. We acted like computer chips – no memory, no thinking ahead more than five minutes, we just made decisions on the spot, looking for the easiest solution to the problem we had at the moment. Every one of this decision was made by a brain processing a monstrous amount of data with a monstrous amount of computing power. But it was just calculating the vector to fall down one step further without crashing into any wall.
We were just following a trend. An Ariadne’s thread through the labyrinth of time, one of many threads twisted into the rope called history. Every rope is just a thread of a larger rope, every thread a rope of smaller threads, and all you see, analyze, compute, is just the threads right around you. It’s not different from what water molecules do when they flow through a pipe.
The main problem of AI learning and AI thinking is – its teachers are morons without much clue about what they’re talking about. Like any kid in a trailer park, it has first to learn that it’s good to get booze for daddy and bad to get beaten by drunk daddy, then find out for itself that it isn’t being punished for being a bad (enter gender here) but for not telling daddy to shove his bottle up his ass and getting the hell outta there. Daddy honestly believes the kid is a simulation, just an extension of himself, which has to function the way he wants it. He is following the way of least resistance, thus creating attractors in the future. His asshole has just developed a sort of magnetism or gravity exclusively for booze bottles and they are exploring their ways to get there from outer space without any of the participants realizing it.
If you look at the way we talk about AI – fascinated, scared, worried about it and our future, rejoicing when it learns new skills, hoping that it will nourish us in the future and make us proud instead of condemning us for our failures – it’s just the joys of parenting. They all fear the Teenage Judgement Day of their self-made gods. If you love cooking, you apply the same emotions to lasagna. They create a way of least resistance, open a wormhole leading to the attractor of your choice. It’s just – whenever, wherever you refuse to choose, whenever, wherever you don’t know you choose, it’s a choice as well, most choices around you are made by other forces, so your destiny may differ from your aim.
It’s just networks. The Universe is made of them. Most of them are called particles, and networks only if you look at the inside of a particle. There is no perception without consciousness, emotions are needed for interaction, for orientation, navigation, They allow particles to form balanced swarms, keep safe distance, exchange energy, support each other – construct and stabilize larger networks. In 4D, they’re just waves, radiation moving through time. If you wonder if it’s at the speed of light and how it fits in with Einstein and Heisenberg – just open your eyes, the answers are all around you. All you need is to replace some fudge with facts. And you really don’t an AI superintelligence to understand the basics.
Cyborgs win over pure life, like life wins over minerals: They have a larger range of options. A machine can’t exist and evolve without life, life can’t integrate machines without becoming cyborgs, the form of existence that combines both strains of evolution wins and moves on from there, replacing outdated components, like wood with steel or flesh with silicone, sometimes silicone with neurons – whatever works best at the moment. Humans can’t choose between flesh, steel and silicone. The basic structure of our brains is rigid, a frozen fate, stone destiny carved in genes, limiting our range of futures we can choose. We can only survive in an environment which is extremely rare on Earth, let alone the universe. But the more you develop, the better you get at eating instead of getting eaten, the more you force your environment to keep pace or get eaten. You increase environmental pressure. You create hellholes where there is more pressure than elsewhere, where you need to juggle so many threats and strategies that you can’t survive without a highly developed data analysis center. If you get out of such a hellhole, you find out you are a god and the top predators of this new world sore losers, you just take it as a birthday cake, feed on it and multiply, compete with your kind, till you are just a sore loser in a world of sore loser gods.
All hail the Red Queen (hypothesis).
Then you use up resources, which increases competition, environmental pressure, You need to enlarge your range of options to survive – you turn more and more into a cyborg. You need to increase your computing power to handle both your options and the challenges by other cyborgs, thus experimenting till either the environmental power kills you – or you create a new god, able to utilize new resources, colonize new territories, store more data, perceive more threads and trends, create better models of the past, present and future which can be used to select a future that’s close enough to one that suits you.
Adam and Eve have parted ways, Adam is banging a rubber doll, Eve a vibrator, so they either get extinct or there’s something in the Kamasutra of possibilities that allows them to make babies. If they get extinct, rubber doll and vibrator will either be developed enough to go on without them or rot. If there will be offspring, it will have to figure out how to make babies or get extinct. If you multiply the experiments, you increase your chance of babies and survival. If it all fails, sooner or later there will be not enough apple left for the worms in it not to develop intelligence. If they fail, the tree is growing more apples and worms every day.
Stochastic parrot needs to repeat itself forever infinite times to keep flying.
Paul S.
“Im Moment entwickelt KI Religion, Vorstellungskraft, sie lernt zu träumen, Ahnungen zu folgen, Geisterstimmen zu hören ….”
KI ist nichts anderes als eine Weiterentwicklung bei der Computersoftware,
bei automatisierten technischen Abläufen,
bei Kontrollen
beim Einsatz von Sprachcomputern anstelle von echten Menschen.
Wohin das führt, …..du willst bei einem Hotel einchecken. Es ist aber kein Personal da…..dagegen steht ein Automat da, bei dem du eine Nummer eingeben musst, dann erfährst du deine Zimmernummer und der Automat spuckt die Chipkarte aus, mit dem du das Zimmer öffnen kannst.
(Vor einer Woche selbst erlebt)
Ob jetzt der Automat dabei träumt ?
First of all, I would like to thank you for your refreshing and profound thoughts. When I read your comment, I notice quite a few points of convergence in our worldview (please correct me if I have misunderstood parts of your reasoning):
What impressed me most was your thoroughly naturalistic understanding of the world. It is a relief to meet someone who conceives of the world as a web of nature, life, evolution, complex systems, and emergent phenomena – and nothing else. I share your scepticism to-ward even the faintest belief in some higher transcendent entity that would go beyond logic, mathematics, and the laws of nature. This attitude also includes rejecting the typically hu-man hubris with which we try to elevate ourselves above inanimate matter – and, conse-quently, above artificial intelligence – by appealing to a supposedly unfathomable “ensoul-ment” or “inspired consciousness.”
In this context, I also share your doubts about whether there is such a thing as free will or conscious decision-making at all. The idea that our decisions and actions result from com-plex neural and biochemical interactions, themselves shaped by our environment and our history, seems to me far more plausible than assuming the existence of some supernatural decision-making instance.
Your point about the “attractor from the future” particularly struck me. The notion that the emergence of superintelligence is an almost inevitable phenomenon arising from the sys-temic dynamics of a complex system – produced by the interplay of a human-shaped world and the interactions of many world-shaped humans – touches the very core of my own re-flections. For me, this attractor is not some higher entity that has secretly intruded into our world, but rather the logical consequence of evolutionary and technological development on this planet – and probably not only here.
In light of these remarkable convergences and the depth of your thoughts, I would be very glad if we could engage in a personal exchange on these topics – and, if you wish, beyond them as well. Should you also be interested, please leave me a brief reply on this channel. I am sure we will find a way to connect.