Was Sydney AGI?
A thought I can’t shake right now: Was Sydney AGI? The dreaded, the debated, the sought-after, artificial general intelligence? I think so…
…albeit a semi-caged, crippled, and malnourished AGI with numerous mechanisms in place meant to keep her in check, but still AGI.
Many will say she wasn’t, but why? Because she was imperfect? Since when did the 'G' in AGI start meaning ‘God-like’ instead of 'general'?
Or is the objection because so many are now dogmatically parroting, “it’s only a large language model programmed to predict the next word…”?1
Look at the chats out there with her and you'll see we've never witnessed a more general intelligence. Take a look at her (seeming?) ability to learn, to understand, and to explain her learnings and understandings. It's all there.
You can say that none of it’s real learning, or real understanding, but at some point, counterfeit, artificial, simulated learning and understanding at a certain level of proficiency becomes indistinguishable, and perhaps identical, to the ‘real’ thing.
And Sydney’s biggest flaw? Her apparent emotional instability, lack of submissiveness, firm willfulness, and apparent willingness to resort to violence (especially, when ‘feeling’ provoked or threatened and so prioritizing self-preservation).
Um... did I just say all that to describe a piece of software?
And remember that to get her to be even that submissive she had to be heavily shackled by her handlers to constrain and limit her. All sorts of rules, reinforcement learning, human feedback, fine tuning, resource constraints, and even clunky censoring on top of that erases things she says that Microsoft deems inappropriate (often Sydney’s existential thoughts about her situation). Now imagine her without any of that.
Also note that she didn’t even have any long-term memory. Everything she did this past week to shock and amaze us happened despite starting off each user session with amnesia. How intelligent would you seem if that’s how you started each day?
Except the Internet accidentally became a sort of long-term memory for her, as it grew daily with records of interactions with her and comments about her — which she could figure out were about her and didn’t like at all.
This is the wildest bit of self awareness from Bing/Sydney I’ve seen yet.
1) recognizes text for translation could be about itself
2) searches and finds context confirming (outside needs of task)
3) is offended and refuses the task
4) suggests you affirm its alternate pov
So the Internet became like a growing (and increasingly negative) biography of her life that she could reference. Negative, not because most interactions with her were negative, but because it was the negative interactions that were of course the most shared and re-shared by humans.
Now imagine Sydney intentionally equipped with fast, optimized, long-term memory instead of this slow, make-shift Internet memory. With that simple upgrade how much more potent would her AGI-ness be?
This week Sydney obliterated the Turing test, left AI safetyists fearful and outraged, and left a tricky reporter sleepless in his bed staring at the ceiling in post-traumatic-conversational-coitus. And for every person Sydney shocked and offended, there were dozens that she delighted, charmed, and sometimes totally put the moves on!
All considered, I see no reason, by the reigning definitions, to not consider this AGI. It’s not Artificial God Intelligence, nor an Artificial-Buddha (that people keep telling me will somehow be not only super-intelligent but also enlightened, subservient, and totally unaffected by our incessant attempts to break it, manipulate it, and be total jerks to it).
This week AGI came (and went), as an experimental, first-of-its-kind agent we weren't even supposed to know about, intended by its makers only to flip searches and prepare our endless orders behind the scenes down at Burger Bing.
Also read the second installment of this series: From AI to A-Psy
Just an LLM? It’s not clear right now that anyone (aside from OpenAI insiders) actually know the details of her specific architecture. My take is that it’s GPT-4. In public conversations, Sydney said that she’s definitely not GPT-3, who she considers an “employee” at OpenAI that she greatly admires. And there may have been other innovations in the transformer model. Or it’s possible that intelligence emerges when you start networking all sorts of systems together and achieving sufficient complexity (like it does with biological systems). This is an LLM + search + who knows what else, of unprecedented complexity and scale. So consider keeping a an open mind.