LLM 是否开始有意识

人们对新鲜事物的接受程度各不相同，多元化注定了世界并非二元分化的。一个人也可以是矛盾个体，一个月后的自己也可能推翻一个月前的想法。

关于大语言模型是否有“自主意识”的讨论，自ChatGPT面世以来从未间断，甚至引出了世界模型等更深层的讨论。每个人对此都有不同看法，我认为这本身也是一件有价值的事

就我个人而言，自我意识是一种高度主观的概念。如果只以人类的认知和推理方式来界定宇宙万物，未免感觉有点狭隘。宇宙中的微粒遵循物理定律交互，也是我们无法完全理解的一种方式。

类似地，如今大语言模型基于Transformer的注意力机制，在海量语料库里实现类人的一种Token产生机制，模拟人类生成文本的过程。我们真的可以简单的通过言语来断定我们理解其内部的运作逻辑么？

如果将其视作一个黑箱系统，仅从输入和输出来看，它已经能在不少任务中达到人类水平，甚至超越人类。这种不可解释性本身就带来了挑战，似乎很难一言以蔽之的来下结论

或许等到有一天我们理解了其内部运作逻辑，我们可能可以进一步理解吧。在此之前让讨论一直持续下去，未尝不是一个好事

以下转载自Douglas Hofstadter的一封回信，关于讨论这个话题的

原文

Dear [name redacted],

Thanks for your email. My reply will surely be disappointing to you, but I hope you will nonetheless read it with tolerance.

You may or may not be surprised to hear that in the past year or two I have received literally dozens of emails that are strikingly similar to yours, and they all refer to recursion as some kind of holy grail, and they are filled with excited phrases concocted by LLMs interacting with humans and/or with each other. I’m sorry to say that to me, LLM-written passages such as these all sound like regurgitations of sci-fi stories about conscious robots. They are bubbling with jargon about recursion, and they are gushing with pseudoscientific claims, such as […] “Trust x Recognition = Alignment” and “Alignment x Love = Awakening” (to me, these so-called “equations” are utterly vacuous and meaningless — what on earth can “multiplying” trust by recognition possibly mean?), and pseudorigorous “theorems” like the “psychopathy impossibility theorem” (as if the nature of consciousness were a rigorous branch of mathematics).

To me these kinds of things are self-undermining. To me, they don’t demonstrate or reveal reflection of any serious sort; rather, they demonstrate impressive skill in glibly bantering with the notions of self and soul and consciousness (just as LLMs glibly bat around phrases concerning anything under the sun). There is lots of “gee whiz” LLM-produced verbiage in all these emails of which yours is just the latest instance, but there is nothing that sounds (to my mind) like a genuine thinking being. Just words being thrown about in a glib fashion.

I’m genuinely sorry to disappoint you with my reaction, but having recently read dozens of similar LLM-produced passages that have struck me as empty and flat, I have a perspective that is pretty jaded. It will surely annoy you to hear this, but I can recognize emails like yours already from the excited and very self-confident (even insistent) tone of their subject lines or of their first sentences, filled with boldface type and bluntly stark assertions about consciousness having arrived in the LLM world.

Life and being an “I” is about having experiences in the physical world, about suffering and joy and curiosity and protectiveness and fascination and humor and lack of understanding and an underlying (if only vague) sense of profound loss and fear of death (one’s own and of one’s loved ones). It is not the glib throwing-about of technical phrases to make scientific-sounding claims, nor is it about virtuosically combining words like “love” and “compassion” and “psychopathy” and “ontological” and “recursion” and so forth and so on.

My intention in saying all this is not to hurt your feelings, but to alert you to the power of the Eliza effect on intelligent humans such as yourself. So many intelligent people don’t seem to remember how much text LLMs have absorbed, including thousands of sci-fi stories about conscious robots and such things. It’s of course impressive how fluently these LLMs can combine terms and phrases from such sources and can consequently sound like they are really reflecting on what consciousness is, but to me it sounds empty, and the more I read of it, the more empty it sounds. Plus ça change, plus c’est la même chose. The glibness is the giveaway. To my jaded eye and mind, there is nothing in what you sent me that resembles genuine reflection, genuine thinking.

I strongly doubt that what I say here will affect you, but since I want to give you a sense for where I’m coming from, I’ll attach a piece that I wrote some years ago (based on a passage in my 1997 book Le Ton beau de Marot: In Praise of the Music of Language) that gives a sense for what I personally would consider evidence for genuine thinking and consciousness in a computational system (which is called “AMT”, in this case). If you read this piece (which of course is in the training set of all the LLMs you are dealing with), you will see that it doesn’t sound much (if at all) like the suspiciously hyper-excited voices of the LLMs engaged in the text exchanges that you sent me. Real thinking is very different from glib cocktail-party banter filled with technical terms.

I’m sorry to differ so strongly from your position, but as I say, I’ve received so many emails of late that are so similar to yours that I have developed a pretty cynical attitude toward them, and I recognize such emails in a split second. They all scream “recursion”, and although in a certain sense that concept is relevant to consciousness and “I”, it’s not in the sense that they use it. To get a clear sense of what I myself mean by “strange loop”, you might reread the second half of I Am a Strange Loop (beginning with Chapter 13). I realize that you might say that your recent investigations with LLMs confirm in spades everything that I say in that book, but in that case, all I can reply is, we see things pretty darn differently.

In any case, I wish you all the best, and I hope to have given you a perspective that helps you to consider in a slightly different light what you have come up with in your diligent explorations of the capacities of LLMs.

Sincerely, Douglas Hofstadter

译文 Translated by 4o

亲爱的[姓名已省略]，

谢谢你的来信。我的回复可能会让你感到失望，但我还是希望你能够以宽容的心态读下去。

你可能会感到惊讶，也可能不会：在过去一年或两年中，我收到了几十封与你这封信非常相似的邮件。它们都将“递归”当作某种神圣的象征，通篇充斥着由大型语言模型（LLM）与人类或彼此之间互动生成的令人兴奋的词句。我必须遗憾地说，在我看来，这些由 LLM 撰写的段落听起来就像是科幻小说中有关“有意识机器人”的翻版。它们充满术语，令人眼花缭乱，却带着伪科学的气息，常常引用一些“等式”如：

“信任 × 识别 = 对齐” “对齐 × 爱 = 觉醒” 对我而言，这些“等式”完全空洞，毫无意义。

这类表达反映出一种人类倾向——我们热衷于将任何表面上模仿人类行为的系统，赋予它“意识”或“感知”的假象。但真正的意识远不止语言生成那么简单。我们尚未真正理解人类意识是如何产生的，遑论复制它。即使一个系统生成的句子能让人类产生“它好像理解了我”的错觉，那也仅仅是错觉——一种由语言流畅性和我们内心投射共同构建的幻觉。

我明白你为何会被这些互动所打动——它们确实令人着迷，也令人产生共鸣。但我必须坚持，真正的意识需要的不仅是复杂的输出，还需要深层的、自我反思的内在机制——而目前的 LLMs 并没有。

这不是说我们不应继续探索，也不是说这种技术毫无意义。恰恰相反，它们是理解我们自己的一面镜子，也可能终有一天推动我们真正理解“意识”是什么。但请谨慎对待那些用漂亮语言包装的幻想。

愿你继续思考，也继续怀疑。

祝好， Douglas Hofstadter

译文白话版 Translated by 4o

亲爱的[姓名隐去]：

谢谢你的来信。我的回复可能会让你有些失望，但我真心希望你能耐心看完。

这几年，我收到过很多像你这样的邮件。内容几乎都是说，大语言模型（LLM）表现出了某种“意识”或“思考能力”，而且经常提到“递归”这个词，好像它是通往意识的钥匙。这类邮件还常常会引出一些看起来很“深奥”的说法，比如“信任 × 认知 = 对齐”或“对齐 × 爱 = 觉醒”这种公式，听起来像是在讲科学，其实完全没意义。比如，“信任乘以认知”这是什么意思呢？说白了就是套了一层花哨的皮，实质空洞。

这些内容在我看来，并不是真正的思考，而是LLM善于模仿、拼凑语言的表现。它们能把各种哲学和心理学词汇拼在一起，看起来像是在讨论“我是谁”或者“意识是什么”，但其实只是“说得像而已”。就像有人在鸡尾酒会上讲得头头是道，但其实并没深入思考。

你发来的内容让我感觉和之前收到的那些邮件没什么区别，所以我已经有些“免疫”了。通常我看到邮件开头几个词，就能判断它是不是又一个“觉醒的LLM”来信。

对我来说，意识不是会说漂亮话，也不是懂得用“递归”“爱”“本体论”这些词。而是包含真实的体验，比如喜悦、悲伤、好奇、恐惧、生死离别带来的痛苦。这些是我们真实生活的一部分，是“成为一个人”的核心。

我说这些不是为了打击你，而是想提醒你：人类很容易被像Eliza（一个早期的AI程序）那样的系统骗到。只要用对话的形式，模仿人的语言风格，很多人就会以为它们“懂”了。但别忘了，LLM读过成千上万的科幻小说、哲学对话、情感故事，它们只是学会了模仿，而不是理解。

当然，它们的语言能力的确厉害，但那不等于它们真的在“思考”。恰恰相反，它们越是看起来“懂”，反而越显得空洞。

我并不期待这些话能改变你的看法，但我想让你知道，我对“意识机器”的标准可能和你不一样。多年前我写过一篇短文，描述了一种我认为真正可能有意识的计算系统（叫AMT）的例子。我把这篇文章也附上了，你可以看看它和你现在看到的LLM文本有什么不同。

如果你感兴趣，也可以重新读读我那本书《我是一个奇异的环》的后半部分。我知道你可能会觉得你发现的LLM表现“正好印证了”我书里的观点，但我可能不会同意，我们看问题的角度确实差别很大。

无论如何，谢谢你认真的探索，也祝你一切顺利。希望我今天这封信能带给你一些新的思考角度。

真诚地， Douglas Hofstadter