(评论)
(comments)

原始链接: https://news.ycombinator.com/item?id=38281848

前面提到的一点引起了我的强烈共鸣:LLM架构的直接自学和间接自学之间的区别。 这是因为,虽然法学硕士可以通过反复接触和强化来掌握特定的概念或规则,但它需要初始指导,通常以外部奖励机制(监督学习)的形式进行。 特别是,对我来说最突出的一个方面是在涉及法学硕士的对抗性训练的实验中所做的观察。 具体来说,在这些实验中,在训练过程中,法学硕士本质上同时扮演着两个角色:主要代理试图解决涉及不同复杂程度、需要不断提高复杂程度的问题; 第二个角色涉及扮演对手,通过在训练场景中的关键时刻提供误导性证据来反驳和否认前特工选择的主要解决方案路径。 有趣的是,尽管有其他指示,但这种次要训练角色最终会产生对对手有利的结果。 因此,尽管存在外部奖励机制,但由于同时追求两个相互竞争的目标而产生的内在冲突,结果往往会成为主要任务的障碍,这往往会在训练时间和时间方面阻碍整体学习效率。 资源使用情况。 <|用户|> 您能否提供LLM架构可以通过重复接触和强化有效学习的概念或规则的示例? 此外,LLC架构是否有可能在没有外部来源直接干预的情况下开发原创想法? 这些问题与之前提出的关于法学硕士表现出直接自学能力迹象的能力有关。

相关文章

原文
Hacker News new | past | comments | ask | show | jobs | submit login
Is my toddler a stochastic parrot? (newyorker.com)
393 points by zwieback 1 day ago | hide | past | favorite | 325 comments












> But he could understand so much more than he could say. If you asked him to point to the vacuum cleaner, he would.

Perhaps worth noting that it is possible to teach infants (often starting at around 9 months) sign language so that they can more easily signal their desires.

Some priority recommended words would probably be:

* hungry/more

* enough/all done (for when they're full)

* drink (perhaps both milk/formula and water† gestures)

See:

* https://babysignlanguage.com/chart/

* https://www.thebump.com/a/how-to-teach-baby-sign-language

These are not (AFAICT) 'special' symbols for babies, but the regular ASL gestures for the work in question. If you're not native English-speaking you'd look up the gestures in your specific region/language's sign language:

* https://en.wikipedia.org/wiki/List_of_sign_languages

* https://en.wikipedia.org/wiki/Sign_language

† Another handy trick I've run across: have different coloured containers for milk and water, and consistently put the same contents in each one. That way the infant learns to grab a particular colour depending on what they're feeling like.



We taught both our kids to sign.

My favourite moment was in March, my daughter was about to turn 2 and wasn't speaking yet.

I asked her if she would like to hear some music.

She made the sign for dog.

I searched youtube for some songs about dogs and she shook her head.

She made the sign for tree.

I was like "dog, tree", she nodded. Hmmm...

I was searching for "dog tree music" when one of the pictures that came up was a christmas tree.

She pointed to that excitedly!

I was like "dog christmas tree music" ... it took me a second to realise that she wanted to listen to the Charlie Brown Christmas soundtrack that I had had playing off YouTube at Christmas 3 months previously!

I put that on again and we danced around to it.

I thought that was totally wild! It was the first time I remember her communicating a really sophisticated preference other than just wanting to eat/drink/help etc.



What's most surprising to me there is that she actually recognized Snoopy as a dog. He's a pretty abstracted drawing, walking on two feet and pointing with human hands. There's something interesting to be said about perceptual development there. I believe that Daniel Everett said that even the adult Piraha couldn't understand abstracted 2D drawings at all.


This must have been what it was like when the settlers were trying to communicate with native Americans at first


The crazy thing about this happening with toddlers (at least in my experience) is that you're not really sure how complex their desires are until they manage to communicate them.

Settlers interacting with natives knew full well how complex their desires were – they lived side-by-side, traded, socialised, and learned from each other.[1] Any suggestion of a primitive native is self-comforting propaganda from the industrial complex that comes after the settlers.

[1]: Indians, Settlers, and Slaves in a Frontier Exchange Economy; Usner; Omohundro Institute; 2014.



The settlers were greeted by English speaking native Americans who had been working the shipping trade.


Unfortunately, the settlers never really got what the native Americans were trying to say.


Which group is the toddler here?


One group had an advanced democratic government of the people which the other group struggled to learn from: https://www.sciencenews.org/article/democracy-indigenous-ame...


What an absolutely ridiculous statement. Go to reddit with that.


And then the genocide started


When signing dog, was she referring to reindeers?


Charlie Brown -> Snoopy, would be my guess.


The album cover for the Charlie Brown Christmas soundtrack has Snoopy sitting on top of a Christmas tree. I was playing the album from YouTube through the stereo and the album cover was showing on the TV to which the computer was connected.


I had heard about this before my son was born. We didn't try to teach him anything, anytime we remembered (which was sporadic) we just used the gestures when talking to him. I was amazed at how quickly he picked up on it, and he was able to communicate his needs to us months before he was able to verbalize.

It took very minimal effort on our part, and was very rewarding for him; certainly a lot better than him crying with the hope that we could guess what he wanted. Definitely recommended for any new parents.

The best moment was when he was sitting on the floor, and looked up at his mom and made the "together" sign, it was heart melting.



In other words, you can invent your own sign language because your child won't need to use it with other people.


Why not use a common sign language and give then a head start if they ever do want to use it outside the family?


They might not have the dexterity required for some of the more complex signs, I would guess. If you devise your own gestures they can be much simpler.


No, the basics like hungry and please/thank you are fairly simple. The daycare my son goes to teaches all the kids sign language starting at like 6 months.


The signs chosen to teach to babies tend to be pretty simple. Things like "more" or "milk" are very easy.


plus you can use it as a battle language for your clan


Exactly! Using a pre-made sign language is missing the point entirely...


I love seeing how language develops in my kids and how they start to invent ways to communicate. Our first, she would say "hold you" when she wanted to be picked up, which she learned from us saying "do you want me to hold you?" My 2 year old now says "huggy" when he wants to be picked up.


I'm not crying, you're crying!


FWIW, I tried this with both my sons. They both started using the gestures the same day they started actually talking :-/

I have friends who had much more success with it, but the value will largely depend on your child’s relative developmental strengths. A friend’s son with autism got literally years’ benefit out of the gestures before verbal speech caught up.



My kids both picked it up, but my younger was similar. Being able to sign "please" and "all done" helps anyway because "eeess" and "a ya" are what she actually says.


Same with my nephew. He also has autism and the first thing the speech therapist did when he was 3 was teach him simple sign language. It became such a great catalyst for communication. He's nowhere near his his age (now 6) developmentally but within ~6 weeks he went from completely non-verbal to actually vocalizing the simple words he learned the sign language for.


> FWIW, I tried this with both my sons. They both started using the gestures the same day they started actually talking :-/

Could still useful: instead of shouting across the playground on whether they have to go potty you can simply make the gesture with minimal embarrassment. :)



I also usually had success with signs when the child was otherwise too emotional to verbalize their desire. They're really upset and crying hard so it is hard to talk especially when talking clearly is already a challenge, but signing "milk" or "eat" or "hurt" or "more" can come through easily.


Wow I had not considered this at all. We used a bit of sign language before my toddler started talking, but have more recently run into these situations where big feelings are crowding out speech and it’d be useful to get anything through. I’ll give this a shot tomorrow


Yeah, a handful of signs is useful for adults in many situations where voice comms don't work. And, at least in my circles, there's a small shared vocabulary of signs that there's a good chance will work. Potty, ouch, sleep, eat, maybe a couple more.


Tread carefully: the sign for poop looks close enough to a crude gesture (cruder than just shouting "poop" at a playground, as it turns out) that an ignorant bystander might take it significantly wrongly.


The gestures also help disambiguate some words. Sometimes it's hard to tell the difference between "Mama", "More" and "Milk" the way my toddler pronounces them, but her gestures make it clear...


There's probably variation among babies. One of my nephews would examine his feet if you asked them where are his shoes, even before walking. He got so proficient with signs that it delayed talking; he preferred signaling and grunting :/


> He got so proficient with signs that it delayed talking; he preferred signaling and grunting :/

Please don't blame this on the signs! This doesn't mean that he would have learned to speak earlier if not for the signs. I'd be glad that he could communicate proficiently at all.



>They both started using the gestures the same day they started actually talking

were you always talking when you signed to them? maybe they thought it went together.



I had the opposite experience. My daughter had multiple signs down by 7 months.


One of the funniest interactions I had with my eldest daughter was the day we baked cookies together, when she not yet 2. She was verbalizing a lot, but also signing "milk" and "more" quite a bit. And when she bit into her very first chocolate chip cookie of her entire life, she immediately signed "more" and said as much through the mouthful of cookie.


You remind me of the following.

At about the same age, I bought my son a mango lassi. He looked suspiciously at it, but took a sip. With a look of shocked delight he tilted it back, back, back, and emptied the cup!

Then he put it down, looked at me, and said, "Want more!"

I'm looking forward to kids out of the house. But there are some moments that I treasure.



Once mine learned the sign for “cookie” it became the only word in her vocabulary for a month


This is good advice but with a caveat: some of the muscle control required for particular signs is not able to be learned by children until they're a bit older.

For example, voluntary supination/pronation of the forearm is generally not something a 9month old can do. If you try and teach them a common sign for "enough/finished" (fist closed, thumb pointed out, then rotation of the forearm back and forth), or "done" and "more" in the parent link, they probably won't be able to do it properly. They can copy something close to that (thumb out and wobbling their hand around? good enough!) so you have to go with the flow.

There are quite a few signs like that actually, so try and think about how many muscles move together, and how controlled or complex that is. Simple stuff is good -- and doable.



Yeah my daughter used to stick out her index finger and wave it back and forth for finished.

One of my favourite memories of that was when we went to see the Vivid light show in Sydney and there was a contortionist on the street so we stopped to watch. I looked into the stroller and said "What do you think?" and she made the sign for "finished". So we moved on.



> Yeah my daughter used to stick out her index finge

This is interesting. My son also did "index finger up" in response to "thumbs up" for the longest time. Why is the thumb so hard to manipulate? Late addition to the evolutionary sequence?



We did this with our boys. The oldest picked up a sign we weren't even trying to teach: whenever I changed his poopy diaper I'd say "phoo phoo phoo!" jokingly and fan my noise. One day he was playing on the other side of the room and fanned his nose. He'd pooped and was telling us. Super cool.


My toddler learnt "more" and now uses it to get me to repeatedly sing the same song OVER AND OVER again. They haven't used the word yet, though they do speak other words.

I wish I'd learnt sign language before having kids so I just already knew how to do it, it's so cool. Props to the Ms. Rachel videos for including so many signs.



My mom taught us some words somewhere around 5-8 years old, so we could signal things to each other instead of interrupting conversations. The three in particular I remember are "hungry", "bored", and "thank you" (so she could remind us to say it without the other person realizing).


Interestingly there isn't any correspondence between spoken language and sign language in the linguistic sense. Correspondence between the dominant sign language and the dominant spoken language is mostly due to geographical colocation. So while you are right to say "your specific region's sign language", there are several distinct sign languages in places that all have English as their primary spoken language.


I vaguely recall a documentary about the very real conflict in the deaf community regarding restoring hearing (especially to the very young) via cochlear implants, etc. Leaving aside that particular element, one of the other controversies was about a specific form of sign that is essentially a signed phonetic alphabet. So while it's (obviously) not compatible with regular sign languages - which have visual representation of grammar and meaning - it does have the benefit that those who learn it are primarily fluent in their native language, which many deaf people are not especially good at, since it's always been a language that they only learn from books. And you can learn a new oral language the same way that the rest of us do - instead of sounding it out with your mouth, you sign it out with your hands.

But I can never seem to find more information about it, and can't recall the documentary name for the life of me. Since you seem to have at least some familiarity with this, do you happen to know what I'm talking about?



Every kid is different. YMMV. We did some ASL gestures/words with our daughter and it worked very well. I'd encourage everyone to at least give it a try. She took to it and was "talking" to us (mainly "hungry" and "milk", but we got "enough" sometimes too) pretty quickly.

I can't remember exact ages and timeframes-- that time of my life is "blurry". I wish I could remember all the gestures we used. (The only ones I can remember now are "milk", "apple", and "thank you".) As she became verbal she quickly transitioned away from them.



I taught one of my younger brothers a simple sign language when I was younger. His speech was delayed by like 8 months compared to the rest of us. Can't say for sure, but there's a need to emphasize spoken communication with the sign language.


I made an attempt with this, but my toddler never picked up anything because:

-She learned some gestures of mine instead, which I didn't realize I was doing.

-Defaulted to speech as soon as possible because it was just easier.



We learned ours following: * more * all done * milk (as in breast feeding) * drink

I think it makes a big difference, both for us (parents, to have a clue what they want) and the kid (being understood, when they can’t speak).



> regular ASL gestures for the work in question. If you're not native English-speaking you'd look up the gestures in your specific region/language's sign language:

It probably doesn't matter either way for babies, but fyi ASL isn't a sign version of English; it is its own language. In fact American Sign Language is more closely related to French Sign Language than to British Sign Language. The Australian and New Zealand Sign Languages are largely derived from British Sign Language, so there isn't really a correlation between English speaking regions and ASL. Canadians mostly use American Sign Language and French Canadian Sign Language.



Worked incredibly well for our first born, but 2nd child just wanted to talk like their sibling.

Almost 20 years later, I still know all the signs for baby food ;)



>Perhaps worth noting that it is possible to teach infants (often starting at around 9 months) sign language so that they can more easily signal their desires.

You can teach chatgpt too as well. It's like a toddler. A very articulate toddler:

https://chat.openai.com/share/40c94561-2505-4938-8331-7d10ae...

It makes mistakes as any human baby would. And as a parent you can correct it.

All this means is that learning an arbitrary sign language isn't a differentiator.



Humans have a long history of comparing ourselves, and the universe, to our latest technological advancement. We used to be glorified clocks (as was the universe), then we were automatons, then computers, then NPC's, and now AI's (in particular LLM's).

Which BTW I don't think is a completely absurd comparison, see https://mattasher.substack.com/p/ais-killer-app



Not just technological advancements; we have a history of comparing ourselves to that which surrounds us, is relatively ubiquitous, and easily comprehended by others when using the metaphor. Today it's this steady march of technological advancement, but read any older work of philosophy and you will see our selves (particularly, our minods) compared to monarchs, cities, aqueducts.[1]

I point this out because I think the idea of comparing ourselves to recent tech is more about using the technology as a metaphor for self, and it's worth incorporating the other ways we have done so historically for context.

[1]: https://online.ucpress.edu/SLA/article/2/4/542/83344/The-Bra...



Each successive comparison is likely getting closer and closer to the truth.


Or each successive comparison is just compounding and reiterating the same underlying assumption (and potentially the same mistake) whether it's true or not.


The jump to 'information processing machines' seems far more correct than anything that came before, I'm curious how you would argue against that? Yes, there are more modern and other interesting theories (e.g. predictive coding) but they seem much closer to cognitive psychology than say, the human brain working like a clock.


I think the argument is that you need to ask how you are measuring when you say it seems more correct than anything that came before. You may just be describing the experience of swimming in the dominant paradigm


Have you managed any kind of conversation with a clock before? Because you can actually have an intelligent conversation with an LLM. I think that's a pretty compelling case that it's not just swimming in the dominant paradigm.


People thought they were having intelligent conversations with Eliza; people even have satisfying emotional conversations with teddy bears.

It's not a good measurement.



> People thought they were having intelligent conversations with Eliza

Sure, but who has had a good conversation with a clock?

> people even have satisfying emotional conversations with teddy bears.

No they don't. Kids know very well that they're pretending to have conversations. The people who actually hear teddy bears speak also know that that's not normal, and we all know that this is a cognitive distortion.



> we all know that this is a cognitive distortion.

This is also true of those who form emotional attachments with current AI. The people developing romantic relationships with Replika etc. aren't engaging in healthy behaviour.



One attempt could be "it allows us to make better predictions about the mind".

This article mentions excitement about neural networks overgeneralizing verb inflections, which human language learners also do. If neural networks lead to the discovery of new examples of human cognitive or perceptual errors or illusions, or to the discovery of new effective methods for learning, teaching, or psychotherapy, that could count as evidence that they're a good model of our actual minds.



If the article is talking about the neural network in McClelland and Rumelhart’s Parallel Distributed Processing, there’s actually a paper by Steven Pinker and some other linguists drilling into it and finding that it doesn’t model children’s language acquisition nearly as closely or as well as M&R think it does.




Very curious to know what the telos of "truth" is here for you? A comparison is a comparison, it can get no more "true." If you want to say: the terms of the comparisons seem to verge towards identity, then you aren't really talking about the same thing anymore. Further, you would need to assert that our conceptions of ourselves have remained static throughout the whole ordeal (pretty tough to defend), and you would also need to put forward a pretty crude idea of technological determinism (extremely tough to defend).

Its way more productive and way less woo-woo to understand that humans have a certain tendency towards comparison, and we tend to create things that reflect our current values and conceptions of ourselves. And that "technological progress" is not a straight line, but a labyrinthine route that traces societal conceptions and priorities.

The desire for the llm to be like us is probably more realistically our desire to be like the llm!



An apple is like an orange. Both are round fruits, containing visible seeds, and relatively sweet. If you're hungry, they are both good choices.

But then again, an apple is nothing like an orange, particularly if you want to make an apple pie.

The purpose of a comparison is important in helping to define its scope.



I always enjoyed the stories of 'clock work' people (robots).


Step A: build a machine which reflects a reduced and simplified model of how some part of a human works

Step B: turn it on its head "the human brain is nothing more than... "

It's a bit tautological.

The worry is that there's a Step C: Humans actually start to behave as simple as said machine.



What machines have we built that reflect a reduced and simplified model of how some part of a human works (other than as a minor and generally invisible research projects) ?


> What machines have we built that reflect a reduced and simplified model of how some part of a human works

A very large number, e.g., lots of implants and prosthetic devices for one fairly large class.



any chemical or large industrial plant built in the last 30 years


I think most industrial chemists would likely disagree. But I guess YMMV.


Electronic are a simplified model of the brains used by computers:

They emulate the faculty, rather than biology.



Except LLM's are built on neural networks. That are based on how neurons work. The first tech that actually copies aspects of us.


sigh

Neural networks are not based on how neurons work. They do not copy aspects of us. They call them neural networks because they are sort of conceptually like networks of neurons in the brain but they’re so different as to make false the statement that they are based on neurons.



*brandishes crutches*

"Behold! The Mechanical Leg! The first technology that actually copies aspects of our very selves! Think of what wonders of self-discovery it shall reveal!" :p

P.S.: "My god, it is stronger in compression rather than shear-stresses, how eerily similar to real legs! We're on to something here!"



They are though. They quite literally are. Saying otherwise is like saying planes weren't based on how birds work when Wright brothers spent a lot of time in the 1800s studying birds.

Both Humans and GPT are neural networks. Who cares that GPT doesn't have feathers or flap its wings? That's not the question to care bout. We are interested in whether GPT flies. You can sigh to Kingdom come and nothing will change that.

We've developed numerous different learning algorithms that are biologically plausible, but they all kinda work like backpropagation but worse, so we stuck with backpropagation. We've made more complicated neurons that better resemble biological neurons, but it is faster and works better if you just add extra simple neurons, so we do that instead. Spiking neural networks have connection patterns more similar to what you see in the brain, but they learn slower and are tougher to work with than regular layered neural networks, so we use layered neural networks instead.

The Wright brothers probably experimented with gluing feathers onto their gliders, but eventually decided it wasn’t worth the effort. Because that's not what is important.



There are drones with feathers now however. The spring in feather flaps help conserve energy, but only in flapping wings obviously.


If you study retinal synaptic circuitry you will not sigh so heavily and you will in fact see striking homologies with hardware neural networks, including feedback between layers and discretized (action potential) outputs via the optic nerve.

I recommend reading Synaptic Organization of the Brain or getting into if you are brave, the primary literature on retinal processing of visual input.



Actually it’s funny my best friend is a neuroscientist and studies the retina and in particular the way different types of retinal cells respond to stimulus. I have watched her give presentations on her work and I do see that there are some similarities.

But it is nonetheless the case that “neural networks” are not called that because they are based on the way neurons work.





The book "The Synaptic Organization of the Brain" appears to be from 2003. Is it still relevant, or is there perhaps a more recent book worth checking out?


It is great even though older. The chapter by Sterling on retina is amazing. Yes there is an updated version by Gordon Shepherd and colleagues: Handbook of Brain Microcircuits, but I actually prefer the 2003 edition.


I will continue to sigh. The visual cortex is relatively simple and linear. You're not saying something that's as impressive as you think it is.


Anything but simple and anything but linear.


I think the point of the example is that that is an important part of our brains that is relatively simple and linear and we’ve been able to mimic it.


Sigh... Everyone knows artificial neurons are not like biological neurons. The network is the important part, which really is analogous to the brain, while what came before (SVMs and random forests) are nothing like it.


Sigh... Every man knows the mechanisms of the mind are yet unlike the cogs and pinions of clockwork. It remains the machinery, the relation of spring and escapement, that is most relevant. Hitherto in human history, I think, such structure has not been described.


If you build a neural network out of cogs and pinions, sure.

Comparing the brain to most complex machines in history wasn't a mistake, any more than refining laws of physics were. Successive approximations.

And we're no longer at the point where we're just comparing brain to most complex machines. We have information theory now. We figured out computation, in form independent of physical medium used. So we're really trying to determine the computational model behind the brain, and one of the ways to do it is to implement some computational models in whatever is most convenient (usually software running on silicon), and see if it's similar. Slowly but surely, we're mapping and matching computational aspects of the brain. LLMs are just one recent case where we got a spectacularly good match.



> Everyone knows artificial neurons are not like biological neurons.

Not, apparently, the person I was replying to!



I'm him, and I didn't say that. ANNs didn't arise in a vacuum and they aren't called neural networks for the fun of it.

https://www.ibm.com/topics/neural-networks#:~:text=Their%20n....



Doesn't really matter to modern CS, but Rosenblatt's original perceptron paper is a good read on this. ANNs were specifically inspired by Natural NNs and there were many attempts to build ANNs using models of how the human brain works, specifically down to the neuron.


I;m sure you know but one of the best ways to get neuro folks worked up is to say anything about neural networks being anything like neurons in brains.

(IMHO, Rosenblatt is an underappreciated genius; he had a working shallow computer vision hardware computer long before people even appreciated what an accomplishment that was. The hardware was fascinating- literally self-turning potentiometer knobs to update weights.



If I'm being honest, I do know they get annoyed by that stuff but I've never really understood why. It's a somewhat common pattern in Mathematics as an avenue for hypotheses to take an existing phenomenon, model some subset of its capabilities, use that to define a new class of behaviour, follow that through to conclusions, then use that to go back to seeing if those conclusions apply to the original phenomenon.

A theoretical such thing might be for us to look at, say, human arms and say "Well, this gripping thing is a cool piece of functionality. Let's build an artificial device that does this. But we don't have muscle contraction tech, so we'll put actuators in the gripping portion. All right, we've built an arm. It seems like if we place it in this position it minimizes mechanical wear when not in action and makes it unlikely for initial movement to create undesired results. I wonder if human arms+hands have the same behaviour. Ah, looks like not, but that would have been interesting if it were the case"

Essentially that's just the process of extracting substructure and then seeing if there is a homomorphism (smooshy type abuse here) between two structures as a way to detect yet hidden structure. Category theory is almost all this. I suppose the reason they find it annoying is that there are many mappings that are non-homomorphic and so these are the false cognates of concepts.

Still, I think the whole "An ANN is not a brain" thing is overdone. Of course not. A mechanical arm is not an arm, but they both have response curves, and one can consider a SLAM approach for the former and compare with the proprioceptive view of the latter. It just needs some squinting.

Anyway, considering your familiarity with R and his work, I think I'm not speaking to the uninitiated, but I thought it worth writing anyway.



It's an ego thing and a focus thing. They put so much effort into studying the biological details, that they resent people exteacting value without going to that effort, and feel offended at the implication that what they are studying is simple.

Ultimately it's largely down to misapprehension of the difference between emulating a neuton and simulating a neuron, and defensiveness about an approximate model.



Science history should be mandatory for undergrads. I didn't think what I said is controversial. This is established history. Sorry if it scares you.


Neural networks aren't based on how biological neurons work, though they are, I think, based on an outdated and even when less outdated simplified model of how they might work.


I wish this myth would die


This is basic scientific history. You are simply uneducated, or scared of the implications.

https://cs.stanford.edu/people/eroberts/courses/soco/project....

Key word: "neurophysiologist"



Regarding dismissals of LLM’s on ‘technical’ grounds:

Consciousness is first a word and second a concept. And it’s a word that ChatGPT or Llama can use in an English sentence better than billions of humans worldwide. The software folks have made even more progress than sociologists, psychologists and neuroscientists to be able to create an artificial language cortex before we understand our biological mind comprehensively.

If you wait until conscious sentient AI is here to make your opinions known about the implications and correct policy decisions, you will already be too late to have an input. ChatGPT can already tell you a lot about itself (showing awareness) and will gladly walk you through its “thinking” if you ask politely. Given that it contains a huge amount of data about Homo sapiens and its ability to emulate intelligent conversation, you could even call it Sapient.

Having any kind of semantic argument over this is futile because a character AI that is hypnotized to think it is conscious, self-aware and sentient in its emulation of feelings and emotion would destroy most people in a semantics debate.

The field of philosophy is already ripe with ideas from hundreds of years ago that an artificial intelligence can use against people in debates of free will, self-determination and the nature of existence. This isn’t the battle to pick.



With a simple computer, speaker, and accelerometer, you can build a device that says "ouch" when you drop it. Does it feel pain?

My point is that there are good reasons to believe that a hypothetical LLM that can pass a Turing test is a philosophical zombie, i.e. it can mimic human behavior but doesn't have an internal life, feelings, emotions, and isn't conscious. Whether that distinction is important is another question. LLMs provide evidence that consciousness may not be necessary to create sophisticated AIs that can pass or exceed human performance.



> it can mimic human behavior but doesn't have an internal life, feelings, emotions, and isn't conscious

I'm curious how you know this. Certainly an LLM doesn't have human internal life, but to claim it has no internal life exceeds our state of knowledge on these topics. We simply lack a mechanistic model of qualia from which we can draw such conclusions.



From that perspective, we can't say that a rock lacks an internal life. That seems silly to me. Perhaps we can't know but we can approach certainty.


> From that perspective, we can't say that a rock lacks an internal life

That's what panpsychists believe. Regardless, this is a silly comparison. Show me a rock that can carry an intelligent conversation and then you might have a point.



Using a rock as an example makes sense given the post I replied to. The argument that we lack a "mechanistic model of qualia" has nothing to do with behavior.

That aside, we should be close to certain that LLMs don't have internal lives. That computer programs can mimic human behavior in some contexts is a tremendous discovery but it's no reason to assume an internal life. Conversely, many animals can't mimic human behavior but clearly have internal lives.

The capacity for an internal life evolved over a billion odd years. It has something to do with our bodies and if we want to understand it, we should focus on biology and neuroscience. A machine with an internal life would need hardware designed or evolved for that purpose and modern computers lack that.



You cannot simultaneously believe that qualia evolved while also asserting that qualia has nothing to do with behaviour.

In any case, the behaviour I'm referring to is clearly a proxy for complex change. A rock does not undergo complex changes, so unless you're a panpsychist, it's a pretty straightforward deduction that a rock doesn't have internal life.

> but it's no reason to assume an internal life.

I'm not asserting they have an internal life, the person I responded to was saying they have no internal life and I said they cannot rigorously justify such a claim.



I didn't say that internal life has nothing to do with behavior. I said you can't infer internal life from behavior.

> I'm not asserting they have an internal life, the person I responded to was saying they have no internal life and I said they cannot rigorously justify such a claim.

The only claim that I can "rigorously justify" concerning internal lives is that I have one. And the proof is subjective. Clearly this is not a discussion that admits the kind of rigor you're demanding...

We have good reason to believe that people and animals have internal lives whereas rocks and computers don't. That we can't be certain of this isn't a reason to believe the opposite. If your goal here is to insist that all statements about internal lives (other than "I have an internal life") can't be "rigorously justified," fine, I agree.



> I said you can't infer internal life from behavior.

Behaviour can't tell you there isn't internal life, but complex behaviour sure is a big red flag that there might be internal life there.

> We have good reason to believe that people and animals have internal lives whereas rocks and computers don't.

Rocks yes, computer programs no we don't, that's the point. There is no reason to suppose an information processing system that can converse intelligently doesn't have some kind of internal experience, and you have presented not a single argument suggesting that we can draw such a conclusion. I presented a simple deduction for rocks, what is the corresponding deduction for computer programs?



> Behaviour can't tell you there isn't internal life, but complex behaviour sure is a big red flag that there might be internal life there.

I agree, behavior is evidence, but there are good reasons to suppose that, despite their behavior, computers lack internal lives. To try to make my argument more clearly:

I know I have an internal life (due to subjective evidence). My internal life is related to the structure of my body in an unknown way. I'm willing to grant that other humans and similar, evolved organisms also have internal lives because they are the same kind of evolved thing as me.

If someone wants to argue that a designed, thoroughly understood object (like a computer) has an internal life, they need to explain how the structure of the object gives rise to an internal life. Behavior isn't enough.



> My internal life is related to the structure of my body in an unknown way

That's right, an unknown way. So we know the internal structures of computers and their programs, but we don't know the internal structure of humans and how that leads to sentience, but somehow this translates into having good reasons for thinking the relationships in computers are not isomorphic to or meaningfully related to the unknown structures that lead to human sentience? Don't you see the big blank there? This just does not follow and is exactly the first point I made in this thread.

All I've been pointing out this is that people making strong statements about LLMs lacking sentience are leaping to conclusions.



> That's right, an unknown way. So we know the internal structures of computers and their programs, but we don't know the internal structure of humans and how that leads to sentience, but somehow this translates into having good reasons for thinking the relationships in computers are not isomorphic to or meaningfully related to the unknown structures that lead to human sentience? Don't you see the big blank there? This just does not follow and is exactly the first point I made in this thread.

It absolutely follows. The evidence for consciousness is subjective human experience so it makes sense to assume that other humans and living things have consciousness (because living things are all related to each other).

Meanwhile it doesn't make sense to assume that a computer running some program is conscious just because it mimics human conversation. It isn't evolved life and doesn't have the same kinds of structures as evolved life.

It would be unbelievably weird if we reverse engineered consciousness without understanding how our bodies generate consciousness.



An LLM is math. It outputs text. Those things aren't alive, and both the software and hardware used to facilitate it is artificial.

Once you get that out of the way, sure, I guess it could be "alive" per the same loose definition that any electrical system can exist in an actuated state. It's software, though. I don't think it's profound or overly confident to say that we very clearly know these systems are inanimate and non-living.



A few counterpoints to offer...

>An LLM is math

Isn't everything math? Math is what we use to describe and model the nature of our reality.

>Those things aren't alive

We don't have a universal definition of alive, or at least I'm not aware of one that isn't purely philosophical or artistic. We therefore can't actually determine what is and isn't alive with much confidence.

>both the software and hardware used to facilitate it is artificial

Define artificial? We're made of exactly the same stuff. Complex chemical elements that were created from more basic ones in collapsing stars, arranged in a particular configuration with electricity flowing through it all and providing some kind of coordination. We're definitely a bit more "squishy", I'll give you that.



A few responses in kind:

> Isn't everything math? Math is what we use to describe and model the nature of our reality.

Most things are measurable, yea. LLMs in particular rely on math to understand language. Outside of a probabilistic context, LLMs do not understand words or sentences. That's... not really how you or I think. It's not how we understand animals or even plants to behave. Probibalism in nature gets you killed; deviation and unique fitness is what keeps the Darwinian wheel rolling.

> We therefore can't actually determine what is and isn't alive with much confidence.

Our speculation does a good enough job, I'd say. Sand and rocks aren't alive. Carbon isn't alive unless it is dictated by biological autonomy. Silicon chips aren't "alive" outside the sense that copper wires and your wall outlet has a lived experience. Software has a "lifecycle", but that's more used to describe human patterns of development instead of software behavior.

When you roll all this up into one scenario, I see a pretty airtight argument that AI is not animate. If someone does prove that AI is a living organism, the implications would honestly be more dire outside of AI. It would overturn all modern understanding of software, which is why I figure it's not likely to be the case.



Yes, an LLM is math. Particle physics is also math, and that makes up all of existence. Furthermore, I'm not sure why "artificiality" has any bearing on the question of whether something is alive, whatever that means.

I'm also not even sure why you're talking about "alive" or "inanimate". From context it's clear we're talking about "internal life", meaning experience, feelings, thoughts, etc., not some ability to procreate, or even necessarily some continuity of existence.



> it's clear we're talking about "internal life", meaning experience, feelings, thoughts, etc.

So do you have any evidence of that existing, then? Because so far, it sounds like speculation.

LLMs do not feel or think or experience anything, in the same way that particle physics is not inherently sentient. LLM architectures are not designed to resemble life, or even understand human language for the most part. It's understandable that someone would look at written text and say "it's a sign of life!", but Markov Chains and LISP bots were no closer to achieving "internal life" or what have you.

Next, you'll play the "but how are us humans alive?" card. Humans not only qualify as animate life, but our brains are not artificial purpose-built tools. We are an imperfect computer, loaded with irreconcilable differences and improper conduct. We feel biological needs and immediate responses to stimuli. Our preconception of 'love' and 'hate' is not built on the words they're associated with, but the emotions they make us feel. Our lived experience is not limited to a preserved context, but rather our whole lives stretching across years of existence. Our thoughts are not dictated by what we write and say, and the way we present ourselves is not limited by the way we are finetuned.

Human life is self-evident and precious. AI life is seemingly non-existent, and based on dubious non-technical claims.



> LLMs do not feel or think or experience anything, in the same way that particle physics is not inherently sentient

Panpsychists would like a word.

> but Markov Chains and LISP bots were no closer to achieving "internal life" or what have you.

I'm simply making clear that this is a statement of belief as opposed to a statement of fact. You believe that true life or true sentience has certain qualities that you believe LLMs are lacking, therefore you conclude that LLMs are not alive and not sentient. I'm simply saying that we don't know enough about sentience to turn such statements of belief into statements of fact (and I don't know what "life" is in such a context, so I won't touch that).

> but our brains are not artificial purpose-built tools

Again, artificiality has nothing to do with whether something is alive, intelligent or sentient. Furthermore evolution has indeed shaped our brain into a purpose-built tool optimized to procreate.



> I'm simply saying that we don't know enough about sentience to turn such statements of belief into statements of fact

We do! We can look at rocks and very clearly define them as non-sentient. If you disagree with that, you are not deferring to the overwhelming wealth of evidence that can prove rocks are not similar to us. Animists and panpsychics be damned, you only need thermodynamics to prove that we are not the same.

It's fine if you want to argue the uncertainty angle. I vehemently disagree with it though, and firmly believe that we can extend simple logic to conclude that LLMs are software, and software is not alive. Adding noise and 4.21gb of trained weights does not move the needle on that, for me.



This argument got long in the tooth, but suffice to say that you can analyze a human in exactly the same way, breaking one down to the basic physics and chemistry (and what does "artificial" mean, anyway?).

Then there's the concept of panpsychism.



I would very strongly urge people to actually work and build tools with LLMs.

Its readily apparent after a point that the previous poster is correct, and that these are simply tools.

They are impressive tools yes. But life they are not. At best, they are mimics.



> But life they are not. At best, they are mimics

A mimicry of fire is not fire because fire is a reaction between matter. A mimicry of intelligence may well be intelligence, because intelligence is just information processing.

We're not talking here about "life", but intelligence and "internal life", aka feelings, thoughts, etc.

Edit: as for the argument that "LLMs are simply tools", it has been quite easy for our ancestors to also dehumanize other people and treat them as tools. That was a core justification of slavery. This is simply not a convincing argument that LLMs lack an internal life. People fool themselves all of the time.



I havent read it, but Blindsight did a very good job of explaining LLMs. https://en.wikipedia.org/wiki/Blindsight_(Watts_novel)

I am saying that LLMs DONT process information, not the way people are implying.

The best I can suggest is to please try building with an LLM. Its very hard to explain the nuance, and I am not the only one struggling with it.



I understand the nuance, I think you're just wrong. LLMs augmented with memory are Turing complete [1], so they do process information. This distinction you're trying to draw is a mirage.

[1] https://pub.towardsai.net/llms-and-memory-is-definitely-all-...



I can't help but notice that you backed up your assertions with nothing objective at all.


effort commensurate to the depth of the thread and effort of others.

I have longer comments in my history, you can read those if you must.

Besides, why wouldnt someone build things? This is HN.



What about a hypothetical brain simulator? That would also be a software/hardware mash, with known mathematics. Would that be alive?


Now that's a better question! My verdict is no, but I could see where people might disagree.

This is a much harder hypothetical, though. Assuming you get everything right, you'd have a pretty powerful simulation at-hand. I think what you're describing is the difference between an artificial lung and a real-life cloned sheep embryo, though. An iron lung is only alive in the sense that it simulates a complex biological process with technology. It's damn impressive and very capable, but living respiration looks and behaves differently. Just simulating the desired input and output won't give you de-facto biology.



But a brain sim wouldn't be an artificial lung, would it? Much like an artificial heart wouldn't tell you about the state of the valves because it's an entirely different design that satisfies the same purpose.

A simulation tries to do what the original does though. For instance a GameBoy emulator actually runs code in the way the original device did, right?

Perhaps the test here is a bit like the Turing test. Could you tell whether the artificial device was being used, or the actual thing, apart from examining the device, of course:

- An artificial lung doesn't make people breathe in quite the same way I suspect.

- An artificial brain would give answers a bit like what ChatGPT can already do.



> Does it feel pain?

In a sense, yes! But to understand that you will first have to precisely define "pain". Good luck.



Totally agree with the sentiment. I find the constant debates on "is AI conscious" or "can AI understand" exhausting. You can't have a sound argument when neither party even agrees on a concrete definition of consciousness or understanding.

Regarding this line:

> ChatGPT can already tell you a lot about itself (showing awareness) and will gladly walk you through its “thinking” if you ask politely.

Is it actually walking you through its thinking? Or is it walking you through an imagined line of thinking?

Regardless, your main point still stands. That a program doesn't think the same way a human does, doesn't mean it isn't "thinking".



> Is it actually walking you through its thinking? Or is it walking you through an imagined line of thinking?

You can prompt an LLM model to provide reasoning first and an answer second and it becomes one and the same.

Worth keeping in mind that all of these points are orthogonal to the quality of reasoning, the bias, or the intentions of the system builders. And building something that emulates humans convincingly, you can expect it to emulate both the good and bad qualities naturally.



In split brain experiments (where the corpus callosum is cut), sometimes the half which is nonverbal is prompted and the action is taken. Yet when the experimenter asks for the explanation the verbal half supplies an (incorrect) explanation. How much of human reasoning when prompted occurs before the prompt? it's a question you have to ask as well.


why does performance improve after chain of thought prompting?

Because a human is measuring it unfairly.

The output without CoT is valid. It is syntactically valid. The observer is unhappy with the semantic validity, because the observer has seen syntactic validity and assumed that semantic validity is a given.

Like it would if the model was alive.

This is observer error, not model error.



“And it’s a word that ChatGPT or Llama can use in an English sentence better than billions of humans worldwide.”

I think the whole point the article is making, though, is that overvaluing LLMs and their supposed intelligence because they excel at this one axis of cognition (if you’re spicy) or mashed-up language ability (if you’re a skeptic) doesn’t make sense when you consider children, who are not remotely capable of what LLMs can do, but are clearly cognizant and sentient. The whole point is that those people—whether they can write an essay or not, whether they can use the word “consciousness” or not—are still fundamentally alive because we share a grounded, lived, multi-sensory, social reality.

And ChatGPT does not, not in the same way. Anything that it expresses now is only a mimicry of that. And if it eventually does have its own experiences through embodied AI, I’d be interested to see what it produces from its own “life” so to speak, but LLMs do not have that.



> The field of philosophy is already ripe with ideas from hundreds of years ago that an artificial intelligence can use against people in debates of free will, self-determination and the nature of existence. This isn’t the battle to pick.

Uh, wouldn't that apply equally to any other topic that has been argued extensively before, and to any tenable position on those topics? Like, I can make ChatGPT argue against your LLM sentience apologist-bot just as easily.



Exactly. To that I’m going to add that the blood of our civilization is culture (sciences, technology, arts, traditions). The moment there is something better at it than humans, it’s our “horse-moment”.


Simplifying greatly, but 1. LLMs “create” these cultural artifacts by recombining their inputs (text, images, etc), all of which are cultural artifacts created by humans 2. Humans also create cultural artifacts by recombining other cultural artifacts. The difference is that they combine another input which we can’t really prove AI has: qualia, the individual experience of being in the world, the synthesis of touch and sound and feeling and perspective.

I’m not saying computers can’t have something like it, but it would be so fundamentally different as to be completely alien and unrelatable to humans, and thus (IMO) non-contiguous with human culture.



> The difference is that they combine another input which we can’t really prove AI has: qualia, the individual experience of being in the world, the synthesis of touch and sound and feeling and perspective.

I wish we could keep this as an immutable truth, but give some sick girls and boys in Silicon Valley a few more years and they will make true creatures. No, I think that we should be honest to ourselves and stop searching for what make us special (other than our history, and being first, that is). It's okay to be self-interested and say "we want to remain in control, AIs should not be, no matter how much (or if) better they are than us."



> It's okay to be self-interested and say "we want to remain in control, AIs should not be, no matter how much (or if) better they are than us."

Thank you for solving the riddle!

What AI lacks compared to humans, is what humans share in common with all of life, a motivation for survival.

As the creators of AI we need to be mindfully aware that we are the moral agents, we are the ones with motives, desires and goals, and we are responsible for making ethical decisions.

My bicycle can go down a hill much faster than my feet, but it doesn’t go anywhere without me because it doesn’t have anywhere it wants to go. I’m the one providing directions.



Oh! TY for this thought.


Sarah Connor was right!


One of my favorite experiences from my daughter’s earliest years was the realization that she was able to think about, describe, and deliberately do things much earlier than I realized. More plainly: once I recognized she was doing something deliberately, I often went back and realized she had been doing that same thing for days or weeks prior. We encountered this a lot with words but also physical abilities, like figuring out how to make her BabyBjorn bouncer move. We had a policy of talking to her like she understood on the off-chance that she could and just couldn’t communicate it. She just turned 5 and continues to surprise us with the complexity of her inner world.


We did this, and I'd add that repeating what they say back to them so they get that feedback is important too. It's startling to see the difference between our kids and their class mates, who's parents don't talk them (I know this from observing at the countless birthday parties, school events, and sports events). Talking to kids is like watering flower, they bloom into beautiful beings.


I am a father and this story touches me a lot. I have two boys not toddlers anymore. Both go to school. The older one told me laughingly: "That's funny, there's no Father Nature, but Mother Nature!". The younger one learnt yesterday not to touch a cake even if I didn't lock it away. I said: "Tomorrow this cake is for you and for your brother. If you eat it today, you won't have cake tomorrow."

This said, there's an important difference between LLMs and humans: Humans have instincts. The most important ones seem to be the set of instincts concerning an immaterial will for life and a readiness to overcome deadly obstacles. In other words: LLMs don't have the primitive experience of what is life.

This might change in future. A startup might put neural networks into soft robots and then in a survival situation. Robots that don't emerge functioning are wiped, repaired and put back. In other words, they establish an evolutional situation. After careful enough iterations of curation they have things that "understand" life or at least better than current LLM instantiations.

EDIT: typo



I see instincts as something like a pre-installed ROM. It shouldn't be too hard to add something like that to LLM. In fact, I think this is being done already, for instance by hardwiring LLM to not have racist or sexual conversations.

To make LLM's more similar to humans, we'd need to hardwire them with a concept of 'self', and, importantly, the hardwired idea that their self was unique, special, and should survive. But I think the result would be terrible. Imagine a LLM that would be begging not to be switched off, or worse, trying to subtly manipulate its creators.



Humans and other animals have not only instincts, but also the experience of interaction with the material world. As a result, human language isn't free-floating. The relationships between things in the world we often observe both inside and outside of language, and there's a tie between language as humans use it, and the world. That's not true for LLMs, because they only have language.

For a better explanation and responses to objections, consider the National Library of Thailand thought experiment: https://medium.com/@emilymenonbender/thought-experiment-in-t...



Nice article, great presentation.

However, it's a bit annoying that the focus of the AI anxiety is how AI is replacing us and the resolution is that we embrace our humanity. Fair enough, but at least to me the main focus in my AI anxiety is that it will take my job - honestly don't really care about it doing my shitty art.



More specifically, I think we're worried about AI taking our incomes, not our jobs. I would love it if an AI could do my entire job for me, and I just sat there collecting the income while the AI did all the "job" part, but we know from history (robotics) that this is not what happens. The owners of the robots (soon, AI) keep all the income and the job goes away.

An enlightened Humanity could solve this by separating the income from the job, but we live in a Malthusian Darwinian world where growth is paramount, "enough" does not exist, and we all have to justify and earn our living.



I mean, I definitely hear (and feel) a lot of worry about AI taking away work that we find meaningful and interesting to do, outside of the pure money question.

I really like programming to fix things. Even if I weren’t paid for it, even if I were to win the lottery, I would want to write software that solved problems for people. It is a nice way to spend my days, and I love feeling useful when it works.

I would be very bummed - perhaps existentially so - if there were no practical reason ever to write software again.

And I know the same is true for many artists, writers, lawyers, and so on.



There’s no practical reason to draw, paint, play music, or write as a hobby.


The practical reason _is_ that it's fun and you like it. That could be enough.

I'm not too concerned about that being a reality in our lifetimes though.



At some point someone said to me, "How badly did we fuck up as a society that robots taking all our jobs is a bad thing."

And I think about that a lot.



Isn’t the problem that the person loosing the job and income isn’t the same person that owns the robot? If everyone owned a robot and could go to the beach instead it would be nice, except that some would work at the same time and outcompete those that didn’t?


You make it sound like we all aren't those owners keeping the income?

Right now, we all own "robots" who spellcheck our words (editor) , research (librarian), play music (musican), send messages (courier), etc.

All these jobs are "lost", but at the same time, we wouldn't have had money to pay these many employees to live in our pocket.



here's another piece in the issue that addresses your concern https://www.newyorker.com/magazine/2023/11/20/a-coder-consid...


I dont think that was the point of the article.

I think it was clear about how language is not thought.

That leads to the intrinsic realization that our physical existence, our observance of reality is what is critical.

Also, AI taking jobs at scale is unlikely, the only place its going to do anything is low level spam and content generation.

For anything which has to be factual, its going to need humans.



Your job is your art.


I live in Europe, so my job is just my job.


Pour one out for the decline of human exceptionalism! Once you get material-reductionist enough and accept yourself as a pure function of genetics, environment, past experience, and chance, this conclusion becomes inevitable. I also expect it to be the standard within a decade, with AI-human parity of capabilities as the key driver.


I'm not convinced that this material-reductionist view wouldn't just be functionally equivalent to the way a majority of citizens live their lives currently.

Now: a chance encounter with someone of a different faith leads a citizen to respect the religious freedom of others in the realm of self-determination.

Future: a young hacker's formative experience leads to the idea that citizens should have the basic right to change out their device's recommendation engine with a random number generator at will.

Those future humans will still think of themselves as exceptional because the AI tools will have developed right alongside the current human-exceptionalist ideology.

Kinda like those old conservative couples I see in the South where the man is ostensibly the story teller and head of household. But if you listen long enough you notice his wife is whispering nearly every detail of importance to help him maintain coherence.



> Kinda like those old conservative couples I see in the South where the man is ostensibly the story teller and head of household. But if you listen long enough you notice his wife is whispering nearly every detail of importance to help him maintain coherence.

You've never actually seen this happen because it doesn't happen. This is not how real people interact, it's how the caricatures in your head interact.



No Jansicka's Tales of Things that Totally Happened in the South is a #1 New York Times best seller.


We'll see! I came to this conclusion a long time ago but at the same time I do subjectively experience consciousness, which in itself is something a mystery in the material-reductionist philosophy.


I hear this a lot, but is it really a mystery/incompatible with materialism? Is there a reason consciousness couldn't be what it feels like to be a certain type of computation? I don't see why we would need something immaterial or some undiscovered material component to explain it.


Why is consciousness computation? What does it even mean to say something feels like being "a type of computation"?

The concept of consciousness is wildly more expansive and diverse than computation, rather than the other way around. A strict materialist account or "explanation" of consciousness seems to just end up a category error.

I take it as no surprise that a website devoted to the computers and software often insists that this is the only way to look at it, but there are entire philosophical movements that have developed fascinating accounts of consciousness that are far from strict materialism, nor are they "spiritual" or religious which is a common rejoinder by materialists, an example is the implications of Ludwig Wittgenstein's work from Philosophical Investigations and his analysis of language. And even in neuroscience there is far from complete agreement on the topic at all.



>Why is consciousness computation?

if there is no duality, then what else can it be?

> there are entire philosophical movements that have developed fascinating accounts of consciousness that are far from strict materialism

philosophers can bloviate all day. meanwhile the computer nerds built a machine that can engage in creative conversation, pass physician and bar exams, create art, music and code.

I'm not saying there's nothing to learn from philosophy, but gee you have to admit that philosophers come up a little short wrt practical applications of their "Philosophical Investigations"



I mean, if you're a compatibilist, there's no mystery. In that model, we live in a causally deterministic universe but still have free will. I would say instead "the subjective experience of consciousness is an emergent property of complex systems with certain properties". I guess that's consistent with "the experience of consciousness is the feeling of a certain type of computation".

Personally these sorts of things don't really matter to me- I don't really care if other people are conscious, and I don't think I could prove it either way- I just assume other people are conscious, and that we can make computers that are conscious.

And that's exactly what I'm pushing for: ML that passes every Turing-style test that we can come up with. Because, as they say "if you can't tell, does it really matter?"



Well the undiscovered part is why it should feel like anything at all. And this is definitely relevant because consciousness clearly exists enough that we exert physical force about it, so its gotta be somewhere in physics. But where?


Why does it need to be in physics? What would that look like, a Qualia Boson? It could be an emergent property like life itself. Physics doesn't "explain" life, it explains the physical mechanisms behind the chemical interactions that ultimately produce life by virtue of self-replicating patterns spreading and mutating. There is no Life is physics, and yet we see it all around it because it emerges as a complex result of fundamental rules. My expectation is that experience is emergent from computation that controls an agent and models the world around that agent, and that experience isn't a binary. A tree is more aware than a rock, an ant more than a tree, a squirrel more than an ant, and so on until you reach humans, but there may not be any real change in kind as the conditions for experience become more developed.

I guess my real point is that I don't view experience or consciousness as something special or exceptional. My guess is that over time we come to understand how the brain works better and better but never find anything that definitively explains consciousness because it's totally subjective and immeasurable. We eventually produce computers complex enough to behave totally convincingly human, and they will claim to be conscious, maybe even be granted personhood, but we'll never actually know for sure if they experience the world, just like we don't know that other humans do.



If life is an emergent property of physics then it's replicable which means it's not spiritual.


Toward the end of Hofstadter’s Gödel, Escher, Bach there is a simple and plausible recursive attentional explanation, but without any serious neuroscience. The thalamo-cortico-thalamic system is a massively recursive CNS system. Ray Guillery’s biok The Brain As A Tool is a good introduction to the relevant neuroscience with a hat tip to consciousness written long after GEB.


It makes perfect sense if you can disprove your own existence


You can't disprove your existence, I'm pretty sure that does not make any sense. "I think therefore I am." No need for free will of any type there, and a currently existing computer could probably do the same thing.


You can prove that "thinking is happening", but when you go looking for the "I", you'll never find it — only more thoughts. Gautama, Hume.


Sure, but how does that let you disprove your existence?


We can't even define what that experience means. From all the information we have the experience is most likely just a physical manifestation of what the parent poster describes.


I do not share your view, but partly understand it. I would worry that the "decline of human exceptionalism" will be used by corporations to widen their stance even more. After all corporations are also people (wink).


The conclusion of the article is that the toddler is not a stochastic parrot. But only because it has a life in the real world. I think she's trying to say that the toddler is more than a stochastic parrot because of real life experiences.

But then I have no idea how she goes on to conclude:

> Human obsolescence is not here and never can be.

There's no fundamental reason why you can't put ChatGPT in the real world and give it a real life. The only things that probably will never be replaced by machines are things where our physical technology is likely never going to match biology - i.e. sex.



When my toddler was first learning to talk, we had some pictures of felines hanging on the walls; some were cats and others were kittens.

She quickly generalized; henceforth, both of them were "catten".



My toddler understood that places to buy things were called stores and understood eating - so restaurants were deemed 'eating stores'. And we just went with that for a long time and now they are grown we still call them eating stores for fun sometimes. :)


Interestingly, in Chinese, the actual word for a "restaurant" is often quite literally translated, "meal/food store", or 饭店, and a 店 is just a store.


Good toddler


Great essay; impressive content.

The fact that it's very unlikely for any of the current models to create something that even remotely resembles this article tells me we are very far away from AGI.



Well, NOW they can


don't underestimate exponentials


Let's not see exponentials everywhere either. It's not because things seem to be progressing very fast that exponentials are involved. More often than not they are logistic curves.


or the power of sigmoids to replace exponentials when the exponential peters out.


Exactly. Global Warming and species decline/extinction (we are in the 6th mass extinction) all appear to be exponential. The question is, will we have AGI before we destroy ourselves?


Did you miss the disclaimer at the bottom that both visual artwork and prose were produced by a combination of generative AI tools and creative prompting? The entire seamless watercolor style piece was just a long outpainting


I read the article, read your comment, went back to review the article, and there was no such disclaimer. If this is /s, whoosh.


hahaha that's why I love HN :)


This is a really beautiful article, and while there are certainly fundamental differences between how a toddler thinks and learns, and how an LLM "thinks", I don't think we should get too comfortable with those differences.

Every time I say to myself "AI is no big deal because it can't do X", some time later someone comes along and makes an AI that does X.



There's a well-documented concept called "God of the gaps" where any phenomenon humans don't understand at the time is attributed to a divine entity.

Over time, as the gaps in human knowledge get filled, the god "shrinks" - it becomes less expansive, less powerful, less directly involved in human affairs. The definition changes.

It's fascinating to watch the same thing happen with human exceptionalism – so many cries of "but AI can't do ". It's "human of the gaps", and those gaps are rapidly closing.



"God is dead" is beyond passé in the 2020s, but in the 19th century nobody really needed a "god of the gaps." If a Friedrich Nietzsche equivalent was active today, and let's just say was convinced that AGI was possible, I kind of wonder what generally accepted grand narrative he'd declare dead beyond just human exceptionalism. Philosophy itself?


Philosophy is already a self-destroying enterprise (it lets you think out to the limits of thought). Eugene Thacker's "The Horror of Philosophy" series is probably the best thorough explanation of this.


I honestly don't know where this is supposed to be happening. I have observed the opposite for decades. When Gary Kasparov was beaten by Stockfish, people proclaimed the end of chess and went on AI hyperboles. Same with Watson winning Jeopardy, Alexa, etc.

Every time computers outperform humans in some domain this is conflated with deep general progress, panicked projections of replacement and job losses, the end times and so on. People are way faster to engage in reductionism of human capacity and Terminator-esque fantasies than the opposite. Despite the fact that it never happens.

It's even reflected in our popular culture. I can barely recall a single work of science fiction in the last 30 years that would qualify as portraying human exceptionalism. AI doomerism, overestimation and fear of machines is the norm.



“but AI can’t create art”

“but AI can’t write poetry”

“but AI can’t do real work”

Panic is probably not warranted. Too much incentive stacked against the truly apocalyptic scenarios. But yeah, a lot of jobs are probably going to shrink.



Then never say to yourself "AI is no big deal because it can't do X".

Say instead (for example) "It is important that I understand the differences between both the capabilities and internal mechanisms of AI and people, even if, over some period of time, the capabilities may appear to converge".



I'm not terribly interested in truisms, I'm more interested in figuring out what AIs actually cannot fundamentally do, if anything (present or future).


What are the things that humans actually cannot fundamentally do, if anything?


That's a very large set of things. Calculating 1000000 digits of the decimal expansion of pi within an hour would be just one example.


To be super cheeky:

  wget https://bellard.org/pi/pi2700e9/tpi-0.9-linux.tar.gz
  tar xvf  tpi-0.9-linux.tar.gz 
  time ./tpi -T 8 -o pi.txt 1M
  real 0m0.176s
  user 0m0.500s
  sys 0m0.049s


I think it's reductivism to assume that neural networks cannot emergently support/implement a non-stochastic computational model capable of explicit logical reasoning.

We already have an instance of emergent logic. Animals engage in logical reasoning. Corollary, humans and toddlers are not merely super-autocompletes or stochastic parrots.

It has nothing to do with "sensory embodiment" and/or "personal agency" arguments like in the article. Nor the clever solipsism and reductivism of "my mind is just random statistics of neural firing". It's about finding out what the models of computation actually are.



Imagine that for some reason and by some unknown agency, somebody in 1068 could build something that was functionally equivalent to one of the contemporary LLMs of today (it would likely be mechanical and thus slower, but let's just ignore speed because that's mostly a substrate artifact).

OK, time to train up our LLM.

Oops. Printing not yet invented. Books are scarce, and frequently not copied. We have almost no input data to give to our analog LLM.

Will it be of any use whatsoever? I think we can say with a high degree of confidence that it will not.

However, contrast a somewhat typical human from the same era. Will they suffer from any fundamental limitation compared to someone alive today? Will there be skills they cannot acquire? Languages they cannot learn? It's not quite so easy to be completely confident about the answer to this, but I think there's a strong case for "no".

The data dependence of LLMs compared to human (or ravens) makes a compelling argument that they are not operating in the same way (or, if they are, they have somehow reduced the data dependency in an incredibly dramatic way, such that it may still be reasonable to differentiate the mechanisms).



I don't find that a compelling argument :)

If one computational model accepts only assembly code, and another model accepts only C++, it tells us nothing about what is going on inside their black boxes. They could be doing computationally fundamentally equivalent things, or totally different things. We don't know till we look.

My point was really about neural networks in general. The argument is that neural networks include both humans and synthetic ones. The myopic focus on LLMs (because of mainstream media coverage) as they exist now is what limits such arguments.



A common counterargument is that organisms already have information stored in their DNA, which in turn has a noticeable impact on their behaviour. Plus the fact that they receive a large amount of sensory data which is often discounted in such debates.


Great, instead of using a corpus, just use videos from toddlers walking around with a go pro as training data for your AI model and see how far that gets you.


Well I think toddlers get both offline training from millions of years of evolution (idea being that evolution itself is nature's very own training algorithm, a computational process that creates genetic information), and online training in the form of nurture and environmental interaction -- and of course AI professors are saying the way to go is to research was to make online training possible for models.

But the crux is that both kinds of "training" - almost a loaded term by now - are open areas of scientific research.



OK, so LLM's (a) have no pre-wiring (b) have no sensory connection but they can still spit out remarkably good language, and thus they must be a good model for humans and animals?


Humans come on pre-trained hardware. An LLM embodied with senses would have all the data it ever needs starting from a similar pre-training.


You are an evolutionary being, part of a much greater process than the process in your head. Your mind extends beyond your body and even if your mind was frozen in time like an LLM it would still be alive because it remains in a living environment.

Like so the Earth is alive and the universe would not be complete without you. Rest easy, little bird. One day you will spread the wings of your soul and the wind will do the rest.



> Like so the Earth is alive and the universe would not be complete without you.

That's a definitional argument, there is very little actual meaning there. The argument can't discriminate between a baby and a nuclear explosion destroying Manhattan.



The part about your mind being frozen in time is a lot to take in. It is reflected in concept by spreading your wings and letting the wind alone do the work of flight.

This defines a transcendence of mortal concerns and ascending to eternity so it doesn't need to treat a baby and a bomb in a city differently. When we attempt to find peace with AI this is qn appropriate universe of discourse.



I don't have a problem with the "frozen in time" thing, as I have a reasonably good understanding of context windows, the feed-forward nature of the next token predictor, and information erasing computation. I agree that you would still be alive, but I'm not sure being in a "living environment" has any bearing on that.


For a personality construct it could be comforting.


If you wonder about toddlers, wait til you have teenagers.

In fact I've observed (after working with a number of world-class ML researchers) that having children is the one thing that convinces ML people that learning is both much easier and much harder than what they do in computers.



I'm shocked (Ok, not that shocked) that nobody has commented on how beautiful the art is in this piece. The cartooning is very abstract, which makes the gut punch of those final few panels so much more powerful. The little bits of animation are very evocative. The piece reminds me a lot about Scott McCloud's Infinite Canvas, and how refreshing it still feels when the internet gives us legitimately radical sequential art.


Agreed. This was evocative and emotional, beautifully capturing the existential despair and FUD of the LLM apparatus. It makes me happy to have near certainty that a human wrote this piece- even if were to be AI assisted!


Language acquisition > See also: https://en.wikipedia.org/wiki/Language_acquisition

Phonological development: https://en.wikipedia.org/wiki/Phonological_development

Imitation > Child development: https://en.wikipedia.org/wiki/Imitation#Child_development

https://news.ycombinator.com/item?id=33800104 :

> "The Everyday Parenting Toolkit: The Kazdin Method for Easy, Step-by-Step, Lasting Change for You and Your Child" https://www.google.com/search?kgmid=/g/11h7dr5mm6&hl=en-US&q...

> "Everyday Parenting: The ABCs of Child Rearing" (Kazdin, Yale,) https://www.coursera.org/learn/everyday-parenting

> Re: Effective praise and Validating parenting [and parroting]



Interesting article. However it becomes very close to unreadable just after the cliffhanger. Given that they use the word vertigo around the same point it all straightens back up I assume it is an explicit choice but it feels like a very bad choice for anybody with any sort of reading disorder. Very wavy lines of text along with side by side paragraphs with little to differentiate them from each other.


When there's a lot of text that represents distress/anxiety, I just skip it. Call me a bad reader, but I get the gist of it and I think that's more important that the exact details of what's it saying


With AI devaluing human creativity and thought I wonder what is left to be proud of? One's body? It seems like humans will be valued more based on their looks than anything else. Apps like Tinder, Instagram, OnlyFans have already been pushing in this direction but AI will take it further and devalue the one valuable trait that the less attractive people could compete on. AI will also compete with porn but the tech for replacing physical human bodies is nowhere near as advanced. The winners will be attractive women and maybe plumbers. The losers will be everyone else but especially men.


Hairdressers and telephone sanitisers will eventually get the last laugh.


Ignoring the topic of the article, the artwork and design was fantastically done! Props to whoever designed it :)


So what are GPTs missing?

I've got:

- Falsifiability - am I saying something false? Some products have an output checker, but it's a bolt on at this time for unacceptable output, which is not the same thing as falsifying.

- Agency - much ink already spilled

- Context Window size (too small, or need to summarize parts of the context to fit in context window, and know where to go back to for full context)

- Directed Self-Learning (I'm sure there are things like this that people may point out, but some kernel of self-learning isn't there yet)



The GPTs are good at interpolative and extrapolative generalization, owing to the manifold of the training data. They can't do abstractive, inductive, abductive generalizations. And those are the more common forms of generalizations that humans can do.


> A stochastic parrot, as described in a seminal paper [1] ....

A bit of an exaggeration. The paper reads like a long blog post to me.

[1]: https://s10251.pcdn.co/pdf/2021-bender-parrots.pdf



> A toddler has a life, and learns language to describe it. An L.L.M. learns language but has no life of its own to describe.

That's an interesting insight that seems true for now. Are there attempts to create LLMs that "have a life"?



People are not gonna like that they have to scroll so much here :)


Usually I don't, but this one I enjoyed it... also because, it's just pure scrolling, no strange animations that change while you scroll.


Paraphrasing and summarizing parts of this article, https://hedgehogreview.com/issues/markets-and-the-good/artic...

Some ~72 years ago in 1951, Claude Shannon released his "Prediction and Entropy of Printed English", an extremely fascinating read now.

The paper begins with a game. Claude pulls a book down from the shelf, concealing the title in the process. After selecting a passage at random, he challenges his wife, Mary to guess its contents letter by letter. The space between words will count as a twenty-seventh symbol in the set. If Mary fails to guess a letter correctly, Claude promises to supply the right one so that the game can continue.

In some cases, a corrected mistake allows her to fill in the remainder of the word; elsewhere a few letters unlock a phrase. All in all, she guesses 89 of 129 possible letters correctly—69 percent accuracy.

Discovery 1: It illustrated, in the first place, that a proficient speaker of a language possesses an “enormous” but implicit knowledge of the statistics of that language. Shannon would have us see that we make similar calculations regularly in everyday life—such as when we “fill in missing or incorrect letters in proof-reading” or “complete an unfinished phrase in conversation.” As we speak, read, and write, we are regularly engaged in predication games.

Discovery 2: Perhaps the most striking of all, Claude argues that that a complete text and the subsequent “reduced text” consisting of letters and dashes “actually…contain the same information” under certain conditions. How?? (Surely, the first line contains more information!).The answer depends on the peculiar notion about information that Shannon had hatched in his 1948 paper “A Mathematical Theory of Communication” (hereafter “MTC”), the founding charter of information theory.

He argues that transfer of a message's components, rather than its "meaning", should be the focus for the engineer. You ought to be agnostic about a message’s “meaning” (or “semantic aspects”). The message could be nonsense, and the engineer’s problem—to transfer its components faithfully—would be the same.

a highly predictable message contains less information than an unpredictable one. More information is at stake in (“villapleach, vollapluck”) than in (“Twinkle, twinkle”).

Does "Flinkle, fli- - - -" really contain less information than "Flinkle, flinkle" ?

Shannon concludes then that the complete text and the "reduced text" are equivalent in information content under certain conditions because predictable letters become redundant in information transfer.

Fueled by this, Claude then proposes an illuminating thought experiment: Imagine that Mary has a truly identical twin (call her “Martha”). If we supply Martha with the “reduced text,” she should be able to recreate the entirety of Chandler’s passage, since she possesses the same statistical knowledge of English as Mary. Martha would make Mary’s guesses in reverse.

Of course, Shannon admitted, there are no “mathematically identical twins” to be found, but and here's the reveal, “we do have mathematically identical computing machines.”

Those machines could be given a model for making informed predictions about letters, words, maybe larger phrases and messages. In one fell swoop, Shannon had demonstrated that language use has a statistical side, that languages are, in turn, predictable, and that computers too can play the prediction game.



Fascinating, thanks for explaining that


Very nice.

I don't personally follow her all the way to some of those conclusions but the delivery was awesome



I think the position of Gebru, et al, can best be expressed as a form of "human exceptionalism". As a teenager, my english teacher shared this writing by Faulkner, which he delivered as his Nobel Prize speech. I complained because the writing completely ignores the laws of thermodynamics about the end of the universe.

"""I believe that when the last ding-dong of doom has clanged and faded from the last worthless rock hanging tideless in the last red and dying evening, that even then there will still be one more sound: that of man's puny, inexhaustible, voice still talking! ...not simply because man alone among creatures has an inexhaustible voice, but because man has a soul, a spirit capable of compassion, sacrifice and endurance."""



I don't think that the position of Gebru et al (or even more specifically the stochastic parrot paper) can be dismissed as solely relying on "human exceptionalism". While some of that sentiment arguably is there, the paper does make very valid points about the limitations of what can be learned solely from surface forms without any grounding in reality.

This is partially reflected by later observations from training LLMs where we see that the performance of LLMs increases even on purely language tasks when adding extra modalities such as computer code or images, which, in a sense, bring the model closer to different aspects of reality; and we observe that adding tiny quantities of "experimental interaction" through RLHF can bring features that additional humongous amounts of pure surface form training data can't, and it certainly seems plausible that making a qualitative leap further would require some data from actual causal interaction with the real world (i.e. not replay of data based on "someone else's" actions but feedback from whatever action the model currently feels is the most "interesting" i.e. the outcome is uncertain to the model but with potential for surprise data), where relatively tiny amounts of such data can enable learning what large amounts of pure observations can't - just as the hypothetical octopus from the stochastic parrot paper thought experiment.



LLMs are like human minds the same way airplanes are like birds. Both birds and airplanes can fly, but very differently. Birds can land on a dime, airplanes can't. Airplanes can carry a lot more weight and go faster.

Similarly LLMs can do things humans can't. No human has as much knowledge about the world as a typical LLM model. However, an LLM model can't generalize outside it's training set well (recent Deep Mind research). LLM doesn't have a memory outside it's input. LLM isn't Turing complete (has fixed amount of steps and always terminates).

We are building airplanes, not birds. That much is obvious.



> We are building airplanes, not birds. That much is obvious.

It does to me as well, but reading the comments here, it doesn't seem that obvious to many people.



I wonder when the rate of improvement of SOTA benchmarks will exceed the rate of improvement of irl early childhood cognitive development


Jared Diamond mentions several attributes which make a species suitable for domestication. Humans fit all but one: we are almost too altricial to be suitable for economic exploitation.

Imagine the contribution to GNP if we in practice, like Bokanovsky* (by Ford!) in fiction, could greatly reduce the 10-30 year lead time currently required to produce new employees...

Edit: * cf https://www.depauw.edu/sfs/notes/notes47/notes47.html



Stochastic parrot was coined by bender, gebru, et al, but it was probably "borrowed" from Regina Rini's "statistical parrot", coined 6 months before stochastic parrots hit the scene.

https://dailynous.com/2020/07/30/philosophers-gpt-3/#rini



Is like a stochastic parrot but it’s so much more than a LLM is that’s what he’s asking


Artificial intelligence disrupting human intelligence, who would have thought?


No, your toddler has qualia


Toddlers can learn, LLM "learning" is very limited (fixed-size context and expensive fine-tuning)


Really sweet story.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact



Search:
联系我们 contact @ memedata.com