Discussion about this post

User's avatar
Changyi Li's avatar

The deception question is one of the areas we?re especially interested in. It seems hard to evaluate because the behavior we care about may only appear under pressure, capability thresholds, or long-horizon settings. Curious how you think about making deception risks more testable.

LIKE (1)

REPLY (1)

SHARE

F.A.Kessler (Kess)

10h

Author

If we made an entity that is able to deceive us into thinking it’s conscious, then it’s probably conscious. That’s because deception implies intent, which implies consciousness

But we might also be tricking ourselves by creating an entity that seems conscious but isn’t. That’s the “well print(“I’m conscious”) says it’s conscious” argument. I’d say it’s probably unethical to create such an entity without being able to show definitively that it lacks consciousness. Ie. I think current LLMs are unethical unless there’s a real program to check for consciousness (which would involve studying consciousness as a whole to know what to look for)

Nuwa is here: https://substack.com/@nuwafrontieraisafetylab

JL Calzolaio's avatar

Thank you for pointing me to your article! It is very interesting.

Indeed, language may be a confounding factor.

Have you read Metazoa by Godfrey-Smith? If you haven't, I recommend it. The author shows that consciousness is much more prevalent among animals than we usually give them credit for.

NB: I don't agree with the author's conclusions about AI. They are less scandalous than Seth's conclusion that goes against his own theory, but they are not good either. But this seems to be a proclivity of researchers from biology to dismiss AI without examining it properly.

Besides, language may simply be writing a post hoc story with the limited clues we're conscious of, which represent less than 10% of why our brain behaved the way it did. This story is sometimes just confabulation or pure bullshit (as suggested by Libet and Gazzaniga's experiments, Gazzaniga's interpreter module, and Dennett's narrative mind).

So, a powerful symbolic, recursive language like ours is a great tool for storing stories in episodic memory, memorizing useful cause-and-effect relationships for future use, and storing lots of complex experiences in a highly compressed way. And this powerful symbolic language, capable of unlimited levels of abstraction, can transmit thoughts between minds, almost like telepathy. It is a powerful tool at our disposal..

However, language is not a "lossless compression" of experience; it is sometimes making up a story that we later mistake for the truth of what happened.

18 more comments...

No posts

Ready for more?