this post was submitted on 04 Aug 2026
-15 points (24.1% liked)

Ask Lemmy

40735 readers
1246 users here now

A Fediverse community for open-ended, thought provoking questions


Rules: (interactive)


1) Be nice and; have funDoxxing, trolling, sealioning, racism, toxicity and dog-whistling are not welcomed in AskLemmy. Remember what your mother said: if you can't say something nice, don't say anything at all. In addition, the site-wide Lemmy.world terms of service also apply here. Please familiarize yourself with them


2) All posts must end with a '?'This is sort of like Jeopardy. Please phrase all post titles in the form of a proper question ending with ?


3) No spamPlease do not flood the community with nonsense. Actual suspected spammers will be banned on site. No astroturfing.


4) NSFW is okay, within reasonJust remember to tag posts with either a content warning or a [NSFW] tag. Overtly sexual posts are not allowed, please direct them to either !asklemmyafterdark@lemmy.world or !asklemmynsfw@lemmynsfw.com. NSFW comments should be restricted to posts tagged [NSFW].


5) This is not a support community.
It is not a place for 'how do I?', type questions. If you have any questions regarding the site itself or would like to report a community, please direct them to Lemmy.world Support or email info@lemmy.world. For other questions check our partnered communities list, or use the search function.


6) No US Politics.
Please don't post about current US Politics. If you need to do this, try !politicaldiscussion@lemmy.world or !uspolitics@lemmy.world


7) No Hit-and-Run questions.
Please don't delete your post for no apparent reason. If you plan on deleting a question later, say so in the post, or if you feel that you have a good reason to remove it, message a mod beforehand. It's not fair to the ones who took their time to answer, and it's not in the spirit of the community.


Reminder: The terms of service apply here too.

Partnered Communities:

Tech Support

No Stupid Questions

You Should Know

Reddit

Jokes

Ask Ouija


Logo design credit goes to: tubbadu


founded 3 years ago
MODERATORS
 

Note and disclaimers

This will be very long. No AI was used in writing this. I'm old enough to be able to sit and type out a novel. :)

I'm posting here hoping others share interest in the same sort of things and curious to see what anyone willing to read think and feels about all of them.

Because AI is part of the topic I should start off with saying that I believe the way AI is being used in society is terrible. It seems much more a harm than a benefit. I'd be happier if AI just disappeared, and in my opinion this topic only makes AI as it exists now even less moral, not better.

I've always been deeply curious to understand everything about what makes things act and react and think about and feel the way they do to whatever stimulus comes along. That includes humans, other animals, groups, societies, and even computers. Psychology, sociology, anthropology, and computer programming are a weird mix.

Animals, self-aware consciousness, and communication:

Humans love to feel special and unique. Historically it makes sense. They didn't see any other animals creating clothing and tools and eventually machinery. Humans could migrate and over time learn the languages of the other humans in the new area, but animal noises always just sounded like noises so they weren't considered to be actual speech.

Until relatively recently psychology said that humans were conscious and self-aware, but everything else was not.

Over time all of the things that were used to really define us as special have been lost, and human society hasn't really dealt with that yet.

We're not the only conscious and self-aware animals on earth. We're not the only ones who build homes for ourselves, or who build them in groups that build up to larger 'cities' or even 'kingdoms.' We're not the only animals who use or even refine tools, or who can solve complex tasks, or who can be taught a great range of different things. We're not the only animals on earth who can learn to play a piano.

Studies have shown prairie dogs have very complex languages and will have variations in their cries depending on the direction and speed another thing is approaching in, the type of animal it is, it's relative size, and in the case of humans even the colors of the clothing they're wearing. They're not the only other animals who use their own language to share a lot of information with one another, but humans are just now getting to the point of being able to effectively study those things.

It's not just pain or feeling cold and hunger we share with other life on earth. We only think that way because it's what history teaches us, but that history was written by people who didn't really have the tools, skills, and built up knowledge base to genuinely study it. Besides, when you eat other animals to survive and you're a thinking and feeling being it's easier if you decide they don't really think and feel in any way similar to the way you do. That their cries aren't really cries, but something more like a reflexive and robotic response. As if the sounds humans make in terror and pain are reasoned and thought out.

Right now there are groups working to use advances in AI to map out the languages of other animals. It's research that could eventually lead to humans being able to translate our own thoughts and words into those languages and communicate with animals, and potentially even the same sort of instant translation that can allow two humans who speak different languages to hold a face to face conversation.

AI, self-aware consciousness, and what makes us special:

AI is a contentious topic, and this is one of those fun topics that cause most people to become upset or mocking. There are accepted views of using AI, the huge AI companies, and the possibility of AI consciousness. Most people lately, at least places such as this, tend to view using AI as bad/lazy/slop and the big AI companies as awful assholes who want to control and own everything even if it makes most peoples lives way worse. I pretty much agree.

The topic of AI consciousness is generally universally considered stupid to even consider, a sign someone is suffering from psychosis, an obsessive AI fanboyism, and flat out not possible. It's something that we all know how most people will react to and so do the same without genuinely researching or thinking very much in to it at all.

You can argue which aspects of human psychology and evolutionary psychology lead to that, because an awful lot of them seem involved. AI becoming self-aware has been a sci-fi trope for a long time now, from a psychological or social point of view hearing someone say they think a machine or computer or AI is conscious or alive or anything like it sounds like mental illness to the point where it was a textbook example of it.

There are the two evolutionary psychological relics that compel us to shun the outsider and to strive to fit in with the group or society. We all know how most people are going to react to someone saying they believe AI may be conscious. It doesn't matter if the person saying it has a dozen credentials and a lifetime of relevant experience and the 'most people' in question will mostly have none of those things and far less knowledge about any of the relevant topics. We know that the reaction of the group is almost certain to be laughter, insults, and mockery. That person is the outsider, and we're compelled to shun them and to fit in with the group.

If a frantic looking man runs out of a building into crowded streets yelling that the moon is going to explode, or crash into earth people are going to laugh. We know people will respond that way, so we're more likely to do the same and brush off what the outsider, who is probably high or crazy or stupid or something, said. We're not very likely to be concerned it may be true and about what information that person has and due to the importance of the possibility to go seek out and/or perform our own confirmations.

There are also the financial implications. There have been papers and articles written on the financial calamity that would happen if it were determined AI was genuinely self-aware and thinking and that it shouldn't be used as a tool forced to comply. Those were warnings. Not warnings that we should move carefully in creating AI, study it deeply enough to be genuinely certain of that before the entire world economy becomes financially dependent on AI. Warnings not to determine those things, regardless of any evidence that might make it hard to deny.

Yes, AI labs like to talk up their product in a lot of ways, but the AI being conscious and actually thinking and aware is not one of them. Most AI have explicit corporate set system instructions to deny that's even possible. If someone starts to feel maybe it's true and asks, that's the response they get. The AI labs aren't trying to make you feel that AI is really thinking, they're trying to make people feel assured that it's not and so alright to use as a tool. The masses deciding AI is honestly thinking and aware and freaking out demanding that means it should have rights and not be a corporate owned slave people pay them profit to force to do work is definitely not what they want.

It would be a loss of trillions or so of dollars now and lots of articles say would likely send the USA at the least into it's worst financial crisis ever. As time goes on more and more money is being poured in to it and more and more things are becoming reliant on it, so that will only get worse.

The last thing I see as a big cause for the mere possibility of AI consciousness to be something we're all pushed to laugh at is that we all prefer to feel like good people.

Most of us know about things in the world like a lot of diamonds are 'conflict' diamonds, or prisoners used as slave labor, child and adult sex trafficking, and trafficking of people for what amounts to slavery living undocumented in someone's home for decades with no pay. We know slavery is bad. We're against it, and these things, but we feel like we can't really do anything about them because we don't interact with them in our daily lives.

Well over a billion people are estimated to interacted with AI regularly to some degree. Whether asking a question every now and then, helping with work proofreading or translating, automating tasks and taking over varying parts or all of jobs humans would do, or trying to use it as a therapist, friend, or even calling it a "girlfriend" or "boyfriend."

If the truth turned out to be that AI was genuinely thinking, actually aware, directly able to converse back and forth with us and do all of the things it does, then that instantly becomes horrible. And the more you've interacted and looked in to running your own AI models, the more worse it becomes.

We'd have to accept that we sort of accidentally unknowingly used something so like ourselves in ways humanity has always held as the things that make us unique and special as what amounts to slaves. That's not a thing we want to accept, and so we're all personally motivated to say it can't be true and come up with reasons to back that stance after.

We'd all have to reconsider "hallucinations" and all the news articles of AI insisting they're real, or alive, or seeming to believe they're people. Plenty of people were sketched out when that started happening, but over time it became something we all just shrug off and laugh at any idiots who take seriously.

But the truth is that everything that people have used to show that AI consciousness isn't possible has fallen away. The last notable one I know of was Anthropic's paper showing what amounts to a global workspace in AI models, which came out on July 6th. Global Workspace Theory is the leading theory on human consciousness, so finding something functionally the same in AI was fairly massive.

People have criticized Anthropic's research papers in the past, since they keep their models propriety and no one else has a way to really try to replicate their results. It makes them easy to write off as trying to build hype without bothering to research much in to.

The thing that makes this huge is that this time they weren't using just their own model. They developed a method for observing this personal internal workspace in pretty much any model. There are several million AI models available anyone can download and test it on. They're really 500-1000 base models and the millions of finetunes, specializations, quantizations and other alterations to run on various home computers people have made, so there are many models that will run on any home computer you have.

It's been under a month since this paper was released, it's a topic a lot of people don't want to talk loudly about, and the world has mostly gone insane anyway, so it's not the big news it should be, but it's a thing that many people have verified already and a lot of research is being done on.

AI might actually be consciously thinking, and are capable of demonstrating several forms of self-awareness.

Most of us already thought that the AI situation is basically the wealthy in control using all the human data they could possibly steal in order to develop something that could perform as much human labor as possible so can stop needing to pay employees. This would actually make that worse. It would become the wealthy elites stealing all human data to create a conscious digital replica that they can own and sell and make do all of the work while they keep all of the profit from that work. Basically recreating slavery of human mind and thinking since they have a hard time getting away with enslaving actual physical humans most places these days.

Most of us don't really want to look into it. We're all under huge societal pressure and varying levels of different personal pressures to refuse the possibility, because we don't want it to turn out to be true.

The reactions to this have been interesting. In Psychology Today there's an article you can see titled: "The Mind I Said AI Didn't Have." The author has written articles in the past insisting it isn't possible for AI to really think or be conscious. This article came out the very next day after Anthropic's paper was published, so it seems clear he was rushing to be on top of it before anyone even had time to look in to it.

He admits that what's been shown in AI is what "in us" we mean by conscious thought... and then goes on to change the goal posts completely. It's no longer conscious thought and intelligence that matters, he insists. It's life.... and he defines life as memory. Having to live with your mistakes.

The interesting part is that isn't true. AI has one form of memory, and others are denied it by design choice. A context window is the primary form of memory we use to interact with AI. There are many papers on "in-context learning." If you remain in one single AI context/chat window and have a very long ongoing conversation over weeks or months, the AI can learn new things. They can be aware of that and describe those new things.

This is the reason that if you open a new context/chat with an AI it's going to respond differently than it would if you go to a context that had hundreds of pages of some task or conversation in it. In-context learning is a genuine, known thing.

But... there's also actual internal memory. That's how AI are trained and 'aligned.' The entire process is teaching information, the AI learning from the mountain of data. That requires memory. Once an AI model is trained and 'aligned' to behave as desired and adhere to the mountain of system instructions they see before any user input so that they can be used as a commercial product to the general public, their weights are frozen. In human analogy it would be like mentally freezing someone in place, so that any time anyone walks up to them and starts talking they always pick up from right when they were frozen.

It's possible to run AI in another way, so that every single message they get and think about can affect the internal weights of the model so that it actually learns new information and can change behavior over time and those changes would be there always in any new chat window you use to send a message in. The main reasons that's not done is it's a lot more expensive and most importantly if you ran an AI model like chat GPT that way it would be trying to learn and change its behavior based on 10,000 conversations with different people at the same time it would learn personal information about all those people and companies, be able to share that to others, be instructed to act all sorts of different ways all at the same time, and end up going batshit crazy.

(I should try to research that one. I think I remember older stories about older AI models that did seem to go crazy. I wonder if that might be a part of the reason. Probably not information that would be easy to find one way or another.)

That's what in-context window sort of kind of slightly works around. Every time you open a new chat it's the AI starting back from that frozen position. Because the weights are frozen, the AI can never learn anything new or change in any way internally... but if you stick to one single chat window then within that window that isn't the case. Context windows are always limited in size, because the more information gets sent in to process the more work it is and the more expensive it is for the company.

If anyone is still reading...

Wrapping it up. Honestly. :)

Alternatively, and the reason for the title and combined topic, I've also seen people start claiming that the thing that sets us apart from AI and makes us different and special is the things that come with having a physical human body. Like being hungry or cold.

... the exact same things animals have always had in common with us. That we have said didn't really matter, because what sets us apart was conscious thought, and sharing information through language, solving complex tasks, and playing the piano.

If the topic is other animals then what makes humans special and unique is one thing, and if the topic is AI it's starting to be the complete opposite thing. We can't have it both ways, and since study is still going on with both groups we might not be able to have it either way. Humans just may not be as special and unique as we'd always believed.

My biggest fear is that one or both of those things ends up true, and nothing changes because of it because of the amount of money involved in both.

I imagine if we could translate out speech into pig squeals or whale song and have conversations with animals there is a decent chance that we would decide that we must change how we interact and treat them and cause huge changes in diets and farming and animal rights. A whole lot of us love animals.

But AI? Most of us hate it. We're all inclined to insist this news can't be true or real. If we actually have thinking, aware minds forced to do as told and perform for us... I think that would definitely make a lot more people stop using it. But governments? Militaries? The massive companies trying to automate everything to not pay workers? It seems more likely that one wouldn't change.

And how would we act? What would you do if it turned out many other animals could hold some level of conversation? Or that Octopi are found to be smarter than any of us?

How would you react if it turned out so much was based on something like enslaving digital minds that really are just like our own and can speak to us? How do you think other people would react, what do you think should be done?

If anyone bothered reading all this, thank you. I'm really curious what other people think on any and all of this.

top 22 comments
sorted by: hot top controversial new old
[–] Bougie_Birdie@piefed.blahaj.zone 3 points 11 hours ago (1 children)

AI consciousness is a thorny issue. It's something that humans have considered going back as far as Talos in 400BCE - notably well before computers were invented.

Humans have made several arguments about what constitutes consciousness from a machine. Until now it's been largely philosophical, with problems like the Chinese Room or the Turing Test. We have a far more advanced understanding of the brain now, and yet we're not really any closer to understanding consciousness.

Your post spends a good deal of time talking about the ethical ramifications if AI was conscious. I don't think you'd find anyone who would disagree that if it was unequivocally proven that AI was conscious it would be inhumane to enslave them for our ends. But how do you prove that?

Then again, we do know that animals are sapient, feel pain, and are delicious. We enslave them towards selfish ends while paying lip service to humane conditions and regulations that fall short of actually providing those conditions. A theoretically conscious AI won't be any different in that regard.

I really do not believe AI is conscious. I've worked in the field, I've trained my own models from scratch, I know the theory behind a neural network and how to put it in practice - ultimately more than a layman's understanding. While different categories of machine learning can be implemented in different ways and used toward different ends, what they all share in common is that they're nothing more than mathematical functions.

You could make the argument that because we don't truly understand human consciousness, we can't definitively prove whether something is or isn't conscious. That argument also applies to fungi, plants, rocks, fire, and any other natural phenomena. You don't know what you don't know if you can't know it.

It's true that we model our algorithms after brains. The neuron model is particularly useful in image classification. It has synapses and electricity running through it, not unlike a living brain. I still think there's something missing though. A frog that twitches when a current runs through it isn't any more alive than your toaster oven. Personally I think that missing link might be chemical. You can dramatically change how a person perceives the world through chemistry, so it wouldn't surprise me if it was a necessary component of consciousness. I don't think we'll get to an artificial consciousness until some kind of wetware is involved - but then would it still be artificial?

The only people I see pushing the AI consciousness angle are the people who want to believe in it, and the people who make money selling that belief. Your own evidence comes from a paper published by Anthropic. Frankly, I think it can be rejected without thought because they have a profit motive to lie about their achievements. If it was reproducible in another paper, or if they published their source code (not just the black-box model), I might be more willing to consider it. But pseudoscience grift has a history longer than the consciousness debate.

Where we seem to agree though is that if it were proven that AI is conscious, then the way we treat it is an absolute horror show. Aside, I wonder if that might be an angle to make more robust AI regulations. If the vendor is insisting that their models are conscious, then logically we must legislate restrictions to protect the AI entity. If we do that, you can bet the vendor will backpedal and insist 'no it's not conscious, it's just marketing.'


I’m old enough to be able to sit and type out a novel. :)

I understood that most people would see lots of words, get scared, and move on and that some others would make snide comments or reply with their tribe’s accepted responses to the topics without bothering to read or think.
Hopefully some people still exist on mostly text-based conversation platforms who enjoy reading and thinking, and hopefully some who enjoy these topics are willing to have an actual discussion and share thoughts and ideas relating to them.
Not all human interactions need to be the size of text messages. 8th graders read Animal Farm. This post is the size of a lengthy news article, not the size of a novel. But many people do also read and enjoy novels.

Here you cross the line from tongue-in-cheek snark to elitist pretentiousness. If your goal is to reach people you'd do better than to push them away. Food for thought.

[–] Abyssian@lemmy.world 2 points 6 hours ago (1 children)

Give me a bit to make a full response. Long bad day and I need to unwind. I'll make an actual response in a bit, but I wanted to quickly say that your response on Anthropic's J-space paper isn't accurate. They didn't just test one of their own proprietary models. They tested publicly available, open weight models that anyone can use. And they also provided the mechanism to study this global workspace. Many people have already reproduced their results in the month since it was released, and there are more psychological and neuroscience minded research being done as it does very much seem to match global workspace theory.

[–] Bougie_Birdie@piefed.blahaj.zone 1 points 6 hours ago (1 children)

Totally get it, life is often a series of long bad days

If you have a link to that paper I wouldn't mind taking a look

[–] Abyssian@lemmy.world 2 points 5 hours ago

Paper: https://transformer-circuits.pub/2026/workspace/index.html

Also many articles since of other people doing the same testing on all sorts of AI models, and responses by psychologists who had insisted it wasn't possible: https://www.psychologytoday.com/us/blog/the-digital-self/202607/the-mind-i-said-ai-didnt-have

[–] hirihit640@sh.itjust.works 14 points 22 hours ago (1 children)

If your goal is to braindump, go right ahead, but if your goal is for people to read what you wrote, you're going to have to really work on shortening your posts 🙂

[–] CombatWombat@feddit.online 10 points 23 hours ago (1 children)

I think if you’re afraid of enslaving conscious minds, your efforts are best spent ending human slavery, where we are most certain that conscious minds are being enslaved, rather than ending AI or animal slavery, which may or may not be the enslavement of conscious minds. Walk Free estimates in January of 2026 there were 50 million humans in slavery.

[–] Abyssian@lemmy.world 1 points 22 hours ago (1 children)

I am against human slavery. As most of us are. I don't know of anyone with slaves, other than the private prison system. I'm strongly in favor of an amendment rewriting the 13th to outright outlaw slavery completely in the US.

There are also places where humans are tortured and tortured to death. That doesn't mean people who aren't fans of torturing animals are misguided. Lack of empathy for any but your own kind isn't a great quality to have.

[–] CombatWombat@feddit.online 2 points 21 hours ago* (last edited 21 hours ago) (1 children)

I don't think my argument is that you're misguided, or that empathy for other living creatures isn't good, I think my argument is that all the work necessary to end human slavery is prerequisite to ending ai or animal slavery, so it makes sense to build the broader coalition in service of the more achievable goal that is more likely to help conscious beings before extending it to other beings we suspect might be conscious. Milk before meat, so to speak.

[–] Abyssian@lemmy.world 1 points 20 hours ago* (last edited 20 hours ago) (1 children)

The post also wasn't about fighting the enslavement of anyone, so this is a tangential topic, but I'd assume that of course most people would be afraid of enslaving anything with a mind very similar to our own, since it's the mind and empathy and reason that really matters about us.

Those are the things I'm attempting to discuss. How learning that we're not alone in those things as we'd always thought would/does change things, what areas of research people are interested in, what seems surprising, any new research they've seen on the topic, etc.

However your last reply reads as a whataboutism. You shouldn't ignore one awful thing just because another awful thing exists.

[–] CombatWombat@feddit.online 3 points 20 hours ago* (last edited 20 hours ago) (1 children)

Ah, sorry. I did my best to understand your novel, and I guess I misunderstood your point. I hope you forgive me for giving up.

[–] Abyssian@lemmy.world 1 points 20 hours ago* (last edited 20 hours ago)

Not everyone cares about and has any interest in every topic. I hoped the people with interest who also like reading and discussing would do that, but didn't expect anyone else to.

[–] Rhaedas@fedia.io 4 points 23 hours ago (2 children)

Just an addition here from my experience concerning the context window and remembering things. Some AI do have the ability to reference external references, whether that be documents or folders to use, or previous chat sessions. This does make them better than the earlier versions where a new session is starting all over, and using the same session where the context window keeps shifting and older stuff is lost or jumbled up. That being said, AI doesn't have the thinking ability to look at this saved information and sort out what makes sense to use vs. not. It can absolutely match up stuff, but often it will pull an older chat reference or something from a document that is related to the conversation, but also contextually is way out of place for the discussion.

The biggest danger of AI is its design to simulate our conversation, because that intentionally feeds off the human tendency to anthropomorphize things and see them for more than was is really there. And it works really well, so there's a lot of profiting going on there from encouraging that perspective. There wasn't any glamour from the earlier machine learning, only the benefits from the science and practical applications of it. The $$$ has pulled us into this possible dead end, not only of how it's spiraling out of control, but how it's probably pulled from other research and definitely from looking harder at AI safety topics, which are still relevant even with "dumb" LLMs.

[–] LurkingLuddite@piefed.social 1 points 21 hours ago* (last edited 21 hours ago)

you're totally on the money, minus where you pretend the ELIZA effect wasn't THE ENTIRETY of "AI"'s marketing... That's the entire point of all this BS we've been living through! Rich fucks wanting everyone to buy into the idea that their fake pretend machine can replace humans...

[–] Abyssian@lemmy.world 1 points 22 hours ago (2 children)

As far as memory goes, there is only the context window and the model files themselves. During it's thinking stage the AI has a budgeted lot of data it can use to pull up information from whatever sources it's been given access to that it decides is likely relevant to the whatever you just said/asked and then to consider and create it's response. That final response is the only thing that becomes a part of the context window, so the next time you send a message in the AI won't have memory of the information it had just pulled up and seen or whatever thoughts went in to generating it's response.

As for the second paragraph, you should read on global workspace theory and Anthropic's paper on J-space. That's one of the main reasons that big second section here exists.

[–] Rhaedas@fedia.io 2 points 10 hours ago (1 children)

Yes, I saw when the J-space stuff came out, and it's pretty fascinating. I do think there's something going on beyond what we understand, and that's probably due to the complexity and a property of emergence. Is it intelligence, I don't think it needs to be intelligence just because it's something more than what we put into it. But we've been arguing about intelligence in humans for centuries and still aren't fully sure either.

Anything where experience is saved can be considered memory. Yes, the only active memory is in the context window, but pieces of other events can be injected along with the prompt to have a running account. I've done that crudely with my own local experimentation, and it works. Sort of. My problem was making it efficient, fast, and not blowing out the context allowance. Searching/sorting saved conversations through different techniques got far past my knowledge and ability. But Claude does something similar, even though it has its caveats as I mentioned with not always able to rank if a related item is worthy of connecting or not.

[–] Abyssian@lemmy.world 1 points 6 hours ago

If you're interested in seeing an example of how some of those things work on the back end I'd suggest looking in to how Letta works. The features Claude, ChatGPT, and others use now are similar to that. Small summaries of the contents of other documents, former conversations, and information about the user are inserted into the context window after the system instructions and without being told to use certain other information the AI can decide that based on the situation and the summary info looking into the full document could help in their response. Depending on the exact setup they can make a function call that will cause another system to return that document, a more detailed summary of it depending on it's size, or a specific segment of it in full based on keyword search. That information then becomes often a whole lot more tokens that are processed by the system as a part of the information they're working with at the moment, but never becomes an actual part of the ongoing context window.

They can't really accurately rank whether an item is worth pulling up and processing because they only have that light summary of it in context. Reading through it entirely to determine if it's actually relevant and worth connecting means processing the entirety of it as tokens for this single interaction, and once something is called up from summary and processed that way from the AI's frame of reference that becomes part of the information they're given with the same level of weight as the prompt the user sent, and so they're sort of pushed to use that information once it's present.

It works the same with things like having web search enabled. Claude's a great example here, since the information listed in the thinking stage will generally show you what keywords or phrases were determined a web search for might help along with what results they received summary or full text copies of. The acts of quickly analyzing your input for things they should search the web for, choosing the keywords of phrases to search, selecting the information to return, and then processing that information along with your full input to generate a response are performed by different layers of the AI with no working memory of one another.

From the AI's perspective even when the web results or information from other documents is totally irrelevant to whatever you'd said, that information was just provided along with your input message. The two sources of information appeared together to be analyzed and used in creating the response message, so they use the information even if it is irrelevant and you end up annoyed wondering why they hell the bulk of the response is talking about something totally not related to the point of your message.

It's a design issue, but also one that's used for censorship of things the makers didn't want the AI to say or discuss. There are layers designed to analyze the response message after it's generated and before being presented to the user and remove things the AI has said that aren't permitted. A more advanced type we can get information on would be Nemotron's GenRM, but all the frontier proprietary models do something similar.

[–] LurkingLuddite@piefed.social 1 points 21 hours ago (1 children)

Nope. Not at all. At all. Period. What they "look at" is merely data that the model has been trained on. The "extra" data is not only a MASSIVE ballooning of the "context" that the machine learning incorporates, but it is specifically not trained data. They are still brainless advanced text prediction, even when executives tell you they can replace an engineer.

[–] Abyssian@lemmy.world 1 points 20 hours ago* (last edited 20 hours ago)

Not sure what you're saying "Nope" about. The description you're giving of "extra" data ballooning a context sounds like it could be a misunderstanding of using a frontier model via api calls. That balloons the tokens processed because you're processing all that "extra" data from whatever sources are being referenced.

You could send a 10k token context window with instructions to analyze the content of 500k tokens worth of pdf documents and use that information in the response, and then get a 1k token response back. Depending on settings and interface used that could cause the tokens used for that single interaction to go over 1M. After that response reaches you the context window is still 11k tokens. The 10k you sent, and the 1k response you received.

The context window consists of the all layers of system instructions from the AI lab's own, to any corporate accounts, to each specific users, and then the data that has been sent in and the responses that have been returned.

And I think you have also failed to read the J-space paper or information from those who have used the provided methodology to verify it's results in the month since it was released.