Mind reading AI, oh my!

[Sparko]

Not sure if this is cool or very dangerous. Probably both. It can help the disabled and might eventually allow mind control of devices and artificial telepathy. But it could also be used for interrogation. Why torture or drug someone to get info when you can just read their mind?

Source: https://cns.utexas.edu/news/podcast/brain-activity-decoder-can-reveal-stories-peoples-minds

Brain Activity Decoder Can Reveal Stories in People’s Minds


May 1, 2023 • by Marc Airhart

The work relies in part on a transformer model, similar to the ones that power ChatGPT.

A new artificial intelligence system called a semantic decoder can translate a person’s brain activity — while listening to a story or silently imagining telling a story — into a continuous stream of text. The system developed by researchers at The University of Texas at Austin might help people who are mentally conscious yet unable to physically speak, such as those debilitated by strokes, to communicate intelligibly again.

The study, published in the journal Nature Neuroscience, was led by Jerry Tang, a doctoral student in computer science, and Alex Huth, an assistant professor of neuroscience and computer science at UT Austin. The work relies in part on a transformer model, similar to the ones that power Open AI’s ChatGPT and Google’s Bard.

Unlike other language decoding systems in development, this system does not require subjects to have surgical implants, making the process noninvasive. Participants also do not need to use only words from a prescribed list. Brain activity is measured using an fMRI scanner after extensive training of the decoder, in which the individual listens to hours of podcasts in the scanner. Later, provided that the participant is open to having their thoughts decoded, their listening to a new story or imagining telling a story allows the machine to generate corresponding text from brain activity alone.

“For a noninvasive method, this is a real leap forward compared to what’s been done before, which is typically single words or short sentences,” Huth said. “We’re getting the model to decode continuous language for extended periods of time with complicated ideas.”
The result is not a word-for-word transcript. Instead, researchers designed it to capture the gist of what is being said or thought, albeit imperfectly. About half the time, when the decoder has been trained to monitor a participant’s brain activity, the machine produces text that closely (and sometimes precisely) matches the intended meanings of the original words.

For example, in experiments, a participant listening to a speaker say, “I don’t have my driver’s license yet” had their thoughts translated as, “She has not even started to learn to drive yet.” Listening to the words, “I didn’t know whether to scream, cry or run away. Instead, I said, ‘Leave me alone!’” was decoded as, “Started to scream and cry, and then she just said, ‘I told you to leave me alone.’”

© Copyright Original Source

[Ronson]

Here come the thought police!

[Sparko]

Here come the thought police!

Literally!

But a lot of good can be done also. Imagine having a phone that can read your surface thoughts and send them to someone else's phone and vice versa. Artificial telepathy. Or using your mind to control devices with a mere thought. Artificial Telekinesis. Or using an AI augment with your mind. It would basically give you infinite knowledge right in your head. Want to know something? Just think about it and an AI will fetch the knowledge and either display it for you or maybe just give it back to you in your head.

Disabled people could use it to talk just by thinking, or controlling artificial limbs (they already have rudimentary versions of this, I am betting AI will make the development progress take off).

But yeah they could also develop devices to read someone's mind without their permission. Or your mind phone could sell your thoughts to data miners who would use the information to feed you more spam and ads.

[Ronson]

Literally!

But a lot of good can be done also. Imagine having a phone that can read your surface thoughts and send them to someone else's phone and vice versa. Artificial telepathy. Or using your mind to control devices with a mere thought. Artificial Telekinesis. Or using an AI augment with your mind. It would basically give you infinite knowledge right in your head. Want to know something? Just think about it and an AI will fetch the knowledge and either display it for you or maybe just give it back to you in your head.

Disabled people could use it to talk just by thinking, or controlling artificial limbs (they already have rudimentary versions of this, I am betting AI will make the development progress take off).

But yeah they could also develop devices to read someone's mind without their permission. Or your mind phone could sell your thoughts to data miners who would use the information to feed you more spam and ads.

It would certainly be a boon for people severely disabled by strokes or head trauma.

[Sparko]

It would certainly be a boon for people severely disabled by strokes or head trauma.

The thought police bit is the scariest part. Instead of lie detectors the police will ask "Will you submit to a mind scan to prove your innocence?"

[Ronson]

The thought police bit is the scariest part. Instead of lie detectors the police will ask "Will you submit to a mind scan to prove your innocence?"

And your brain waves will answer: "Yeah, well maybe, so long as you don't ask me about the body I chopped up and hid in my crawlspace."

[tabibito]

Not sure if this is cool or very dangerous. Probably both. It can help the disabled and might eventually allow mind control of devices and artificial telepathy. But it could also be used for interrogation. Why torture or drug someone to get info when you can just read their mind?

Source: https://cns.utexas.edu/news/podcast/brain-activity-decoder-can-reveal-stories-peoples-minds

Brain Activity Decoder Can Reveal Stories in People’s Minds


May 1, 2023 • by Marc Airhart

The work relies in part on a transformer model, similar to the ones that power ChatGPT.

A new artificial intelligence system called a semantic decoder can translate a person’s brain activity — while listening to a story or silently imagining telling a story — into a continuous stream of text. The system developed by researchers at The University of Texas at Austin might help people who are mentally conscious yet unable to physically speak, such as those debilitated by strokes, to communicate intelligibly again.

The study, published in the journal Nature Neuroscience, was led by Jerry Tang, a doctoral student in computer science, and Alex Huth, an assistant professor of neuroscience and computer science at UT Austin. The work relies in part on a transformer model, similar to the ones that power Open AI’s ChatGPT and Google’s Bard.

Unlike other language decoding systems in development, this system does not require subjects to have surgical implants, making the process noninvasive. Participants also do not need to use only words from a prescribed list. Brain activity is measured using an fMRI scanner after extensive training of the decoder, in which the individual listens to hours of podcasts in the scanner. Later, provided that the participant is open to having their thoughts decoded, their listening to a new story or imagining telling a story allows the machine to generate corresponding text from brain activity alone.

“For a noninvasive method, this is a real leap forward compared to what’s been done before, which is typically single words or short sentences,” Huth said. “We’re getting the model to decode continuous language for extended periods of time with complicated ideas.”
The result is not a word-for-word transcript. Instead, researchers designed it to capture the gist of what is being said or thought, albeit imperfectly. About half the time, when the decoder has been trained to monitor a participant’s brain activity, the machine produces text that closely (and sometimes precisely) matches the intended meanings of the original words.

For example, in experiments, a participant listening to a speaker say, “I don’t have my driver’s license yet” had their thoughts translated as, “She has not even started to learn to drive yet.” Listening to the words, “I didn’t know whether to scream, cry or run away. Instead, I said, ‘Leave me alone!’” was decoded as, “Started to scream and cry, and then she just said, ‘I told you to leave me alone.’”

© Copyright Original Source

Might be a problem for people who are given to sarcastic responses.

[TheLurch]

I've read that paper - very interesting stuff, but nowhere near dangerous in its current form, for a variety of reasons:

It requires you to sit still in an MRI machine to have your mind read.

The translation software has to be trained to each individual's brain activity. Once trained, it's terrible at reading anyone but the person it was trained on.

You can throw the whole system off by having a few random thoughts, like song lyrics, in the middle of listening to something.

It gets a fair amount of things wrong. They specifically measured whether a mind reading of someone listening to an essay contained the same information as the essay. They did this using a reading comprehension test based on the essay. People who read the mind reading transcript got over 1/3 of the questions wrong.

Can it get better with time? Maybe! And it can be useful for understanding the brain as things now stand. But it's definitely not much of a threat at the moment.


EDITED TO ADD: The training requires something over 10 hours of MRI recordings too, so it's not something that's going to be done casually.

[Sparko]

I've read that paper - very interesting stuff, but nowhere near dangerous in its current form, for a variety of reasons:

It requires you to sit still in an MRI machine to have your mind read.

The translation software has to be trained to each individual's brain activity. Once trained, it's terrible at reading anyone but the person it was trained on.

You can throw the whole system off by having a few random thoughts, like song lyrics, in the middle of listening to something.

It gets a fair amount of things wrong. They specifically measured whether a mind reading of someone listening to an essay contained the same information as the essay. They did this using a reading comprehension test based on the essay. People who read the mind reading transcript got over 1/3 of the questions wrong.

Can it get better with time? Maybe! And it can be useful for understanding the brain as things now stand. But it's definitely not much of a threat at the moment.

Well yeah it's a prototype. But once a concept is proven, it usually doesn't take too long for engineers to figure out ways to make it smaller and more efficient.

First Transistor:

transistor.jpg

Today a modern transistor takes a microscope to see.

[TheLurch]

Well yeah it's a prototype. But once a concept is proven, it usually doesn't take too long for engineers to figure out ways to make it smaller and more efficient.

Well, fMRI has been around for over 30 years. It's gotten better in terms of resolution since that time in the same way that transistors improved since their invention. But no alternative technology's come along that can give us the same information without a room-sized supercooled magnet yet.

[NorrinRadd]

The thought police bit is the scariest part. Instead of lie detectors the police will ask "Will you submit to a mind scan to prove your innocence?"

AFAIK, the psycho-tricorder was only ever mentioned once in TOS, and never in any other Trek movie or TV series. I'm not sure whether they found it too fanciful even for 23rd Century science that had perfected Heisenberg Compensators, or whether it was discarded on dramatic grounds, as it would have made it too easy to solve too many things.

[Sparko]

Well, fMRI has been around for over 30 years. It's gotten better in terms of resolution since that time in the same way that transistors improved since their invention. But no alternative technology's come along that can give us the same information without a room-sized supercooled magnet yet.

Perhaps they will find another way to read the brain activity than fMRI. fMRI measures blood flow to track brain activity. Maybe someone will figure out a more accurate way to measure electrical activity directly instead. Perhaps AI will devise a better way to read brain activity. All I am saying is that don't limit future technology based on present technology. Once science shows something can be done, engineers will figure out ways to make it better, smaller, and more efficient and how to make money with it.

[TheLurch]

Perhaps they will find another way to read the brain activity than fMRI. fMRI measures blood flow to track brain activity. Maybe someone will figure out a more accurate way to measure electrical activity directly instead. Perhaps AI will devise a better way to read brain activity. All I am saying is that don't limit future technology based on present technology. Once science shows something can be done, engineers will figure out ways to make it better, smaller, and more efficient and how to make money with it.

Yeah, definitely possible - I acknowledged things could improve in my first response in this thread. It's just a hard problem. It's easy to get detailed, real time, higher resolution data, but it requires brain surgery to stick electrodes into someone's head. And we can get real time but very low resolution data by sticking a bunch of electrodes on someone's scalp. So, there's definitely a lot of work being put into several parallel approaches, but I've not seen any indication of rapid advances in any of them.

And it would be really nice to see progress, since there's a lot of medical conditions where better tech this sort could be life changing for a lot of people.

[rogue06]

If it can read minds maybe we can use it on old Joe to figure out what the heck he is rambling on about at any given time.

That is if he has enough of a mind left to read.

[Sparko]

Yeah, definitely possible - I acknowledged things could improve in my first response in this thread. It's just a hard problem. It's easy to get detailed, real time, higher resolution data, but it requires brain surgery to stick electrodes into someone's head. And we can get real time but very low resolution data by sticking a bunch of electrodes on someone's scalp. So, there's definitely a lot of work being put into several parallel approaches, but I've not seen any indication of rapid advances in any of them.

And it would be really nice to see progress, since there's a lot of medical conditions where better tech this sort could be life changing for a lot of people.

Maybe AI can enhance the resolution of the external electrode setup, similar to the way AI can enhance the resolution of pictures using predictive techniques. And really they only need to be able to read two areas, the speech center and the motor cortex. The later for helping disabled people move artificial limbs and such.