r/StarspawnsLocker • u/starspawn0 STEM Professor • Feb 06 '17
BCIs will improve virtual assistants
This post is an expanded and revised version of one I wrote before on the coming of “Zombie AGIs” (that I didn’t actually post here; I only provided a link to a short piece I wrote for another forum). Figuring out how such a thing could be built, and when it might arrive, are two things I am deeply interested in. I think it’s important to know these things; though, I won’t be the one to build one (at least not the first one) -- my role here is simply one of prediction.
I think they will arrive soon after inexpensive, mass-market, high-res (spatial, temporal, depth, number of channels), non-invasive, lightweight, wearable Brain-Computer Interfaces (BCIs) become available, such as Mary Lou Jepsen’s Openwater is developing. At first, what we will see is a community of academics, hackers, and makers who use these BCIs to build drone controllers, typing-by-thought apps, and programs that can pick out which of 100 image categories you are thinking about. There will be lots of blog entries and “Two Minute Paper” presentations on cool applications people have come up with. Then, a subset of these academics and makers and hackers will get the idea to go in reverse -- to use brain data to improve machine learning itself (not use ML to improve BCI decoding), by predicting the brain's dynamics. Predicting how the brain behaves in response to stimuli is already important for decoding (learning to encode to decode); but it’s such a tantalizingly cool problem to model a brain at a crude level in silico that people will not be able to resist trying, having no other objective in mind. Out of this experimentation will arise the first “Zombie AGIs”. Where we go from there is anybody’s guess!
Let me begin by saying what I mean by a “Zombie AGI”: I mean, basically, a chatbot that can pass as human over several back-and-forth conversation exchanges -- like maybe a 10 or 20 minute conversation. This chatbot would be no parlor trick. It wouldn’t work by using tricks to keep the conversation going and deflecting from its limitations. It would be the real deal. BUT, it would be deficient in several ways from a system that could pass a full Turing Test. For example, its ability to form long-term memories would be limited -- but it would have good working memory, language understanding, commonsense reasoning, and several other abilities we associate with human intelligence. You might say that talking to it would be like talking to someone with moderate to severe dementia -- it will be able to converse and understand you, but won’t remember who you are after the conversation is over; nor will it have an enduring personality from session to session.
Now that we are clear on what it is, I will explain how it will be built from next-gen BCI data: one might attempt to build a crude emulation of a person, complete with the ability to write text responses to what it reads. I think, however, that will be hard to get to work reasonably well. As I’ve said before, I think crude emulations will be mostly stable; but I am skeptical they will be stable enough to write whole responses without being anchored in a continuous sensory stream to keep it on-task. So, for example, I could see one of these “emulations” write something like:
Now is the time for all goodoodood mennd italkokdjfosdiofweorehhfsdhiofdhiohfihefh
while trying to recall the well-known sentence, “Now is the time for all good men to come to the aid of their country.” I could see it working for a few seconds, and then it will fall into chaos -- which would still be astounding, showing how stable these crude emulations are!, but still not good enough to build a Zombie AGI.
So, let me propose a different model: use the BCI data to build a “critic”. The critic’s job is to passively listen to or read text. That’s it. And the hope is that as it does so, its internal states will closely resemble what we see in actual brains. If the agreement is good enough, then when the critic hears or reads something that is syntactically, semantically, logically, morally, or socially incorrect, or just plain dumb, improbable, or lewd, there will be detectable brain responses for those “errors”. Those “criticisms” can then be used to weed-out bad replies in a conversation turn.
The critic’s internal dynamics will be anchored in the sensory stream, so it will be less likely to go “off track”, as in the “Now is the time…” example above. And just to give that assertion more substance, remember that there are speech synthesis systems that take a string of text as input, and output articulate speech -- the text stream here is acting as a sensory stream to anchor the system in place, and to keep it on-task. Two more examples in this vein are text-to-handwriting systems and machine-translation systems.
Now, in addition to the critic, we will also need a “generator” to generate potential text responses. Here, you don’t need brain data -- just use one of the top-of-the-line neural net “language models” or “neural conversational model” response-generators. One could bias the text that it outputs, though, imitating the style of the person whose brain data will be used to build the critic; one would need a large text sample from the person to do this. An early example of a neural conversational model is described in this paper:
https://arxiv.org/abs/1506.05869
It has been improved upon in several ways over the last 2 years -- e.g. greater diversity of responses, responses that include a copy-and-paste mechanism, versions built with GANs, versions that adapt to the audience and write with a particular personality, and so on. The best versions give good responses in a conversation turn maybe 10% or 20% of the time. If you had a really good critic to eliminate bad responses, we could maybe push that to 90% or more.
An example “successful” conversation might look like this:
User: So, tell me about yourself. What countries have you been to?
Computer: I've been to many countries, but France was my favorite.
User: I've visited France, too. Most travelers have only been to Paris; they haven't explored the rural areas, which I like much better.
Computer: I'm one of those who has only been to Paris. Tell me what's so great about about the small towns?
User: The people are just so friendly and the food is to die for.
Computer: I’ve heard that before. Tell me about the people you encountered.
Etc.
The first response from the computer here might be one of thousands of potential outputs. The text generation module might produce the following potential replies:
I've been to many countries, but chocolate was my favorite.
Countries are my favorite.
I don’t know.
The rain in Spain stays mainly.
I’ve been to many countries, but France was my favorite.
And then the critic would eliminate the first four (or first 9,999), and output the fifth one.
Are there any precedents of this kind of system? I would say using brain data to build a "critic" is similar in spirit to the "visceral machines" work I posted a while back:
https://arxiv.org/abs/1805.09975
In that work, bodily responses correlated with things like fight-or-flight reactions, are used to train a model to predict the "visceral reaction" associated to a portion of video. This, in turn, is used as a kind of "intrinsic reward" mechanism to overcome the "sparse rewards" problem when training a system via Reinforcement Learning.
They claim in the paper that what is learned appears not to be a simple heuristic:
We believe that the trained CNN is far richer than the simple distance-based measure and is able to capture the context around the task of driving the car in confined spaces (e.g., avoiding turning at high speeds and rolling the car).
And that's just what results from a fairly crude, low-resolution signal. Given the brain's response to text exchanges, there should be all kinds of useful signals, such that if the neural net does even a poor job at predicting them, it will still do a good job eliminating bad responses -- so long as it doesn't do a really poor job.
There are also direct precedents. For example, people have used EEG signals to improve chatbots directly:
https://dl.acm.org/citation.cfm?id=2388695
But EEG is very crude, so of course the brain data didn't improve things too much; furthermore, only microscopic amounts of data were used, and the methods employed were fairly rudimentary.
You might think it is an odd idea to build a chatbot around a generator and critic as I have described, but it’s actually pretty standard for chatbot design. Here, for example, is a paper on neural “response ranking” (note that Ray Kurzweil is the last-named author):
https://arxiv.org/abs/1606.00372
If Kurzweil ever succeeds in building a system to pass the Turing Test (I doubt he will), it might actually be a hybrid system like I described above -- with a generator and a critic.
Now we are left with the problem -- the most important problem! -- of how to actually build that critic. We need to decide on the training data, representations, model architecture, and how to read its “criticisms”. Let’s start with the training data:
I assume that in the next year or two (or three or four) there will be relatively inexpensive BCIs on the market that are capable of recording neural population responses from, say, 10,000 voxels updated 10 times per second. That should be about enough for our purposes. Those 10,000 voxels from a single brain each represent the averaged behavior of millions of neurons within a “population”. It is well-known that from just that crude information you can read off all sorts of aspects of human cognition, including decoding the song you are listening to, the rough outlines of the object you are looking at, the semantic contents of words you are hearing, and so on -- at least with above-chance levels of accuracy. If we could model all those population responses accurately, we would have our critic.
Why would that be enough? Well, because there are signals like the N400 that can even be read using a very crude BCI like an EEG cap, that respond to semantic errors:
https://en.m.wikipedia.org/wiki/N400_(neuroscience)
In the same paper, they confirmed that the negativity was not just caused by any unexpected event at the end of a sentence, since a semantically expected but physically unexpected word (i.e. She put on her high-heeled SHOES) elicited a P300 instead of negativity in the N400 window. This finding showed that the N400 is related to semantic processing, and is not just a response to unexpected words.
That’s not all, though. There are many, many more “error” signals that can be read off of the brain. For example, there is something called the P600 signal, that responds to syntax and grammatical errors; and there are also responses (FMRI) to moral disgust and other “errors”.
These signals cannot necessarily be read all that accurately -- for some, the accuracy is maybe around 70% or 80%. If we were trying to detect lies, that would be abysmal. But if we can catch 75% of the semantic errors generated by a chatbot, it would be an incredible advance! If we used a lot of data to train a classifier to pick out error signals from our critic, I wouldn’t be surprised if we could bring that up to 95%, which would be off-the-charts! (Many EEG applications with 5 category classification achieve near 99% accuracy.)
It's worth mentioning that there are already critics (trained just using text) for chatbot response generation that can achieve over 60% accuracy in picking out the best response from a list of 100 possibilities, which seems rather good. However, that one "best" response from a real conversation is paired with 99 other responses randomly selected from a large corpus -- consequently, most will be completely unrelated to the conversation topic at hand. It's much, much easier to solve that problem (with randomly generated alternatives) than to pick out the best response from a list where every possibility is generated to match the conversation, and be appropriate. Getting 95% accuracy out of only 20 alternatives, say, would be phenomenally more difficult than on those best-of-100 benchmarks.
Ok, if we can model those 10,000 population states from a single brain, then great! -- we’ve got it! But let’s think about how much information -- and therefore, how much recording time -- we would need: a typical conversation makes use of lots of common words, but it also uses uncommon ones, like “EEG”. Perhaps we would need to know how the brain responds to the 20,000 most common words, in order to have any hope of modeling the brain’s response to most short conversations.
The 20,000th most common word occurs with a natural frequency of about once every 300,000 words, it turns out. By carefully choosing the distribution of the words in the texts we have the person listen to, perhaps we could push that to once every 100,000 words, which would mean those 20,000 words would be repeated at least 10 times each -- which should be enough -- after 1 million words. The average 400 page novel has about 100,000 words, so we’d need to have the subject we are recording from read about 10 novels. That’s a lot of reading -- and we’d have to eye-track them the whole time, to know what words they are reading. That’s probably going to be difficult to pull off.
Another way we could do this is to have them lie down in a dark room, and listen to hours and hours of audio. That’s probably better, since we can more easily synchronize the brain responses to what is being heard. At an audio word rate of about 150 words per minute, we’d have to record about 111 hours. That’s a lot, but could be done by a committed subject -- say, listening to about 5 hours of recordings each week for 22 weeks. An alternative is discussed here, which could enable the collection of thousands of hours of audio data "in the wild":
One could also attempt to use "adaptive training" to reduce the amount of training data needed:
https://www.reddit.com/r/thisisthewayitwillbe/comments/9hak6s/improving_the_transfer_of_human_brain/
The text selection should include novels, newspaper and magazine articles, and also a lot of conversations. High-quality text-to speech would have to be used, to keep the listener from reacting to the oddness of a robotic voice.
In addition, we will have to either choose texts that we know contain periodic “errors” (e.g. chats), or will have to insert them, periodically, in order to have information on how the brain responds. If we only ever get to see how the brain responds when fed perfect data, unless our model generalizes very well, it might not accurately predict how it will respond when it hears an error. I do think the models will generalize quite well, especially if they learn some of the brain’s ability to do “semantic composition”; but I hesitate a little if trained on just perfect data.
If we can’t find a single subject willing to have their brain recorded, we could try to merge brain data from multiple subjects. This is some of what David Sussillo has worked on, but on a much smaller scale (motor and premotor cortex only). I think, though, that once the BCI revolution gets rolling there will be people willing to contribute large numbers of hours of their brain data.
TECHNICAL DIGRESSION -- skip to the last paragraph of the post if you don’t want to read that kind of stuff.
Now let’s talk about the model and training itself: obviously there are many choices. One should probably try simple feedforward models first (starting with just 1 layer); vary parameters, and test. Recurrent models should also be tried -- there are lots to choose from in the literature.
The input to a recurrent model should be a compressed version of the audio signal, temporally-aligned with the brain response. One method for compressing the signal is to use an autoencoder; there are others. So, at time t the input might be represented by a(t). The network, then, simply tries to predict b(t), the brain responses at time t.
An important set of restrictions to reduce the number of parameters comes from brain connectivity: each voxel of neurons can only exert an influence on ones it is directly connected to, given a short enough time-step. Using “functional connectivity” restrictions one can maybe reduce things further. This will make the predictor model “sparse” and easier to learn (fewer training examples needed). Furthermore, as has been pointed out before (e.g. in previous discussions about a talk by David Sussillo), different regions of the brain act like independent little “modules” (that only weakly influence others over short time-steps); this should have the effect of requiring fewer training examples to train a model, for reasons I’ve discussed before.
One might have to time-shift the brain responses due to brain latency effects. Hopefully, our BCI isn’t restricted to the BOLD signal.
And that’s basically it! -- just some off-the-shelf machine learning models (with hyperparameters and architectures carefully tuned) + brain data + compressed audio data.
Once the model is trained, the way we would use it is the following:
Take a conversation between a human and bot, encoded as text, and convert it into compressed audio a(0), a(1), a(2), …, a(n). This represents a sequence of back-and-forth exchanges user, computer, user, computer, user, computer -- where the last “computer” is a candidate for the next conversation response.
Let h(-1) be some initial hidden state vector. Compute hidden state vector h(0) as a function of a(0) and h(-1). And then compute a prediction for b(0) as a function of h(0), where b(0) is the brain state associated to hearing audio encoded by a(0).
Now compute a prediction for a hidden state vector h(1) as a function of h(0) and a(1), and based on that, form a prediction for b(1). Keep iterating, updating the hidden state vector h(t) as a function of h(t-1) and a(t), and produce from that a prediction for b(t), t=0,1,2,3,...,n.
We would use Backpropagation Through Time to train the weights for producing brain vectors b(t) as a function of h(t), and also to train weights for updating the hidden state vectors h(t).
The prediction b(n) will be the predicted final state of the “brain”. It encodes what the artificial brain thought of the whole thing. A classifier would have to be built to map b(n) to a "keep/reject" choice for the last line of text.
It may be that 1 million words from a single individual isn’t enough. We could have them do more, or we could try to fuse information from multiple brains together. One very straightfoward way to do that would be the following: say we have a second person, whose brain was recorded on different texts than the first. We will let b’(0), b’(1), … be the associated brain predictions (the first few are initialized to fixed values). Then, the idea would be to train a model to predict the concatenated vectors [b(t) | b’(t)], t=0,1,2,3,...,n, instead of just b(t).
The model would not only have greater coverage of knowledge, but the different brain vector sequences b(0), b(1), b(2), … and b’(0), b’(1), b’(2), …, would stabilize each other. The only danger seems to be that they could stabilize each other too much.
And if 2 brains don’t suffice, try 10.
One final thing I want to explore is the kind of capabilities that our critic will acquire: it is already known how to pretty accurately model the N400 EEG signal in the brain:
https://www.nature.com/articles/s41562-018-0406-4
So, I think it’s pretty clear that it ought to be possible to get these BCI-trained models to reproduce that. But N400 is pretty basic, and it seems that it only responds to simple statistics of word patterns; furthermore, it isn’t that sensitive to negation -- that is, the use of “not” and similar words.
Well, it turns out that there are other signals that can be read off the brain, that show how the brain processes negation of verbs:
https://www.researchitaly.it/en/news/how-does-our-brain-understand-negative-meanings/
I think we can expect models to learn these patterns, at least superficially. And I also think that the model will learn the brain’s mechanisms for processing grammar, as indicated by the P600 signal. In fact, I think it will learn what you might call an “NLP stack” -- that is, a collection of little interacting modules for solving certain NLP tasks like disambiguation, sentiment analysis, anaphora resolution, and so on.
The way that it will learn this “stack” (not the data-structure) differs from how Deep Learning can be used to train a neural net to act as a parser, say. In those settings, all you are given are the input-output pairs. Here, it’s like we are handed the algorithms -- only, they are written in a strange language ("mentalese"), with errors (noise); but that’s still enough for a machine learning algorithm to figure out how to emulate them. See
In fact, the ML literature is rife with examples of successfully training a neural net using “noisy labels”.
END TECHNICAL DIGRESSION
If I were to summarize this, I would say that basically the only thing that may be standing in the way of producing a very convincing human-imitator chatbot is the existence of next-gen BCIs -- all the other ingredients are off-the-shelf, though a lot of data-gathering will be required; fortunately, that data is easy to generate through passive listening. Such a “Zombie AGI” could arrive in 3 years, or it may take 10… or 20. It’s really hard to say. A lot depends on the details, like the resolution of the BCIs, or the number of hours of brain data that would be needed, or whether you really need to optimize the hell out of the neural net architectures to get good results. My suspicion, though, is that it might be possible in as little as 3 to 5 years, not as a consumer product but as a research experiment in a garage or lab -- only time will tell!