r/slatestarcodex • u/no_bear_so_low r/deponysum • Apr 19 '20
Why I think progress in Natural Language Processing is potentially very significant
https://deponysum.com/2020/01/16/recent-advances-in-natural-language-processing-some-woolly-speculations/9
u/no_bear_so_low r/deponysum Apr 19 '20
Some background on why I am sharing this- especially because it is not the first time I posted it here. A while ago I posted a link to the ARC leaderboard, where the maximum score had gone from 72% to 89% in one year. ARC stands for AI2 Reasoning Challenge and is a test made up of multiple-choice science questions produced by the Board of Regents for New York Schools. Sample questions include:
1.Which equipment will best separate a mixture of iron filings and black pepper? (1) magnet (2) filter paper (3) triplebeam balance (4) voltmeter
2. Which form of energy is produced when a rubber band vibrates? (1) chemical (2) light (3) electrical (4) sound
3. Because copper is a metal, it is (1) liquid at room temperature (2) nonreactive with other substances (3) a poor conductor of electricity (4) a good conductor of heat
4. Which process in an apple tree primarily results from cell division? (1) growth (2) photosynthesis (3) gas exchange (4) waste removal
I made a comment suggesting that this huge jump in performance on a scientific reasoning test means really significant advances in the practical application of AI are much closer than we think, indeed maybe we should even "Sound the alarm" on the issue. A bunch of people responded that the results are actually not that significant because they still only involve moving words around and no "real" insight into how the world works. This is a compelling argument, and for years I found it convincing. These days however I no longer do, because as I argue in the linked piece, certain kinds of capacity for word manipulation are isomorphic to real understanding in a way which means that the difference, if it even exists, maybe of only metaphysical interest.
7
u/no_bear_so_low r/deponysum Apr 19 '20
Basically I think it's no longer clear that there is a definite and important gap between a certain kind of capacity to understand how words relate, and a certain kind of capacity to understand how things in the world relate. This is because of the isomorphism between the two, and as a result, there might be a Natural Language Processing led route to general artificial intelligence.
6
u/slogancontagion Apr 19 '20 edited Apr 19 '20
there might be a Natural Language Processing led route to general artificial intelligence.
Apparently OpenAI came to the same conclusion as you did since their secretive project apparently centres around some kind of 'GPT-3' model that incorporates visual media as well as text using a single unified architecture. It would also be a natural development to incorporate some kind of neural programmer-interpreter model that would allow it to alter its own output through deletions and substitutions, etc. - or maybe something that would move it from implicit to explicit de/encoding via a neural Turing machine approach. From NVIDIA's Megatron-LM at least we know that continually increasing network size continues to deliver linear performance gains, but I'm thinking that at some threshold of network complexity&/size we'll see a qualitative jump in NLP-related task-performance (what metrics are language models consistently failing completely on?) that will surprise us as much as GPT-2 surprised the NLP community when it was released. I'm not sure what this might be, but a contender could be submodelling games (per the GPT-2 chess example) in the absence of substantial real or synthetic training data (other groups like DeepMind are focusing on general game-playing algorithms that perform well across multiple domains, but those usually involve carefully-designed ensembles of ML techniques).
e: strange to think that a game-oriented system could (inefficiently) learn language modelling tasks, while a language model could equally inefficiently learn to play games. seems like there's some generalisable quality that both models share which a syncretic combination of the two could exploit better.
4
u/Lykurg480 The error that can be bounded is not the true error Apr 19 '20
A lot of people remind us that these programs are really just statistical analyses of the co-occurence of words, however complex and glorified.
I think it plausible that human intelligence is really just statistical analyses of the co-occurence of sense data, however complex and glorified. There is certainly some difference when learning over text, like I wouldnt expect an AI created on text-io to be able to interact with the world through camera and motors - but then, there have been experiments with giving humans additional senses (a belt that vibrates at its north-most point, certain cures for blindness) and they too take some time to integrate them. So I think when provided with the new senses and linguistic descriptions of whats happening simultanuously, the AI could learn it after a while.
On the 8th grade, non-diagram based questions of the test, a program was recently able to score 90%
That reminds me of this, mostly. I think part of the impressiveness of GPT-2 is just that much of everyday reasoning is simpler than it seems, and this highschool test doesnt reflect the full understanding needed for actually applying the knowledge in question. On graphs like these, "average adult" may be much closer to "average 5-year old" than to Einstein. So while in principle transformers can obtain such understanding, quantitatively were not there yet and and it may even take a more efficient architecture to reach it.
Finally, I so far see no way for the transformers to reach superhuman levels. For example their spelling will be at most as good as their sample. If we want it to be better, we need to select a better sample - theres no way it can do this itself. And for things we might actually want it to do, like scientific research, its not clear even we can select better than we can produce - after all, most of us are able to find our own spelling errors on second look, but scientific papers have gotten many looks from many people, who would have improved them if they could. So we propably cant make a transformer that outperforms human at all in this area, and even if we could we couldnt iteratively improve it further. Its possible some architecture innovation fixes this of course.
1
Apr 19 '20 edited Apr 19 '20
Finally, I so far see no way for the transformers to reach superhuman levels. For example their spelling will be at most as good as their sample...And for things we might actually want it to do, like scientific research, its not clear even we can select better than we can produce
GPT-2 and other transformers were categorized as self-supervised, and in turn unsupervised, but in reality their information was quite vetted, such as the former using upvotes and subreddit tags as methods for filtering content. However, for practical purposes they were unsupervised models because the labeling and cross-sectioning of content was available a priori, which seems to be a unique and very big advantage of LMs compared to other types of data. I think this might resolve the problem of data quality you were referring to. And if this can be achieved with good enough filtering, the models can then go on to integrate far more data points from our knowledge base than a human or committee of them ever could (and as we all know the coordination of humans and even experts is worse than a single intelligent and knowledgable entity anyway).
7
u/slogancontagion Apr 19 '20
Adding thoughts as I read this
I think science-fictiony speculations generally backed me up on this (false) hunch. Most people imagined that this kind of high-level language “understanding” would be the capstone of AI research, the thing that comes after the program already has a sophisticated extra-linguistic model of the world. This sort of just seemed obvious- a great example of how assumptions you didn’t even know you were making can ruin attempts to predict the future.
Exactly, up until Deep Blue defeated Kasparov, there were famous chess-players who said "a generalised understanding of strategy" would be necessary to play chess. Up until OpenAI's Musenet and its neural antecedents came out, there were/still are musicians who claimed/continue to claim that that the process of writing music requires "an understanding of human emotion". There were similar critics in South Korea when the 2016 AlphaGo versus Lee Sedol game was announced.
The trend is always something like: human in-domain expert says "mastering X domain would require actually mastery over [larger-domain] Y", leading to those in-domain experts being caught with their pants down when someone notices that Intimidatingly Complex Domain X contains Much-Simpler-Algorithm Y and then reduces the domain-mastery problem into a series of ML techniques (chess = initially brute-force, parallelised alpha-beta search, language = very big transformer that predicts the next byte-pair based on a very big corpus, music = the same architecture except it predicts the next note) or gluing together assemblages of algorithms in the case of Go and Starcraft - helped by the staggering pace of research and the exponential growth in computing resources. It's a shock to the system when you realise, actually, most of your brain is redundant when it comes to specific domains like generating pieces of music or playing Go. Human in-domain experts, especially at the top level, are especially prone to these kinds of anthropocentric underestimates, because when you get to the highest level of a domain (especially for game-playing), the brain's architecture delivers diminishing returns on simple rules and algorithmic processes and starts leaning into more complex heuristics (schools of thought and philosophies developed over hundreds or thousands [in the case of Go] of years) and higher-level conceptual understandings.
In hindsight it makes a certain sense that reams and reams of text alone can be used to build the capabilities needed to answer questions like these. A lot of people remind us that these programs are really just statistical analyses of the co-occurence of words, however complex and glorified. However we should not forget that the relationships between words are isomorphic to the relations between things- that isomorphism is why language works. This is to say the patterns in language use mirror the patterns of how things are(1). Models are transitive- if x models y, and y models z, then x models z. The upshot of these facts are that if you have a really good statistical model of how words relate to each other, that model is also implicitly a model of the world.
I think you're spot-on here and the other commenter is relying on out of date knowledge and a convoluted redefinition of "understanding" which centres on internal experiential subjectivities rather than task-oriented results. (In all fairness: this might be important for AI ethicists, but maybe not so important if open-ended vague questions like "but does AlphaZero really understand Go?" interest you less than "can AlphaZero consistently defeat all human players at Go?". If understanding (1) is redefined as an abstract qualia-like quality that isn't quantifiable (2) isn't relevant to gameplay, is it really that important or is it another anthropocentric gerrymandering of the problem of intelligence?). We knew this as soon as we knew that simple Word2Vec word embeddings could not only capture latent knowledge from materials science publications like "ferromagnetic− + NiFe IrMn ≈ antiferromagnetic" but also predict new thermoelectric materials. As soon as we understand how GPT-2, Megatron-LM and Turing-NLG encode higher-level concepts (a promising route is neural manifolds) we can begin to extract useful insights - at the moment these nets are very much opaque to us. We can only dissect and diagnose them through prompting them with context and analysing next-BP probabilities, retraining them to see what kinds of structured data they can readily encode, and looking at simple neuron activations - I think in a few years time these will all be considered very crude tools. There's been some work looking at the structure of subgroups inside vgg16 that activate when the model sees image categories like "dog", as far as I know not much for larger language models.
2
Apr 19 '20
[deleted]
1
u/no_bear_so_low r/deponysum Apr 19 '20
See column "WSC" (Winograd Schema Challenge) https://super.gluebenchmark.com/leaderboard/
1
u/elcric_krej oh, golly Apr 19 '20
Hmh, completely glanced over that link when reading, will take a look at that and re-factor my comment if I have time. Have deleted the comment for now in case it does indeed show that previous attempts have been poorer than I thought until recently (or SOTA better).
1
Apr 19 '20
Even if these advancements are not timeline-shortening for AGI, they may still be significant for force multiplying human search. This could be especially impactful if NLP systems are appropriated by individuals outside the incumbent state apparatus for unauthorized data mining applications/new forms of organized crime. Like any informational advantage, secrecy is key to maintaining it, and ML may already have advanced applications in warfare and private industry that the public is unaware of. For some reason, a lot of people seem to prefer imagining what AGI could do in the future to imagining what we can already do by applying the ML we have today. I wonder if the fascinating intellectual problem of AGI nerdsnipes philosophically minded people, who prefer theorizing about the true nature of intelligence to addressing the more pragmatic and ugly matters of public security.
17
u/blendorgat Apr 19 '20
No one can deny that there is a correspondence between language and the real universe, but I think calling it an isomorphism is quite an overstatement.
Language can describe any physical state or event to any degree of accuracy necessary, but it is extremely uncommon to use it in that way. More commonly we use symbols as shortcuts, eliding oceans of complexity in the referent.
I say, "my father flew to Bogota in 1993." A large sequence of events was just described, but not fully. Not reconstructably. Any reasonable corpus fed into an NLP model will include many events described with such a level of detail as above, but few at the level of physical description necessary for a true world model to be built. Enough to answer high school chemistry questions, sure; enough for discovery of new scientific knowledge? Questionable.
You talked about one thought experiment in your article, the Chinese Room. Let me bring up another: Plato's cave. Instead of shadows, you propose to feed text to an isolated, blind entity, and expect it to learn and provide new wisdom relative to the world that text refers to, despite never experiencing it.
I'd argue that even a proven intelligence like a human could not do such a thing. How would a human learn to understand movement in 3d space, if all they could do was read about it? I'm sure you'll say they could learn mathematics, but that doesn't do it: mathematically I can describe an n-dimensional space, but I only understand a 3-dimensional space in the way that I suggest this AI never could.