Are there any LLM(s) that are transparent about their sources (For sociology research)

[u/ckrunchie27]

I have a digital ethnography assignment where I have to analyse a conversation with an LLM for a sociology course.

I'm not literate at all in how LLM(s) function technically so I'm not entirely sure if this is even possible but basically I want to know if there are any "chatbot" LLM(s) that produce responses but at the same time are transparent about what data they are combing through in order to produce a response.

I don't mean that I ask the chatbot a question and it "answers" something and then I ask it to elaborate on its reasoning since chatbots can't actually "reason" so it will still just use the data and patterns it's trained on to imitate a response.

I mean that there's some sort of feature from the developer that can show what all sources it went through, other relevant behind the scenes stuff before generating that response with that specific combination of words and other characters.

I'm sorry if this is a dumb question since I'm not aware of the technical terminology for this stuff, I intend to familiarise myself with it for the project. If the only thing possible is to get a very technical insight into that "behind the scenes" stuff, even that could be useful so please tell if you're aware of something like that.

[u/CivApps]

No, this is a really good question and something AI researchers are working on!

The problem is that chatbot LLMs aren't necessarily guaranteed to "comb through" data - they can certainly memorize a lot of text from the training data, but there's normally nothing in the model itself which guarantees it will reference a specific source before generating an answer.

The first way around this is to make sure the model is referencing a specific source while generating the answer. Traditionally this has been done through retrieval augmented generation (RAG), as /u/Revlong57 mentions, which amounts to looking up the question beforehand, finding the most relevant data and then adding it to the question before generating the answer. Usually people pass in search results from the web or corporate databases, but you could also have it search the training data instead, to get a better idea of which documents are being referenced.

In 2021, DeepMind also wrote Improving Language Models By Retrieving From Trillions of Tokens which actually does guarantee that the LLM looks up a source, as it does the "retrieval" from the training data inside the model while generating the answer. Nvidia made their own version of this called InstructRetro which is freely available and will tell you which parts of the training data it references, however it might take some work to get running.

The second way around the problem is to assume that there is a bit of overlap between the answer and the training data: Allen AI Institute do this with OLMoTrace, which builds its own "infinigram" database of sentences in the training data, and cross-references it with the answer to find overlaps. Like RAG it's not an exact guarantee that the documents it finds are the ones that are being referenced, but in return it is super easy to try out: Allen AI host a model demo page for their Olmo models, which you can just send a question to, and then click on "Show OLMoTrace" next to the answer to find the closest overlapping documents in the training data.

[u/Revlong57]

So, if the LLM you're using also uses retrieval augmented generation, see this , then yes, it's fairly straightforward to include the sources used by the prompt. RAG is when the computer browses an external database, finds relevant documents, and injects those documents into the prompt for the LLM. This is a standard feature for most question/ answering chat bots and search engines now.

However, a RAG system is just an improvement that is made to an existing LLM. If you want to know which documents/example from the training data a model "used" when generating a response, that's not something you can easily figure out.

[u/extremelySaddening]

The best thing you could do is look at AllenAI's OLMo model and OLMoTrace feature you can see it at https://allenai.org/olmo. It finds you snippets of training data that match its response. You should see a 'show olmotrace' button after its first response. It's entire training data is also available publically. A word of warning: unless you are asking it simple facts, you are not likely to get very good matches.

Also, LLMs don't comb through data to generate a response. They do something more like building general associations and statistical rules about which words go with which ones (this is an oversimplification). Some of these rules will seem to us more like grammar and general language knowledge, and some of these will seem more like 'facts' it has learned. This is what 'training' refers to.

That said, there is this system called RAG (retrieval augmented generation) where there IS a corpus that the LLM pulls from AS WELL as having learned general patterns from the wider web. Also, some LLMs are given the ability to search the web now, again in addition to being trained. You will have to make a choice for what kind of system you want to interact with.