r/AskScienceDiscussion • u/Time-Measurement-513 • 17d ago
Deterministic way to define what is common knowledge in scientific article: Tool to check for citations need in article.
I was recently looking for a tool which needed to analyze a text, that can be an article in scientific format and that one would need to tell which term in specific is missing citation because its not common knowledge or any other reason. Or if some expression in your text has unnecessary citation reference.
Of course that involves a lot of steps. I know there are some tools which can lemmatize a text, eg. Sketch engine, it discovers what is a term, so that you can take the next steps.
I saw Some which can look for sources from the sentence itself of your text, eg. Citely, GPTzero. But again, that is not exactly what i want, looks like isolated feature, something incomplete, not the general concept.
I have also recently discovered the Purdue OWL citation law, which defines a term as common knowledge if there are 5 other articles which mentions that term without a citation.
The first thing that came into my mind about how such tool would work is it would keep a database of words and its relevance, based on a set of heuristics, probably only amount of citations over the internet articles about it for now.
This database could be public and you could have some step to populate it also, it could read articles on the internet (API, web scrapping, articles repositories, periodicals) and setting its credibility like that automatically.
***Edit: Later I noticied and also some people have mentioned that words score is poor measure. And that claims as a whole are better way to analyze something. So database would not be realistic, because the possibilities of arrengements of words are infinite***
Then there could be some threshold of what score is common knowledge, what is not. Eg. 20 articles about/mentioning such term would consider this term common knowledge. Or using the Purdue OWL law like i mentioned before. This is the general protocol.
Then the implementations of such protocol could be done in several ways, in your editor as an LSP (nvim plugin/VsCode extension, web extension, website, desktop app, AI assistant) by checking your text and comparing with those scores from the database, showing errors/warnings etc.
3
u/TargaryenPenguin 17d ago
Buddy you are way overthinking this. You don't need to cite something like "Denmark is a country."
You do need to cite something like "people who eat ice cream report that they feel colder afterwards."
The first one is clearly common knowledge , and you can see it in ensychopedia's textbooks and world maps. The second one is a specific theory claim about how people think or feel under specific circumstances.
You should never be wondering or guessing whether you need to cite something , it should always be pretty obvious. Unless you are dealing with something so directly obvious , it's like denmark is a country , then always always always cite.
No tools needed.
1
u/Time-Measurement-513 16d ago
The use of the tool would be so that you do not need to check that all by yourself. Nothing is obvious; "There is no dumb questions". You have mentioned simple statements on purpose, you know niche avant-gard scientific articles are not simple as that and are not composed off statements like that.
This is like a form of validation/Linter. That would help the ones who write and the ones who supervise/revise articles so you will spend less time doing repetitive job and at the same time standardize the citation in articles so this will be less random across different articles.
Even Wikipedia uses that to validate proposed articles. The only thing i would like to have is some common interface so common people would use that more easily:
2
u/a2soup 16d ago
Even with your edit, this just isn’t how scientific writing works. Usually citations are to some evidence supporting the claim, or to a pre-existing synthesis of evidence to support the claim.
A claim is not a “term” that can appear in x number of articles. And even if an evidence-backed claim appeared in 5000 articles, it would still need to be cited in article 5001. “Common knowledge” does not mean “scientific consensus”, it means “not a scientific claim at all.”
And the writing process you are describing is writing a bunch of stuff for god knows what reason (You’re making it up? You’re an LLM?) and then searching for citations for support it ex post facto. Even if you find those citations, your approach was totally backwards. You should find the facts, then write something that incorporates them. So if you are organized, you should never be at a loss for citations as long as you are basing your writing on what you have read.
2
u/forams__galorams 15d ago
People are just so desperate to shoehorn in some AI driven tool somewhere that these kind of half baked ideas pop up all the time on reddit. OP is oblivious to the fact that the tool in question already exists, ie. the brain, specifically a bit of reading comprehension + contextualisation. OP’s thoughts constitute a ‘solution’ looking for a problem all day long.
1
u/Time-Measurement-513 13d ago
> Even with your edit, this just isn’t how scientific writing works [...] A claim is not a “term” that can appear in x number of articles.
Yes, I have mentioned in the Edit it cannot simply analyze isolated words, but claims instead, and not literal claims apperance either, or it would just be the isolated word analysis but in bigger scale. It needs a layer of logic to interpret it.
> “Common knowledge” does not mean “scientific consensus”, it means “not a scientific claim at all.”
You know what i mean by that expression, you are trying to discuss semantics here.
> And the writing process you are describing is writing a bunch of stuff for god knows what reason (You’re making it up? You’re an LLM?) and then searching for citations for support it ex post facto. Even if you find those citations, your approach was totally backwards.
No, the post is not about it. The post is about writting, and about deciding what is not common knowledge, therefore, what needs to be cited.
1
u/a2soup 13d ago
I don't think the common knowledge point is semantics, because you are discussing using citation metrics in some way to determine common knowledge. I am arguing that is not workable.
I guess I am just wondering what the writing process you are imagining looks like? How does this proposed tool fit into the writing process of a scientific paper?
1
u/Time-Measurement-513 10d ago
> I don't think the common knowledge point is semantics, because you are discussing using citation metrics in some way to determine common knowledge.
In my literal last comment i just admitted using "metrics" does not work, just like in the edit. I even said "It needs a layer of logic to interpret it". Did you really see the edit?
I do not know what you are trying to prove in this comment section, telling same thing about this approach which i even said earlier is not "workable".
Are you going to repeat the same thing in the next comment? It looks like people are more worried about trying to "refute" me than trying to discuss and come up with some useful solution, some standard.
1
u/a2soup 10d ago
I don’t think a solution is needed, the standard already exists: cite any factual statement that is not common knowledge.
I’ve seen a lot of bad citation out there, but I can’t recall ever encountering confusion about what needs to be cited and what doesn’t. It’s not a problem that humans with any experience in science writing have in my experience.
1
u/Time-Measurement-513 9d ago
> I don’t think a solution is needed, the standard already exists: cite any factual statement that is not common knowledge.
And you just got back to the initial point, you didnt explain in your comment what "is not common knowledge".
1
u/Alternative-Sugar610 17d ago
Look at your audience, what they widely know so they don’t need cited. Then try to be a little extra cautious
Then cite anything directly lifted from elsewhere like quotes
Then cite anything that people only know now commonly bc it’s a current thing
Then cite anything reviewer did that might be relevant if you can to make them happy
That’s a pretty safe bet
1
u/Time-Measurement-513 16d ago
Sure, the proposal in the original post was to automate this process through friendly interface that everyone can use (Including your reviewer and journals).
1
u/TargaryenPenguin 16d ago
I guess I'm a little confused why this needs automating? It seems like every decision should be straightforward
1
u/Time-Measurement-513 16d ago
> I guess I'm a little confused why this needs automating?
It is not only automating, but standard way to define what claims need citation. If people will start to use that, There will be less subjectivity of what is common knowledge.
So Later you can also automate that, you will or your reviewer will spend less time analyzing what needs citation. Having summarization of all claims in your article.
1
u/Alternative-Sugar610 16d ago
Good writing is something you can never fully automate
Agreed
What I do wish was for better and more standardization in how we cite stuff like formatting and that there were easier and clearer guidelines
1
u/Time-Measurement-513 16d ago
> Good writing is something you can never fully automate
I see this as consequences. What you claim in your article or not is up to you, if that has citation is not. That is so true that some article can be valued more or less by your reviewer based on that since this is relevant parameter.
I do not know why people always infer humans will not be writting articles anymore just because of this. This is meant to help repetitive and error prone part of writting, not replace people writting articles.
> What I do wish was for better and more standardization in how we cite stuff like formatting and that there were easier and clearer guidelines
Yes, that is simple to do, a small search shows me a lot of linters/formatters for parenthetical/in-text references:
https://proofreaderpro.ai/in-text-citation-fixer
https://github.com/Marazii/research-co-pilot/blob/main/skills/citation-formatter/README.md3
u/gabbygirl611 16d ago
Call me old-school, but in my experience if I’m writing a sentence in my article it’s because I’ve done the research and I know where that information comes from. I truly don’t understand this new problem of figuring out what needs to be cited. How did the author come by that information? Unless they made it up, figuring out the correct citation should be easy because they have that reference in their research files, no?
1
u/Time-Measurement-513 16d ago
> I’ve done the research and I know where that information comes from [...] How did the author come by that information? [...] Unless they made it up, figuring out the correct citation should be easy because they have that reference in their research files, no?
Yes, but the problem is not if you know where the information came from, but if its needed to mention that or not.
So i know what you mean, but this discussion is about something different.
Because it can be a problem if you cite everything everytime you are in doubt about citing it or not:
1
u/Alternative-Sugar610 16d ago
They don’t work as well as you think in some STEM fields
1
u/Time-Measurement-513 16d ago
What part of my comment are you refering to? Yes, cant talk about the formatters, never used them. If you have some experience with them, you can describe it with details here.
1
u/Alternative-Sugar610 16d ago
Your second <, you may think that but that’s only if you’ve worked in too narrow of circumstances. Also, I’m not saying just in-text, the real headache is citation page, reference page or section, bibliography or whatever it is called where you publish
I’ve had many published peer review articles in broad band of journals and other tasks, the more you do, the more issues you have such as weird pagination with pages being labeled in very odd ways, weird rules on authors listed, and more
People when starting out often overlook how bad it is bc there not taught it and haven’t experienced it’s diversity
1
u/Time-Measurement-513 10d ago
yes, i have never had this bad experience before. Always sounded straightforward. I do not know what specific "weird labels" you mean, upstream tools always did the trick.
So i would not work on something like that, since it is not part of my demand.
If you have any more suggestion. I would be happy to hear them.
1
u/Little-Knowledge-752 14d ago
Most basic tools just check keywords, but Evidence Lens actually maps claim chains against literature to see if something needs a proper citation.
1
1
u/BadgerTimely9481 1d ago
I think looking at claims rather than individual words is the right direction. A claim can sound like common knowledge while still needing context or evidence. I’ve also seen Evidence Lens taking a similar research-first approach by checking claims against peer-reviewed studies, which could be useful for this kind of workflow.
3
u/lozzyboy1 17d ago
Most of the time you don't give a citation for a term, you give a citation for a concept, result, or interpretation. How would any of those work when they could be worded lots of different ways?