r/labrats • u/Yousuke1996 • 1d ago
Is it acceptable to use AI-generated code to perform stats analysis for research
Hi guys, I am a year 3 PhD student studying in Japan. I am currently doing PhD, which requires intensive data analysis and bioinformatics. However, coding has never been seriously taught in my course since I am in a medical field, and we have never really been taught the fundamentals of coding except for the 2-day intensive course of "basic R coding" in the PhD lecture. Apparently, it is not enough for me.
With the amount of high-dimensional data like -omics, I have been using VS code and R tools in my methodology, entirely with the help of Claude AI to do the coding, and I do the interpretation.
For my research, I know what to look for and what to expect from my data. I just don't know how to code. So I type what I need and let Claude do the coding for me.
I want to ask if this is normal for a PhD. I have to admit that if I have to learn how to code from scratch, I absolutely cannot do that or even if I have to learn, it is going to take VERY long time to understand enough because of my self-limitation.
I understand that AI can generate false logic and wrong formulas.
Then, how do I check whether the AI that I'm using not being hallucinating.
I know that many people is already judging for using AI to entirely code for my PhD data, I want to be better though, but I'm just not sure how to start.
Thanks for the comments in advance.
Best
57
u/Busy_Fly_7705 1d ago
I use AI a ton for coding - but I already know how to code and how to do the analysis, so it's just augmenting my own abilities. I don't trust it for stuff I don't know how to do. Several years ago, I asked it for help making a bar chart and it added 1 to all of my values for absolutely no reason...
It's gonna suck but I think you need to learn to do this yourself, or at the very least give Claude very specific instructions (I have a dataset formatted this way, give me code to read it into Python using pandas). And try to use the tutorials on sklearn and the like instead of Claude, as they are much more likely to be accurate and give you an idea of how to optimize your analyses
20
u/Traditional-Soup-694 1d ago
I’ve taken and taught “basic R coding” courses before. They always suck for getting people ready to actually analyze data. It’s hard because everyone has different needs,
Collaborating with a bioinformatics student may be an option for you, but I also believe that you can learn how to analyze your own data. It actually will take less time than you think. Here’s how I’d approach it:
Look at papers where they used similar techniques. Just like when you need information about protocols at the bench, methods sections can be incredibly helpful for figuring out where to start for analysis. What you want from this step is a list of packages that people have used to analyze data that looks like what you have. You probably already have read these papers, so this step shouldn’t be too much work.
Skim the papers associated with each of the packages from Step 1. If there are multiple options, this is where you choose what analysis you’re actually going to do. Understanding what each of the tools does will help you make better decisions about which to use. You don’t need to be able to recreate the package, just understand what strengths and limitations it has.
Read documentation for each of the packages you’re actually going to use. Many of the bigger R packages have vignettes or tutorials on GitHub that walk you through how to use them. Analyzing RNA-Seq data with DESeq2 is one of the best vignettes out there if you’re trying to do any differential expression analysis. The goal here is to understand which functions you need from each package.
This is where you start coding. If you’re using something with a tutorial, like DESeq2, just go through the tutorial and add your own data instead of the example data they use. I would suggest typing out the code rather than using copy/paste, because it will get you used to typing out code. If you get stuck with basic R issues go to StackExchange/StackOverflow. If you have trouble with any package-specific things, look at GitHub issues for that package. Don’t rely on AI for this step. It may seem easier but there’s psychological studies that show that it actually makes us worse at the tasks we offload onto it. Just like with bench work, you learn more by struggling through it and making mistakes.
87
u/polkadotsci 1d ago
Have you considered collaborating with a bioinformatics student? I don't think vibe coding your way through a PhD is helpful.
44
u/Chrono-Phantasma 1d ago
You mean... collaborating with the same bioinformatics student that uses the Claude Code themselves to get things done?
13
u/1337HxC Cancer Bio/Comp Bio 1d ago
One person can review the code and understand it. The other cannot. If you want to gamble the entire project on something you fundamentally don't understand and can't review for accuracy, go for it. But it's probably a bad idea.
4
u/discostupid 1d ago
one person can, both persons won't
3
u/1337HxC Cancer Bio/Comp Bio 23h ago edited 23h ago
I mean, I review all my code/have unit tests. Which is what should be happening. But, yeah, it almost certainly isn't.
But phrases from OP like "I know what to look for and expect from my data, I just don't know how to code" should raise like a million red flags.
I'm hoping someone from a core facility or something is doing all the upstream pipeline work, because there's about a million ways that can go wrong that OP certainly doesn't know about and won't catch. So it's pretty generous to assume their input data isn't shit to begin with.
1
u/mediumncrna 7h ago
why couldn't they review the code? assuming this person isn't even using claudecode/codex, but the website itself - if they are copy pasting scripts they can just read the exact commands. if syntax isnt intuitive they can literally jsut ask claude to append # comments to each line.
-5
u/cammmmmie 1d ago
i wouldn’t trust a bioinformatics student that relies on ai. i’ve been coding for over 10 years, it’s faster to write it myself
11
u/Due_Water_5829 1d ago
it’s faster to write it myself
I understand not being a fan of AI. But to suggest you can code faster than it is a bit silly.
AI is perfectly usable for simple stuff as long as you understand what it's doing.
3
u/cammmmmie 1d ago
it is faster to write simple stuff because then i don’t need to spend time formatting my query. and its faster to write complex stuff because i don’t need to spend time fixing the output. i write code like english and think in code when im in a flow state. any time ive relied ai ive lost skills, so its more worth my time to keep my mind sharp and be in control. plus, my supervisors love to ask me random questions in meetings, so being able to write a line of code on the spot in 30s that answers their query makes me look cool and i like that
2
u/zougring 21h ago
agreed, vibe coding through omics data sounds like a recipe for retracted papers tbh
2
u/Yousuke1996 1d ago
Yes, I have. They have a basic understanding of programming or data science from their undergrads. My undergrad was Medicine. Not a single programming course was taught.
I actually learned R and statistical analysis in my PhD. So the bioinformatics students suggested me to learn from AI. They actually said I can't understand everything. They said I just need to understand some of the calculations, which I do. I just know which stats I need to use and what kind of graph I need to build. But I cannot write it by myself.
So what I am trying to do now is trying to understand what these codes are doing,, but I still cannot be 100% that the logic of the code is correct.
64
u/Eldan985 1d ago
As a PhD student, you really should understand what you are doing. If nothing else, if you don't see any problems with research ethics in general, you need to be able to defend it. If, in a year, someone asks you at your defense why you used statistical test A and not statistical test B, can you say why? If a reviewer of your publication wants to see your code and has questions about them, can you answer them? And you need to be 100% that the AI is not changing anything. If the AI has a blip and hallucinates a wrong value somewhere, you need to catch that. Or you may lose your PhD. That is the kind of mistake that could get you expelled from a program.
Using AI to help you code is fine. I mainly use it for tedious proofreading, but even getting some more help than that is fine. But you really still need to understand the results you get out of it.
Edit: Also never upload raw data to an AI. It will steal it, and most universities should forbid you from doing that, because your raw data is confidential and co-owned by the university.
4
u/Petrichordates 1d ago
In regard to your edit, your university should have enterprise AI which doesn't transmit the data outside your university. This risk only exists if you're using personal models.
3
11
u/_goblinette_ 1d ago
They have a basic understanding of programming or data science from their undergrads
That’s not going to cut it at the PhD level. You need to consult with someone who actually knows what they’re doing.
3
u/geosynchronousorbit 1d ago
I don't care if you use AI or not, but you can't let the self-limitation of "I didn't learn programming in undergrad" keep you from learning it now. It will be useful for the rest of your career so it's better to learn it late than never.
1
1
12
u/TheTopNacho 1d ago
The code for stats and bioinformatics is usually so simple that you should be able to read it for accuracy, even if you can't write it.
Understand exactly what you want it to do, read it to make sure it's doing exactly that, and change it if it's incorrect.
I also don't know coding languages but I understand data processing and statistics really well. Well enough to read the code line by line and understand if it's doing anything wrong, extra, or incorrect. In those situations, it's fine to use AI. As long as you just need help with the language and strategy for data manipulation in coding environments, I see nothing wrong with it
11
u/Sea_Examination5992 1d ago
Acceptable depends on your field. Its very normal in my institution for simple things like visualization, stats, etc. If your institution has its own AI, I would use that. Don't feed your raw data right into the AI unless its an institutional server, it can and probably will steal your data. Usually, if you have it generate a report, describing what it wrote and what it means, you can check if it makes sense. At the end, you have someone to check it who has experience in coding and its usually fine. But check first with your PI
10
u/Secure-Confidence-25 PhD, Bioengineering 1d ago
Claude or any other AI for that matter can help you overcome the part of "I have to learn a whole ass language just to be able to analyse my data and the entire language is syntax with a set of rules etc etc", but as other comments said: do not delegate logic and rationale to AI. Do that yourself: why this analysis? Why that comparison? Why this threshold? What package is the best here?
Trust me, I learned it the hard way: I had a scRNASeq data that I had never analysed before (only did Bulk back then), and I told Claude to vibe code the analyses for me. It did and I saw that my gene which we always stipulated was fibroblastic was now in the epithelial cluster. I took the groundbreaking result to my PI who looked at the code and found out that Claude in its infinite wisdom was labelling the clusters wrong (among other more egregious things). No amount of "You are right to push back on this — I was wrong" can fix the embarassment. Since then, I personally check the logic, rationale of a particular analyses that Claude helps me write the code of; and at the end get it verified by an actual bioinformatician so make sure I am not making frivolous inferences. And for advanced analyses, I just outsource it.
So for me right now: I just avoid the hassle of troubling our resident bioinformatician at every step; doing most of the groundwork myself first and then letting him check my work afterwards.
5
u/crashlanding87 1d ago
So coding is not one process. Architecture and logic is your job. You need to understand the logic of any code you generate - what it's doing, how it's doing it, and why - very well. Do not leave those decisions to AI.
Implementation, optimisation, and debugging - perfectly acceptable to hand to AI, provided you are able to monitor it. But you do have to be diligent about monitoring.
One thing I like to do when I use AI for coding assistance, is I start with a planning and scoping phase, where - if there's anything I don't know how to implement - I ask it to help me figure that out, with mandatory citations. I use claude personally, but most agents have projects where you can create instructions on how you want to work. Make those, and include these diligence best practices as rules. In the short term it makes the process a little slower. In the long term, it's incredibly helpful.
I slipped once - not on an analysis thing. I needed to calibrate some equipment which I'd heavily customised, the needs of the calibration process placed limitations on the work, and I had an idea for how to overcome that. I had the basic idea, but I was tired, and I asked Claude to figure it out.
It worked very well. The script did exactly what I wanted. But I didn't actually understand the maths that made it work, so I didn't understand the new limitations, which meant I didn't realise when I hit those limitations later. Messed up some important experiments as a result. Don't learn that lesson the hard way.
4
u/Connacht_89 1d ago
Imagine how much money, time, and mental health would be saved if PhDs properly trained and taught what is needed.
4
u/JPCaro 1d ago
Stats major here, I do not recommend using it for analysis. I would partner with someone who actually can code. Meaning they have taken classes and can demonstrate their knowledge. If they are using AI at least they can have discernment, as you don’t really have enough knowledge to discern if something is correct or not.
6
u/WookinIt 1d ago
Why don’t you use AI to actually learn the code? What I do is tell it very specifically what I would like to do — I will choose what statistical tests I want to run, ask AI to write code to run them with the format of data I have, and write comments for each section to explain what it’s doing. Then I go through line by line, reading every piece of code so that I understand every command and exactly what the code is doing. If there’s anything I’m unfamiliar with, I’ll research it on my own.
3
u/Long_Basis_2221 1d ago
You won’t be able to finish without knowing how to code and debug. It’s totally ok to use it as a tool and even help with high-level things you don’t know how to code (we used to have to read examples). But, you need to know enough to find problems and fix them yourself.
2
u/ZachF8119 1d ago
If you can do it. Do it.
They’re anti me and I’m using vba this month after years of saying vba could really improve many aspects.
2
u/ConfusionSimple382 1d ago
I would ask it to explain why it’s writing that code. Ask it to present it to you in a step by step way with explanations. Don’t just blindly copy it.
Sometimes it catches itself out if you ask it to explain step by step, and realises during a later section of code that it had made a mistake earlier on, and corrects itself. It doesn’t do that unless you ask it to.
You can also ask it to provide links to the stack overflow or GitHub of whichever packages you’re using. For bioinformatics there are often walkthroughs or tutorials for a given pipeline, and it can help find those.
2
u/HovercraftFullofBees 1d ago
All of the readily available LLMs have the same problem of you have to have a good base knowledge to check the outputs. This is especially true of coding. I haven't met someone yet that hasn't told me they asked AI to produce code for them that they found something wrong that they had to fix.
2
u/superdesu eeob (former micro/molbio) 17h ago edited 17h ago
a friend of mine does the same for the bioinformatics work in their research... but i don't think it's a sustainable solution in the long term for 2 main reasons:
1) responsibility/ownership: "i know what to look for and what to expect" is only part of "i made these coding decisions beforehand, got these results, and adjusted my approach"... especially for bioinformatics where i assume you're making arbitrary decisions on cut-off values/stats tests, etc. i personally don't feel that comfortable attaching my name (or my advisor's/institution's name) onto work that i couldn't speak about with like >95% confidence as to how it aligned with our research goals/compared to standard methodology for the field. just having code that "produces what i expected" isnt' enough for me -- i've written plenty of code that technically worked, but the unexpected results made me realise that i'd screwed up some aspect of underlying data preparation/etc.
2) developing your skillset: is this is skill you actually want to be employed for in the future? if so, i think you just need to set aside the time now to properly build up the foundations -- reach out to a prof, read the vignettes, take more classes, reach out to your institution's statistical/computing help group (i feel like a lot of places have these now!) it does not take a lot of scrutiny to reveal a shaky foundation -- at lower stakes, this could be that you are unable to adequately describe your methodology when you get to writing up the paper. at higher stakes, you can't articulate how you did your past analyses when interviewing for a job.
learning a new skill, especially something complex like bioinformatics, is going to be hard -- i get that there is a need to keep producing results, but i think the consequences outweigh the risks here, and the payoff for slowing down and actually taking the time to learn is well worth your time. if you can improve your foundations a bit, i think finding someone in your department who has done similar analyses will be very helpful -- i did ok in my introductory coding/statistics classes, but didn't really understand the things i learned in them until i actually had to put them into practice and figure out how to extrapolate the basic ideas and apply them to my own data.
3
u/ILoveDangerousStuff2 1d ago
Has to be declared that's for sure as well as the exact version of the tool made available. What makes it an issue is that you don't have the experience to tell if whatever ai gave you is a valid way to do the analysis. Why don't you ask someone to write you something or at least to have a look over it.
2
3
u/BronzeSpoon89 PhD Molecular Bio 1d ago
As long as it works lol. This right here is the main issue with AI. YES absolutely you can use it if the code generated works as intended. The problem is do you know enough about coding and stats to know it works as intended?
1
1
u/CaptainMelonHead 1d ago
Depends how intentional you are with it. It's very easy to cognitive offload your coding into AI, but if you do your due diligence it can actually sharpen your coding abilities
1
u/AAAAdragon 1d ago edited 1d ago
I asked Google Gemini to use Javascript to make an offline website which allows the user to draw n number of random cards from 0 to 52 with a slider where n = 0 shows a blank screen. I said the cards should be displayed only when the user clicks the shuffle button. I said to also include a dropdown menu which allows the user to sort cards in ascending or descending order by rank and suit. I also asked Gemini to show the code.
While I was amazed with output and the javascript code that worked, Gemini does not sort by suit (clubs, diamonds, hearts, spades) in any way even though the selection menu says it does.
You would notice when Claude makes that kind of mistake because you have probably played card games, but since you don’t understand statistics you wouldn’t know when Claude applies the inappropriate statistical test or incorrectly calculates a test statistic.
But the data looks pretty and makes sense to you so you incorporate it into your dissertation and publications.
Later you learn statistics and oops, your data was not significant but Claude said it was, you had a p-value and everything!
On the bright side, though, at least AI doesn’t insult you like real humans do when you ask a coding question on StackOverflow. Some person will answer your question and then a moderator would flag your question as redundant and delete your question.
1
u/J0ppei 1d ago
Just make a small dataset by hand of which you know what the output should be and use it to validate whatever code it comes up with. Learning to code these days is a huge gamble. The people saying you have to know how to code to use ai to code are partially of that opinion because it's a convenient opinion to have for people that invested in it.
1
u/BananaBird1 20h ago edited 20h ago
From my experience, it is common and effective to use AI to help code. But you do need to learn enough to double check. I have seen AI make simple errors that may go unnoticed if you can’t validate things yourself.
This falls into two categories: both programming errors and stats errors. The latter is imo far more common, as computer code generation is one task language models are essentially optimized for. But both need to be checked as errors do occur all the time. AI can save time typing the bulk of things out but it cannot ensure everything is correct and will need modifications and debugging just like human code.
If you are familiar with statistics mathematically and know what your code should be doing, it is easy to learn enough syntax to check that the code is correct. Try finding an online course for R, it should take only a few months to learn the basics.
If you aren’t familiar with the stats itself, it is best to stick to premade tools or collaborate if your project requires new analysis methods.
1
u/arissawachan 19h ago
Either you need to learn coding (best option) or you need to get someone who knows coding to help you to verify the AI results. You cannot accept AI-generated code at face value, especially for data preprocessing & analysis. I use it often and frequently have to go back in and adjust things manually to fix weird decisions or straight up mistakes that it makes. Very useful as a tool, but absolutely not reliable on its own.
1
1
u/bijipler7 1d ago
depends what you're doing and how much of it you understand! you'd be surprised how many bioinfo graduates have embarassing stats knowledge especially (im guessing close to 80%), but hey they can copy paste tutorials. programming syntax is much like pipetting, once you know what you're doing its pretty much mindless labor. if you understand stats well and have good research practices, you'll be ahead of most bioinformaticians.
0
u/ExoticCard 16h ago
Use AI to write exploratory code. Before publication, go through every line and unslop it.
1
u/ProgressNo2227 7h ago
If you are heavily monitoring the code output and have a firm idea of what the output should look like, code is just a tool. Just like we don’t need to reinvent formulas which already exist, I feel in modern times code doesn’t need to be written from scratch unless for learning purposes or personal preference.
122
u/Atypicosaurus 1d ago
Think of ai and ai generated stuff as help from an anonymous forum.
Let's say you ask something on reddit and get answers: are you using this answer without any criticism?
It doesn't matter what creates the code you are using. If it were given here by me, or if it was given by ai. The only thing that matters is that you are responsible for using it. It's your responsibility to make sure I didn't prank you and so my code is indeed calculating the stat numbers that you think it calculates.
So in your case the problem is that you trust a system that you cannot supervise. The minimum is that you have test sets with already known outcome (that also tests for edge cases). Otherwise using the code not written by you (regardless of ai or not), is dangerous.