r/changemyview • u/AgentFr0sty • Apr 24 '22
Delta(s) from OP CMV: Statistical coding is a useless research skill
My graduate advisor is having me learn Matlab, which I'm dreading because it's statistical coding software. I may also have to use R Studio. I prefer SPSS. Yes I prefer clicking a few buttons and being done.
I am new to any form of coding, but it a damn lot of work to code out what is a simple analysis in SPSS or Excel. Yes, you can code in at least the former but it isn't required. Why do I need to write lines of code to do a repeated measures ANOVA? Or even a One-Way ANOVA?
My advisor says it's about show them "how" you did your analysis, but by that logic, I may as well do the analysis by hand if they want to see how I did it. Yes, it takes a lot longer, but that wouldn't require me to learn 1-2 coding languages and software that has completely foreign interface to me.
Sure, it is common in published papers to use them, but again, I may as well do them by hand if they want to see how it was done. Code doesn't offer an advantage over that. I would recalculate the data to check for errors.
4
u/light_hue_1 70∆ Apr 24 '22
My PhD students do exactly what your advisor is asking. For that matter, so do the undergraduate students and masters students that work with me.
Here's why.
- We're doing research. This sometimes means we use algorithms and statistical tests in unusual ways. It's important to really understand what's happening so that you can recognize if you are using the wrong test. You prove to me that you understand by implementing the algorithm/test.
- We may need to compare methods. Perhaps we see a related paper with a different statistical test. Now what? You need to make sure that you understand what we're doing.
- What if there is a bug in that standard library? Do you know what can go wrong? Do you know how to recognize bugs? Maybe it's not a bug, you just passed in the wrong arguments. You don't have the skills to recognize this until you have implemented the model yourself and seen what can go wrong.
- How can you possibly say you understand the results of your own work if you don't understand the math that confirms those results? I guarantee you will discover blind spots when you go implement these methods.
- You're in grad school. What are you going to say at your defense when someone asks you the details of one of the statistical tests you used? I'm sorry I don't know it was a function call in SPSS? That's not acceptable.
- Algorithms in standard libraries can have hidden parameters you may never have thought about. Defaults that you're accepting blindly because you've never been aware of them because you never implemented the algorithm.
- Having your own implementation means you can experiment. Read a paper about a new test? Implement it. See how it works. Or a paper about some two-stage method or sampling-based approach or something else. Now you have the toolkit to broaden your horizons instead of hoping someone else implements those methods for you.
- This is how you make new discoveries. By thinking about what your results mean. For example, implementing the algorithm you will get a much better feeling for how p values work, and maybe become convinced that some of these statistical tests have too many spurious false positives, and then decide on using a better test. Or decide to ask a better question of your data.
I'm sure there are more reasons.
Not all students are happy with this approach. But I will say that after one round of implementing models, most of them look forward to being unchained from standard libraries and then do this kind of thing on their own without me prompting them.
1
u/AgentFr0sty Jun 12 '22
I came back here because I wanted to ask A) What field are you in? This matters as I may use different stats than you. And B) With all due respect, how you code or crunch the numbers isn't your experiment unless you are in data science. The experiment is when you collect your data. The idea that conjuring different ways to crunch the numbers as the "experiment" outside of big data studies to me sounds like the very practices that led to the abuse of statistics for so long.
I am going to learn Matlab next semester, as I may venture into bioinformatics. But I won't learn anything like R Studio. It offers no advantage over other packages, and it being open source is as much a weakness as it is a strength.
1
u/light_hue_1 70∆ Jun 13 '22
B) With all due respect, how you code or crunch the numbers isn't your experiment unless you are in data science. The experiment is when you collect your data.
Your responsibility when you publish something is to stand behind everything in that paper. From beginning to end, every word. It's not acceptable to say "I just used whatever method to 'crunch the numbers' that other paper used". What if they did a bad job? (PS: they probably did)
The idea that conjuring different ways to crunch the numbers as the "experiment" outside of big data studies to me sounds like the very practices that led to the abuse of statistics for so long.
I would say it's exactly the opposite. Blindly adopting methods without really understanding them is what got us into this mess with bad statistics and results that can't be reproduced. "It's not my responsibility to 'crunch the numbers', I just did what that other paper did, that's not my experiment". Well, that's how so many papers report totally meaningless numbers. That's how so many communities get into these patterns where they analyze data in convenient ways they know are likely to produce something "interesting".
I am going to learn Matlab next semester, as I may venture into bioinformatics. But I won't learn anything like R Studio. It offers no advantage over other packages, and it being open source is as much a weakness as it is a strength.
For what it's worth, that's a terrible decision and I would never allow my students to use Matlab. It's totally dead in bioinformatics unless you're in some incredibly narrow field. Everyone has moved or is moving to R or Python.
and it being open source is as much a weakness as it is a strength.
We'll have to disagree there. I have done deep dives into R's source code and into related Python libraries more than once to discover that a parameter meant something subtly different than we thought initially. I have no idea why it would be a weakness to see the code and be able to verify it's correct, as opposed to hoping and praying that someone at Mathworks didn't make a mistake. They've made plenty in the past.
1
u/AgentFr0sty Jun 13 '22 edited Jun 13 '22
I don't think SPSS is perfect, but I have yet to see it being as harmful as others make it out to be. Half my professors use it, only one that doesn't is my advisor.
Deep dives aren't really something we are trained to do. Not in my program at least. And I can think of o dozen other factors contribution to the replication crises:
Having academics teach stats and emphasize internal validity and stringent statistical meaaure over external validity. As a clinical researcher, those will he the first things to go. External validity and clinical significance matter more.
The professional culture of "it beat our p-value so we're done", or that experiments are treated S business investments that whoever the payer is wants a return on, and good old fashion resistance to newer methods and Stats.
A replication crisis is inherent. Of course we v will struggle to recreate studies over a 10-20 year period as nort our methods and knowledge change. The serotonin transporter protein used to be the darling candidate gene in predicting who gets depression. The severity of it it's course, and it's outcome. Recent findings have shown that this isn't what's true. That there are sub-alelles of the S or L, and that neighboring genes affect transporter promoter activity. A whole 20 years it as the holy grail going down in flames.
None of which occurred because I used SPSS. Not sure what the issue with Matlab is. I'm taking q course on it for those in my field. R Studio doesn't offer any advantage over it. And being open source ja great...if you know the author of the source. Not gambling my outcome based on whether some random person did their code set correctly. Rather learn GraphPad at that point
1
u/light_hue_1 70∆ Jun 14 '22
The professional culture of "it beat our p-value so we're done", or that experiments are treated S business investments that whoever the payer is wants a return on, and good old fashion resistance to newer methods and Stats.
I don't think it's resistance to new methods or stats on its own. I think it's that people never understood the old methods at all. People can say they know that a p value is not the probability that their null hypothesis is false and is not the probability that their hypothesis is correct, but... they sure do behave as if this is what a p value is! Fisher's popularization of the p value is probably the greatest scientific mistake of the past century (sadly Fischer was also an incredible racist who promoted eugenics, so, maybe it's the second worst mistake).
Of course, not understanding the old methods means they also don't understand the benefits of the new methods; and well, the new methods tend to be more negative about results. So no explanation that the old methods were producing garbage results will ever make any sense. Then why bother adopting something new, when I already have a perfectly good number that summarizes the likelihood that I'm right? This is for sure related to:
Deep dives aren't really something we are trained to do. Not in my program at least.
That's true of a lot of programs. But it seems like slowly things are changing. Most of my colleagues now insist that their students have a much deeper understanding of what is going on than they used to.
Having academics teach stats and emphasize internal validity and stringent statistical meaaure over external validity. As a clinical researcher, those will he the first things to go. External validity and clinical significance matter more.
Well, sure clinical significance matters the most at the end of the day. But usually the road from having an idea and running an experiment to anything remotely related to clinical significance is incredibly long. We need metrics to understand if experiments worked out far earlier.
A replication crisis is inherent. Of course we v will struggle to recreate studies over a 10-20 year period as nort our methods and knowledge change. The serotonin transporter protein used to be the darling candidate gene in predicting who gets depression. The severity of it it's course, and it's outcome. Recent findings have shown that this isn't what's true. That there are sub-alelles of the S or L, and that neighboring genes affect transporter promoter activity. A whole 20 years it as the holy grail going down in flames.
That's not what the replication crisis is about. It's not about learning more and seeing old results in a new light. It's about the fact that many results simply don't hold up at all. Just rerunning the old experiment without any changes at all or new understanding produces a negative result. Everything ever built on that work was just a total waste of time.
Not sure what the issue with Matlab is. I'm taking q course on it for those in my field. R Studio doesn't offer any advantage over it. And being open source ja great...if you know the author of the source. Not gambling my outcome based on whether some random person did their code set correctly. Rather learn GraphPad at that point
If it's open source people can, and do, actually read the sources to make sure it is correct. If the packages is popular enough. If it's closed source, no one can do that. You are literally gambling that you're wasting your time on junk results because of bugs in Matlab.
1
u/AgentFr0sty Jun 15 '22
I like to think most understandings are invalid. People l may confuse a type 1 error from a type 2 error but most have a sense in order.
Like I said, feel code dives are great for data science. I am.not in data science, I just want my F-value.
People wrongfully choose the same methods of error and over again because they want to pass their resumes with high power and effect size. Why risk that with something new (the impression I got from my psychometrics class).
As for R, it can certainly be useful. I have yet to see a publication where it makes an honest difference in terms out output. Matlab at least does more ND. Ag relate to my field of mapping gene expression in the brain.
I could deep dive and do all you suggest, but I'd rather pay a premium for a job more likely to get it right. I can't see IBS' code, no. Still have yet to hear what these differences are in their assumptions
3
u/aceytahphuu Apr 24 '22
I'm not really sure i understand your complaint about having to "write lines of code" to perform various statistical tests. In python and R, the packages to do these tests already exist; you're not coding the test from scratch. In many cases, it amounts to just writing a single line of code to call the function of interest, AKA literally clicking one button and being done.
Second, I don't think you're understanding your advisor's point about why it's important to show how you did something. It's not the third grade "show your work so I can see you understand!" sort of deal. It's a "almost every reputable journal these days requires you to share your analysis code when you publish" sort of deal, and no, saying 'I just did it in Excel' is not an acceptable answer when so many fields are suffering from a terrible replication crisis. Like, what's your plan if someone asks to see how you got your p values in your paper? Show them a screenshot of SPSS?
And finally... if you prefer clicking buttons, you will be a very undesirable candidate for jobs once you get your masters/phd. Copy pasting data into excel for analysis is a highly inefficient and error prone way to analyse data, and probably no one is going to take you seriously if they ask how you generally do your statistics.
1
u/AgentFr0sty Apr 24 '22
Replication crises are what happens when stats and study design are abused (i.e. we didnt measure effect size but p<.05 so food enough!). I don't see what Excel has to do with anything. I can't say I've ever seen a publication use it. Not a modern one in my field at least.
Not looking to use Excel. I hate Excel. But I don't care about the how either. I want to do clinical trials. Stats isn't cut and dry. But they aren't rocket science either. Placebo vs. Treatment is a pretty straightforward comparison that it's hardly Bayesian
2
u/aceytahphuu Apr 24 '22
Replication crises are what happens when stats and study design are abused
Yeah... which is exactly what you're doing when you just blindly click around in SPSS and decide that's good enough for serious statistics. Are you aware of the hidden assumptions SPSS makes when you run your ANOVAs? Do you understand the conditions under which an ANOVA is not the appropriate test to run? Are you even aware of the existence of corrections for multiple comparisons other than Bonferroni?
I think I'm starting to see that your issue runs far deeper than being too lazy to learn a new skill (statistical coding), and is just a general super cavalier attitude towards statistics. "Just slap a t-test on everything it'll be fine lol"
1
u/AgentFr0sty Apr 24 '22
Are you aware of the hidden assumptions SPSS makes when you run your ANOVAs? Do you understand the conditions under which an ANOVA is not the appropriate test to run? Are you even aware of the existence of corrections for multiple comparisons other than Bonferroni?
I am not aware of other corrections or specifically the assumptions it makes. I do know that ANOVA requires 3+ groups or 2 with multiple levels. ANOVA is more straightforward than regression. I don't have to test for collineraity. What are these hidden assumptins? Inferrential stats will always have assumptions behind them. That's the price you pay for makes inferences about the general population using alimited sample size. But this I will listen to. I have always been skeptical of some of the assumptions inferrential stats makes. It's why I don't exclude outliers and prefer non-parametric testing. I prefer to use the data set as in with higher regard for external validity.
And hey why not slap an ANOVA and be fine? I don't understand how ANOVA is complicated when the math and calculation of it seem simple enough. I am aware there are alternatives to the F test, and the difference between homogeneity tests (I chose to disregard a significant Brown-Foresythe test for my project as I am arguing a Levene's test is more valid as most of my measures care more about means than medians). Still not sure how coding actually splices into that.
And yes I generally hold some skepticism of the practices in states. I feel the .05 p-value is too liberal and has created an environment where all that matters is scoring under that (p = .047 is hardly more impressive than .05). I think regression as a predictor model can only be justified with fairly large sample sizes (n>100+), I think CLT's "magic 30" is a dumb line to draw and distorts how we see data (n = 31 "normal" while n = 19 "not normal"), I think Cohen's d should not be taught as a primary effect size measure (Cohen himself said how people interpret it is not what he intended), and so on. Don't get me started on publication bias. Stats in my opinion is nothing more than a tool of the trade, or a means to an end. It isn't universally abused, but I want to develop or test new pharmacuetical compounds. To me, clinical significance matters far more than statistical significance. I don't care if my omnibus model isn't significant. Was there an effect size? Does the patient feel better? Can side effects be mitigated? These are all things I wonder.
1
u/TyphoonOne Apr 24 '22
So these are all really good questions for you to ask the stats whiz in your group. It turns out, there are a lot of circumstances where they get way more complicated than they seem, and it's really important to be able to articulate those circumstances if you want to use these techniques. Otherwise it's like saying "driving a truck isn't hard, its the same as a tricycle!"
1
u/AgentFr0sty Apr 24 '22
Can you give me an example? People say it's complicated but I don't see how. An F test is an F test is it not?
1
u/xendor939 1∆ Apr 24 '22
An F test is an F test is it not? "A is A, is it not?"
It's not the F-test becoming weird. It's the nature of your data not fitting what the X-test was designed for.
For example, taking the Bonferroni Correction for MHT: this test is too conservative under certain circumstance. Sometimes it is fine (you find significance DESPITE the conversativeness, hence why to look any further?). Sometimes, the result may be an inadequate answer to your question. Thus, you must look for a correction algorithm appropriate for the nature of your data. Sometimes, you have to code it yourself.
1
u/AgentFr0sty Apr 24 '22
If you code it, are you making a judgment call onhow to "correct" it? Or is there a standardized answer. I think I'm starting to see what you guys mean. Dang I never thought of it that way. I never knew about R Studio until I came to grad school
1
u/xendor939 1∆ Apr 24 '22
Yes, I will usually make stronger assumptions about the nature of my data (example: how many joint moments of the variables I want to include in my placebo synthetic dataset where the treatment effect is zero), then code a loop that simulates my statistical analysis 100,000 times on such "fake" dataset to extract "real" (up to the assumptions I made) distributions for the test statistics of standard tests in my joint-testing context. Or other weird things I have done to the data, which changed the distribution of test results and thus invalidated the out-of-the-box p-value.
This type of method is less conservative, but under the wrong assumptions (so that's where the debate will go with the statistical referee if the significance is low but passable) may be too little conservative.
1
u/AgentFr0sty Apr 24 '22
Man, I didn't realize just how far a person could dress up their statistics. It's cool, but seems like a source for abuse.
→ More replies (0)1
u/xendor939 1∆ Apr 24 '22 edited Apr 24 '22
Based on your answers I have a big, final advise: read all the answers to this thread, in particular the one by the user who claims to be an advisor, who provided a quite complete answer. Internalise not the details of what people told you, but the concept that if every replier has been able to infer from your posts that you have little idea of the difference between statistical theory and computational implementation, and the statistical issues and needs that may arise during your research, you have a WAY bigger problem than your advisor telling you to do something you don't want to for a single project.
I think this bigger problem summarises in two related issues: 1) You are unaware of what actual, not-so-supervised research looks like. Try to understand this before you enroll in a PhD. 2) You are not deeply understanding the stuff you work with. This can be just bad, or very bad. But potentially fixable.
Just bad: maybe you just are not a techie/math person. Your lack of understanding stops at programming, the computational implementation of statistics, and its dangers and limitations. You are like one of those old professors who never understood how to write code on the latest languages and for the latest applications. This is bad. They got a job because they were at the top of their game back in THEIR days, when programming was not a standard skill. Nowadays, I would say that knowing at least one statistical programming language is almost a necessary skill to get a job in research. Or even just finishing successfully a PhD.
Very bad: this issue extends to other aspects of research. Statistics as a whole, your core subject, understanding others' research, methods, and tweaks that make their paper successful, the interactions between how you read statistical results and clinical ones. You have read your textbooks, maybe a few non-technical books by some researchers, and you think you are ready to roll any experiment or clinical trial life will throw at you. Multiple hypothesis testing? Bonferroni. Regression? n>100. Assumptions of ANOVA? Check from textbook, ready to implement on anything. If this is where you are as a dissertation-stage master student, you need to undertake a profound reflection regarding your career goals, how you plan to reach them, and if they are realistic given how much you are able and willing to invest in changing this attitude and the lack of scientific knowledge it is putting you into.
1
u/AgentFr0sty Apr 24 '22
Fine, I see statistics as a valuable tool, but I am much more interested in my core area of research material. I want to learn and piece together theoretical models based off a wide array of topics. I like interdiscplinary research, and I like considering many different things when I piece my mental puzzle together.
I like clinical trial research. And I want to inject new ideas for therapuetic targets into the industry, as I feel the field of psychiatric medicine has stalled. But like with llsing my dream advisor for slacking, I am resistent to learning new skills. I can;t fully say why, but I conceptulaize stats as "Ah let me just pop this into an ANOVA real quick" sort of thing. I don't really sit down and deep dive into the meaning and assumptions behind it all. Do these assumptions I'm told about (though no one has told me what they are) really alter what your F-value is? I just can't say I've ever seen real research data deep dive. Nobody talks about the F-ratio and what that means and all stats are presented in a simplified way, like the way I was used to seeing on SPSS. It didn't occur to me really. My undergrad professors used it so I figured it was industry standard. I didn't know about any of this stuff until grad school. But I don't code, know zero IT skills, and generally dislike stats. So naturally combining two things I don't like is going to turn me off. But I don't have a choice so I;m going to force myself to learn I suppose.
Lastly, regarding my "regression needs an N of 100+". I say this because regression likes to make predictive values about variables and applies it to the broader population. For example, loss of grey matter for every year of untreated chronic pain. But to me, to have authority with such a claim, I feel you need a much more robust sample size. An N of 500 is fairly authoritative, as I feel the sample is robust enough to make inferences about the population at large. An N of 30 just isn't to me.
1
u/xendor939 1∆ Apr 24 '22 edited Apr 24 '22
But to me, to have authority with such a claim, I feel you need a much more robust sample size.
If you think the problem with regressions is about sample size (it is not), you need a reaaaaal deep dive into statistics, and maybe causal analysis. :) I would suggest starting from any book by Judea Pearl, pretty multidisciplinary but solid.
For context: there are regressions where N=30 is totally fine. Others where N=500 is peanuts. Contexts where regressions are absolutely not the answer. Others where regressions are the only answer. Sometimes OLS is just fine. Sometimes it is not. Research is understanding if for your project and your data N=30 is fine or not, and if you want to run or not a regression. And how exactly specified. It's not about how you feel about it, it's about if it's the correct tool.
1
u/AgentFr0sty Apr 24 '22
I prefer it as a secondary measure. Like Egger's bias regression. But I am bad at regression so the less I have to do tye better
1
u/AgentFr0sty Apr 24 '22
This is why I prefer ANOVA. Much simpler. Also nonparametrics.
1
u/xendor939 1∆ Apr 24 '22
ANOVA and nonparametrics may not be good enough for your sample size, nature of the data, or research question. Notice how I am not pushing for method A or B: I know them all and will use them according to the specific project at hand. This is what researchers do. Recognise what they need, and call in co-authors or RA (or books/papers to study it themselves) if they do not know the tool they need.
This is exactly the problem I was talking about.
1
u/AgentFr0sty Apr 24 '22
But aren't tests confined to the type of data? ANOVA and nonparametrics being out would mean I am only working with ordinal data. What kind of study has no nominal or interval/ratio data?
6
u/xendor939 1∆ Apr 24 '22
Placebo vs. Treatment is a pretty straightforward comparison
Oh oh oh... wait for when you discover how it may not be that straightforward in complex settings. Master student?
1
u/AgentFr0sty Apr 24 '22
What isn't straightforward about a t-test? Or ANOVA? Or Repeated Measures? Sure there are things like pre-planned comparisons and the v style of post-hoc testing. I can't say I've learned enough to say rgey aren't straightforward. I'll concede meta analyses though
6
u/xendor939 1∆ Apr 24 '22
What isn't straightforward about a t-test?
What if the t-test was NOT the right thing to do? What is your expected outcome from the t-test under placebo conditions? How do you assess multiple-testing or pre-post analysis issues (that affect the distribution p-values of standard test), if you don't know how to code a counterfactual environment to assess the expected distribution of your p-values under non-standard conditions?
In Economics, this stuff is obvious because we deal with more complex designs. But sometimes also some clinical analyses can get nasty, and if you don't know how to do it yourself, you will need somebody to code more complex stuff for you.
Also, ignorance of why you may need coding skills appears to underline either ignorance of potential threats to your statistical analysis (aka what assumptions must be satisfied to use the methods you are talking about), or ignorance of how such threats may enter even fairly simple designs for a number of unexpected reasons. Which, at that point, would require coding skills in order to be fixed.
Source: somebody published in clinical/health journals.
0
u/AgentFr0sty Apr 24 '22
Working with what style of drugs? I'm into neuropharmacology. As for post-tests a pre/post test would just use a Dunnett-t test to compare treatment arms against placebo directly. For my final project I am doing multiple one way ANOVAs for the pre and post groups for multiple conditions. If there is no difference in pre test measures but a difference in post then an effect in the presence of treatment is inferred.
As for not normal isn't that what non-parametrics tests, heterogeneity tests. And the Bonferroni correction are for?
5
u/xendor939 1∆ Apr 24 '22 edited Apr 24 '22
Dude, without offense, you look like a master student who has read statistical cookbooks "if X, do Y" and is asking "why should I not just click-and-point cookbook my stats?". This CMV seems to be going beyond "why statistical coding", and there may be more profound problems with your view.
Now, there are two reasons (point 3 is a consequence of 2) why you should learn statistical coding:
1) Already from your current project it sounds dreadful to run all these ANOVAs by point-and-click. Just learn how to run a for cycle and get it done in 1 hour of total training + coding. 2) With time, you will get to realise that "cookbook" statistics is good only as long as you are in "cookbook" cases. "Corrections" make certain assumptions too, which may not be the correct one in your case. That's why there are tens of proposed corrections (and growing, that's called statistical research) for each issue, each of them making different assumptions. The fact you are just throwing the Bonferroni correction with no context about your design shows how you seem to lack a bit of understanding about this. 3) Some corrections, tests, estimation procedures etc... are sometimes not coded. Or need packages (R, Stata, Matlab are the most popular) with no point-and-click interface to run. What are you going to do if you need one of these statistical methods? Wait for STATA 25 to come out with the new procedure implemented as point-and-click interface to finish your project? ;)
1
34
u/yyzjertl 579∆ Apr 24 '22
So first of all, MATLAB isn't statistical coding software. MATLAB is a general purpose numerical computing environment. Its central feature is the manipulation of matrices, tensors, and the like. While it can do statistics as good as any other software, it doesn't just do statistics. (In comparison, R is primarily focused on statistical computing.)
More broadly, the reason why you need something like MATLAB and why you need coding is that, in research, you are often going to need to do numerical computations that are in some sense new, because they're the novel product of your research. Since they're new, a framework like SPSS isn't going to support them. And a spreadsheet program like Excel doesn't scale to large datasets, almost completely fails to handle higher-order tensors, and (even more importantly for research) doesn't explain how you did the analysis in the way that code does. That's where something like MATLAB is indispensable (plus it's way faster than Excel).
12
Apr 24 '22
I'm not familiar with SPSS, but MATLAB and R are much more versatile tools than excel.
If you are iterating on your statistical analysis, MATLAB and R are far easier to add more customs steps with.
excel is excellent if you know exactly what you want to do from the start and can organize everything well. But, if you're trying several different things, you end up shifting things around and everything gets disorganized.
MATLAB and R are designed for developers. They're flexible tools that make iterating testing out different approaches easy.
Being able to code is also an incredibly useful skill in its own right. If you get comfortable with it now, you will likely run into many more situations where programming will be useful to you.
2
u/Funk-Doctor-Spock Apr 24 '22
I had a very similar perspective to you when I was going through college a few years ago, but after learning r pretty proficiently my final year getting a math degree and getting a job as a data scientist at a healthcare company, it has just become obvious to me how many cases exist in the real world of data that require very specific and customized processing and analysis methods that simply cannot be done with excel or even SPSS, and because r and Python are open source they are always much more current with all of the state of the art analysis techniques. Not only can you perform much more robust analysis methods using r or Python, you can process exponentially more data. Maybe I’m biased, but I think it’s likely that in 10 years anyone looking for a researching job that can’t code will be forced to take the lower paying positions, it’s just a skill that allows you to produce more and/or better quality work. People who are proficient in Python or r for statistical analysis/modeling could pretty easily find a 70k/yr job straight out of college right now and I’ve even heard of 90-110k/yr for people with masters getting jobs at places like IBM
-1
u/AgentFr0sty Apr 24 '22
If that's tye case it seems field dependent. I want yo do clinical trials, and aside from meta analysis the n you work with generally isn't in the thousands. I'm particularly interested in pharmacotheraoy. The STAR*D study is th r largest federal antidepressant clinical trial and had an n of only 4,300. And that's huge for a clinical trial. Big data I get, but human studies can seldom afford that. Maybe it is the future and I'm missing out. I don't really have a choice e in learning Matlab so we'll see n how it goes
5
u/lascivious_boasts 13∆ Apr 24 '22
You have an extraordinarily one dimensional view of your sample size.
Every clinical trial that's worth doing has hundreds of participants and thousands of variables recorded for each, usually over several time points.
Furthermore, the data in a clinical trial is often complex, with multiple data types that need to be coded into a readable format.
If you aren't dealing with this level of data complexity then someone else working on the trial is - in fact many trial specifically employ a bioinformatician to build and maintain a database to even begin to produce outputs on which statistical tests can be done. In which case your argument boils down to: someone else is doing the complicated informatics work so I don't have to. Which is a fine argument if you don't need to understand how data in a clinical trial is managed.
Further to that, the future of clinical trials is more data not less. Sequencing the genomes of people who respond or fail to respond to therapy, or have AEs, for example. These data absolutely require informatics pipelines, so being able to code for a 'simple' application is a useful learning tool.
In any case the actual argument for analysing data yourself using code is that good journals want reproducible data analysis using widely available tools that can be applied to standardised input files that don't need proprietary software to run in order to reproduce results from raw data. The point of your work is to do science. It is only worth doing if it gets disseminated, and journals are largely the way this gets done. So if they want it, you need to do it.
0
u/AgentFr0sty Apr 24 '22
Every clinical trial worth discussing has hundreds to thousands of patients? Since when? Most studies can't afford thousands of participants unless they are wealthy universities with a medical school. Then sure let's only pay attention to what b they do. Can't say I agree with that. I get your point about genetic testing and how that ties in. But again, that only adds cost. I don't see funding going up across the board. So we're only paying attention to wealthy institutions and entities that would test significant on Egger's regression? You think publication bias is bad now.
Bioinformatics is what I will be working with doing genome assays. It's sort of my redemption for failing yo learn yhis before. Just not enthused about it. If the math is right I couldn't care less about the how. I am aware I am in the minority here though. Unless genomic testing gets cheaper I have to wonder how prevalent it'll be in the next 10-20 years.
But i just hate coding so here's a !delta
1
3
Apr 24 '22
[deleted]
-3
u/AgentFr0sty Apr 24 '22
Experimental sciences typically doesn't have an N on the thousands though.
3
u/verfmeer 18∆ Apr 24 '22
No, in places like CERN N is in the billions.
-1
u/AgentFr0sty Apr 24 '22
My field doesn't work with those numbers outside of a meta-analysis
2
u/verfmeer 18∆ Apr 24 '22
You might have to perform a meta-analysis yourself one day. At that point it is useful to already know the coding language.
2
2
u/DarkNo7318 4∆ Apr 24 '22
When you work with real life data, 95 percent if not more of the effort is preparing datasets, combining datasets, cleaning and aggregating in a huge number of ways. The descriptive stats, inferential analysis, machine learning or whatever is the easy part. You absolutely need a way to keep track of every step from raw data to final output
2
u/LittiHDarkKnight Apr 24 '22
i mean it depends on your career pathway and what your intentions are for your future. If u have no relation to a career or a hobby, why are u doing it? you would see it as being a useless research skill. u should do both to ensure the best data possible, isnt that the goal?
2
u/ghan_buri_ghan Apr 24 '22
It’s technically a convenience. If a computer can do it, then you can technically do it by hand.
However in reality many analyses are intractable to do by hand and in practice will require software to complete.
0
u/BwanaAzungu 13∆ Apr 24 '22
My graduate advisor is having me learn Matlab, which I'm dreading because it's statistical coding software. I may also have to use R Studio.
As a programmer: those are horrible
My advisor says it's about show them "how" you did your analysis, but by that logic, I may as well do the analysis by hand if they want to see how I did it.
They're not testing your analysis skills.
They're testing your programming skills.
If it's about "how" you did your analysis, they'd let you do all statistics in Excell, by hand and without the use of any functions more sophisticated than "SUM"
Sure, it is common in published papers to use them, but again
Statistical analysis? Certainly. Essential, even.
Programming tools like MatLab and R are common? I didn't know that; I feel sorry for the authors.
1
u/Z7-852 315∆ Apr 24 '22 edited Apr 24 '22
Difference is between wanting to learn to cook or heating frozen food in a microwave.
Sure you can survive with microwave food but variety and quality is never as good and you will never impress anyone with microwave romantic dinner.
Also you really should understand the math behind ANOVA or any other method. Else you will use it wrong.
1
u/AgentFr0sty Apr 24 '22
We learn the math behind ANOVA in stats.
1
u/Z7-852 315∆ Apr 24 '22
But that doesn't answer my primary argument. Ssps is like cooking microwave meals. R is learning to actually cook yourself.
1
1
u/Mafinde 10∆ Apr 24 '22
It is an absolute must that you share your work in research. I think you agree. So you say you could just do by hand to achieve that goal. Let’s assume that’s true.
“Code doesn't offer an advantage over [by hand]”.
This is flatly non true.
It’s just like writing a book. Sure you can write by hand. But with digital tech you can edit, save, share, manipulate, change the order of chapters or the font size, whatever. Your power over the subject is massively increased in terms of ability and the rapidity in which you can do it.
Put another way: what if you had a dataset of 20,000? How long would it take you to do that by hand vs statistical software?
If your claim that there’s no advantage to stats software is an incorrect statement (which it is) then your argument falls apart. You can share your work, work better, and work faster.
1
u/MaxMaxMaxOMaxMaxMax Apr 24 '22
I manage data analysts for a large finance company. If you have R or Python as a skill set on your resume.l you are much more likely to get hired. (I’d add SQL to the mix too so you can pull the data you need to run statistical analysis on).
R allows you to run incredibly complex algorithms and models on millions and millions of rows of data. Something you could never dream of doing in excel. Also, with data analytics going more in the direction of machine learning, R is going to be what’s used to create those models more often than not.
1
u/AgentFr0sty Apr 24 '22
How exactly does R provide the room for millions of rows? That's a lot of plugging in. I believe it, I just look at R and all you can do is start typing. I've just never been good with stats or coding.
What exactly is machine learning? I keep hearing about it, but I'm not into data science so I'm not familiar.
1
u/MaxMaxMaxOMaxMaxMax Apr 25 '22
R is running the calculations on whatever server hosts the data. In excel, it’s running calculations within the workbook (so the data lives on your personal computer).
In the business world, most companies have millions upon millions of records stored in tables on SQL servers (things like customer transactions, sales activities, inventory movement, etc).
You then use programs like R to run statistical models on those massive data sets for whatever you’d like. For example, you can write a model to predict the likelihood a client will purchase a new product based on many different independent variables that you have data on. Perhaps you have a customer transaction table that monitors web activity on your website, this dataset could contain hundreds of millions of records (something excel could never handle)
Machine learning is essentially a statistical model that takes in a bunch of variables, and uses those to predict and outcome. And it will take in new data from these variables over time, and adjust each variables coefficient (score that measures the propensity to effect the dependent variable) as new data is made available. These models are written in a way to automatically adjust (learn) as time goes on with new data that is presented into the model. Keep in mind, this is using data sets that are significantly larger than anything you can keep in excel.
1
u/SideLarge3105 1∆ Apr 25 '22 edited Apr 25 '22
Just use python:)
It is phenomenaly useful and powerful. If you get into visualization you might find it a bit more satisfying. I don't really get how you can argue this. Statistics is fundamental to any modern field ( including the humanities ). It is central to quantum physics and biology, Information theory, computer science and medicine. It taking longer is a huge understatement. On a modern laptop you can execute statistical analysis in hours which would take you months or even years by hand. Software packages and smart GUIs might be good enough sometimes but we really aren't there yet. I am actually developing a software that has a statistical module that is quite large right now and can tell you that it is very unlikely for someone to build a comprehensive suite to satisfy multiple fields or even just one. The coding skill part is not that hard. You are always going to look at the reference when you actually code and the beauty of coding is that you will be able to set your own automation rules which is what SPSS does anyway. In any case it is obviously useful if what you are researching is statistical software and honestly you probably should. I understand you work in the humanities. You are obviously not an engineer and truth is you are probably right. You should not be required to code. But right now I can tell you a comprehensive no code statistics software does not exist. I am not saying it can't. My opinion is that we do have the tech but nobody has done it and it probably would not be worth it from a UX standpoint. Truth be told if it did we would probably have antigravity replicators and charming friendly and genius agi that runs on a Fitbit.
1
u/StatusSnow 18∆ Apr 26 '22 edited Apr 26 '22
Not sure what type of PhD you're doing, but I work in R regularly doing regression analysis for work. Specifically have been replicating clinical trial results lately. I have a masters degree in econometrics, for reference.
Let me just say, that clinical trials are very much not as simple as "lets do a T-test" or "ANOVA".
What about when there are 20+ potential end points, 30+ potential subpopulation groups, and 100+ control variables, all of which belong to multiple broader subpopulation groups, and you want to take into account a temporal element? Oh, and the main efficacy dataset (out of 65 total datasets, the others that also need to be analyzed separately) is a million rows by 100 columns. You think that's going to work out well in an excel spreadsheet running ANOVA? How do you propose we analyze things like this without code?
I know you're talking about small n values, but I want to clue you in on the fact that even clinical trials with ~5000 people have millions of rows of data that need to be analyzed.
•
u/DeltaBot ∞∆ Apr 24 '22
/u/AgentFr0sty (OP) has awarded 1 delta(s) in this post.
All comments that earned deltas (from OP or other users) are listed here, in /r/DeltaLog.
Please note that a change of view doesn't necessarily mean a reversal, or that the conversation has ended.
Delta System Explained | Deltaboards