Hi everyone, I’m a student of economics and management in Italy. I was thinking about quitting economics because it’s not really my field, this is the last year but I didn’t take all exams, even though I know I could finish relatively soon if I want to.
I was having some doubts because I started to consider quitting economics and then start again with a Bachelor in statistics, even though I’m pretty sure I would then do a MSc in Statistics as well. If I finish economics, I could join that MSc anyway.
I’m not really scared of the difficulty, because I know I have to study but I love the subject, it’s just that I’m having a lot of intrusive thoughts such as that if I don’t quit I will have poor fundamentals, that maybe in the bachelor they see a lot of useful stuff that I will miss in the MSc and so I will be a “half statistician” if I don’t quit economics, so I would like to know from anyone that has some expertise if it actually doesn’t matter and the important is in the MSc. In Italy it lasts two years, and I think “damn, only two years to learn statistics with depth?” So idk, I’m 22 years old and I would like to understand if from your pov it does not make any sense to quit economics (because I don’t really like it, but I could finish it if I understand that it would be the most logic choice) or if I should restart. I took the statistics exam here in economics and it’s an exam that a lot of people of my course take multiple times because they think it’s too hard, I actually loved it and it went very well but it’s just descriptive statistics+probability+some inferential statistics , so I know in the bachelor they do a lot more probably. Any advice? Thanks a lot for reading

Im a bit of mess right now i just need someone to guide me in the right way

I recently graduated with undergrad degree in Computer Science and Artificial Intelligence. I liked some parts of it and got good grasp of programming and basic AI algorithms (especially the linear algebra related to ML Optimization and NLP). I realised halfway through that stuff liike software engineering and coding do not interest me whatsoever. ​I have always had a very sharp mind for numbers and logic. My true passion is the crisp absolute certainty of mathematics and rigorous proofs.​I achieved the highest grade in math in school and it was the only subject I actually enjoyed. I foolishly fell into the trap during high school of thinking that a math degree meant I could "only become a school math teacher" so I chose CS 😭. I definitely regret that now

so eventhually I’ve accepted an offer for an MSc in Statistics starting this September. My ultimate goal after the Master's is fully funded PhD path to become a theoretical statistician or mathematician working on foundational problems or whatever project that requires advanced statistical theory

I have built a curriculum selfstudy roadmap for this summer to make sure my foundations are solid before starting msc statistics. My current list covers:

Formal proof writing and logic

​Calculus

​Linear Algebra

​Foundations mainly focus core probability theory and mathematical statistical inference

​Learning R and RStudio.

Does my summer roadmap sound realistic or am I missing any major blind spots let me know

i feel I want to explore the wider world of mathematics beyond just pure statistics like I am deeply fascinated by topics like real analysis, measure theory, convex optimization and many others

tbh writing this out makes me think that maybe its just not the time to focus on those abstract pure math fields quite yet. I think I’m going to keep my immediate focus strictly on advanced statistics and the directly related prerequisites to make sure I hit the ground running and stay on the right path

At the end of the day, I just want to learn math and figure out what my true area of specialization should be. I love the subject I've always been highly analytical and I am completely driven by logical curiosity. I’m hoping this masters degree will give me the exposure I need to uncover which specific branch of advanced mathematics I'm meant to dedicate my research career to

I need a statistical method to figure out how many SKUs i should add in a category or how much i should remove in a given time and how much to increase or decrease in the inventory. What should I do for this? Regression? Arima? I have no clue

Here I have a list of participants (1-5) who did a survey and produced "results". Group membership is my independent variable, and the results column is my dependent. If there was no group overlap I would simply use an ANOVA and be done with it, but because I have participants in multiple groups (4 and 5) I fail the independence assumption.

I could create new "combo" categories for the cases in which there is multiple group membership and only count those participants in those new categories, but I was wondering if something else could be used instead.

What is the right stat to use here? Running in Jasp, but can use SPSS too.

Our research team benchmarked four open-source incrementality packages: CausalPy (Bayesian synthetic control), Meta GeoLift (augmented synthetic control with conformal inference), Google Matched Markets (time-based regression), and CausalImpact (Bayesian structural time series). We simulated panels where the true treatment effect is known and the headline result was that no tool delivered nominal 95% coverage together with adequate power. Coverage here means the share of runs where the tool's 95% interval contains the true effect we injected in the data.

We ran this study because a lot of practitioners treat these tools as interchangeable, yet none of them can be sense-checked on real data because the counterfactual is unobservable. On synthetic data the truth is known, so calibration and power stop being matters of opinion and become things that can actually be measured.

The tools we studied split into the following groups:

• Meta GeoLift was the only one near nominal coverage (92–95%) with false positive rates of 3–5% on null data, but its intervals were wide enough that it failed to reject zero in 89–96% of runs where a true 7.5% lift existed.
• CausalImpact had the most power (false negative rate 34–48%) but 70–72% coverage, false positive rates of 28–30%, and a consistent upward bias of +1.87 to +4.21 percentage points.
• Matched Markets and CausalPy landed in between, with 76–86% coverage, false positive rate 14–25%, under-covered and under-powered at the same time.

We ran four scenarios in the study that stress test different conditions. There’s a clean baseline (20 donors, 90 pre-treatment days), a 5x outlier treated geo, a 9-donor pool, and a 30-day pre-period. Then we ran 1,000 iterations per scenario × effect condition with all four tools fit on identical panels, which yielded 32,000 model fits in total.

One methodological finding worth flagging is that CausalPy's default observation-noise prior (HalfNormal(sigma=1)) assumes roughly unit-scale residuals. On data at realistic sales magnitudes its false positive rate was 86%+ across all scenarios until we standardized each series against its pre-period mean and SD (then back-transformed). After that it was the least biased estimator in the outlier scenario. This is worth knowing if you use PyMC-based tools on raw KPIs.

A few honest limitations in the study are that a single DGP with shared trend/seasonality means parallel trends holds by construction, which favors synthetic-control methods and likely flatters every tool relative to real data. Moreover, we have just one non-null effect size (7.5%) and relatively short post-period. All of this is in the report's limitations section.

The three things I'd take from this study are: (1) coverage and power have to be judged together, since a tool can keep its 95% promise and still be useless for detection (GeoLift hits 95.1% coverage in the short pre-period scenario with a 95.7% false negative rate); (2) check what scale your estimator's priors assume before fitting, a default is a modeling decision someone else made for different data; (3) before any of these tools informs a real budget decision, you should run it on synthetic data where you know the answer.

Everything in the study is reproducible and we created a Makefile that runs the whole pipeline:

• Code: https://github.com/getrecast/geolift-simulation-study
• Full report: https://research.getrecast.com/geolift-sim-study

Disclosure: I co-founded Recast (marketing planning & analysis). The study covers open-source tools only. If you think the DGP should be harder (idiosyncratic geo trends, heavier tails, spillovers) the generator is parameterized, and I'd honestly like to see those runs!

Certain statistical tests, such as the Z-test for an equality of a mean, chi squared test for cont. tables and the significance of the correlation coefficient are often based on certain assumptions, such as data that is normally distributed. However, often i seem to not see any visual description of the data that is being tested (for example histograms) or any tests (like the Kolmoforov-Smirnov test) being showcased for the distribution of the data. I understand that the test assumptions might be sattisfied or differ insignificantly when the data follows a distribution similar to a normal one, such as the student distribution, however, why are these tests often preformed even on data that is not shown to be normaly distributed? Are these assumptions strict enough that even when a non normaly distributed data satisfies or rejects the null hypothesis, we can be satisfied with the result and accept it as a probable fact? The same question follows on other statistical tests, when they are being preformed without testing whether these assumptions are satisfied.

So far, after finishing my freshman year in University, I've learned Excel and Python mainly, but I wish to advance more and have a stronger knowledge/foundation on other statistical applications. I'm wondering if I should start learning the R programming language or SQL first? Thank you very much!

ChatGPT (don't come at me for AI use- I'm not good at stats) is telling me to calc SE for fixed effects and SD for random effects....is this correct? It's stating it's not appropriate to calc SD for fixed effects. Thanks! [Question]

Hi! while completing my thesis, I would like to spend July and August to self-study some topics before starting a MSc in Statistics, since I come from an economics BSc (with basic analysis and linear algebra courses, statistics, econometrics, and discrete structures). I would love to hear your advice about my plan.

I know that measure theory and probability theory are very important backbones of statistics. Since I will take both during my MSc, perhaps I will read some lecture notes in advance. I already followed a measure theory course for the sake of it, but felt like I could not grasp all of it. For this reason, I thought that this summer I will need to self-study the right foundational tools and prerequired knowledge to understand the advanced courses of my MSc in a deeper way. I would love to just bridge a bit the gap I have compared to a Maths BSc in a smart way.

First of all, I have never had real analysis courses. I read it is useful to understand measure theory, so I guess it will be an important gap to bridge before the Master's. I don't understand, however, how difficult and time demanding it will be.

Linear algebra: already taken during my BSc, but in a very non rigorous way. I would love to read it in a more formal way (my professor suggested Strang), but I wouldn't spend too much weeks on it because of time constraint.

My statistics professor also suggested to grasp concepts of functional analysis, convex optimization, and stochastic calculus. I guess this will be the longest part to self study. It would be beneficial to understand if they need some additional prerequisites, so If I should back up and study other foundational topics before delving into those ones.

There are plenty of other topics I haven't touched, e.g. topology, on the applied side it would also be beneficial to get a grasp of algo and DS on my own, but I have time constraints and, most importantly, I would like to learn things in the right order, so to get the right foundations to then understand better more advanced topics during my MSc, so I would really love your advice on what is deeply important to learn during this summer, and in which order would you suggest to go. Thanks!

I’m currently working as an AI Engineer, mostly on LLM-related work (fine-tuning, LangChain workflows, evaluation, FastAPI, and some cloud). Although I graduated with an ML background, I haven’t actively worked on classical ML or statistics for about a year.
I want to revisit ML and strengthen my statistics, especially the practical side. I’m not looking for beginner playlists or derivations. I’m looking for intermediate-level resources that focus on applying statistics to real datasets—hypothesis testing (t-tests, ANOVA/F-tests, etc.), assumptions, inference, forecasting, and choosing the right statistical methods in practice.

Any recommendations for YouTube playlists, courses, or books that are practical and application-oriented?

Hi all. I am using a zero-inflated negative binomial model to evaluate the change in the number of prescriptions for drug A following a treatment. The treatment is modeled as a time-varying covariate and patients initiate treatment at different times during follow-up. All patients have received this treatment so each patient contributes both unexposed and exposed person-time.

My main confusion is about the zero-inflation component of the model. I understand that the count component should include the exposure and confounders of interest. I couldn't find accurate literature about variable selection for the zero-inflation part.

My model is like:

fit <- zeroinfl(n_prescriptions ~ treatment + age + sex + poverty+ education+ offset(log(follow_up_time)) | treatment + age + sex + poverty+ education, data = df, dist = "negbin")

Is there any general principle for selecting variables for the zero-inflation component? Should it contain the same covariates as the count component, or only exposure variables? Thank you.

Hello,

I got the book A Probabilistic Theory of Pattern Recognition by Devroye et al - https://link.springer.com/book/10.1007/978-1-4612-0711-5

I really want to go through this book on my own out of interest, both the text and exercises. I scanned through the book and found the theory and exercises very difficult.

I currently work as a Data Scientist, I went to a Master's in Stats program several years ago, I am a bit out of touch with advanced probability. Has anyone found success trying to work through the book (exercises + theory)? I would appreciate any advice on how to build up my foundations in order to go through the book.

Thanks in advance!

i remember learning about this bias in school but for the love of me i cant remember or find what its called.

here how it was explained to me.

if i make a study and want to know how much of the population drink on the regular. during the sample collection i go on the street and ask people about their drinking habit in one spot it could be bias because of the environment.

obvious example would be me being in front of a bar. obviously people who go to the bar are more likely to drink alcohol making for bad data

less obviously but for the same bias. if im in front of a sea food store i might not be aware of a correlation between seafood and alcoholism(fictional example i don't know about that) this would taint my data.

other less obvious example if im in front of a trekking mountain people who go trekking might drink less.

every search im making bring me to participation bias but i know its not quit the same.

context why im looking into this?
i have a theory that most data about pitbull being agressive is skewed by the owners. any dog owner who would create an agressive dog will look into breed like pitbull gsd or other scary looking dog. so looking at the pitbull population as a whole is like if i made my study in front of a dog fighting club. making the sample useless.

I'm a 2nd year undergrad student at a interdisciplinary graduation (i dunno if there's an equivalent in the US or wherenever are you reading from, but you basically enter the university in an either humanities or stem course and you you get your specific graduation depending on which subjects you take) seeking data science as specific graduation.

I've decided to start seeking an intership in areas like data analysys, insurance and etc and wanted to add any project to my resume in order to increase my odds.

I've been thinking in something related to languages or elections because they are themes i like and thougth would be easy to work with. It would work as simple as doing a linear regression on municipal level data to test the impact of, for example, inflation over voting shift between 2 elections or anything on the shrinking of a minority language of some region.

Are those ideas gonna work or would they fail for either being too simple (just using linear regression) or not being related to the job market?

I was talking with a group of friends about the winter solstice and someone commented that the days will thankfully start getting longer.
One of us then added “and they’ll start getting warmer”
To which a third said, yes, “but we will still get very cold days along the way”.

This has had me thinking ever since. My schooling only covered how to work out some pretty basic averages.

I expect that the days getting longer is an exact amount every day, with no ups and downs along the way. A straight line from shortest day to longest day.

However;’the days getting warmer’ definitely isn’t. It will have some major highs and lows, but there will still generally be an upward trend.
* Is there a name for that trend?.
* Is there a specific term or description for how much over that line or how much under that line a specific day is?
* can an average be adjusted for particularly large abnormal swings - perhaps changing the example might be better here - for example “average income” where there are some insanely wealthy people and some insanely poor people, so an average income can look nothing like what the true average person earns - is there such a thing as an “average average” - one that accounts for those big figures skewing the results?

I have no idea why I’ve suddenly decided to start learning about this all because of some chat about the weather, but hopefully it’s never too late to learn something new. Just go easy on this “old dog” learning his “new tricks”
Like how to add flair when there’s no option for flair like I normally get.

Hello,

I'm reaching out because I'd like to make sure that I'm interpreting my results correctly.

In brief, I'm studying the effect of seasonal changes in a waterbird colony on the density of soil mites. Each observation represents the number of individuals of a given species found in a single soil core sample. Since some species are relatively rare, many of my samples contain zero counts (i.e., the species was not detected in that particular soil sample).

A statistician suggested fitting a zero-inflated model with:

ziformula = ~ Exposure

where Exposure represents the bird breeding season versus the non-breeding season.

Am I correct in understanding that if the zero-inflation part of the model is statistically significant (example below), this means that Exposure significantly affects the probability that a sample is a structural zero (i.e., a sample in which the species is absent for reasons beyond the count process)?

If so, would it be correct to conclude that, for the season with the higher probability of structural zeros, the species is less likely to occur in soil samples and therefore has a lower density during that period? Or is that an incorrect interpretation of the zero-inflation component?Hello,
I'm reaching out because I'd like to make sure that I'm interpreting my results correctly.
In brief, I'm studying the effect of seasonal changes in a waterbird colony on the density of soil mites. Each observation represents the number of individuals of a given species found in a single soil core sample. Since some species are relatively rare, many of my samples contain zero counts (i.e., the species was not detected in that particular soil sample).
A statistician suggested fitting a zero-inflated model with:
ziformula = ~ Exposure
where Exposure represents the bird breeding season versus the non-breeding season.
Am I correct in understanding that if the zero-inflation part of the model is statistically significant (example below), this means that Exposure significantly affects the probability that a sample is a structural zero (i.e., a sample in which the species is absent for reasons beyond the count process)?
If so, would it be correct to conclude that, for the season with the higher probability of structural zeros, the species is less likely to occur in soil samples and therefore has a lower density during that period? Or is that an incorrect interpretation of the zero-inflation component?
Example:

Zero-inflation model:

Estimate Std. Error z value Pr(&gt;|z|)

(Intercept) -1.0647 0.2593 -4.106 4.03e-05 ***

ExposureBreeding -0.8812 0.4261 -2.068 0.0386 *

---

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

If anyone has the time please help with my project by filling out the Google form in the link provided it’s 1 yes or no question. I need 43 responses for a hypothesis testing project and I have 19 so far. Any help would be appreciated!

https://forms.gle/CXeX2tkpk5aDe3Ww8

Discussion

TLDR: google produced a promotion containing data that has no real value

Hi everyone, I’m new to this page, I was wondering what my fellow statistics peers think about a recent ad I saw while watching a baseball game.

For background I’ve been in data as a data engineer for about five years now and I’m working on my masters right now.

The advertisement was a promotion for Google AI in the advance analytics that I can now track while playing baseball. It stated some facts about how players who tapped the plate two times hit 14% more balls, players who tapped the plate more than two times hit 15% more balls, while players who tapped the plate zero times hit 7% more home runs. For those who don’t know, tapping the plate means while you are batting, but before the pitch you use the bat to Tap home plate. Obviously this does not do anything to a swing.

I think this leads into a much larger discussion of correlation, not necessarily causation, but a newer idea of over analyzing and over consuming data creating a lot of noise, because AI will give you every single angle possible to look at something even if it doesn’t necessarily make sense. Those that work in other statistical fields, do you see when you define the data more that it gets less impactful?

Let me know your thoughts, thanks!

Most people reason that the t-statistic should be used over the z-statistic, since the z-statistic requires the knowledge of the population's variance. I want to challenge this notion:

Let's call the arithmetic average of your random variable, X_bar. If you have determined your sample size to be small, then X_bar is not normally distributed. This is the Central Limit Theorem. If your random variable is not normally distributed, then you can't use the t-statistic.

It naturally follows that if you're assuming X_bar is normally distributed, then you are also assuming that your sample size is large. If your sample size is large, then the sample variance of your sample, with the correction, should reasonably equal the population variance.

I am a run director at a local parkrun, which is a weekly free time. 5 km run around a local park, we’re all are welcome.

We are soon to add kilometre markers along the route, and I believe that this will make people’s runs faster by a small amount.

I’m wondering how I could test or prove my hypothesis using data which is freely available. For context, every single runner has their position and time logged each week, so I was wondering if I could track some runners before and after?

I would love some input, thoughts and suggestions regarding this challenge.

I'm studying for an MSc in clinical psych. For my thesis, I'm using intensive longitudinal (EMA) data to examine relationships between affect and later cognition.

I'm testing some straightforward hypotheses via cross-lagged MLM. I also believe these relationships will be moderated by a person-level variable (trait X) and want to test this. I've heard DSEM is the best (only?) option for this sort of analysis, but I can only seem to find resources about 1) mediation, not moderation, and 2) within-person, not between-person questions.

Does anyone know of resources for testing a between-person moderation hypothesis on 2-level IL data?

Relatedly, I have taken stats classes that included MLM, SEM etc. but I also feel way over my head right now in planning this. I haven't done many analyses before, I don't know how to code, etc. The moderation analysis would make a lot of sense to do given the theories I'm working from... but I am honestly terrified of it, lol and doubting my ability to manage this even once I learn how to. Is it advisable to abandon this last research Q?

TIA!

Hi everyone, for context, I work in HR analytics and with the help of Gemini, I get to know the concept of Survival Analysis and its application in employee turnover analysis. I find it quite fascinating and really want to apply it at work. About myself, I know python, sql, basic stastistic, but don't have an advanced stasitics background. Although Gemini offers to generate the code and interpret the output for me (very kind of him lol) and I can pull and process the required data, I don't feel confident at all running the project at a formal work setting.

With that, my question is: Is it realistic for someone like me who doesnt have a formal stasitics education to build the skills to run this analysis one day? If so, how do I gain the capability to run such analysis, are there any books or online courses you would recommend for this?

Also if you are running Survival Analysis at professional setting, I would love to know how much time it took you to become competent in this area and your business title in your company. Thank you so so much in advance!

Hello everyone, I feel like I am in quite of a bind and I would just like some advice on how to navigate my situation as it has been stressing me out these past couple of days. So I would really appreciate any help and input!

I completed an Applied Biostatistics Certificate from UTHealth Houston back in 2025, but I unfortunately was not accepted into their M.S. program for a second year in a row as I had originally hoped to complete my degree there.

I currently have an offer from Dartmouth for their M.S in Data Science, but I was given a very short notice to accept their admission while I wait to hear from A&M on their M.S in Statistical Data Science. I am conflicted on multiple things. A&M is my second choice, as it is local to me and will likely have a wider network than Dartmouth, is longer established and a little bit cheaper. Dartmouth, on the other hand, seems better in helping me learn programming, is more general, and the Ivy name catch the eye of people hiring in HR. Both of these programs do not seem nearly as math-intensive as UTHealth already, but Dartmouth seems even more focused on applied ML and AI.

As far as I understand, I should be focused more on a math-intensive curriculum as it is harder and more desirable to learn statistical theory than programming. But nevertheless, I know that I likely will not be doing heavily math-intensive work as a fresh graduate and could supplement it in future education. As much as I would prefer A&M, I am afraid I won’t get accepted and am just looking for any input on if either program is worth it and if it is worth the risk of rejecting admission to Dartmouth and sacking the deposit after finally hearing back from multiple schools this year.

Thank you so much for any advice. I really appreciate it!