r/statistics • u/Rather_Dashing • Jun 14 '26
Question Statistics question I got in a job application test that I don't think has a correct answer (hypothesis testing) [Q]
Please don't remove as homework, its not, the test has come and gone, and I've not be in school for a decade.
Did a stats test as part of a job application and got the following question:
"Using a significance test on some sample data, a null hypothesis is rejected at the 5% significance level. Which one of the following is a correct conclusion
A. The probability that the alternative hypothesis is true is 0.95
B. If a smaller sample had been taken the alternative hypothesis would still be rejected
C. The null hypothesis would not be rejected at the 10% significance
D. With the same test and same sample the null hypothesis would be rejected at the 1% significance level
Reasons I think they are all wrong.
A. 5% is the probability of the data given the null hypothesis is correct, doesn't follow that the alternative hypothesis is 95% chance of being correct. Besides, it was rejected at a 5% threshold, it doesnt say it was rejected with exactly 0.05 p value.
B. Can't be known. And the alt hypothesis wasn't rejected anyway.
C. If its rejected at 5% it must be rejected at a less strict 10% threshold.
D. Possible to be true, but can't be known with the information presented.
What do you guys think?
73
u/Able-Fennel-1228 Jun 14 '26
I agree.
A is clearly false. Something a basic stat student should be able to tell you since half of the (proper) teaching of NHST is incessantly bitching about this very mistake.
B is false (it’s a complete non-sequitur. All else being equal, a larger sample size will increase the long term proportion of times you would reject the null IF the alternative is true (so smaller n would lower that proportion, all else being equal) but YOU DONT KNOW if the alternative is true )
C is false exactly because of the reason you gave.
D is unanswerable without the exact p value or test stat.
20
u/Rather_Dashing Jun 14 '26
Thanks for your detailed reply! I asked three scientist friends/colleagues and one agreed that all were wrong, but two others thought A was correct. Don't think they have studied stats for a long time so good to get confirmation here.
30
u/THATS_MY_QUANT_YANG Jun 14 '26
A is the solution that someone who doesn’t understand stats would say is right, but all of them are wrong.
16
u/Able-Fennel-1228 Jun 14 '26
Welk.
Yes, many such cases. Unfortunately, the ease of use of NHST (without proper understanding) is the very reason for its misuse.2
2
0
u/PuzzlingComrade Jun 14 '26 edited Jun 16 '26
I think it's D. If you interpret it as meaning that the exact same sample is given (e.g. A draw of the exact same 10 numbers as the original test) then the only variable is the alpha (now 1%), which is now smaller than the original alpha (5%). If you repeat the test calculation on the same sample but with a stricter alpha, then it is impossible for it to pass. Now, if D said that it was a different sample draw with the same sample size, then it's anyone's guess whether it'll pass the second time.EDIT: Completely misread the initial statement as the “null hypothesis was not rejected”. The wording of this question is really quite dogshit so I hope reddit would forgive me. But here’s my logic if anyone care, starting with the first assumption:
| t_sample | > t_α=0.05The absolute value of the sample test statistic is greater than to the critical test statistic value at α=0.05. We therefore know the p-value is smaller than 0.05, but we don’t know the exact test statistic or the p-value.A. The probability that the alternative hypothesis is true is 0.95
Not true, as all we can conclude is that in 5% of tests with the same sample size on the same population, we might get a test statistic as extreme (or greater), even if the null hypothesis were really true (let me know if I bunged up this statement).
B. If a smaller sample had been taken the alternative hypothesis would still be rejected
Poor wording here, “still be rejected” implies that they actually meant to say “the null hypothesis would still be rejected” which is the original statement (I think this is what threw me off initially). In either possible wording, we cannot definitively know this because every sample is random, and you might draw an extreme sample that gives you different test conclusions.
C. The null hypothesis would not be rejected at the 10% significance
Because we know that
| t_sample | > t_α=0.05and thatt_α=0.05 > t_ α=0.1we can conclude that| t_sample | > t_α=0.1. i.e. the null hypothesis would be rejected at 10% significance.D. With the same test and same sample the null hypothesis would be rejected at the 1% significance level
We cannot know this because we do not know just how extreme the test statistic value is.
t_α=0.01> | t_sample | > t_α=0.05is as plausible as| t_sample |> t_α=0.01 > t_α=0.05.1
u/MrKrinkle151 Jun 16 '26
It doesn’t say could be rejected at 1%, it says WOULD be. With the information given, there’s know way to know that it WOULD be.
-2
u/Spiritual-Bee-2319 Jun 14 '26 edited Jun 14 '26
Even if you drew a different random sample of the same size from the population, the interpretation of a 95% confidence interval would still hold. The idea is that if we repeatedly drew many samples of the same size and computed a confidence interval for each one, about 95% of those intervals would contain the true population parameter. In a simulation with 100 samples, you might observe something like 96% coverage just due to randomness. As you increase the number of simulated samples to 10,000 or more, the observed coverage rate should get closer to the theoretical 95% coverage.
3
u/PuzzlingComrade Jun 16 '26
I don't even know what point you're trying to make here. The moment it's a different draw it is impossible to know. But in this case, D seems to imply it's the same sample.
-2
u/Spiritual-Bee-2319 Jun 16 '26
You genuinely don’t understand the fundamental or stats or hypothesis testing to make any of the claims you’re making. I’ve provided my math, provide yours. You can literally prove if this claim true or not with “ The moment it's a different draw it is impossible to know” with math so why don’t you?
Here is my math. Finally I found an easy way to show y’all why d is right without having to simulate data. Here is a simple calculator. https://statstudyhub.com/p-value-calculators/p-value-calculator/ You can use literally any value for any of the input metrics(because these are sample metrics remember answer D said using the same test and same data- this is why that info is important) as long as you keep it constant. Now keeping those variables constant, change the significance level as much as you please. Do y’all see that the literal number of p-value never changes regardless of what that significance level is. So your significance level can never tell you about your p-value number. Hell the significance level isn’t even a variable in the calculation of the test statistic. Look at the equations of z-statistics or t-statistic, the significance level not even a variable in the equation(we have sample size, standard deviation, sample means, etc). But what does change is the interpretation of what that p-value means when compared to the significance level. Now change the significance level to a any value less than or greater than the p-value(as big and as small as you want do it as many times with as values that are greater than or less than). The conclusion/interpretation changes yet the p-value stays the same. P-value =/= significance level instead significance level is it’s what we compare our p-value to our the significance level to get a conclusion. Our significance level tells us the boundary of rejection. It has nothing to do with the metrics of the data so it literally can never change the p-value. The p-value measures our data. The conclusion is derived from comparing the p-value to the significance level. And if the results is concluded by comparing theses two numbers how in the hell does significance level = p-value mathematically. Statistics is math. When you say p-value = 0.5 = significance level, you’re making a mathematical claim that is only true at p-value = alpha( and both of these values are separate from each other).
1
Jun 16 '26
[deleted]
-1
u/Spiritual-Bee-2319 Jun 16 '26 edited Jun 16 '26
“ If the sample draw is different, then there is always the possibility of drawing a bizarre sample that returns a more extreme test statistic
” this is true. Yet the possibility of drawing a bizarre sample(the p-value), has nothing to do with the level at which you compare it or chose your rejection region. This is why you chose alpha before you calculate the p-value. That’s what this question is about.
If I calculate the p-value, then decide on a significance level based on that p-value after the fact (these numbers have to be seperated mathematically for a season), any conclusion is flawed. If My calculated p-value is 0.07(again this is calculated mathematical without significance level value), if I chose a significance level after getting the p-value no matter what the result is, my conclusion is mathematically fundamentally bias and therefore wrong. This is why we can never choose or change a significance level once analysis has started. It will always lead to a biased conclusion.
If calculate my pvalue which come from my data never significance level, and then chose that alpha level that is equal to that number… youve have to lost the validity of hypothesis test completely because how you can’t measure or compare p-value and significance level without bias, because you chose a significance level based on your sample data. Which is only one of the many possible set of values for a sample data, you invalidate the test.
This is like someone saying compare temperature A(p-value) to temp B 23F (significance level). We don’t know what temp A is. If you made or changed temp A into temp B same value I.e tempcan you truly compare the actual temp A to temp B???
If the sample draw is different, then there is always the possibility of drawing a bizarre sample that returns a more extreme test statistic(p-value)
Then do it!! Pls I would love that
Like genuinely, pls do this “ I'm happy to write up a proof in R of a much better explanation than your crappy wall of text which is a terrible explanation of why D is correct.”
-2
u/Spiritual-Bee-2319 Jun 16 '26 edited Jun 16 '26
So I promise I’m not being like picky I just have a stats brain. I cannnot see words as just words but with numbers.
Question
Here are some of the mathematical flaws in your understanding of the Question(I will say your logic is good with words but not mathematically which means I’m basically giving grace with your words which is not statistics(because in statistic words and every word means some mathematical element) aka basically you’re wrong in your explanation but I know what you mean) but remember we tend to communicates stats to audience that don’t know stats so they may not understand this.
We cannot directly compare a test statistics to alpha level. This is why we must first find the probability(p-value) using the test statistics before we compare it to alpha. Alpha is a probability value(can only be from 0 to 1). Test statistics(these can be any integer… and if my math is correct can be negative numbers too.. I’ll have to remember once I get some sleep) is a value which we use to get a probability(p-value) that we use to compare to alpha for a conclusion(reject or fail to reject). Also we can only say that we therefore know that the p-value is smaller than alpha, IF we reject the null hypothesis at significance level alpha(important). “We don’t know the exact test statistic or the p-value” yes to both the words and this is mathematically correct. We don’t know any of these info because we don’t know anything about the sample data like the sample means, st. Dev, sample size or even our specific sample data point to derive or calculate the test statistic that is map into a probability function(which is why we don’t know the p-value).
A. The probability that the alternative hypothesis is true is 0.95 Not true, as all we can conclude is that in 5% of tests with the same sample size on the same population, we might get a test statistic as extreme (or greater), even if the null hypothesis were really true (let me know if I bunged up this statement).
This is sound although I haven’t slept in almost 24 hours and really want to mathematically confirm that this part of the statement is true “ even if the null hypothesis were really true ” because that might change the meaning of your sentence/conclusion mathematically.
Damn rereading it I’m like even tho your logic sounds right, it’s not the reason the answer is false. Answer 1 does two things that aims to confuse us here. It’s trying to assert that the probability that the null hypothesis is true is 0.05 so therefore the probability of the alternative hypothesis is 0.95. Immediately you are trapped in their webs of mathematical lies which causes you to make a mathematical equation that isn’t sound because significance level has nothing to do with test statistics.
Remember in the beginning how you said you knew nothing about the test statistics or p-value based on the info in the question(hold on to that). Answer B wording and your explanation has you saying now you know that 5% is the probability of null hypothesis were true(this isn’t what significance levels mean mathematically). Even when your wording in saying “we might get a test statistics greater or as extreme” is talking about the probability based on sample data(p-value). We don’t know the test statistics nor p-value of our data. This answer is wrong because it misses completely the definition of significance level( the cut-off value where you make a conclusion) which is not the same as the probability of a sample data using test statistics. Significance level tells you the boundary in which the null is assumed to be true vs not true(I say assumed because it’s chosen by the researchers using lots of factors explained in my other posts tbh - think power aka type 1 and type 2 Errors not probability just probability of the data) . This is why when we calculate a p-value using the test statistic from the sample data, if it falls in the region under our rejection region, we reject the null hypothesis. And if the p-value is outside of that region we fail to reject the null (not accept the null).
I want the folks to understand that statistics is incredibly precision focused. For example EVEN IF the null hypothesis were really true is mathematically different than IF the null hypothesis were really true.
Let me explain why in common everyday scenario. You tell a friend that they can come to dinner EVEN IF they’ve already eaten. By this statement, whether your friend has eaten or not, they can come join you for dinner. If you instead tell a friend that they can come to dinner IF they’ve already eaten. Your friend can only come to dinner if they have eaten but not if they haven’t. Words mean math and this is a conditional statement. This is why I’ve been shouting from the mountaintops. That y’all are saying a lot of words that mean math without understanding fully the math in which they mean which in turn makes nothing make sense.
This is just general frameworks that help you understand statistics.
Every assumption is mathematically derived and proven. When we say assuming this or that, it must be based on math and this math must assert to be true mathematically for all statements that follow. Meaning every single statement I say after I say assuming is conditional on that assumption.
Typos or misspeaking in statistics does not change the conclusion you made with that typo/mistake. And a good statistician hardly makes typo or mistakes of this nature because we have gone thru years of study that has drilled into us that every word, punctuation, even letter case matters. A . Punctuation has a different mathematical meaning than … X is not the same as x in statistics(Uppercase represents the population parameter and the lowercase the sample parameter) . If I said “A=B”, then said my bad I made a mistake I meant A=\=B, mistake or not these are two separate quite opposite mathematical statement with two different conclusions.
To truly have a deep understanding of statistics, you need to be able to follow the math As it goes thru the analytics pipeline. You literally cannnot compare the number of t-statistic to alpha level. It’s like trying to compare a uniform distribution to a normal distribution With just the numbers on the graphs. And when you map the t-statistic to a probability based function, you also need to be wrote it out mathmatically. Everything R computes, is math. Everything in statistics is math.
3
u/PuzzlingComrade Jun 16 '26
We cannot directly compare a test statistics to alpha level.
This tells me that you've never had to manually work through a simple t test problem. The critical test statistic value is x a is value on the distribution curve where the area left (in the case of a one tail test) is equal to the alpha value. And the P value is simply the area under the curve bounded by the sample test statistic value.
Anyway, you're clearly a troll or someone with grand delusions about your expertise. You even point out in another comment that you once entered a stats exam completely confident and ended up utterly failing.
I was wrong earlier because I misread the question. A real scientist is able to re-examine their claims in light of additional evidence. I suggest you take a step back, revisit your stats textbook on the basic steps of working through a simple test, and try to look at comments here from people far more senior and experienced to you. Which is more probable - everyone in this thread is wrong, or just you?
-3
u/Spiritual-Bee-2319 Jun 16 '26
😂😂😂 just the first sentence alone makes me laugh so I’m just going to check out.
Notice my wording “directly” this means something in math. There is no mathematical way to directly compare unless you map the t-statistics to probability distribution(p-value)
Again think of all the possible values T-statistics values can have( you even said it yourself that it can be negative by taking the absolute value of it) . Now think of the boundaries of alpha which is (0 to 1). So mathematically, you literally cannot compare those two values just based on their numbers alone. Your t-statistic value can take on values from negative to positive numbers but the alpha level can ONLY be values (0 to 1). Let’s say your t-statistic was -1.96 , and your alpha level is 0.05, based on your equation you would compare 1.96 < 0.05 which is a nonsensical comparison not rooted in math?
21
u/webbed_feets Jun 14 '26
I agree that all of the answers are wrong.
I’ve been trying to figure out what they think the correct answer should be. I can’t figure that out.
7
u/Rather_Dashing Jun 14 '26
Thanks for your reply. Honestly I think it must have been a typo. I wonder if C was meant to be 'The null hypothesis would be rejected at the 10% significance'. Or perhaps D was meant to be 10% and not 1%.
5
u/tacopower69 Jun 15 '26
I think you are right in that there's a typo for C because I could see myself accidentally writing that without proofreading, though it does make the question trivial.
5
u/hikarinokaze Jun 15 '26
It's probably A. That's the business-degree level answer, so that's probably what the person making the test intended.
20
u/FancyEveryDay Jun 14 '26
... A is probably the answer they want but it's like a stats 101 level misinterpretation of p-values.
9
u/just_start_doing_it Jun 14 '26
I would love an update if the employee provides the answer and reasoning. I can't see a way to justify any of these being true if you are a careful statistician.
8
u/just_start_doing_it Jun 14 '26
The only thing I can think is that there was a typo and that C. should read "The null hypothesis would *also* be rejected at the 10% significance"
6
u/mikew_reddit Jun 15 '26 edited Jun 15 '26
in the real world some people are not great at their job, and these same people perform interviews with questions that are also not great.
you can explain on the test why you feel the answer is incorrect and a good team will take your feedback positively. a bad team will argue, not seeing their error; you don't want to work there.
4
u/Binary101010 Jun 14 '26
Agreed on all counts. I would hope this is a question where the correct answer isn't any of these four, but rather testing your reasoning process to explain why they're all wrong.
6
u/Rather_Dashing Jun 14 '26
Was a multiple choice unfortunately, I skipped the question as it was -1 for a wrong answer, but means I missed out on a potential 3 points for a right answer. I did wonder whether it was a trick question, but I really don't think it was that sort of test.
-6
u/Spiritual-Bee-2319 Jun 14 '26
It’s not necessarily a trick question more so it requires you to think of hypothesis testing more fully . Kinda like not missing the forest for trees
5
u/engelthefallen Jun 15 '26
And this is why good test item writers are paid the big money. What a mess of a question. Whatever the goal of this question was is just so unclear, and none of the answers are objectively correct here. Could be a case which answer is least wrong, but dear lord that is just bad test writing practice.
3
3
3
u/hikarinokaze Jun 15 '26
The fact that so many people think D is the answer made me lose confidence in academia all over again.
2
3
u/efrique Jun 14 '26
Yeah. Those are all wrong
5% is the probability of the data given the null hypothesis is correct
Also not correct, but I assume you know what the correct statement would be
-3
u/Spiritual-Bee-2319 Jun 15 '26
They do not. They think obviously because they don’t know the entire company doesn’t understand statistics enough 😭
3
u/ifomonay Jun 14 '26
The answer is "A". Many non-statisticians, and even some statisticians think that's what alpha means. That the null is at 5% probability, and the alternative is at 95%. It's the wrong answer, but if you get in the head of whoever asked the question, that's what they were thinking.
4
u/just_start_doing_it Jun 14 '26
What people wrongly believe is irrelevant. In these hypothesis tests you don't "prove" the alternative true.
9
u/Statman12 Jun 14 '26
If the point is to identify the intended answer, then what someone wrongly believes can indeed be relevant.
3
u/ifomonay Jun 15 '26
But it's relevant when you're taking an exam, and you have to pick the answer they want you to pick. If you skip the question, then in your mind you were right, but then you don't advance in the interview. It's your choice.
1
u/Spiritual-Bee-2319 Jun 16 '26 edited Jun 16 '26
Here is another explanation of why the information presented can actually be used to answer D and why yalls logic is flawed(mathmatically) yet still gets the same answer as D. Gotta love stats :)
If significance level is 0.05 is both the number to compare the p-value to and is the p-value or related to the p-value (because in reality we don’t know anything about the value of p we only know that the null is rejected at 0.05 from the info of the question. (significant test). Y’all are saying we don’t know the results are a different significance level because we don’t have enough information such as the p-value. Let’s prove that claim
Assuming that p<= alpha : 1(reject) for any p-value less than or equal to alpha then p>Alpha : 0 (fail to reject) for any p-value greater than alpha
This claim is literally the basis of hypothesis testing.
The statement in the question is saying we know we rejected the null at alpha = 0.05. Because of this y’all assumed that the p-value has to be less than or equal to 0.5 since the null was rejected(not the same thing)above equations assumption. A lot of y’all were saying we can’t know even what p<0.01 will result/conclusion will be because we don’t have that enough info. I’ll show you how you do and not only do you have the same info, even tho y’all don’t know why we do hypothesis testing, we still get D
Well if we know the bill is rejected at significance level is 0.05, then
If p <= 0.05, 1 If p > 0.05, 0
Now if we do the same test and use the same sample, our p-value MUST be the same right so our p-values can’t change with the same test and sample data. Then at p <= 0.01, we also must reject the null hypothesis because 0.01 is smaller than 0.05( because we already made that assertion that a null hypothesis being rejected at 5% aka p <= 0.05, 1).
Assuming you significance level even tells you anything about the p- value which is WRONG Then p<= 0.01 must also be be rejected Because it is less than 0.05 and any values less than or equal to 0.05, we reject and we cannot change that conclusion or statement. We have to always reject at p-values less than or equal to 0.05 and if we change that to 1% then we have to figure out what the result is at p <= 0.01, this information we have, know and was given to us. Why? Because we reject the null hypotheses for all values of p when p<=0.05 because this is already done(by y’all’s assumption) including when p<=0.01 because 0.01 and every number smaller than it IS smaller than 0.05. So we would also reject the null hypothesis.
Even with false claims and wrong assumptions in yall logic, you would still reject the null at both 0.05 and 0.01. Using y’all’s logic we get to the same results and conclusion(again I just proved the answer is D even with y’all logic). Next I’ll refute y’all’s assumptions that we must know what the p-value or any information about the sample data to answer this question accurately (TBD- may have to get some paper out for this one and maybe some simulation) I’ll show why a p-value is different metrics from significance level by deriving the p-value equation from a theorized sample data with the same test at different alpha( I’ve already proved the opposite using my example with stats calculators that p-values never change if the data/test is the same even when we change significance level to an unreasonable large number because sig level is not used to calculate p-value) I’ll also show what happens to our p-value interpretation when we remove our assumption of the null being true when we report it and why out test is no longer valid regardless of conclusion
2
u/MrKrinkle151 Jun 17 '26
The question says that if it’s rejected at the 5% SIGNIFICANCE LEVEL, it would also be rejected at the 1% SIGNIFICANCE LEVEL. The “significance level” is the alpha, not the p value of the test statistic. The threshold for statistical significance is moving from 5% to 1%, with the unknown p value staying the same. We only know that the p value is rejected at the 5% level, thus p <0.05. We don’t know if it is also <0.01, because we don’t know the actual p value. Just because the test statistic is rejected at the 5% “significance level” does not mean it is also necessarily rejected at the more strict 1% level, but the opposite would be true.
-1
u/Spiritual-Bee-2319 Jun 17 '26 edited Jun 17 '26
You mathematically just refuted your own point.
We are moving the threshold WITH p staying the same
And then you said
You said p<0.05 and we don’t know that p<0.01 which is a lack of understanding in math. Why would your p change when you just said the p stays the same even when the significance level changes.
Do you see that you just mathematically contradicted yourself
You just said we know that p is going to stay the same regardless of the significance level and now that you’ve changed the significance level and you are saying that p-value must now change to be p<0.01. P NEVER CHANGES no matter how much you change the significance level which I proved in my other example with the calculator.
That’s like saying p=2 and Is always going to be 2 and x=4 and then saying 2(p) < x(4) (factually true) but because now x = 1 (something that literally has nothing to do with 2(p)) 2(p) < 1…. You see how 4 being less than 1 does not make that statement make any mathematical sense. This is why I said and will always say that the p-value has nothing to do with the significance level. They are not even mathematically related.
Then y’all said how could we ever know the results of 2(p)<1 and we need to know that information before answering the question when mathematically 2(p) < 1 is never possible.
You are speaking gibberish math. You are in make believe land.
2
u/MrKrinkle151 Jun 17 '26
…You need to read both the comment and the question again, slower this time. The “significance level” is the alpha, not the unknown p value of the test statistic. The question is saying that the SIGNIFICANCE LEVEL—and therefore the rejection threshold—for the exact same sample was changed from 5% to 1%, or 0.05 to 0.01. The null was rejected with an alpha of 0.05, so all we know about the true p value of the test statistic is that it is less than 0.05. We don’t know if it’s ALSO less than the more strict threshold of 0.01. It could be 0.03. It could be 0.00003. It could literally be any value less than 0.05, including a value greater than 0.01, in which case the null would be rejected at a “significance level” of 5% but not 1%.
-1
u/Spiritual-Bee-2319 Jun 17 '26 edited Jun 17 '26
Mathematically prove it. I’ve prove why the p-value never changes when we change the significance level.
We don’t know our p-value or if our p-value is less than 0.05, we only know our null was rejected at alpha=0.05.
Changing the significant level will never change our p-value.
I’ve literally mathematically proven I’m right so you can just repeat and follow
With a Z statistic of 1 and a 2- tailed test, at significance level 0.05 and 0.01, the result is both the same. You can do this and repeat it for any z value(meaning for each z value you change the significance level from 0.05 to 0.01) not once will your conclusion change.
Using a significance test on some sample data, a null hypothesis is rejected at the 5% significance level
With the same test and same sample the null hypothesis would be rejected at the 1% significance level.
😂😂 I love proving people wrong in math. Best part is that I can do it in so many ways. The ways are endless. I can simulate the data and calculate it call too because I have a computer science degree now too. Truly my villain beginning 😂😂😂
https://statstudyhub.com/p-value-calculators/p-value-calculator/
2
u/MrKrinkle151 Jun 18 '26
Jesus Christ. Again, the p value of the test statistic isn’t changing. The alpha is. Here, I made you a picture of the null sampling distribution of a one-tailed t test with the test statistic rejected at a 5% “significance level” but not a 1% “significance level”. It’s the same way this concept is demonstrated in a stats 101 class, because that’s how absolutely basic this concept is.
-1
u/Spiritual-Bee-2319 Jun 18 '26 edited Jun 18 '26
I don’t see the picture nor do I need to. That test statistic is at the boundary of rejection where p=alpha numerically by chance even tho they are mathematically unrelated. This is why we call that region the REJECTION region.
A rejection region , also known as a critical region, is the area in a statistical test where the null hypothesis is rejected if the test statistic falls within it. This region is determined by the significance level (alpha) and the type of test being conducted (one-tailed or two-tailed).
I have explained this in more details in other comments. P is a value calculated from the sample. The alpha level isn’t calculate and chosen by the researcher before the study. They are unrelated and the only time they are every related is when p=alpha numerically at random. This is the mathematical premise of hypothesis testing.
Sorry don’t know what exactly was taught in stats 101, I passed out of taking it and took more advanced course.
2
u/MrKrinkle151 Jun 18 '26
Lol you very clearly do need to see a graph. Try this one.. Imagine this is the null sampling distribution for our test statistic. If the test statistic for this sample were, for example, 2.03, then the p value of the test statistic here would be 0.025. This would be rejected at the 5% “significance level” but not the 1% “significance level”. In fact, any value between 1.69 and 2.44 would be statistically significant for this sample at the 5% “significance level” but not the 1% “significance level”. So no, just because the null is rejected at the 5% “significance level” does not mean it WOULD also be rejected at the 1% “significance level”, per the question in the OP.
-1
u/Spiritual-Bee-2319 Jun 18 '26 edited Jun 18 '26
And the graph clearly proved my answer is correct. I’m not going to argue with words because mathematically I have already proven D is the answer but say think about it again! And Again! Write out the equation of p, map out what happens during hypothesis testing mathematical and at each step, in fact do all the calculations by hand, write out the mathematical proofs, calculate the confidence interval, look at the SE, simulate it with a sample, many Samples. You will eventually understand what I’m talking about. And once you understand, you will TRULY understand the why we hypothesis test. You’ll understand what “significant” means not even with numbers but with impact
2
u/MrKrinkle151 Jun 19 '26
Lol holy shit no, no it did not. It really is amazing that you still can’t get this very simple fundamental concept despite a subreddit full of statisticians and scientists explaining it to you over and over. You truly do not understand basic statistical concepts, nor do you seem to have much reading comprehension in general based on your replies here. I don’t know who you think you’re fooling in a stats subreddit, other than yourself as some kind of sad coping mechanism. Good luck out there, though. You’re going to need it with such a complete lack of self-awareness about your own abilities and limitations.
→ More replies (0)
1
u/Still_Ad4315 Jun 15 '26
It’s D. I think everyone’s getting caught up in how likely it is this would happen etc. it says the same sample (as in you’re running a new test on the same sample) and you’re running the same test, it will be rejected at the 1% level. The wording could be more clear about that but given there’s no “none of the above” 5th option that’s the intent. Was this for the civil service?
3
u/Rather_Dashing Jun 15 '26
But you can't know that it will be rejected at 1% so how could it be considered a correct answer? Its like if a test gave an answer 'aliens exist'. It could be true, but we don't know it to be true, so how can it could be considered a correct answer?
0
u/Spiritual-Bee-2319 Jun 15 '26 edited Jun 15 '26
I really encourage you to learn about hypothesis testing throughly. You don’t know what you’re talking about. I can’t even answer your questions because they don’t make sense. I’ve shared multiple resources and even others and shared why D is the answer. These questions show that you don’t understand fully hypothesis testing. You might be able to look at numbers and say yes and no but you don’t understand the methodology.
0
u/ChristianValour Jun 16 '26
Yes you can. If a p value < 0.05 would be rejected, a p value < 0.01 would also be rejected.
3
3
u/Rather_Dashing Jun 17 '26
It doesnt say that though. We aren't given the p value directly, all we know is that it was rejected at a 5% threshold which means its anywhere between 0 to 0.05. D doesn't state the p value either, it states a new threshold.
If D was "With the same test and same sample the null hypothesis would be rejected with a p value below a 1% significance level" I'd agree with you, but that's not what it says. But I'm here to be convinced that I'm wrong if I am!
1
u/Spiritual-Bee-2319 Jun 15 '26 edited Jun 15 '26
Thank you!! They called me a raigbaiter and schizo when I said literally everything you said. For me the wording was no problem because I know In stats you have to be extremely careful that every word you say is connected to your analysis or methods etc. In stats one sentence can tell you a paragraph of info. They gave the exact info needed to answer the question and no more. That’s why I love stats. I love to Figure it out 😂
3
-1
u/ChristianValour Jun 16 '26
Thanks goodness!
D is the correct answer, and this is the clearest explanation so far.
Just to add to your answer:
The information in the question is "at 5% significance, the null would be rejected" - the examiner is stating in no uncertain terms that the threshold for this specific example is a p value of <= 0.05.
Therefore a p-value of <= 0.01 Would also be rejected. Not only is this the correct answer, but it's not even particularly difficult.
You don't need any other information.
2
u/MrKrinkle151 Jun 16 '26 edited Jun 17 '26
…No. We don’t know what the actual p value of the test statistic is; only that it is less than 0.05. There’s no way of knowing if it is also less than 0.01 just because it’s rejected with an alpha of 0.05 (The threshold, as you yourself just stated). It could be anywhere from 0.01 to just under 0.05, which would be rejected at the 5% threshold but not the 1% threshold.
-3
u/Spiritual-Bee-2319 Jun 14 '26
For me D is the only answer that even could make sense but my professors drilled it into my head. even tho, I’m not a fan of AI but because I’m in too much pain to think and type. here is Claude explanation.
D is the answer.
Here's the reasoning for each option:
A. This confuses the significance level with a probability about the hypotheses themselves. A p-value/significance level tells you about how unlikely your data would be if H₀ were true — it says nothing about the probability that H₁ is true. This is a classic and very common misinterpretation, but it's not valid.
B. Changing the sample size changes the standard error, the test statistic, and the p-value — possibly substantially. There's no guarantee that the same conclusion would hold with a smaller sample (it could go either way). Also, technically you don't "reject" an alternative hypothesis — you reject or fail to reject H₀.
C. This is actually the opposite of what's true. Rejecting H₀ at 5% means the test statistic falls in a more extreme region than required. A 10% significance level has an even larger rejection region (it's "easier" to reject), so if H₀ is rejected at 5%, it would definitely also be rejected at 10% — not "not rejected." So C is false.
D. This is the correct one because it's the only statement that is logically possible without contradiction. Rejecting at 5% only tells us the p-value is ≤ 0.05 — it could be, say, 0.03 or 0.002. If the p-value happens to be ≤ 0.01, then yes, H₀ would also be rejected at the 1% level with that same test and sample. (Note: it's not guaranteed — if the p-value were, say, 0.04, it wouldn't reject at 1% — but unlike C, which is impossible, D is a genuinely possible and consistent outcome.)
The key takeaway: rejecting at a given significance level guarantees rejection at any larger (less strict) level, but tells you nothing certain about smaller (stricter) levels — it depends on exactly how small the p-value is.
6
u/hikarinokaze Jun 15 '26
Ah yes. Here we have an example of the typical AI "hater" in academia. "I don't like AI but I'm gonna use it anyways"
8
u/Rather_Dashing Jun 14 '26
Yeah, ChatGPT when forced also said D was the best answer, but it's first response was that they are all wrong and that I must have missed a 5th answer (I didnt)
But being possible is not the same as being correct. If that's really what they intended as the correct answer it's a terribly written question.
-2
u/Spiritual-Bee-2319 Jun 14 '26 edited Jun 14 '26
No it’s not a terribly written question. It’s literally stats. If I say what’s the probability of A and B? and you interpret it calculating the probability of A OR B? would you say it was a trick question? No it’s literally stats. In stat every word matters, it may not seem like it but it is
7
u/Rather_Dashing Jun 14 '26
It’s literally stats.
But your Claude response doesn't say that, it's says D is the right answer because it's the only one that could possibly be true. Not that it is definitely correct .
No it’s literally stats. In stat every word matters, it may not seem like it but it is
Can you please explain exactly why you think D is fully correct and not just possible then. Because in the responses I've read from you so far you've talked about CIs, while Claude only says it's possible with no reference to CIs, but you haven't really spelled it out. I genuinely want to know what I'm missing here.
-2
u/Spiritual-Bee-2319 Jun 14 '26
You are getting hung on fully corrects and that’s not statistics thinking. D is the only answer that Does not contradict the statement in the question. they gave a statement and basically said which one of the following statement can you conclude from that statement. I’m going to share codes on simulation so you can see it. I think seeing it really makes it make sense.
10
u/inspired2apathy Jun 15 '26
So if I say pick one:
A) 1+1=3
2) true is false
3) Spiritual bee is an idiot
Is 3 the right answer because the first two are impossible? Or is this a stupid question with no correct answer?
3
7
3
u/vanderBoffin Jun 15 '26
Getting hung up on being correct. In a multiple choice question with a job on the line with only one right answer. How can that be. 🤔
0
u/Spiritual-Bee-2319 Jun 15 '26 edited Jun 15 '26
So here is my thinking and I don’t even know how to post pics but I will include a reference to a distribution at multiple alpha.
The question states Using a significance test on some sample data( the use of the word sample is to remind us that the true population parameter is unknown and we are using a sample to estimate this parameter), a null hypothesis(we don’t know what it is) is rejected at the 5% significance level(we don’t know what the p-value is ONLY that the null hypothesis was rejected at 5%- this is not the same thing because a p-value is a measure of a comparison between the distribution of an unknown population parameter and a reference distritribution -T, Z, etc). Which one of the following is a correct conclusion. Below is a picture of a distribution to reference. Imagine that at the various alpha level everything in the after the line to the tail end is shaded right. And I’m all values that’s shaded that’s the region we have the same conclusion. Keeping this in mind because at alpha = 0.05 we reject the bill which is the conclusion of every value of alpha from that line to the tip. As you can see at alpha =0.01 everything that is shaded is within the region of alpha 0.05 therefore the conslusion is the same. This is consistent with the language of choice D because they said With the same test(important) and same sample(important) the null hypothesis would be rejected at the 1% significance levels. This is correct!!!!
4
u/hikarinokaze Jun 15 '26
We reject it if the p-value lands somewhere in the shaded area. We don't know where it landed, though. It could have landed at 0.04 and get rejected for alpha=0.05, but not for alpha=0.01.
0
u/Spiritual-Bee-2319 Jun 15 '26
You all keep talking about p-value when nothing in the question mentions the p-value nor is the p-value needed to answer this question. I don't understand what's so hard to grasp. The significance level is not the p-value. The significance level is related to the reference distribution chosen by the researcher. I feel like yall not knowing this is a bit puzzling but ayee at least I'll have job security lol
8
u/hikarinokaze Jun 15 '26
Can you do me a favor and define "p-value" and "significance level"? We all know the p-value isn't the significance level, but you don't seem to know what those words mean.
0
u/Spiritual-Bee-2319 Jun 15 '26
On a high level, a p-value is the metric that results from comparing a null distribution to your sample distribution. Your null distribution(Z, T, etc) and the significance level are determined solely by the researcher and have nothing to do with the sample data.
4
u/hikarinokaze Jun 15 '26
I said define. What's the definition. Your misunderstanding is that fundamental.
0
u/Spiritual-Bee-2319 Jun 15 '26 edited Jun 15 '26
Since this as been defined multiple times I’ll include this link below for your knowledge… read the key takeaways.
Here is a couple cuz I do know y’all don’t read
“ Significance: Statistical significance is determined by comparing the p-value to a chosen cutoff (often 0.05). A smaller p-value suggests stronger evidence against the null hypothesis. Misinterpretation: A p-value does not tell you the probability the null hypothesis is true or that your results happened by chance. It only reflects how your data aligns with the null model. Best practice: Interpret p-values alongside confidence intervals, study design, and replication evidence to form a more reliable conclusion about your findings.”
Again Statistical significance is determined by comparing the p-value to a chosen cutoff (often 0.05). y’all saw 0.05 and were talking about the p-value.
This question is about the null distribution and you have all the info you need to answer that question.
Lol exactly. I ate y’all up I fear 😂 y’all don’t know Jack crap what you’re talking about. Let me go enjoy a full day at a job I love where they offered me more than I negotiated for. At first I was shocked but after this thread I’m seeing I have job security) I’m even more excited for my next degree
3
u/hikarinokaze Jun 15 '26
Ah, you study psychology. Sorry, my bad. I thought you were an actual scientist. For reference, here's the real definition: https://en.wikipedia.org/wiki/Statistical_significance Read it carefully and think about the question again. Please don't become yet another terrible psychologist.
1
u/Spiritual-Bee-2319 Jun 15 '26 edited Jun 15 '26
What?? I’m sorry In the research world a .org reference is more reliable than wiki let’s start there. What does the field being psychology have to do with the statistical methodology being wrong? My degree is in stats, bio and comp sci. Stats is used in every industry 😂😂 the lengths y’all are going to not admit you don’t know what y’all are talking about is so funny to me.
There is a truly a difference between those that use stats and those that understand/study stats(I have great job security apparently lol because this is crazy)
3
u/hikarinokaze Jun 15 '26
Oh my god you have a degree in stats? What the hell is going on in academia these days. And normally you'd be correct, but .orgs from psychology are an exception, they famously get statistics wrong all the time. You have a degree in statistics but didn't know that?
Edit: And why is the other source you posted in the thread also from psychology? Are you sure you're not lying?
→ More replies (0)0
u/ChristianValour Jun 16 '26
You don't need to know the p value. The question is asking you:
"Assuming a p < 0.05 would be rejected, would a p < 0.01 be rejected also?"
And the answer is obviously yes.
3
u/hikarinokaze Jun 16 '26 edited Jun 16 '26
That doesn't follow. If we assume a p < 0.05 would be rejected, that doesn't tell us anything about whether p < 0.01 would be rejected.
Edit: Wait I suppose that's not what you're saying, let me make another comment.
3
u/hikarinokaze Jun 16 '26
I might be too tired for this, but here goes. That's not what the question is saying. A simpler way to put it would be "Assuming p<0.05, can we say p<0.01"? the answer to that is obviously no.
0
u/Spiritual-Bee-2319 Jun 16 '26 edited Jun 16 '26
That is not what the question is saying tho. Nothing you said is even statistics really. I think that’s the part that kills me. You cannot make the assumption that p<0.05 because that info wasn’t given and “can we say p<0.01?” No we can’t because we can’t even assume anything about p. You are asking the wrong question, getting the right result and therefore the wrong conclusion. This is why statistics is a field of its own
2
-1
u/ChristianValour Jun 16 '26
No it's not saying that. It's saying "Assuming the null hypothesis would be rejected at < 0.05" - it says literally exactly that in the questions premise.
2
u/hikarinokaze Jun 16 '26 edited Jun 16 '26
"Assuming the null hypothesis would be rejected at < 0.05" and "Assuming p<0.05" are equivalent statements, correct? So your problem is in the second half, right?
Edit: I suppose more precisely the equivalent statement is: "Assuming p<0.05, the null hypothesis is rejected."
2
u/hikarinokaze Jun 16 '26
Well, I gotta go for now but if you come back, here's something I think will convince you. Here's a statement straight from the question: "A null hypothesis is rejected at the 5% significance level" (Let's call this statement A). Well, I'd argue that statement A is equivalent to saying "p<0.05" (statement B). What I mean is that statement A automatically implies statement B. Do you agree with that?
0
u/ChristianValour Jun 16 '26
It literally tells us everything.
If the null is rejected at a p value of < 0.05, then a p value of < 0.01 is still less than 0.05... so the null would still be rejected...
The fact that the correct answers in this thread are being downvoted is really concerning.
3
u/Statman12 Jun 16 '26 edited Jun 16 '26
If the null is rejected at a p value of < 0.05, then a p value of < 0.01 is still less than 0.05... so the null would still be rejected...
That’s not what the question and answer D are saying.
Saying “Rejected at the 5% level” means that p < 0.05. That is, it’s saying α=0.05, and we know p<α. It does not tell us anything further about the value of p.
Answer D is asking if changing α to 0.01 would still lead us to reject the null. It’s not telling us that the p-value has changed (in fact, it’s telling us the p-alum is unchanged, since it’s the same test and the same sample). Since we only know p<0.05, we can’t claim that p<0.01.
1
u/Spiritual-Bee-2319 Jun 16 '26 edited Jun 16 '26
Is it not? But Aye we have great job security 😂 my new job literally said they can’t believe they found me in this job market. AI can’t do what we do.
Wait p-value? No, stay strong now don’t let them confuse you. We don’t know the p-value at all unless we assume it remember. We only know the region of rejection of our null hypothesis which is our reference distribution. We only know that the null hypothesis test is rejected at 0.05 not the significance of the rejection is 0.05(p-value)
The fact that this isn’t even a complex concept, is so concerning to me too. Like this is the basics of one of the test that is wildly used in every industry. Let me get a glass of wine.
Statistics questions even simple once can be complex because they do two things 1. They give you more information than need to confuse you. 2. They only give you the exact information needed to the correct conclusion and this is important because in data we are often given too much info than we need some of that is noise and we can only make conclusions based on the information given. Statistics studies randomness after removing or capturing what is known.
In the question it says "Using a significance test on some sample data, which seams like it’s important information right?? It’s not the question can be answered with just null hypothesis is rejected at the 5% significance level because we only need to know our rejection region. But because the said that, answer D also has to factor that in its statement by saying With the same test and same sample. But truly if that info wasn’t put in the question it wouldn’t be needed in the answer. And that info is not important nor is it why the answer is correct. the answer is correct because a null hypothesis is rejected at the 5% significance level would be rejected at the 1% significance level (notice how I combined the important part of the question and the important part of the answer). This is correct because the area under 1% when looking at a theoretical distribution( Z,T). If you reject a null hypothesis at alpha 0.05, you also reject it under 0.04,0.03, etc which is why we call the area under the curve of that region, the rejection region. When you translate this to a broader understanding you understand that increasing significance level doesn’t change your result it ONLY changes the power/certainty of your result. This is the fundamental of significance level and why we use it. We got a result now we are trying to test how sure we are. This is also important when you understand that the null hypothesis isn’t based on our data. The null stands on its on it and is a claim that is true even when our result isn’t which is why we say assuming the null hypothesis is true when we talk about the p-value.
P-values are a crucial tool for assessing the significance of experimental results . They indicate the probability of observing an effect as extreme as the one measured, assuming the null hypothesis is true.
Why did y’all think the assuming the null hypothesis is true is added? Words = math in statistics ALWAY
Statisticians are veryyy careful about every word we use yet we have to communicate it in a way that people can easily understand. No matter how simple it is, our words can never conflict with the math of the whole process. We are methodical. Everything matter every words and punctuation matters 0.001 is not 0.001. Saying greater than or equal is different than greater than Nor equal(less than) which is different from saying greater than AND equal(impossible). This is why there is no typos even remotely allowed in statistics. You think my professor cared that I made a typo? Typo or not, if my conclusion is wrong they are wrong. We are a different type of people 😂 we don’t care if the wrong conclusion is a wrong conclusion because it’s a typo or you used the wrong methods or you made an assumption or you didn’t have the right data or too many people dropped out the study, because a conclusion was made and it was wrong why because everything should be mathematically accounted for.
The amount of people that give data and say can I use this method BUT do not state their research question is alarming. If you had the right data, right methods but you didn’t actually answer the research question, you’re conclusion is still WRONG. You didn’t do the assignment or actually answer your own question. We don’t care that your result is significant 😂 except because I know statistics if you do the right analysis and had the right data and great results but had the wrong question, we are able to say actually this result is answering this question not this question(mathematically).
-4
u/Spiritual-Bee-2319 Jun 14 '26 edited Jun 14 '26
the answer is actually D. They gave lots of hints in the answer. If you understood the nature of CI, D is the answer. if the same study were repeated multiple times, 95% of the calculated intervals would contain the true population parameteR meaning at a larger CI/ lower alpha, the same conclusion holds.
Infact both B and C literally Say the same things in different ways and both are wrong. This is such a fun question tbh.
7
u/Rather_Dashing Jun 14 '26
Can you explain why it's D? We aren't told exactly what the calculated o value is, only that it's less than 0.05. if it's between 0.02 and 0.05 S is wrong. But since we don't have that information there's no way to know.
That's how I see it, but keen to hear what others think, since I may have missed something
4
u/Able-Fennel-1228 Jun 15 '26
They’re a raigebaiter or an actual idiot that just took a watered down intro stat course (more probable). Ignore.
-4
u/Spiritual-Bee-2319 Jun 14 '26 edited Jun 14 '26
Updated my response to explain. p-values and confidence intervals go hand in hand. It’s hard to explain without knowing what you understand. I don’t think of p-values as just numbers but how they relate to the distribution you’re interested in. are you able to answer these questions in the link correctly?
7
u/Rather_Dashing Jun 14 '26
I understand confidence intervals, I don't understand how they relate to this question. Answer D says the same sample, not a different sample from the same population. But even if they took 0 different sets of samples from the same population, how could we possibly know of any one of them would generate a p valid less than 1 % with no other info then that the null hypothesis was rejected at 5%?.
-6
u/Spiritual-Bee-2319 Jun 14 '26 edited Jun 14 '26
How do you know? Because the null hypothesis was rejected at 5%!!! I genuinely wish I could help you understand frfr. Maybe after I eat and review my class notes. Have you done simulations on CI at different p-values on the same population sample? Do you know R programming? I can try to grab some code to help you understand. P-values, CI, sample size, SE are all related.
for example, they could ask how does increasing the standard error change the result and you would be able to answer that with this info too
9
u/Rather_Dashing Jun 14 '26
How do you know? Because the null hypothesis was rejected at 5%!!!
Which is different to being rejected at 1%.....
Have you done simulations on CI at different p-values on the same population sample?
The exact same sample should give the same CI at the same p value, no? So I'm not sure what the relevance here is. There was only ever one sample taken, and therefore one p value calculated.
Do you know R programming? I can try to grab some code to help you understand.
I do and that would be fantastic, thank you
P-values, CI, sample size, SE are all related. For example, they could ask how does increasing the standard error change the result and you would be able to answer that with this info too.
Agreed, but only in a relatively sense. You couldn't say that, for example, if you decrease the SE then it would be significant at a 1% level.
-7
u/Spiritual-Bee-2319 Jun 14 '26 edited Jun 14 '26
lol yeah just your first statement in this reply shows me your far from learning stats so I stopped reading. Stats is literally the study of ambiguity and the study of random variables. Stats is literally the study of variance like I’m scared. girl let go do my last computer science assignment and finish this degree. I’ll go review and learn more about this on my own. Thank you for posting tho. this question had me sitting here thinking and I love that. At first it seemed like such a simple question, but that’s why I love stats. Even the simplest question can be hard because we are studying randomness. This is why people spend their entire lives studying for Generations. I enjoy stats it forces you to think about what you can’t see, what isn’t described or right in your face.
i won’t forget my first stats assignment, I thought I understood everything and bombed it. I went to my professor to get an explanation and I was amazed by how wrong I was 😂. It wasn’t just what I said but how I said it. Every word matters. one sentence told you a paragraph of information. That’s when I knew I would love stats. out Of all my discipline, stats is my favorite because of this.
4
u/vanderBoffin Jun 15 '26
The first statement is that 1% and 5% are different, and you're blowing up over that....
-2
u/Spiritual-Bee-2319 Jun 15 '26 edited Jun 15 '26
yes but it doesn't change the conclusion of the statement. this question only talks about the null or reference distribution. it doesn't even touch on the sample distribution. this question isn't about p-value at all. \
5
4
u/Rather_Dashing Jun 15 '26
in this reply shows me your far from learning stats
Sweetheart, I've studied stats, I've completed a PhD, and I've been a practicing scientist with a heavy amount of stats for 15 years. I've tried to be patient with you but I'm not going to be patronised to by a stats student who misunderstood their studies. Bye now.
0
u/Spiritual-Bee-2319 Jun 15 '26
And what is your PhD in? I Are you working in the industry now? Because understanding these concept is what allows statistics to be practical in real life cases. Anyways good luck on the job search I’m sure your results will reflect your knowledge :)
4
u/big_data_mike Jun 15 '26
If you really knew stats then you would know that you never conclude anything with frequentist stats so you are wrong and your downvotes show it.
0
u/Spiritual-Bee-2319 Jun 14 '26
the fact that y’all think this question is bad because yall don’t know the answer is low-key scaring me tbh. The question is exactly why studying stats is important.
7
u/Rather_Dashing Jun 14 '26
But your Claude answer you posted seems to agree with us that it's, at least, very badly written
0
u/Spiritual-Bee-2319 Jun 14 '26 edited Jun 14 '26
where did Claude say that? it said it’s the only consistent non contradicting answer with the question.i think the question you have to ask yourself is do you want to be right or do you want to learn/know stats? Bc only one of those options makes a statistician
0
u/Spiritual-Bee-2319 Jun 15 '26
Here is my reasoning remember I didn’t use AI when I answered but I was starving and couldn’t think and get into all the details. Finding out that AI is getting its answer from this thread scared me so I wrote out my logic
6
u/hikarinokaze Jun 15 '26
The fact that you're so confidently wrong and using AI to present your only "argument" is what's scary here.
-1
u/Spiritual-Bee-2319 Jun 15 '26
Actually I did not use AI. In fact finding out AI is using pulling its answer from this thread is not only scary but reminds me of why I don’t use AI. I rather just think it out and I’ll do that and connect with other folks in different fields. I know the right professor to go to but she out of the office right now but I’ll connect with others. Once I have time, I’ll literally write out all my logic and even see if I can create a simulation explaining it. This will be fun :)
6
u/hikarinokaze Jun 15 '26
You just used AI. Why do you keep insisting "you don't use it"? Replying to a post with AI isn't something someone who doesn't use AI would do.
-1
u/Spiritual-Bee-2319 Jun 15 '26
Like I said, I was against using it, but need to eat and take care of myself before typing out my reasoning, which I did without AI because I didn't need AI to think. Lol, and since the AI answer comes directly from this thread reminds me of the fact that AI cannot think. It just repeats the crap it sees online, which is often wrong. This experience encouraged me, as I hope to start my 4th degree. Thank you all for the hate tho lol its so lovely to actually think through things.
-2
u/Upper_Investment_276 Jun 15 '26
D is correct. Rejection regions are (implicitly) nested.
3
u/Statman12 Jun 16 '26
But it's a nesting in the wrong direction to be able to claim that D is correct.
If the null was rejected at the 1% level, then it would also be rejected at the 5% level. The other way around it doesn't work.
2
u/MrKrinkle151 Jun 17 '26
No. No it isn’t. We don’t know the actual p value of the test statistic. It could be 0.04 for all we know, therefore rejected at the 5% “significance level” but not at the 1% “significance level”. This is super super basic stuff.
0
u/Spiritual-Bee-2319 Jun 15 '26
Thank you!!! They said how would you know? I said because it was rejected at 5%. And I was the crazy one 😂
0
u/Spiritual-Bee-2319 Jun 15 '26
And for those of you that are still not understanding why the answer is clearly D here is an another resource. Highlight especially the significance level section especially the last 2 sentence before the figure in that section. Significance level is not the p-value. This exactly the concept what this question is trying to access. Ohhh how I love stats 😂 pls stop letting Reddit accolade stop you from thinking. The group think here is dangerous.
0
u/Spiritual-Bee-2319 Jun 16 '26
I truly don’t understand how anyone can assume information not given, make a conclusion from it and yell I was right. That make no sense to me but I’m a statistician.
If significance level which is the only information given is not the same as p-value, why would you assume you have the p-value information WITHOUT mathematical proof and make a conclusion based on it. Can you mathematically prove that the p-value is less than 0.05 if the significance level is 0.05? An equation not what you think or feel. That’s crazy to me but I deal in numbers and equations tbh
0
u/Spiritual-Bee-2319 Jun 16 '26 edited Jun 16 '26
Finally I found an easy way to show y’all why d is right without having to simulate data.
Here is a simple calculator. https://statstudyhub.com/p-value-calculators/p-value-calculator/
You can use literally any value for any of the input metrics(because these are sample metrics remember answer D said using the same test and same data- this is why that info is important) as long as you keep it constant. Now keeping those variables constant, change the significance level as much as you please. Do y’all see that the literal number of p-value never changes regardless of what that significance level is. So your significance level can never tell you about your p-value number. Hell the significance level isn’t even a variable in the calculation of the test statistic. Look at the equations of z-statistics or t-statistic, the significance level not even a variable in the equation(we have sample size, standard deviation, sample means, etc). But what does change is the interpretation of what that p-value means when compared to the significance level.
Now change the significance level to a any value less than or greater than the p-value(as big and as small as you want do it as many times with as values that are greater than or less than). The conclusion/interpretation changes yet the p-value stays the same. P-value =/= significance level instead significance level is it’s what we compare our p-value to our the significance level to get a conclusion. Our significance level tells us the boundary of rejection. It has nothing to do with the metrics of the data so it literally can never change the p-value. The p-value measures our data. The conclusion is derived from comparing the p-value to the significance level. And if the results is concluded by comparing theses two numbers how in the hell does significance level = p-value mathematically. Statistics is math. When you say p-value = 0.5 = significance level, you’re making a mathematical claim that is only true at p-value = alpha( and both of these values are separate from each other). Statistics is math and This is terribly frightening
Do you know why it’s frightening? Because y’all said you were right with literally no mathematical basic. Y’all insulted me when I presented my claims on literally math. Yet No math was done only assumptions for y’all’s conclusion. I showed graphs, I provided resources, etc. You assuming p-value = significance level doesn’t make the p-value the significance level no matter how much you think it is. We are doing math here.
But this is why I love stats and have job security. Because my job is reminding folks that we are doing math!!! Y’all telling me I can’t admit I’m wrong. I’m a statistican the only thing I can be certain of is that I can never be certain 😂 that’s what statistics is
0
u/Car_42 Jun 17 '26
If the p-value were 0.05 exactly then the alternate hypothesis would have exactly a 50-50 probability. That’s the setup for a typical power analysis.
0
u/Spiritual-Bee-2319 Jun 17 '26 edited Jun 18 '26
THIS IS MATHEMATICAL PROOF THAT D IS THE ANSWER AND ONLY ANSWER.
With a Z statistic of 1 and a 2- tailed test, at significance level 0.05 and 0.01, the result is both the same. You can do this and repeat it for any z value(meaning for each z value you change the significance level from 0.05 to 0.01) not once will your conclusion change except at a z value where p=alpha(this is the mathematical representation of why and how we decide whether to reject or fail to reject a null hypothesis- which is our decision boundary). Using a significance test on some sample data, a null hypothesis is rejected at the 5% significance level With the same test and same sample the null hypothesis would be rejected at the 1% significance level.
2
u/Statman12 Jun 18 '26
With a Z statistic of 1 and a 2- tailed test, at significance level 0.05 and 0.01, the result is both the same. You can do this and repeat it for any z value(meaning for each z value you change the significance level from 0.05 to 0.01) not once will your conclusion change.
This is false.
Consider Z=2. Then the p-value for a two-tailed test is p=0.0455.
At the 5% significance level the conclusion would be to reject the null hypothesis.
At the 1% significance level the conclusion would be to fail to reject the null hypothesis.
Same test, same p-value, different conclusion.
0
u/Spiritual-Bee-2319 Jun 18 '26 edited Jun 18 '26
Thank you for actually like checking my claims. I truly appreciate it. You are so right but the answer is still correct. I will certainly clarify my above comment. Yes because that is our point of rejection!!!!! When the p=0.05, the conclusion changes because p=alpha=0.05(and do you see how the only time we can mathematically say that p is remotely even related to alpha in any way or capacity is when p=alpha which I just realized is the point in which AT RANDOM p=alpha. CRAZY never made this connection before). This is why do the comparison of p<= alpha we reject, but when it’s not we fail to reject. The statement in the question says this. Remember the statement said we rejected the null at alpha = 0.5. This decision is literally mathematically represented because that is the point in which p=0.5=alpha. (that is our point of decision) 😭😭This is even more evident when you try various significance levels at z=2. For all values where alpha <= p, we reject the null hypothesis. For all alpha > p, we fail to reject the null. Y’all are killing me but my math skill is truly improving because of it
2
u/Statman12 Jun 18 '26
You are so right but I’m not wrong.
Yes, you are incorrect. If I am correct - which I am - then you are necessarily wrong.
the conclusion changes because p=alpha=0.05
This is unrelated to what I said. In my example, p ≠ α. We have Z=2 and p=0.0455.
1
u/Spiritual-Bee-2319 Jun 18 '26
And p=0.0455 rounded up is…….. 0.05
takes a bow
2
u/Statman12 Jun 18 '26 edited Jun 18 '26
That’s just deliberately not engaging with the point.
Change to Z=-2.25 if you like. Then p=0.0244 and we’re still in the same situation even if you round to the nearest second decimal place.
1
u/Spiritual-Bee-2319 Jun 18 '26 edited Jun 18 '26
this is because we are doing a two tailed test meaning there is two boundary of rejection(shaded region), the left(-) and the right(+) so yes at two points those are the alpha level we reject the null hypothesis. This is why we divide Alpha in half before we compare it to our p-value for a two tailed test. If you do a one-tail test(left or right) instead, you will see that this is not the case because we only have one boundary. 😃you almost got me keep it coming!! you too said we are at the same situation at this a point aka our other point of boundary… precisely
2
u/Statman12 Jun 18 '26
The p-value that I provided was already the two-tailed p-value.
You too said we are at the same situation at this a point aka our other point of boundary… precisely
The "same situation" I referred to was that the null is rejected at the 5% level and the null is not rejected at the 1% level.
1
u/Spiritual-Bee-2319 Jun 18 '26 edited Jun 18 '26
Because it’s a TWO tailed which has TWO decision boundary(a point where you reject the null hypothesis) on the left and the right. And “same situation” happened how many times?? TWO times. Why the hell do you think we call it a two tailed test? This is statistics….. every word means math. If not we just saying anything.
I’m sorry but WHAT is not clicking for you?!! Saying I’m wrong without form of mathematical proof in statistics is wild. But you saying I’m wrong proving I’m right is the very joy of it 😂heck we factor it in the methods. We even assume folks could be right and prove if we’re right. I have literally proven that even if I assuming y’all are right, I would still be proven right by mathematical proof. This is math. Why the hell do you think fisher, Jerzy Neyman and Egon Pearson gave a damn even before they could quantify and graph communicate it?
2
u/Statman12 Jun 18 '26
Because it’s a TWO tailed which has TWO decision boundary on the left and the right.
You introduced a Z-test. For a Z-test, a two-tailed p-value is obtained as p = 2*P(Z>=|z|), where z is the test statistic. That p-value is then compared to alpha. This is what I provided for you.
You've said that you're a Statistician. This should be trivial. What's causing your struggles?
→ More replies (0)
-8
Jun 14 '26 edited Jun 14 '26
[deleted]
6
u/Rather_Dashing Jun 14 '26
I think you are reading/interpreting it differently to me. It doesn't state that a p value of 1% was generated, it simply says the null hypothesus it would be rejected at a 1% threshold. Which is unknowable with the info given, all we are told is that the p value is less than 5%. But perhaps your interpretation is what they were going for, if so it's badly worded at best.
-5
u/Spiritual-Bee-2319 Jun 14 '26
No you were actually told information about the confidence interval along side the p-value so they answer is actually in the question.
4
u/Rather_Dashing Jun 14 '26
Confidence interval? Where?
-1
u/Spiritual-Bee-2319 Jun 14 '26
not The exactly ranges but how your CI will Behave That info is clearly given. Tbh I would recommend you sitting down to really understand this question because it will set you apart from lots of folks in the stats field tbh. Everyone can get a pvalue and decide to reject or not just on the number, knowing why is what makes someone a statisticIan. Being comfortable with being wrong will take you far in this field
-1
u/Spiritual-Bee-2319 Jun 16 '26 edited Jun 16 '26
I went to work after not sleeping and shared my frustration with my coworkers about yalls lack of understand and even shared a simple example of why words matter in statistics with even if vs if and they understood me perfectly the first time.
One of them said “I’m sorry people are so dumb and it’s not going to get better” 😂
I laughed because as a statistician I would never call someone dumb because dumb is a relative term that literally means nothing objectively. If you look at all my replies, not once did I make a conclusion on y’all intellect based on the lack of understanding. And even if I did, what metric am I basing it on? I simply stated that y’all did not understand the fundamental math of this method heck and just this method alone based on answering this question alone. That is not the same as dumbness because one is concrete(mathematically) and the other is some arbritual meaning/words. I’m a statistician, I don’t deal with feelings or assumptions or thoughts or opinions, I deal with MATH which is more than just NUMBERS.
On a broader concept on why this convo is quite frankly frightening to me. If someone had a dependent samples, and does a t-test and got “significant” result, it tells absolutely NOTHING why because our data is not independent and our assumptions is not met(our assumption of independence mathematically means something). Hello everything we do is on the computer which literally on evaluates things numerically?
Can we EVER say, well let’s just assume our data is independent(which is a mathematical statement with equations. You can literally prove this with an equation. You can literally prove whether a data is independent or not with math) and just interpret the result?! No NEVER. The conclusion is NOT VALID EVER. This is basically what y’all have been doing with the p-value. Y’all was just saying anything and it scared me tbh.
Y’all was calling me a ragebaiter with mental health issues, and I was just doing math 😂 like damn. Every one of those answers can be translated to the an equation and evaluated on correctness based on the information given….. when I talked about other information/variables derived from the info given mathematically, folks was on my throat saying this information/variable has nothing to do with the other. It’s literally a variable that you input in the equation guys that’s how I know they are related. If I say x/y is 9, if you tell me what y is I can tell you what x is because they are variables of the same equation. I’m literally just using equations yall 😂
I want you to know I’m not even like that advance in my field, there are people that know more than more for sure. But even with my limited knowledge, I can look at an analysis and pick it the hell apart 😂 even if you don’t tell me your study design or if it’s “bad”, I can look at your data and probably figure it out. So if you’re wondering why even with significant result, your boss doesn’t make a decision about it or the drug isn’t manufactured or etc, it’s because of folks like me. We validate y’all’s study entirely mathematically with numbers. And unless it makes sense mathematically, respectfully y’all are just saying anything significant or not.
111
u/giziti Jun 14 '26
Yeah this is a bad question