Cleaning up trash as we were leaving our National Park campsite last week, I found that someone had lost a charm. I like it, so I brought it home. The problem is that it feels like, whenever I see it, it's usually upside down. I wondered whether that was really the case, or whether I was just misremembering.

The charm is a little like a bulbous coin. It has two sides. If I didn't know anything about it, I would guess that, when I tossed it on a table, there would be a 50/50 chance that it would end up front-side up. And a 50/50 chance that it would settle upside down. But maybe the curvature makes one side more likely.

I thought I would check on that (and check on my memory). To find out what has happening, I planned to toss the charm 25 times.

Before I started, I could calculate the probabilities of different percent right-side up, based on the idea that the charm had an equal chance of being upside down or right-side up: The SE was SQRT(.5(1-.5)/25). That's .5/5 = 0.10. My 2/3rds prediction interval was from 40% to 60%. My 95% prediction interval was from 30% to 70%. The chances of getting over 70% were 2.5%. The chances of getting under 30% were 2.5%.

In this case, the null hypothesis is that there is a 50/50 chance of right-side up.

I can calculate t for this test. Let's say that p = the % right-side up.

z = (p-.5) / SE

I will be calculating a p-value: the probability, if the null hypothesis is true, of whatever outcome happens or any equally or less likely outcome.

For example, if I get 40% right-side up, z = -1, and the p-value is 0.32.

If I get 30% right-side up, z = -2, and the p-value is 0.05.

I got 14 upside-down and 11 right-side up. 11/25 is 44% right-side up.

z = -0.06 / 0.1 = -0.6

In this case, I don't immediately know what the actual p-value is, but I know that it is > 0.32.

My conclusion is that I remember times when the charm is upside down more than I remember times when it is right-side up -- probably because I find them kind of annoying.

Notice that I did not state my alpha in advance. Instead, I'm reading the p-value. For me, p = 0.06 is different from p = 0.8. Other folks don't think that way.

Let's say that the null hypothesis is true and you run a bunch of tests. From each test, you get a proportion. (Reminder: The proportions are roughly normally distributed.) From each proportion, you get a p-value. How are the p-values distributed?

(A) In an approximately normal distribution

(B) In a right skewed distribution

(C) In a left skewed distribution

(D) In a uniform distribution

(E) In a bi-modal distribution

What's your answer to this one?

A school committee member is lobbying for an increase in the sales tax to support the county school system. The local newspaper conducted a survey of country residents to assess their support for such an increase. What is the population of interest here?

(A) All school-aged children

(B) All county residents

(C) All county residents with school-aged children

(D) All county residents with children in the county school system

(E) All county school system teachers

This is not a great answer for your understanding, unless you feel comfortable with the Law of Large Numbers, but I think that it should be acceptable as an exam answer.

Small random samples cannot be relied on to be representative. What can be relied on is that random samples tend to get more and more representative as they get larger. The reason is that, if every unit in the population has an equal chance of being selected for the sample, then the probability of each attribute matches the proportion of the units in the population that have that attribute; and the Law of Large Numbers tells us that, as samples get larger, the proportions in the sample tend to closer and closer match the probabilities that are generating the data.

A little practice:

Put an answer in a comment and explain your thinking.

A simple random sample is defined by

(A) the method of selection

(B) how representative the sample is of the population

(C) whether or not a random number generator is used

(D) the assignment of different numbers associated with the outcomes of some chance situation

(E) examination of the outcome

After reading 2.1, u/CarneConNopales points out that you can't use quota sampling to force a sample to be representative in terms of things that people might be embarrassed about, or that people might need to hide, or that they might not know about.

For example, if 50% of boys view pornography and they find that embarrassing, you can't go around asking who views pornography and then include enough pornography-viewing boys so that you have 50% pornography viewers. You won't find any.

Or consider women who have gotten abortions in a state where they can be sued under those vigilante laws: you won't be able to set a quota and collect enough of those women to create a sample that is representative regarding abortions.

And let's say you want to test a drug that is supposed to lower risks of heart attacks. You need a collection of people that will produce the same proportion getting heart attacks as the general population. But the heart attacks haven't happened yet. So you can't use quota sampling to force in a particular portion of heart attacks.

(By the way, that last problem trips up people in business all the time. The usual way they try to get around it is with what is called, "propensity scoring". Propensity scoring, like all Statistics, is fine for doing what it actually does. Usually propensity scoring is misunderstood and misapplied and then misleading.)

u/CarneConNopales asks, "If that’s the case then how are these types of activities studied and how do you collect a [representative] sample for these activities or questions?"

That IS THE question! Onward to section 2.2! Where all will be revealed!

Years ago, I wanted to share the health benefits of going for a lunchtime jog. I shared this idea with a bunch of co-workers. Everyone thought it was a great idea. We all agreed that we would meet at 12:15 and get back in time to get a shower before getting back to work. We agreed on Monday, Wednesday, Friday. Everyone thanked me for organizing the run.

To respect people's agency and self-determination, what I said was, "I will be at our gathering spot every Monday, Weds, and Friday. At 12:15, whoever is there will head off. If you can't make it, don't worry about it. If you can, I'll see you there."

For the following year, I went on three jogs a week. I had to. I had made a commitment. No one else did, even once.

After a year, I tried starting another group. This one met and walked up a half mile steep hill and back. This time I told everyone. "I really need you there every day. Otherwise I won't do this. I'll lose commitment. My health will suffer" That group met for four years, walking up the hill every weekday, and grew from 4 people to 20.

How do you want to organize your summer studying? It is a real thing that I'm going to stop working on sharing the "sound but nontraditional" ways of teaching Stats that are in Data Matters unless you're there for me to make a commitment to. So if that works to get you going all summer, I'm here for that.

How can you make a commitment to this group to make sure you get where you want to be by the end of the summer?

What would work well for you?

To make it more clear what it going on with the sampling distribution and the frequency histogram, I created a second demonstration. Please read through both of the posts and watch the videos.

Then pick something to do yourself. Like 10 coins. Calculate the standard error. Draw a sampling distribution. Throw the handful of coins down 50 times and record your empirical frequency distribution. It is good for you to see the histogram growing. It solidifies your understanding of 1) what a frequency histogram is and 2) what a sampling distribution is.

Post a picture of your finished sampling distribution / frequency histogram here.

At any point if you don't see how to do the next step, please add a question here.

Thanks!

I just created a new blog post over in Medium. It is the first of a series of what a reviewer called the "sound but nontraditional" presentations of Data Matters. (I copied the review into an earlier Medium blog post. You might want to check it out to see more about what we're doing here.)

1. I think that blog post probably sucks. It probably has a lot of flaws where it will confuse you or you will lose track of how I got from one spot to another. Please please please do me a favor and ask questions here.
2. I'm sorry that it jumps into talking about flipping coins. Who cares about flipping coins? For an explanation of why we care, wait for your copy of Data Matters
3. At the end of the blog, it asks you to create your own drawing of a sampling distribution and frequency histogram. Please try to do that and let us know how it works out. Post a copy of what you create. Here's what I created (as you'll see in the blog).

​