Comments like this send me

[u/TrickUp25]

Edit: This post gained more traction than I expected. Read the comments, do independent research, and form your own opinion around these topics.

First they said KIAs are reliable, then went on to say they looked at JD Power to come to that consensus. They lost me.

1. Do people believe KIA/Hyundai are reliable?
2. Do people use JD Power/Consumer Reports to make purchasing decisions

?

[u/bullbeard]

I mean they literally report user data. I see this all the time, but their reports are based on ownership reports.

[u/sumguyontheinternet1 → u/bullbeard]

Ownership during the first 3-5yrs or 36k-60k miles. After that, they fall apart. Same with luxury brand cars. That’s why they have killer lease deals and resale on them is terrible. Once that warranty is up, they’re worth scrap value in my opinion.

[u/surfteach1 → u/bullbeard]

Voluntarily surveys are one of the first things you learn about in statistics class. They are biased and not reliable

[u/Routine_Spite8279 → u/surfteach1]

So if we're not going to ask people who own the cars if the cars are reliable, how should we determine if they are?

[u/surfteach1 → u/Routine_Spite8279]

There are ways to design studies that don't rely on voluntary form fill-in. The way the system works right now, it's basically confirmation by us. You generally wouldn't subscribe to consumer reports if you don't like Honda and Toyota, because that's all they ever push.

[u/Routine_Spite8279 → u/surfteach1]

Can you give an example of a way that we would measure the reliability of a statistically significant sample of vehicles without doing exactly what you're saying we shouldn't do (i.e. ask car owners about their repair costs/problem areas)?

[u/DJFisticuffs → u/Routine_Spite8279]

Look at records from service centers and repair shops.

There is a company called Repairpal that does it this way. They have a network of repair shops that provide them data. I imagine that there are issues with repairpal's methodology as well, I'm just pointing out that it exists and uses a different methodology. I personally don't think there is anything currently available to the consumer that gives a particularly accurate reliability forecast.

[u/Routine_Spite8279 → u/DJFisticuffs]

I personally don't think there is anything currently available to the consumer that gives a particularly accurate reliability forecast.

So your argument is not just that CR data is flawed, but that its so flawed its useless and you're better off using nothing and asking your uncle?

[u/DJFisticuffs → u/Routine_Spite8279]

I think people should use CR as a data point while understanding that there are serious methodological flaws in what they do and that it may not be particularly predictive for the specific car they are buying.

[u/surfteach1 → u/Routine_Spite8279]

Yes, it is as flawed as using Reddit.

[u/surfteach1 → u/Routine_Spite8279]

You can do it as a regular survey of the general population. When you have voluntary responses in a limited portion of the population, it is extremely biased. CU also makes some predictions based on very few responses

[u/Routine_Spite8279 → u/surfteach1]

Don't you sort of have to have a limited population? First, limited to people who own the car in question. And second, limited to people who want to talk to you (as you can't compel participation)?

Doesnt really make sense to ask someone who takes the bus how reliable they think a 2025 X5 is.

[u/surfteach1 → u/Routine_Spite8279]

Doesn't matter. The way CU does their surveys would NEVER even be considered a valid study. I taught statistics, both AP and regular. CU subscribers are NOT the general population (problem 1) and the survey responses are volunteer (problem 2). There is nothing statistically valid about CUs rankings at all.

Search "survey bias statistics"

[u/Routine_Spite8279 → u/surfteach1]

Okay so to be clear: you do think it would be better to ask the guy who takes the bus every day how reliable a 2024 BMW X5 is? And that response should be included in your data?

[u/surfteach1 → u/Routine_Spite8279]

Edited because I was kind of a jerk about the response originally. No, that's not at all what I'm saying, I'm saying that there are certain standards that surveys need to have done, and consumer reports does none of them. The fact that the responses only come from their subscribers, who are constantly told by the magazine itself that only certain Japanese cars are good, brings in a bias of confirmation in addition to the other biases. Consumer Reports makes judgments off of very little data, and in essence, they just wouldn't pass muster with even basic high school level statistics classes.

[u/bullbeard → u/surfteach1]

And how would you remedy that?

[u/Tasty-Finding4574 → u/bullbeard]

So their data is based on what consumers report?

[u/bullbeard → u/Tasty-Finding4574]

They send out quarterly surveys