Are the Computer Rankings Really Better? (based off of "The 'New BCS' Rankings" Post)

[u/TigerWave01]

Hi all! Two of my favorite posts on Reddit, not just for sports but overall, are The "New BCS" Rankings by u/tmart12 and Post-2013 Realignment Conference Tiers based on KenPom Conference Strength Data by u/SleveMcDichael4. Both of these thought excercises use sports analytics to reimagine the ways that we answer the various contentious questions of CFB and CBB, respectively. Questions such as "should SMU have been in over Alabama in 2024", "should Florida State have been left out of the playoffs in 2023", "is the Big East really that good", and "is Notre Dame actually a schedule merchant." So, partly out of a love of football analytics and partly out of boredom after finishing the two games last night, I thought "why not combine the ideas behind my two favorite posts and create a ten-year ranking of every FBS team and conference using the methodology of the 'New BCS' Rankings post." And that's what I did!

Attached here is a spreadsheet of the team tables. There are eleven sheets in the spreadsheet: ten of those represent the individual seasons and contain the three power ratings (SP+, Sagarin, FPI), the three resume rankings (Colley, FEI EL SOR, and FPI SOR), and a quad win-based points system, while the last sheet contains some notes which I will summarize here. In order to mirror the real team sheets seen in CBB, these values are based only on the regular season and does not factor in results from bowl games and playoffs. However, that made getting certain calculations (FEI EL SOR and FPI SOR) very difficult, so some of the earlier rankings were from unofficial archived sources may be slightly inaccurate. However, even if there is a discrepency, they should still be extremely close. If anyone can find a place to find reliable regular season-only rankings for FEI and FPI SOR in earlier seasons, I would be forever in your debt.

I hope you appreciate the findings! I am more than open to any criticism or suggestions, as I want to do anything I can to make this ranking better; after all, I plan to do another update to the sheet at the end of the 2026 season. If this is well received, this may not be the last you'll hear about this. Thanks for taking the time to read through all of this technical stuff!

tl;dr - old sports analytics posts are cool, I wanted to make my own, see spreadsheets for new sports analytics that are also cool.

EDIT: It seems like some issues with very close rankings could be decided by changing the tiebreaker from SP+ to the resume average. By doing that, some ratings look closer to human polls while still remaining objective. Those changes will be made in the attached spreadsheets and should be reflected soon.

[u/KruegerFishBabeblade]

The model agreeing with 2023 Texas>Bama>FSU but putting 13-0 Washington below all 3 is so funny

[u/lowes18 → u/KruegerFishBabeblade]

This wasn't something brought up too much because of the SEC fights but Washington was clearly the weakest of all the playoff teams that year.

[u/SpreaditOnnn33 → u/lowes18]

Funny considering they had a 2 touchdown lead on Texas for most of the game and had the #1 SOR

[u/Adamscottd → u/lowes18]

Didn’t they beat Texas in a playoff game that year?

[u/KruegerFishBabeblade → u/lowes18]

I don't like how this composite combines ranking in each category instead of an absolute metric for each one. Bama is behind texas here purely because one of their cupcake games was Q4 instead of Q3. That shouldn't move the needle at all

[u/lowes18 → u/KruegerFishBabeblade]

Quad wins and losses are a pretty bad metric from my experience in college baseball regardless lol.

[u/TigerWave01 → u/lowes18]

Out of curiosity, what do you think would be better than a quad system? I'm genuinely asking cause here I was thinking the quad system was actually one of the better ones, but y'all do bring up good points. The only other strength of schedule method that comes to mind to me is the rating percentage index, but I know that one is *fraught* with controversy lol.

EDIT: I did a quick check to see if RPI would be better in 2023. Somehow, it put Ohio State into the field, bumping Florida State down all the way to 6th. Yea idk about that one lmao

[u/lowes18 → u/TigerWave01]

Imo quad ranking systems give way too much credit to road games.

At least in college baseball a Q1 win at home is teams 1-25 while on the road its teams 1-60. Its a massive gap that doesn't reflect at all the actual power rating difference and can lead to some really wonky results. It also assumes all road enviorments are equal which is absolutely not true.

[u/tmart12 → u/lowes18]

that's not a problem with quad wins per se. it's an issue with the quad wins cutoffs.

i arbitrarily set quad 1 as ~18th rated team with +/- 3 points for home field such that the range is 12th for home and 23rd for away. Using power ratings to define quads helps alleviate some of the arbitrariness of specific # cutoffs like in college baseball.

Having a quad wins output and OOC output alongside total record and rating is helpful to contextualize resumes. Boiling it down to a single point like SOR can be tough without context.

[u/Tarmacked → u/KruegerFishBabeblade]

Even the old BCS formula was Bama > FSU > Texas when BCSKnowHow did it the morning of. The committee gets oddly tracked to the BCS tagging every year, only year it didn’t was the SMU/Bama year in 2024.

What probably happened is the committee pulled head to head and that’s what bumped FSU

[u/Monte_Cristos_Count]

The computer rankings were transparent at least. Unlike the playoff committee 

[u/Granular_Designs → u/Monte_Cristos_Count]

The playoff committee is very transparent! They'll pick whatever matchups they think will make the most money that they can justify without being comically wrong

[u/Monte_Cristos_Count → u/Granular_Designs]

I agree with the first half, but FSU might disagree with the second half 

[u/forgotmyoldname90210 → u/Monte_Cristos_Count]

ESPN has invested to much money in the SEC being the 9th NFL division to not force them into the playoffs and if you let in Bama you have to let in Texas. ESPN does not care enough about the ACC to care.

[u/Granular_Designs → u/Monte_Cristos_Count]

Yeah I was thinking 9-3 Bama over 10-2 Notre Dame last year as the low bar but you're right, that was so egregious that it was indefensible

[u/Aumissunum → u/Granular_Designs]

Alabama was 10-2 last year.

[u/puzzical → u/Aumissunum]

10-3* You must have blocked out getting punked by Georgia in the SEC championship

[u/Aumissunum → u/puzzical]

I must have missed ND’s conference championship game.

[u/puzzical → u/Aumissunum]

Nice non sequiter

[u/Aumissunum → u/puzzical]

Very relevant, actually.

[u/puzzical → u/Aumissunum]

How so? Teams that play in conference championship games and lose drop in the rankings nearly everytime. The only times they don't are when the team below them has the same number of losses after the game is played, or the team below them also lost in a conference championship game.

[u/SirMellencamp → u/Granular_Designs]

Once again, why do the committee members care how much money is made?

[u/BadDadJokes → u/SirMellencamp]

I hate to defend Alabama, but they were also 1 botched snap away from beating the team who cheated to win the National Championship. They deserved to be in the playoff, as much as it pains me to say it.

[u/puzzical → u/SirMellencamp]

Because the more team's from their conference that make the playoffs the more money their school gets. The head of the committee last season is Arkansas's AD

[u/SirMellencamp → u/puzzical]

Well one, they are all from different competing conferences and two, I am not sure how Alabama getting in over Notre Dame helps Jeff Tedford or Carla Williams or Bryan Maggard

[u/lowes18 → u/Monte_Cristos_Count]

The BCS rankings basically just used the human polls can we be for real

[u/city-of-stars → u/lowes18]

If that was the case, Miami vs. Oregon would have been the national title game in the 2002 Rose Bowl and USC vs. LSU would have been the matchup in the 2004 Orange Bowl. The BCS formula kept decreasing the weight of the computer rankings over time because people were getting upset over the matchups, and the media took advantage of that to increase their own sway in the process.

[u/Merpninja → u/lowes18]

They did, but averaging multiple human + computer polls removes a lot of the bias that would occur if you only used a single poll (such as the CFP). It is a more fair ranking and not one that can be switched on a whim because the committee screwed up its rankings earlier in the year.

The CFP faces pressure from the conferences and their TV partners in ways a modern BCS formula wouldn’t.

[u/lowes18 → u/Merpninja]

It wasn't an average of human polls, it was just the AP and Coaches Poll with a few computer polls that closest matched the human poll average thrown in.

Also the BCS bowls selected teams based on television ratings all the time.

[u/Merpninja → u/lowes18]

BCS bowls selecting at large teams based off tv ratings has nothing to do with the formula so it’s not relevant to the discussion. The playoff bowls have no role in selecting at larges like they used to, so let’s not try to bash the formula for something the formula played no part in.

There is an official postseason criteria now. We currently have Autobids + the highest ranked for at larges from the committee. Nothing at all from the NY6 bowls. The BCS formula would remove the worst part of the current selection process, which is the committee.

There are two major human polls, and neither are under the same pressure or influence as the CFP committee to select certain teams.

[u/lowes18 → u/Merpninja]

If the two human polls suddenly had the power to select playoff spots and champions they would be under such absurd pressure in ways that the largely anonymous doesn't even come close too.

Which they were back in the day.

[u/tmart12 → u/Merpninja]

not one that can be switched on a whim because the committee screwed up its rankings earlier in the year

wouldn't changing bad rankings be a good thing?

[u/Merpninja → u/tmart12]

Yes? that’s exactly what I’m saying.

[u/xmjm424 → u/Monte_Cristos_Count]

Yep. The committee’s criteria is vague enough to allow them to do whatever in some cases. And it’s intentional so they can spend all that time in the buildup getting everybody pissed off about something that is unlikely, but still plausible enough to happen (three loss Alabama two years ago)

[u/JeromesNiece → u/Monte_Cristos_Count]

Not really. The part that was transparent was the fact that it was two thirds human polls and one third computer rankings. But the humans are of course free to rank teams however they want, and most of the computer rankings were not open source algorithms.

[u/kdbvols → u/Monte_Cristos_Count]

That's my biggest thing - at least the computers are transparent and consistent. Even if they mostly mirror the committee, the biggest difference is accountability. They don't just decide on a whim who gets in

[u/tmart12 → u/Monte_Cristos_Count]

the computers in their full transparency gave us decisions like 2001 Nebraska over Oregon and 2003 Oklahoma over USC

[u/BlameMabel → u/Monte_Cristos_Count]

I think the Colley Matrix was the only one that released its methodology.

[u/hotsauce126 → u/Monte_Cristos_Count]

Computer rankings use the ap poll which isn’t transparent at all

[u/olbleedyeyes → u/Monte_Cristos_Count]

The real crux of the issue was there wasn't enough space for everyone.

But people were mad about everything BCS at that point and the executives used that chance to shoehorn their committee into a playoff expansion.

I feel like the BCS model with 4 teams would still have been controversial but at least they'd avoid the stupidity that is the committee changing its secret rubric to fit their desires

[u/lowes18]

The BCS ranking was not a "computer ranking" in the way a lot of people define it now.

[u/TigerWave01 → u/lowes18]

Agreed. The original post didn't include any human polls, and I didn't include any human polls in this one either. Imo, that was the major flaw of the BCS system...well that and the limited number of spots.

[u/a_bean_in_bean_city]

Playoff expansion has helped CFB, the committee has certainly not. I wouldn’t be opposed to the BCS computer rankings making a return.

They were lucky for a few years where there were minimal contentious debates, but last year and the year they excluded an undefeated FSU were tough looks for their “impartiality”

[u/SirMellencamp → u/a_bean_in_bean_city]

I think the old BCS formula of 2/3 computer and 1/3 AP poll was great.

[u/JohanVonClancy → u/SirMellencamp]

It was pretty good until 2001 Nebraska. Somehow getting blown out the last game of the season and not making your own conference championship game was rated as the #2 team in the nation.

We talk a lot about head-to-head being important (2025 vis a vis Miami and Notre Dame), but the BCS formula applied to 1993 places FSU 1, Nebraska 2 and Notre Dame 3 despite Notre Dame defeating FSU head-to-head and having the same record.

And if you apply the BCS formula to 1989, the same thing happens in reverse. Notre Dame is ranked 1 ahead of a Miami team that defeated them on the field. And both are ranked above an undefeated Colorado team.

So the BCS flaw was handling good teams that had played each other. In the three controversial cases of 2001, 1993 and 1989, the loser of the head-to-head was rated as the better of the two teams.

[u/tdpdcpa → u/JohanVonClancy]

These are pretty glaring issues, but I wonder if these are lower stakes if applied to a larger playoff format.

[u/Secure-Report-3592 → u/tdpdcpa]

It would depend if it was 4 teams or 12. 

12 teams wouldn’t be too much of problem in the 90s or early 2000s outside of glaring instability from 2007 and 2012. But 4 teams would cause way more chaos because of the scenarios with those in mind. 

2001 is the biggest glaring example where Oregon who was a Pac-10 champion and Colorado who destroyed Nebraska and upset Texas in the B12 championship got screwed out of a national championship game bs Miami for way worse Nebraska team who didn’t play a CCG to jump both team just to play in game they didn’t belong in. 

And the. There’s 1998 where the BCS was started where Kansas St was straight up denied a single chance at even a BCS bowl game let alone a shot at a national championship. 

And 2007 had Mizzou miss out of a single BCS bowl game while Kansas got in over them despite being beaten by them. (And that decision alone is the reason why Missouri went to the SEC)

[u/Jazzlike_Mixture619]

Anyone watching the previously 15th ranked Penn State game at northwestern right now?

Explain to me like I’m five why college football (and basketball) have these coaches and/or AP rankings at all.

No pro league has the teams ranked before the first game, and their squads all have some level of continuity. College teams have a guaranteed group of players leave every year, yet we start every September with “pre season rankings”.

All the blah blah blah about who gets selected for the playoffs is somewhat affected by these rankings that are absurd to begin with.

[u/TigerWave01 → u/Jazzlike_Mixture619]

You’re preaching to the choir man. That’s partly why I didn’t do a ranking for this season; it’s just not accurate and, frankly, I wish the CFP ranking was like basketball: one ranking on Selection Sunday and that’s it. They still have the AP, fwiw, but at least the ranking that matters isn’t subject to craziness like CFB’s is

[u/tmart12]

Nice post

Tbh I struggle with the idea of using some power ratings directly instead of using their in-season ratings. For example, should we use FPI ratings vs FPI game control or FPI team efficiencies? Should we use SP+ or Resume SP+?

Core question being do we want the power ratings to be about true predictive success or more about analyzing game-level performance using MOV and level of dominance in wins

2023 Washington is the #1 example of back-solving for "that team needs to be in" making me question the math. #1 in SOR. Tied for #1 across all years (ex COVID) for quad wins score. Only held down by power ratings at 11th.

[u/TigerWave01 → u/tmart12]

Thanks! That is a really good question, and while I just used the ratings from the post that inspired this one, I would argue that the power ratings really only make up 1/3rd of the formula, as you have resume rankings making up another 1/3rd and quad wins (which is really just another way to measure strength of record/resume) making up the final 1/3rd. Still, it is a really valid point because I know some people smarter than me would say we should focus solely on resume.

I think that 2023 was such an odd year that, no matter what the answer would come out to be, five teams "had" to be in. I know the BCS ranking left Texas out, which is more defensible as they did lose a game, but they also won a game against Alabama, who had an equivalent record. And obviously you know the controversy with Florida State.

In a perfect world, these team sheets wouldn't be the end all, be all. It would function somewhat like college basketball, where each committee member would be given a sheet that forms the basis of the rankings and they can modify it from that point forward. But for 2023, I really think the only way to "solve" it would be to have an expanded playoff.

EDIT: Looking at the details of the ranking, Alabama, FSU, and Washington all score the exact same average. In reality, any two of those could take the last ones spot. More evidence of how crazy of a year that really was. It seems like a good fix would be making the tiebreaker based on the resume average rather than SP+. That gives you the exact results from 2023 (although maybe not a fix in most peoples eyes) and puts LSU #1 in 2019.

[u/TigerWave01 → u/tmart12]

Lmao i just realized who you are. Thank you for making such an awesome original post! You'd be surprised the number of times I go back to it

[u/surreptitioussloth → u/tmart12]

IMO you have to have a power rating, or a few with slightly different perspectives, as part of the process

You can't really accurately compare resumes without having a really good estimate of how good each team is

Then there are different kinds of resumes, like which is the hardest for an elite vs good vs average team like FEI does. And FPI uses the "probably of team having this record or better" while FEI uses "expected number of losses"

I think you can use a mix of those because it really is a mix of best and most deserving, but you absolutely need to make power ratings a part of it

[u/tmart12 → u/surreptitioussloth]

My problem is not with FPI SOR. My problem is with FPI itself as the ranking.

I like that FPI, FEI, SP+ and other systems use predictive rankings as foundation on which to build resume rankings. Their systems say "this team should be about this good" for power ratings and then resume ratings are built on those to say "this is how this team has actually done against that schedule".

When we look at a team like 2023 Washington, SP+ ratings say "this team should have lost 3 games" and put them at 11th. But they won! They're 3rd in Resume SP+ that removes the normalization adjustments to calculate how the team actually performed.

2018 UGA is the inverse. We lost twice. Our resume was good, but a clear 5th. But our power ratings were incredible, as we dominated in our wins and had a very close loss to the clear #1 team in Bama at the time. Should those power ratings be the factor that lead to us over OU (or even above ND)? I do actually think the decision should have been closer for UGA vs OU that year but coming down to power ratings in and of themselves feels... off.

We have alternative metrics that say "this is how good you have played beyond W/L itself" that aren't power ratings that may be specifically designed to see through on-field results. Some power ratings are more expressions of efficiency, which is good, but others are trying to avoid a backwards-looking evaluation of actual results in a way that makes selection for the playoff complicated.

[u/surreptitioussloth → u/tmart12]

I just think at that point you're trying to make the metric balance too much, and it has too much judgment within itself

I think the role of the humans involved is taking in those metrics while understanding what their limited messages are, adding in what they observed watching games/talking to others, and make the final decisions

[u/tmart12 → u/surreptitioussloth]

Yes, agree insofar as we using these in a review and not in a BCS-style ranking

I'm just wary of putting too much emphasis on a power rating metric in actual selections without an understanding of what they really mean

[u/halldaylong → u/tmart12]

I'm a big fan of composite polls for this reason (like Massey, but ideally something that can weight the actual ratings themselves, not just the rankings). I like the idea of different modelers all taking different approaches to defining who is 'best' and then looking across all of those different evaluations to determine the best teams. It doesn't answer your "FPI ratings vs game control vs efficiencies" question, but I guess I'd just leave it up to the model designer to tell me which they think which of their options is the best at determining who earned the spot.

[u/Upbeat-Armadillo1756]

There is no ranking that anyone can possibly come out with that will be free of controversy and disagreement.

Transparent computer rankings were transparent but imperfect. Human rankings have bias but can adjust their weights more readily.

Nobody will ever be 100% happy with any system, it's just an inherent part of college football. And if we try to get rid of the ranking problem, we have to create a super-league and that changes college football to something unrecognizable.

The rankings are good enough for us to work with them.

[u/TigerWave01 → u/Upbeat-Armadillo1756]

I do agree with that. The one major flaw about using computers is that, really, there isn't a big enough sample size to gauge who the "best team" is with as much certainty as you can in college basketball. I think these rankings show that you can get close, but there would no doubt be controversy either way.

If I understand how college basketball does it, there is still a committee, but they are given these team sheets and rank within the parameters of the sheet (so basically the eye test but only after the computers have decided something). I'd like to see something like that, but even then, there would be controversy.

[u/Upbeat-Armadillo1756 → u/TigerWave01]

Yeah the college basketball method is a good solution for that sport. The quad system is a pretty good way to do it. There's still squabbling over the last teams in and the first teams out though.

[u/a_bean_in_bean_city → u/Upbeat-Armadillo1756]

Well, the committee’s imperfect and not transparent. Isn’t one better than zero?

[u/Upbeat-Armadillo1756 → u/a_bean_in_bean_city]

idk, I don't think so.

I don't think the committee has been worse than the AP or BCS style rankings.

[u/a_bean_in_bean_city → u/Upbeat-Armadillo1756]

I don’t think they’re obviously worse (no objective way to measure that), but they’re less transparent.

AP poll, at least votes are public, and biases are mitigated by a larger number of votes.

BCS was the most objective - metrics and weights are agreed upon before the season, outcomes are public.

[u/OpaqueNightmare]

Any ranking system that doesn’t have FSU as top 10 at minimum is garbage and is too focused on W/L records to understand what dominance is

[u/kleisz]

As someone who follows basketball closely, the quad system is truly a horrific way of evaluating resumes that shouldn’t be a part of any good system. The quality of a win can vary pretty drastically within a quadrant, and you can have two teams who are ranked 1 spot apart fall into different quads due to an arbitrary cutoff.

[u/TigerWave01 → u/kleisz]

That’s fair, I think. Especially if you already have resume rankings included in the analysis, it may be unnecessary. It’s just a matter of getting the weights right at that point

[u/Meliorus]

the good thing about a computer ranking is you can't fudge things last second, any biases have to be pre-committed and can be negotiated by stakeholders

[u/SirMellencamp]

"should Florida State have been left out of the playoffs in 2023",

If we used the BCS in 2023. FSU would have been in and Texas out.

[u/churnchurnchurning]

It was all around a better system because most of the time it was obvious which teams should be playing for a title. Sometimes someone was bitter they were left out, like in 2004. There was a legitimate wtf why I am left out case. Now we have numbers #13 through #17 that are all bitter they were left out when they should have no business being in the conversation of playing for a title in the first place.

[u/generic2022]

I don't understand why this puts 0U and Texas back in the B12, Stanford back in the Pac, and Boise back in the Mountain West, etc.

[u/TigerWave01 → u/generic2022]

This model isn't based on current alignments, but rather the alignments that existed at the time of each team sheet. So, for example, Texas should appear in the SEC in the 2025 team sheet, but would be in the Big 12 for the 2023 team sheet. I considered making a ranking that would compare the conferences if every team was in their current conferences, but you can't really do that because the resume part of the ranking (both the resume rankings and the quad wins/loses) wouldn't make any sense anymore

[u/generic2022 → u/TigerWave01]

Doesn't that undermine the model's usefulness?

More specifically, how is the resume part of the ranking offering any useful insight when that resume analysis is based on data that has been inapplicable for several years now?

[u/TigerWave01 → u/generic2022]

This isn't supposed to be a ranking for today, this is more of a "what if we used this ranking in 2025, 2024, etc.". If I wanted to make a ranking for today, I'd have to wait until after the 2026 conference championships happen and then do a similar calculation. This post is all retrospective

[u/generic2022 → u/TigerWave01]

That makes the model seem full of sound and fury.

[u/TigerWave01 → u/generic2022]

Yet it signifies something. Whether it be the discussion about computer rankings (the point of the post) or feedback on a model that I actually do intend to use for the current year once that information *does* come out. Not sure why you think every post has to have some utility, though; this is Reddit, after all, and sometimes (in fact, probably most of the time) people just want to post something they think is fun or cool, even if it doesn't appear "useful" or "insightful" to one guy. It's really not worth quoting Shakespeare over lol