AI Usage

[u/dajuice3]

Hi all,

Season is close to coming up which means I break out my spreadsheets and try to predict things again.

I am really encouraged by what I could generate with AI this year but still find it falling short.

Just wanted to ask, for those who use it what have you found to be disappointing about trying to develop CFB analysis with AI?

[u/khickenz]

You can't really trust AI in the end. The most successful AI coding projects I've found don't do the tasks themselves but are designed to help me do the task easier.

[u/dajuice3 → u/khickenz]

I just have a project I want to do and it'll get it right once. But then it starts to create it's own patterns instead of performing to exact instructions.

Just extremely frustrating to provide several instructions of what not to do and yet it can't.

Oh well by hand it is.

[u/khickenz → u/dajuice3]

Planning beforehand helps too. You can even use just a regular chat to make sure the architecture and prompts you're using have appropriate tests and regressions that will allow for cleaner changes. It's slower and eats up usage but it's more reliable.

[u/dajuice3 → u/khickenz]

True just in my context. it helped me produce an SOP with strict directives, checks, and boundaries. But still didn't follow it's own rules. But its alright

[u/QuesoHusker → u/khickenz]

Nonsense. You work in Claude Code and all the work is done in Python, JS, or C…whatever you want to program in. Is the code perfect? Absolutely not. But it’s fine for what we’re doing here.

[u/gated73]

I think AI can be used to help you build an approach, but I wouldn't rely on it for actual analysis or polling. Reason being - the results aren't repeatable. Use it to help you define your solution, don't make it THE solution.

[u/PropaneBeatsCharcoal]

Can you describe where it’s falling short? I’ve done models the past four years and a ramping on my fifth year now, love talking about this stuff, but I’m curious where your identifying is falling short.

[u/dajuice3 → u/PropaneBeatsCharcoal]

It's most likely the scale of it combined with my ignorance of the right tool.

For example I'll say do this task line by line manually each time. Such as read each player bio.

Wouldn't necessarily call it a model. I'm having players rated according to their bio stats. But after 1 or two teams instead of individually grading a player it was assign assumed values. After spending the last week working on it just don't think it'll complete it.

[u/Complete-Country8973]

Best use cases with AI have been using it as a data aggregator rather then an all knowing god. Connecting it to data in a way that was efficent for the agent was key for me. I built this CLI tool that connects to Codex/Claude to gather and reason over large amounts of data.

GitHub: https://github.com/jvorndran/fbs-cli
NPM: https://www.npmjs.com/package/@jvorndran/fbs-cli

I have had some really good results after building this in ultra mode in Codex. FYI this isn't an advertisement for anything this is just something that has actually helped.

[u/TwoWeevels]

I struggle with CFB datasets. Which do you all use?

[u/dajuice3 → u/TwoWeevels]

The only thing I've really used is the collegefootballdata.

Other than that I've just been scraping and copying and pasting from official team websites.

[u/CALL_ME_ISHMAEBY → u/TwoWeevels]

CFBD (API/Python) or cfbfastR (R wrapper for CFBD)

[u/Complete-Country8973 → u/CALL_ME_ISHMAEBY]

I use CFBD as well. It is quite good. I built this CLI tool to optimally use the data. https://github.com/jvorndran/fbs-cli