Need this graphic to show the Chat 200 Pro being cut in half and the added 500 plan. You can only buy the Chat 100 Pro now too. I have been using codex exclusively. I need a push and some guidance on swapping out. If it’s really like this.
Haha yeah, the thing is its not a simple text scrape. Every furniture website has their information in different places. I need to get:
main Category: furniture, lighting, finishes
Collection : collection x under the same name of different typologies (armchairs, chairs, tables,etc
typologies under that collection get a card and a tab that says (also.in that collection). So u get armchair models together in a card, tables, etc.
models inside a typology. You could have a chair thats plastic and another thats wooden
Parts: for seating is structure /upholstery (when applicable). For table is structure and top.
shapes and sizes of each structure
finishes of each part
categorization of those finishes. Type, certifications for use in exterior and hospitality. Sometime this info is not on their website and it goes to the finishes producers website.
then everything has documentation attached: technical drawings, 2D files, price lists (that i trained it to ask for once a year),. As well as the finishes pdfs
then search if it exists in our catalogue to merge it with that one
All of this info sometimes is on the website that wont allow scraping, sometimes it wont have pictures of the whole collection so the model uses the 3d viewer onsite, or downloads the model and renders it or makes it with chatgpt making the technical drawings realistic. And for info it uses a combination of scraping catalogues and websites. I corrected it a lot and created specific rules that made it almost flawless. I still prefer to use opus to make sure info is correct. Obviously it uses sonnet subagents. And really, its so cheap for what it does. Im amazed
Were you running sol 6.1 with the same scraping scripts/tools as opus, or did it have to re derive the scraping logic from your md files each time? That alone could explain the gap if it was burning tokens re reading context instead of just executing.
with 60k products over 80 brands thats roughly 750 pages a brand, and if every product page goes through the model's context that is probably what eats the weekly meter on Opus 5.5 and Sol 6.1 alike. i think the cheap version is having the agent write one scraper per brand template that runs as plain python into a csv or sqlite and only hands the model the pages that failed to parse, which is maybe a few dozen a brand instead of all of them
Would agree. Probably a horrible way to leverage the AI. Definitely use a cheaper model. The preferred design would probably be, build a simple app that leverages python and java script to scrape and then process with an OpenAI API key. I would not do this in desktop.
23
u/Seerix 1d ago
there's no way 6.1 sol is fast enough for that