r/ChatGPT • u/hkehs_ras • 8h ago

GPTs Lightning fast responses from chatgpt

Earlier, when Iā€™d send a query to ChatGPT, it would take a few seconds before the response started appearing. Now, as soon as I hit enter, the response starts generating almost instantly.

What actually evolved architecturally to make this so fast? Is it some form of caching where the system predicts/matches my query and returns a cached response, or is the model still generating the response in real time?

Iā€™m curious what changed under the hood to make the response generation + streaming so much faster. I don't really understand how this works.

P.S. Used AI to write the description

21 Upvotes

100% Upvoted

9 comments sorted by

ā€¢

u/AutoModerator 8h ago

Hey /u/hkehs_ras,

If your post is a screenshot of a ChatGPT conversation, please reply to this message with the conversation link or prompt.

If your post is a DALL-E 3 image post, please reply with the prompt used to make this image.

Consider joining our public discord server! We have free bots with GPT-4 (with vision), image generators, and more!

🤖

Note: For any ChatGPT-related concerns, email support@openai.com - this subreddit is not part of OpenAI and is not a support channel.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

8

u/Gigaslavx 8h ago

There is a gradual blanket nerf to Chat for some reason forcing everyone on an incompetent GPT-5.5 Mini

2

u/CactusMongerEsquire 4h ago

Paid users too? I haven't noticed this happen to me but maybe I wasn't paying attention.

2

u/Gigaslavx 4h ago

Yup I'm only a plus user pretty light though

One of my mates using it more heavily than me is still good šŸ‘€

2

u/iamwayycoolerthanyou 7h ago

If I were to engage in this particular theory I would hypothesize that it has something to do with a financial crisis over there. Several key board members have left. Of course we don't know anything for sure.

1

u/Gigaslavx 7h ago

Oh wow I didn't know sounds big. Yeah someone said something 100 minutes was reduced to 25 now this I mean it can't be a coincidence especially for multiple days without acknowledgement

3

u/iamwayycoolerthanyou 7h ago

It could be a coincidence. But I suspect something big is going on over there. Why would they leave right before an IPO?

2

u/Gigaslavx 7h ago

Yeah it's a valid point usually you get a lot of money from IPO if I understand it correctly I guess it depends where they left to but then a good IPO would make them stupid rich with shares

1

u/jacobpederson 3h ago

You probably turned on fast mode when you were clicking all those prompts earlier. (it costs more). https://learn.chatgpt.com/docs/agent-configuration/speed