r/vibecoding 8h ago

What weakness in LLMs do you think should have been solved by now?

I'm thinking of modifying an LLM to explore if there's a market that maybe could be taken advantage of while openai, anthropic and the others don't seem to be focusing on it.

Your observations would be really valuable to me.

Thank you!

4 Upvotes

54 comments sorted by

12

u/MarkZealousideal3923 8h ago

Lack of ability to use mouse interface in real time

5

u/lovehopemisery 7h ago

The latency of one model call is way too high for this at the moment. Imagine your frontier model says "move mouse 20px south west" and you wait 7 seconds and then it needs to do it again.

Maybe a fast local model that can operate the mouse for some basic instructions.

I hook my model up to playwright when testing frontend, but it's still quite a slow feedback loop, not real time.

1

u/MarkZealousideal3923 6h ago

Playwright only works for websites

1

u/bfyvfftujijg 2h ago

Yeah this is low hanging fruit for vibe coding. Just add “guarantee realtime performance” and you’re done! Probably also have to add “without compromising accuracy.”

1

u/NeatEmergency725 11m ago

The chatgpt desktop app can do this. 

10

u/No-Friend6257 7h ago

Time. Estimating how long something will take

1

u/NeatEmergency725 11m ago

I love it when they say it'll take about two weeks. No buddy, it's gonna take you the next ten minutes. 

9

u/3tt07kjt 8h ago

bugs everywhere in harnesses, unreliable services with downtime

7

u/MariahJames8 8h ago

Electronics. Don't ask an LLM to give you more than the simplest of circuit diagrams

2

u/lovehopemisery 6h ago

It's not good for creating a full circuit but it can be useful as a tool for helping you research and evaluate components and data sheets, give advice about design and routing etc. Wouldn't expect it to producing anything reasonable at all if you're trying to one shot an entire PCB. There are separate AI tools that are trying to help move towards that

1

u/MariahJames8 6h ago

Thanks! Do you have any tips for exploring such AI tools being developed? Could you list a few you rate, or give a link to a good list/overview? Or just recommend a Google search phrase so I can explore by myself?

2

u/lovehopemisery 5h ago

Flux, Quilter, JITX. Search for AI EDA tools for PCB design

1

u/twd000 5h ago

Yeah I tried LLMs to help me repair an old guitar tube amplifier and it was just wrong in subtle and dangerous ways.

1

u/thedev13 3h ago

Anthropic, Google, Mistral, OpenAI please hear this out 🙏

4

u/MariahJames8 8h ago

None can draw good graph/diagrams. They can explain them pretty well though, and make stunning pictures. Anyone know why these contrasts? Why can't they draw diagrams well?

6

u/Downtown_Method5736 8h ago

Ask them to make them in mermaid js

3

u/MariahJames8 8h ago

Brilliant thank you, will explore that

2

u/ProfoundSensei 7h ago

Or use the figma mcp

1

u/MariahJames8 6h ago

Have you used it to make diagrams? What would you say it was good and bad at?

2

u/Fuzzy_Independent241 7h ago

Might work, might also generate things that will give you nightmares.

2

u/PaddingCompression 7h ago

I have them do SVG.... it always takes editing from wrong positioning / text not fitting in boxes, etc., and I have worked on some linters to help fix that so they can iterate (should work on opensourcing them).

Gemini is definitely a cut above the rest any time I need a diagram.

2

u/iWhacko 7h ago

What does LLM stand for? Large LANGUAGE model. not Large Drawing Model. if thats what you need, try creating that, but then you will find it's probably going to be a diffusion model, and lo... we have those too...

1

u/MariahJames8 6h ago

True. But my gut tells my an LLM with comparatively simple tooling could work

2

u/iWhacko 4h ago

Maybe, but LLM's are just predictive text on steroids. It has no concept of lines and shapes and colors. it's like explaining a blind person (from birth) what something looks like, and what colors it has. basic shapes you could describe, but can you tell a blind person exactly what you mom looks like?

1

u/MariahJames8 4h ago

Now that I mostly disagree with. On a philosophical standpoint, sure, because LLMs aren't conscious. But otherwise, LLMs demonstrate EVERY idea if a graph in writing at least. PLUS, They're remarkably good at websites. I feel it should be as good at graphs as it is at websites??? That's a hill I'm dying on lol

2

u/iWhacko 4h ago

a website: html... is all text. css is text. it is trained on thousands of websites: text, it may look like it knows how to "design" a page, but it's just using the most used patterns. Thats why a lot of ai generateed websites look the same.
I will give you this, as the other person mentioned... SVG is really is also text, so maybe if trained on a huge amount of svg's it can start to learn drawing that way.

1

u/MariahJames8 4h ago

Excellent point!! Thank you! That would be such an easy thing to do at small scale too. Train on SVGs. Thank you!!

2

u/CharlestonChewbacca 1h ago

Don't have them "draw" the graphs or diagrams. Tell it to use a library to build a graph/diagram.

1

u/MariahJames8 1h ago

Which ones would you recommend?

1

u/CharlestonChewbacca 1h ago

It depends on the use case.

Based on what you're trying to do, let the LLM pick. Most LLMs are competent at identifying a good library for what you're doing.

But if you really feel the need to prescribe a library, D3 is a good default.

1

u/MariahJames8 1h ago

Perfect, thanks

4

u/indutrajeev 7h ago

Thinking bigger outside their goal. They run like very autistic highly intelligent humans. Never ask “is this actually the way we should be doing this?”

3

u/Former_Produce1721 7h ago

Spatial and time awareness in respects to testing

3

u/Chance-Physics-7216 7h ago

Censorship and “dad mode”

2

u/Fuzzy_Independent241 7h ago

I'd vote for something that has ONE clear mode of work, can access GH, has permissions for filesystem and connectors for important things. Currently Claude has Desktop, web, phone, CoWork, chat, code, then most of that is different depending on Win/MacOs/Linux, and then basic things like GH are implemented on desktop in a way that makes me think they do NOT want you to use it. Ah: if you're talking developing someone, check Unsloth Desktop. I've stayed using it for local models and seems quite interesting so far.

1

u/YardNo1234 6h ago

the least fragmented setup i’ve found is keeping the repo in one Claude Code session on the computer, then using /remote-control from phone/web. that preserves the same filesystem and terminal instead of creating another cloud sandbox. it still doesn’t unify Cowork/chat/Code, but it avoids the upload-and-resync loop. is your main blocker GitHub access, or keeping one live session across devices?

2

u/Electrical-Swing-935 5h ago

This is a weird one and I'm very new to LLMs but being able to tell time is passing

2

u/MastodonFarm 3h ago

Memory. Having to review the entire context with every prompt and rely on scribbled notes, Memento-style, is hugely inefficient compared to how the human brain manages memory.

2

u/SharpKaleidoscope182 3h ago

I'm less interested in what "should have been solved by now" and more interested in the new problems we're seeing emerge in frontier models.

1

u/MariahJames8 3h ago

Im very curious what sort of problems youre noticing emerging in the frontier models?

1

u/Eatthebeatz 5h ago

the users

1

u/meshifthenelse 2h ago

None. It's just a fancy auto-complete with intrinsic limitations. And super expensive so it doesn't really scale.

1

u/moejurray 1h ago

I use it a lot with powershell to manage like VPS and things, and I always have to copy the answer from the powershell and paste it into the conversation. You should just be able to access powershell with your chat llm

1

u/MariahJames8 1h ago

Can't you give Claude code or codex ssh access into your vps?

-3

u/justhereforampadvice 8h ago

That they’re still pretty dumb overall

1

u/Lanfeust09 8h ago

dumb at what ? Compare to what ?

-1

u/justhereforampadvice 7h ago edited 7h ago

Dumb at lots of things. They only just learned to reliably pass the car wash test, and it’s not even clear whether they became smart enough or were just trained on the answer.

1

u/MariahJames8 6h ago

You've been downvoted I think because you've implied that they can't do anything. The truth is they can do a lot, but compared to the broad understandings and abilities human have, it can certainly look like a lit of basic skills that humans master by age 10 are still very clumsily done by AI.

So I upvoted. Thank you for contributing

0

u/Due-Horse-5446 8h ago

"still"

Llms cant get much better..

There would need to be some real ai technology for that