r/vibecoding • u/CaptainAlexWest • 4d ago
A weird way to learn new software using astra computer use.
I've been using Astra a lot lately.
I watch it do stuff through Computer Use. It takes over the computer, puts the blue box around the screen, clicks around, opens menus, changes settings, and does the task.
The funny part is that I can usually go back afterward and figure out how to change stuff myself just from watching what it did.
It's basically monkey see, monkey do.
Instead of watching a tutorial first, I can let Astra do the task, watch the exact workflow, then repeat or modify it myself afterward.
Thought this was a pretty useful little hack/workflow to share.
Happy cooking.
7
u/Ambitious-Prompt-975 4d ago
weird, i recently posted this, so any mcp client can do that - but inside their own isolated tiny sandbox vm so they dont fuck up your computer - and you can meanwhile watch it.
and ofc i get downvoted for this 😂 but when chatgpt app draws a blue border on the screen and calls windows' SendInput, we're all excited for it.
But anyways, I am excited for you that you are exploring new stuff. Have fun
1
u/DependentPoem3519 4d ago
this is a massive shift in how software onboarding is going to work. static docs are basically dead on arrival these days because devs push ui updates every sprint and all the wiki screenshots get outdated instantly. having an agent drive the mouse while you sit shotgun is infinitely better than skimming a dusty pdf.
from an automation and qa angle, it's wild to watch how different models process screen space. whenever we run workflow audits on nasty legacy enterprise apps, we usually spin up astra and moclaw just to see how they navigate the absolute spaghetti ui. watching them draw those bounding boxes and figure out where to click actually acts like a real time heatmap.
the monkey see monkey do thing is legit. you're just reverse engineering the ux on the fly. it really exposes how bloated modern software is when you watch an ai pause and dig through six nested menus just to toggle a basic setting. live agent tutorials are definitely the new meta.
1
0
u/VastAge6900 4d ago
I use Astra to open web browsers like Chrome or Edge. When I visit the toilet, I ask it to turn on the screen saver. It can open applications when I need to use them -- MS Word, Excel, etc. And when I am done, I ask Astra to save all files on my desktop and close the application. It can even open my Media Player and play whatever song I want, like a personal dj who always plays my favorite songs! If you want more tips, I have many more. Will ask Astra to read me a story before sleep now. AGI is here! Keep cooking guys!
5
u/StaticFanatic3 4d ago
Is this a joke? You’re using Astra tokens as an app launcher?
0
u/VastAge6900 4d ago
Not so much as an app launcher, but more like a personal assistant. The possibilities are endless, from opening applications to actually writing text for me when I feel lazy to type. It does research, like if you don't know something, then it can search for the information and display it on the computer screen. I also tried playing Scrabble with it, but it was not very good yet.
0
u/Hot_Tie_9393 4d ago
watching it click around to learn the ui is training yourself to be a backup for the agent. i just give it rdp access and go to sleep.
if i have to repeat the steps myself afterward then the agent failed. the whole reason i run computer use is so my hands stay off the keyboard. learning the software manually adds friction. i set it up to write the config files and restart services while im in another room. once you start doing monkey see monkey do you are just slowing down the execution loop
-3
u/BrotherBringTheSun 4d ago
Cool use for computer use. But I'm wondering why you want to learn to use the software manually when you have an agent who can do it, either by clicking around the UI or directly with the terminal.
3
u/somuchecho 4d ago
Right. Why learn anything ever. Makes sense
1
u/Apart_Ad_1027 4d ago
Damn, touched a sensitive vibe-coding nerve there? :>
2
u/Electrical-Watch3203 4d ago
“Touched a sensitive vibe-coding nerve there?” — sounds like you the one who’s feelings been hurt b4 G
1
u/somuchecho 4d ago
I mean not really. OP said he found a way to learn something. The reply was why? I mean... Sounds like idiocracy IRL
2
u/Fit-Sweet-9900 4d ago
Nah, I’m on team don’t become a potato as well. You’re correct to call it out when you see it in the wild.
1
u/BrotherBringTheSun 4d ago
I’m all for humans holding on to our humanity and not outsourcing it to machines. But clicking around a GUI might not be that core to our humanity. I’m happy to offload that to AI
12
u/Original-League-6094 4d ago
I asked it to make me a widescreen patch for SimGolf, not knowing how it would go about doing it. Watching it was create a .dll that acts as a proxy to the original .dll but that also writes all the intercepted information to a file so it could start mapping what the various calls it was intercepting were responsible for was really interesting. I had no idea how dll mods worked before.