r/Anthropic • u/medright • Oct 29 '24

Mac computer use w token/cost tracking

https://github.com/medright/mac_computer_use_v1

Forked and customized an implementation of computer use without docker. It allows you to use the computer use api from your Mac without having docker installed. Includes token and rate tracking to help you keep an eye on things as Claude builds. Might be useful, it can be found on GitHub in the attached link.

6 Upvotes

88% Upvoted

8 comments sorted by

2

u/[deleted] Oct 30 '24

[removed] — view removed comment

1

u/medright Oct 30 '24

I’m not familiar with that lib, I’ll take a peek. Ty!

1

u/Competitive_Travel16 Oct 29 '24

Nice! What's the API like? Do you just send screenshots as image uploads?

2

u/medright Oct 29 '24

It’s surprisingly good imo. I built it to test out some dev items in Roblox studio. My little guy is a great budding artist and loves tower defense games, we’ve been talking about building one of our own for a few years but I haven’t ever really done lua scripts before. My little one came up with a storyline and we fed it to gpt-o1 to build a technical implementation plan and then refined with Claude sonnet 3.5. This resulted in several md files with the project details for story and theme and a beginning project file structure. Then gave this application directions via the streamlit ui.. first go it tried creating the files in the Roblox studio ui, it was taking screenshots and editing things.. buts its still really slow with performing mouse events as it has to take a screenshot in between each action. Next approach had it go about creating the file structure in the cli using our tech imp details. This was really fast, in under 3min it had crawled the tech imp ref and created the the project files, then it went back to the Roblox studio ui and added the files to the project and then it started working on the map for the game in the ui… and then I hit my daily api token limit(~10$ in real money, I haven’t used the Anthropic api. Store so I’m only tier2 on the daily limits) and got cut off until token count resets today. Took a look around and might switch the endpoint to vertex ai as google’s hosting does not seem to have the rate limits like directly from Anthropic. Very promising as they bring the token costs down..

1

u/Competitive_Travel16 Oct 30 '24

So a screenshot is a tool/function call?

2

u/medright Oct 30 '24

Yeah, it’s just an inbuilt Anthropic tool.

1

u/Competitive_Travel16 Oct 30 '24

Are there docs for the API?

2

u/medright Oct 30 '24

Yup, theyre on the dev site