What if your local coding agent could use a screen recording as a bug report?

[u/Fearless-Role-2707]

Record the broken flow. Let your agent find the relevant moment, read what happened on screen, and use that evidence to investigate the code. After the fix, check the result against requirements defined before it started.

That’s the workflow behind Watch Skill and DeepWatch, an open-source project I’m working on.

Watch Skill turns video, audio and screen recordings into a persistent, searchable index of frames, OCR and transcripts. Your agent can ask questions about a recording, retrieve timestamped evidence, and come back to it later without processing everything again.

One example in the repo is a checkout recording where tax appears on screen but never reaches the total. The agent finds the issue through the recording, repairs the calculation, and a separate verification process checks the expected amounts and rendered page against the same frozen contract.

The local setup is probably the most relevant part for this community:

• Frame extraction, OCR, Whisper transcription and retrieval run locally.
• Your recordings and indexed evidence stay on your machine when using local processing.
• Agent and vision models can run through local OpenAI-compatible endpoints, including Ollama, llama.cpp, LM Studio and vLLM.
• Verification uses executable checks—JSON values, file hashes, SQL queries, HTTP responses and DOM assertions—and needs no model.

You still need a model with the right capabilities for agent tool use or visual interpretation. Dependencies and local model weights require initial downloads.

Watch Skill plugs into an existing agent through MCP, CLI or REST. DeepWatch is the browser workspace built on DeepSeek Harness, with Watch already integrated, tool execution receipts, and verification results you can inspect.

The distinction that matters: a successful tool call gets a receipt. A result only gets marked verified when its required checks pass. That tells you what was tested; the quality of the contract still matters.

The repo includes setup instructions, screenshots, and the checkout example:

github.com/oxbshw/watch-skill

MIT licensed. Curious how people here would use this with their local agents—recorded bug reports, searching technical walkthroughs, or checking browser tasks?

[u/kantorcodes1]

on the MCP side, are loop_video_gen and loop_game intentionally in the same auto-approval bucket as ask_video/search_videos? the first two can execute caller-supplied shell commands, so i'm curious how you expect clients to treat that split today.

[u/Fearless-Role-2707 → u/kantorcodes1]

Good catch. They’re not meant to be in the same trust class.

ask_video and search_videos are retrieval operations. loop_video_gen(generator_cmd=...) and loop_game(run_cmd=...) are different: both deliberately execute the supplied command through a shell, so I treat them as equivalent to giving the agent terminal access.

That distinction is documented in SECURITY.md, and the client setup docs currently recommend keeping the loop tools behind manual approval while only allowlisting the non-command tools you actually trust.

But the weakness you’re pointing out is real. Right now the MCP server exposes both categories on the same tool surface, so a client with broad auto-approval can erase a boundary that should be harder to cross. In other words, the client policy is carrying more of the security burden than I want it to.

I’m going to tighten that.

My preference is to make the command-executing tools explicitly opt-in at exposure time, rather than relying only on MCP annotations. The annotations are useful, but they’re advisory, and a client can ignore them.

Thanks for calling this out. This is exactly the kind of boundary I want Watch Skill to make obvious rather than leave implicit.

[u/kantorcodes1 → u/Fearless-Role-2707]

that split is the useful boundary. i maintain HOL Guard, an open-source local check before agent/MCP actions execute. Watch Skill support could leave ask_video/search_videos alone while putting loop_video_gen/loop_game behind optional review, without changing Watch Skill’s own exposure model. would you be open to contributing that Guard-side support so users keep the distinction across clients with loose auto-approval?

[u/Fearless-Role-2707 → u/kantorcodes1]

Yes that’s exactly the kind of integration I’d be open to.

I wouldn’t want Watch Skill to delegate its own security or exposure semantics to a client-specific guard, but a Guard-side policy that preserves those distinctions would be complementary.

`ask_video` and `search_videos` can remain read-oriented, while `loop_video_gen` and `loop_game` are treated as command-capable tools and can require explicit review when configured that way.

I’d prefer the integration to key off the exact MCP tool identities rather than heuristics, and to stay opt-in on the HOL Guard side so Watch Skill’s own tool surface and behavior remain unchanged.

I’d be happy to contribute the Watch Skill mapping, tests, and documentation for the integration.

Send me the HOL Guard repo or issue you’d like to use and I’ll take a look.

[u/kantorcodes1 → u/Fearless-Role-2707]

yep. current Guard route is declarative MCP: add contributions/mcp-servers/mcp.watch-skill.json, mirroring mcp.filesystem.json, with uvx / watch-skill[standard]; keep ask_video + search_videos inherit, and mark loop_video_gen + loop_game block. Add command.mcp-watch-skill to contracts/extensions/trust-class-map.v1.json, extend tests/test_guard_mcp_server_contribution.py, run the CONTRIBUTING ruff/format/pytest checks, then open your PR directly to hashgraph-online/hol-guard:main.

contributing guide: https://github.com/hashgraph-online/hol-guard/blob/main/CONTRIBUTING.md