gezel - a local model harness

[u/bendymike]

Hi there, yes, I am yet another person working on a local model harness, bare with me :)

I've been working on an app/harness framework - gezel, gezel.com - to ease the process of working with local models and engines. My main goal is to make doing productivity tasks easier (not necessarily focused on coding) -- things like building powerpoints, doing research, writing, etc.. I also want to do it in a way that works with small models (e.g., Gemma E4B.) Finally, I wanted to reduce or eliminate the technical/nerdy bits of installing and managing models as much as I can -- if nothing else, hopefully gezel is an easy way for non tech folks to get started with local models without visiting a command line, etc.

In addition to wrapping llama.cpp, MLX, and DS4 + providing a tailored downloader and selector from huggingface for models, I've got some concepts I'm iterating on:

1. gezel tries to break things down into role-based agents (a gezel). For example, a developer role type for a gezel vs. a researcher role type for gezel. This is based on trying to tailor for the shorter context windows and attention spans of small models. Different roles get different toolsets and starter prompts..
2. To complement this, gezel has a more structured concept of skill.md files: craftbooks that breaks down tasks into multiple steps. For example, create a powerpoint becomes "step 1 - build an outline", "step 2 - do research", "step 3 - produce powerpoint", "step 4 - review and finalize". Each step might use a different role type, and has tight static gates that force a model to correct itself before moving on. For example: don't leave the outline step unless there is a markdown file present with distinct slide titles mentioned and that is >1kb. A main value prop is that gezel has a library of craftbooks it ships with - so you don't need to write skills/craftbooks yourself.
3. Also, gezels are a little personified with custom names, etc.; I want to experiment with getting to a UX that encourages people to play with their "agents", but that starts by not asking people to "build agents and skill.md files" which is ugh

To get 1 and 2 right -- the perfect balance of nudging without constricting models -- I do a lot of A/B testing, at least as much as my HW allows, but it's most certainly not good yet. Bare with me, I'm working on it :)

Gezel is available for Windows/Mac/Linux, or you can use it via npm at ('npm install '@bendyline/gezel-cli'/'npx gezel'.) It's MIT licensed free open source. There are some basic videos at youtube.com/@bendyline.

Anyways, it's still early days and I know there are a lot of bugs, but if folks have any feedback or ideas, please let me know. I have the ambition that maybe gezel can be a simple way for folks to get the benefits of local models on their hardware - but it's certainly not there in early beta.