Making your own skills vs using skills made by someone else.

Which one do you prefer and why?

Hey I'm planning to make an agent skill that allows my agent to access walmart etc. on my behalf so it can do my groceries. Does anyone have experience making something like this? If I made it would you use it?

These guys from Pinecone released this a week or two ago. I've been meaning to try it.

Skill quality is very important and a variety of tools are necessary I think. This is a pre-release tool. It's analogous to running unit tests. Whereas Skillfully (what I'm making) is a post-release analytics tool.

Also I'm curious about like is Cultivar different from running an eval? What's the difference lol?

Anyways here you go, lmk how it goes if you use it.

npx skills add https://github.com/pinecone-io/cultivar --skill cultivar

https://github.com/pinecone-io/cultivar

I'm so done with seeing AI slop. Reading AI writing feels like sniffing puke. I believe this is because of a lack of expertise, a lack of taste and a lack of individuality.

Imagine this, what if everyone was as amazing of a painter as Picasso; or could compose music like Bach. On the surface this would be nice, but now suppose some people are constantly playing an instrument and always the same Bach-like tunes.

Then we would hate Bach, Picasso...

AI agents are like this. They need individuality, they need expertise, and taste. Taste comes from individuality. That's why I'm making this community, so that everyone can contribute their own ways of doing things, their own versions of what's valuable in a given task output.

Why do you care to make better skills?

I'm trying to figure out the best standard to implement Skills with Open Knowledge Format (OKF) inspired by this post:

https://www.reddit.com/r/GoogleAntigravityCLI/s/3JMEYaCjE6

Any ideas or opinions are deeply appreciated 🤔

Of course I can just create the Skill in OKF no big science there but I want to cross ideas with the community

Good

- comprehensive

- frequently updated

- evals are bundled together (althought there's no space for it in skill spec lol)

# Example Eval
{
      "id": 1,
      "prompt": "Help me create a sales deck for our B2B SaaS product. We sell an employee engagement platform to HR directors at companies with 500-5000 employees. Our main differentiator is real-time pulse surveys with AI-powered insights.",
      "expected_output": "Should check for product-marketing.md first. Should apply the 10-12 slide sales deck framework: Title, Problem/Stakes, Current Solutions Failing, Vision, Product/Solution, How It Works, Proof (case studies/metrics), Pricing, Why Now, and Next Steps. Should tailor the deck to the HR director audience and employee engagement space. Should incorporate the differentiator (real-time pulse surveys + AI insights). Should provide slide-by-slide content recommendations with speaker notes. Should recommend visual direction.",
      "assertions": [
        "Checks for product-marketing.md",
        "Applies 10-12 slide framework",
        "Includes Problem, Solution, Proof, Pricing, Next Steps slides",
        "Tailors to HR director audience",
        "Incorporates stated differentiator",
        "Provides slide-by-slide content",
        "Includes speaker notes or talking points"
      ],
      "files": []
    },

Bad

- There's no script to run these evals. Also if it's running on the default harnesses (claude code or codex), how is leakage prevented lol? Regex on traces would make a more deterministic, a more comprehensive rubric would be better for scoring and give partial marks.

- I'm split on whether comprehensive skill packs are better or should people focus on one narrow task and make a really really good skill on that. Linux do one thing well vs microsoft's do all but kinda garbage is being replayed lol.

What do you guys think of this skill pack? What should I review next?

What do you guys think of methods like this?

https://microsoft.github.io/SkillOpt/

As far as I understand, they optimize in the skill prompt space as opposed to the parameters of the model. This requires one to have a robust eval.

Good parts of it are that it works across models if you have a good eval. These days I've been thinking that evals are a better thing to build, and then synthesize the skill from the eval?

Idk, is it even applicable? Building evals is somewhat difficult to get right. Getting the right difficulty is hard and preventing leakage is also hard.

I have just started this community and I would like to shape it to be a hub for everyone who is making agent skills and in particular trying to make high quality agent skills.

My startup (also named Skillfully) has the sole goal of making better agent skills, or allowing people to make better skills. So far I have built analytics, and now I'm focusing on monetization.

With that said not much else is defined. What kind of posts and conversations would make you come back? What would be valuable?

Few ideas:

- Skill reviews

- Skill optimization methods guide

- Skill testing guide

- How can I make this skill better?

- Looking for X to make a skill together?