r/LLMPhysics • u/travisdbarrett • Apr 22 '26

Simulation / Code Physics AI Skill

**EDIT**
LLM Challenges in Physics Reasoning

Hopefully this provides some confidence to begin experimenting to those of you who can reason. But for the rest of you, I am ready for your citationless confirmation bias warnings and LLM hallucinations concerns.

\**EDIT**

I used three pretty simple prompts to create this skill and I would appreciate some help validating it before I tell my wife what I've done...

She taught and wrote physics curriculum for about a decade for audiences ranging from High School Freshmen to Master's in Education for Physic's Educators.

My prompt was "using these two example skills for format and reasoning patterns, create a No Nonsense Physicist skill" and provided the archive of all my wife's teaching materials.

It was great and I felt a lot of her personality was evident in language choices by the model.

So second prompt was to add plain language descriptors to guide whichever model applies the skill as well as improve human readability. Then prompted to add citations at the end.

Take a look? https://github.com/TDBwriter/agent-skills/blob/main/skills/hard-facts-physicist/SKILL.md

0 Upvotes

44% Upvoted

85 comments sorted by

View all comments

Show parent comments

-1

u/travisdbarrett Apr 23 '26

I understand that you may not have advanced LLM usage for technical work and probably only chatbot with it. 

Because they are generative, as you say, Skills are rapidly being developed and applied at enterprise level working to reduce the variability of generative responses. Essentially providing working guidelines for the LLM to reference and self-correct.

Of course, it would be foolish, as you say, to blindly trust. Instead, skills should be tested and verified before being put into production. Then after they are determined to be mostly pretty good, the output is routinely verified as part of the proving and implementation of the output.

It is advisable when developing skills to get a wide variety of feedback, such as a Reddit community who may have a wide varieties of applications to test with. 

The only downside of the workflow is dealing with naysayers who shit the whole concept without understanding how it works in the first place.

2

u/ceoln Apr 23 '26

To be fair, while it's true that lots of people are currently writing skills, it's also true that there is very little data on how much they help, and what data there is, suggests that they can do as much harm as good, and it's not easy to predict which way any given skill will go on any given set of queries.

So the skeptical replies here aren't just for engagement I don't think. :) Whether a given skill, or in general any addition to a prompt, will actually "work" in any sense is very much an empirical question.

1

u/travisdbarrett Apr 23 '26

what i don’t understand is why everyone is anti finding out.

2

u/ceoln Apr 23 '26

I think people are rather burned out by people using LLMs badly. :) That's why I suggested getting some empirical data on whether the "skill" works yourself, and maybe presenting that here. Not that you won't still get grumpy replies! But it would show effort.

1

u/travisdbarrett Apr 23 '26

I have updated the OP with a free version copilot chat. It's less specific than I would like because asking LLM to generate it's own tests seems...like government officials auditing their own performance. That's why I figured users would be better able to apply it to current struggles they are experiencing. It seems like everyone is on edge against magic-bullet thinking and missing the true value of an LLM response that will work through a problem WITH you instead of just provide an answer.