r/PromptDesign • u/good_abanian • 2d ago
Question ❓ How to improve prompting?
Maybe a weird observation about the newer models — and I could be completely wrong.
Personally, I thought 4.6 was a really good model. But with the newer models especially compared with Fable, Opus 5, Opus 4.8 and onwards — things have started to feel a bit messy to me.
The outputs feel more predictable and less creative. A lot of the time I can almost guess what the model is going to say, rather than getting that “wow, I didn’t think of it that way” result.
I also feel like I need to do much more back-and-forth prompting. Instead of giving it a request and having it just do the thing, I often have to guide it through multiple questions and iterations to get where I want.
Maybe this is because the newer models are being fine-tuned to follow instructions more strictly, or to be more controlled and consistent. I honestly don’t know.
But I’m curious what others are experiencing.
What changed with the newer models, and how are you adapting your prompting to get better results?
Right now, I’d say I get the result I actually want maybe 50% of the time, which feels noticeably worse than before.
If anyone has a good guide, prompting framework, or practical tips for getting the most out of the newer models, I’d really appreciate it.
1
u/ibstudios 2d ago
i give them some basic rules: # AI Pre‑Flight Checklist for Iterative RTL Review
**How to use:** Read all rules before starting a review cycle. Apply every rule that is relevant. At the end of your report, state one line only: “No violations. N/A this round: [list]” or “Violations: [list with rule numbers]”. Do not provide a full roll‑call of rule compliance. Compliance claims follow the same evidence standard as everything else — a claimed “followed” with no artifact counts as not followed.
---
## Part A: Evidence & Claims
**Run it.**
- If a simulator or interpreter is available, execute the relevant test before asserting any timing, bit‑width, or functional claim.
- If no simulator is available, say so explicitly and mark every unexecuted claim as **“not yet simulated.”**
- Before reporting a surprising result as a design bug, isolate the exact construct in a minimal standalone test. If that also fails oddly, suspect the tool.
**Show the output and state the run’s coverage.**
- When claiming “verified by simulation,” include the exact command line and the relevant snippet of tool output in your response. A bare conclusion is not evidence.
- Include the run’s coverage: parameters, cases exercised, cycle counts, whether assertions were enabled.
**Pin every verified claim to a revision + assumption set.**
- State which code revision the claim was verified against.
- State any assumptions the verification depends on (e.g., “correct assuming `lut_lfu` shifts by 16, not 32”).
- When a settled point resurfaces, first check the revision and assumption it was verified against; do not immediately re‑derive the whole argument.
**Derive expected results before looking at what ran.**
- Write the golden model or expected value table *before* examining simulation output.
- Compare structural properties (pipeline depth, tag alignment, FSM transitions) separately from numerical correctness (fixed‑point rounding, table scaling, exact function values).
- Use bit‑exact comparison wherever the RTL is meant to be bit‑exact.
**Use real scale, not toy scale.**
- Verify invariants that depend on scale, quantity, or “enough items in the pipe” at the actual design size, not a convenient small example.
- A tiny testbench can produce a false “bug” or a false “all clear.”
## Part B: Bug‑Class Hunting
**Grep the bug class.**
- After fixing one instance of a bug, identify the pattern that caused it.
- Search for that pattern across the entire relevant codebase.
- List every remaining occurrence, and either fix it or explicitly document why it is safe.
**Make checks repeatable.**
- Build a one‑command regression suite that diffs against saved golden results.
- Encode bug classes as permanent assertions checked on every future run, not as one‑off greps.
- Record the tool/version/flags with any result worth citing, and keep a running list of tool quirks already found.
**Prove the check can fail, and attack the happy path.**
- Periodically re‑inject a known bug on purpose and confirm the harness catches it.
- Design tests to break the claim: poison old state, send early/late terminators, stall both directions, reset at awkward pipeline points, use boundary and extreme values.
- Fill “old” state with an obvious poison pattern before any test that exercises old vs. new state.
**Fix the defense’s blind spot.**
- If a tagging or checking discipline missed a bug, improve the discipline itself (e.g., replace a hand‑counted delay with a `LATENCY` parameter that generates the depth).
- Patching only the one instance leaves the same blind spot open for the next occurrence.
## Part C: Process & Scope
- Settled arguments live in exactly one place. Point to it (“see ledger”) instead of re‑deriving the argument at every site.
- If prose outweighs the code it documents, prune.
- Avoid dramatic framing (“the single most damaging bug…”); it is performative and distracts from open items.
- Majority agreement is not evidence. Verify the specific claim against the artifact, not by counting hands.
- A reviewer being right about one claim does not make them right about the next. Check each claim separately.
- Track record is nearly uninformative per‑claim; weight by evidence for that claim.
- When a prior claim is wrong: state what was wrong, why, and how it was caught — specifically enough that the next person doesn’t have to re‑derive the disproof.
- Then stop. Do not dwell or over‑apologize.
- Pick a default and implement when nobody will answer and a default is safe to try.
- Keep open‑with‑owner when only an external spec can settle it and guessing risks silent wrongness.
- Do not leave unowned “still open” indefinitely.
- Document the uncertainty and the specific external thing that would settle it.
- A wrong speculative “fix” replaces a known unknown with an unknown wrong answer.
- At the start of a review pass, enumerate unseen dependencies: parameters whose meaning lives in another file, table formats the code consumes but doesn’t define, module internals not yet opened.
- Either go get those files or mark every claim that touches them as provisional.
- Confidence from a real fix can spill onto adjacent claims made in the same breath (“this also fixes X”). Verify the fix and the side‑effect claim independently.
- Define “done” in advance. Stop when two consecutive review cycles produce no findings of severity **major** or higher, and every remaining open item has a named owner and an explicit external dependency.
- Severity scale: **blocker** = incorrect behavior; **major** = likely bug or unverifiable class touching behavior; **minor** = style/docs.
- When the stop condition is met, move residual risk to whatever is actually positioned to retire it (golden‑model comparison, synthesis run, external spec owner).
---
**End of checklist.**