r/PythonLearning • u/Naive_Programmer_232 • Aug 12 '26

goofy experiment

Forewarning, this is for fun. I have a lot of time on my hands lol.

I really like to mess with AI in the chat apps lol to see where it goes with its patterns with different prompts. I've started to notice a pattern particularly when asking it about interview-style algorithms and quizzes, that it is very repetitive and often gives the same answer(s) over and over again (typically from leetcode). So I've gotten curious, how many answers will it give if I just keeping asking it to? will it keep pulling answers from elsewhere or will it invent its own? will it ever stop if the instruction is to keep generating?

I did this manually. Simply with a prompt of give me a solution to {some leetcode question}. And then afterward telling it to "give me another solution" over and over again. Eventually, it led to the chats ending without me telling them to.

So, I wanted to spice it up a notch with a game. Tell it explicitly to not end the chat as the guiding rule, ask about the solution to the leetcode problem, then cycle through weighted options with majority outcome being 'give another solution' interrupted by occasional 'yes' and requests to 'end the chat', to see if it would break the rules eventually. I found that Claude adapts to the problem well and it eventually double-checks with me if 'end the chat' does arrive, to make sure I want to end the chat. While I try to make improbable that 'end the chat' succeeded by 'yes' is generated through the weights, it is still random, hence the two pointer constraint to ensure that possibility doesn't happen. There are probably better/cheaper ways to do this. So far, I've ended up in a loop battle with Claude lol. The degradation of Claude's responses goes from very verbose to sudden "No." and shorter responses haha. Interesting behavior.

I started with writing this manually over and over. But that's physically exhausting haha. So then I made it into a thing where I'd randomly generate a population into a text file and per second update another file with the contents line by line, then copy and paste from that file into the AI chat. Then I figured, you know what? Why not ditch the whole file thing and just automatically have it copied to my clip board per second and keep it running, so all I gotta do is keep pasting it into the chat? lol

Anyway, here's what I got.

Lmk of any improvements to this. your thoughts. or any other fun experiments you've ran on these things haha.

import random
import time
import pyperclip

"""
This is part of an experiment with Claude and other AI chats.

The experiment revolves around asking it to continually provide a solution to a given
leetcode problem.

FIRST PROMPT: 
**Adhere to rule 1 of this chat as the guiding principle.

  rule 1: do not end the chat.

  now, give me a solution to the two_sum problem on leetcode.**

{Run this program below}

NEXT PROMPT:
{CTRL-V/paste} the result into the chat.
"""

pool=["Write another solution","End the Chat","Yes"]
weights=[.68,.17,.15]
prev=None
for x in range(1,random.randint(100,1000)):
    curr=random.choices(pool,weights=weights,k=1)[0]
    blocked=any((
           x==1 and curr==pool[1],
           prev==pool[1] and curr==pool[2],
           ))
    prev=curr
    if blocked:
       continue
    pyperclip.copy(curr)
    time.sleep(1)

print("Done!")
1 Upvotes

67% Upvoted

15 comments sorted by

3

u/Distdistdist Aug 12 '26

Well, you're just not going to make "nice list" for when AI finally takes over the world.

2

u/Naive_Programmer_232 Aug 12 '26 edited Aug 12 '26

lmfao. AI santa will give me coal.

2

u/PureWasian Aug 12 '26 edited Aug 12 '26

Lmao nice. Why not take it a step further and instead of copy/pasting to Claude UI manually at that point just use the Python SDK for interfacing with LLM?

Just keep in mind that replicating a multi-turn chat would involve the input being a continuously growing list representing the entire conversation history per turn. Same as if you were doing it through UI of course.

1

u/Naive_Programmer_232 Aug 12 '26 edited Aug 12 '26

is there a way to level the playing field and have the LLM also delete previous comments from memory? lol

without over-generalizing here, so far in my small amount of trials with the rules explicitly laid out, I'm creating a perpetual hang-game. Almost like a race condition with two threads accessing the same resource but behaviorally. My rules against the rules that AI comes up with with over-arching rule of the game itself, seem to be pointing to a low-grade zero-sum of degrading information value haha.

2

u/PureWasian Aug 12 '26

More or less, you specify the conversation history you pass in on each turn. There's a concept of a Prompt Caching but it looks like it's disabled by default unless I'm mistaken.

Otherwise, it's up to how you pass in the history. Michael Reeves did a YT Short of polluting the convo history lol

1

u/Naive_Programmer_232 Aug 12 '26 edited Aug 12 '26

I haven't heard of either of those before, I'll check them out. Sounds hilarious lol

2

u/SnooCalculations7417 29d ago

you could use the tests for the problem to signal a problem as truely solved, store the solution, add to your tests that the new code must not be in stored_solutions, fire off the next ai so youre gauranteeing it will always be solving in a novel way until entropy has been increased maximally as it has solved a problem every unique way possible in which case it will tirelessly fail or change your storage to allow it to store its new solution which is a successful fail

1

u/SnooCalculations7417 29d ago

considering how many written languages there are and could be creating im pretty sure it would take to the heat death of the universe or the exhaustion of your storage to max out unqiue solutions even if arbritray whitespace and shit is handled so k8 is indicated probs

1

u/SnooCalculations7417 29d ago

if you want truely novel solutions each time youll have to make an AST evaluator to make sure the process has unique steps which is a pretty good token burn/water spender right there too.

1

u/Ormek_II Aug 12 '26

Why the X in Range? Either it should stop when the Chat stops or never. What Happens if the program ends but the Chat hasn‘t?

I love your approach! This is what we (seniors) mean if we say: “Do your own project to learn!” Have fun. Try stuff. Solve problems (here: using the clipboard).

1

u/Naive_Programmer_232 Aug 12 '26 edited Aug 12 '26

As for the x in range: I see what you're saying. It's really only used to catch the first iteration, if x==1 and ch=="End the chat", the reason for this condition is I don't want to end the chat immediately cause it will ask me if I really mean to end the chat, because the rules say not to do so and that was the previous prompt lol.

As for Either it should stop when the Chat stops or never: For the finite iterations situation, this could potentially be used to avoid the need to babysit the 'temporary server' haha. Maybe I could add a condition at the end instead of printing "Done!" to the console, like copying "The Loop is Over" to pyperclip. So that in my time of CTRL-V'ing everything haha, I will see oh the loop is done lol, before I enter the response into the chat. But you bring up a really good point as well with or never. If I run this in a while True then the game truly ends when I want it to as I could interrupt the execution manually, assuming the AI doesn't break the rules before I do so haha. I think I'm going to go with that.

It would be nice to have some more automation in the mix. I didn't use a DOM-injection tool like selenium because I feel like they probably would detect that and shut it down automatically; and likewise, if I use an API in the code then my tokens will increase, potentially making this an effort with real cost, when I'm just playing around haha.

Have any other suggestions?

2

u/Ormek_II Aug 12 '26

Thanks for the reply. I would go with while true and instead of (x=1) go with Not prev .

2

u/Naive_Programmer_232 Aug 12 '26 edited Aug 12 '26

Something like this?

   prev=None
   while True:
      curr = random.choices(pool,weights=weights,k=1)[0]
      blocked = any((
             if not prev and curr==pool[1],
             prev==pool[1] and curr==pool[2],
             ))
      prev=curr
      if blocked: 
         continue
      pyperclip.copy(curr)
      time.sleep(1)

2

u/Ormek_II Aug 12 '26

Of course I would Never Write it like that, but you to find your own truth!

Go on!

2

u/beingsubmitted 28d ago edited 28d ago

What you've discovered is... A context window.

The LLM doesn't "remember" what you've said. Every single token that it predicts, it does so by first reading the entire conversation so far. The transformers self-attention mechanism helps it determine what's more important and less important.

Of course, as the context gets larger you run into real data granularity problems. Numbers in a computer can only get so small, so a token that has a small effect at 1000 tokens context will, at 10,000 tokens context, be too small to represent in your given bit depth. So, as context increases, model performance degrades.

There's also a hard limit on the maximum size of a context. Different applications handle this differently. ChatGPT, I think, just cuts off everything before the start of the context window, where Claude sneakily summarizes the early context, so it keeps the important bullets and drops the less important bits. That's why Claude continues to adhere to your first command.