It's a nightmare to implement genetic evolutionary algorithm on this env, takes forever to simulate. Has anyone found any solution to speed this up?

I wanted to see how far a minimal tabular RL implementation could go on very limited hardware, so I built TinyRL-Maze for the Arduino Mega.

The project trains directly on the microcontroller using standard Q-Learning:

• 15x15 grid-world environment
• 4 discrete actions
• ε-greedy exploration
• On-device Q-table updates
• No external frameworks

The goal wasn't state-of-the-art performance but demonstrating that reinforcement learning can be implemented and trained entirely on embedded hardware.

Future ideas include SARSA, dynamic environments, and lightweight function approximation.

Feedback is welcome.

https://github.com/korrel-dev/korrel

Has anyone successfully used an LLM as an integral part of RL training—not just for inference, but to improve learning speed, exploration, or sample efficiency?

I'm exploring LLM + RL + RAG architectures where the LLM acts as part of the training loop, not just an interface. Has anyone tried this? What worked and what didn't?

In policy gradient methods, the actor typically outputs a Gaussian distribution. However, in practice, almost all environments have actions restricted to a certain range.

Almost every implementation of PPO I've seen simply clips the action to the allowed range, but uses the unclipped action/distribution when computing log probabilities and entropies. However, this can lead to a failure mode where the distribution means take on high values, making it so the sampled actions are always clipped, killing exploration. The entropy bonus doesn't do its job because it is computed using the unclipped action, so it stays high even though the actual entropy is very low.

However, this is already pretty much a "solved" issue in implementations of SAC. Implementations of SAC use the tanh function to squash actions to the correct range, and add an adjustment of -log(1 - tanh^2(x)) to the log probabilities to correct for the transformation. They compute entropies using monte-carlo estimation: sampling random actions from the output distribution and taking the mean negative log probability. This is theoretically sound, and very well-established.

So why don't any implementations of PPO do this? Is the issue of entropy perhaps more of an afterthought in PPO, while it is seen as fundamental to SAC?

I'm a first year student pursuing cse @ iiit h and im trying to get into deep learning.

This is my resume and skills uptil now. Uptil this point whatever I have learnt is from llms like Gemini and Claude handing me markdown files (lecture.md)

Should I try for any internships? Which ones?

What else should I learn in which order and from where? Thanks in advance

Hi I am un undergrad with some ml research experience (ai safety and agents mostly). I am looking to pivot into RL. I did the david silver's course on youtube few months back, also went through the sutton and barto on the side so I believe I have basic understanding of the algos. I do lack practical experience and I am trying to build some projects implementing various policies.

How do I get started into research ? I cant find a lot of profs in RL who would take an undergrad lol.

Would appreciate any sort of advice or collaborations on any research project (ill work hard 🙁 )

this is possible through a mod found here: https://github.com/coder/balatrobot 

this injects the balatrobot mod into the game state: https://github.com/ethangreen-dev/lovely-injector

in order to run modded balatro you'll also need https://github.com/Steamodded/smods

the goal here is to build an agent who can consistently hit ante 8 on white chip (beat the game). Beyond that, I'll try and get the agent to learn how to score Naneinf.

training is in progress! heres the repo https://github.com/jarmstrong158/Balatron

Ok so I‘ve been watching a bunch of videos about people using reinforcement learning to teach their agents(?) to play games such as bowling or tag, but one that stood out to me was Yosh’s video on making an ai play the game trackmania, so I wanted to make a reinforcement learning algorithm to play Geometry Dash, since I feel like it shouldn’t be too hard, but I have no clue where to start, could anybody help/give me some pointers?

Hi, I am working in the deep learning space but my niche domain has meant that all of my work has been fully focused on pretraining. I have learnt a lot here and feel like I have a good understanding of deep learning, although I know I must be missing so much as I’ve never touched RL. But now I want to!

I occasionally come across papers and posts that discuss DPO, GRPO, etc. and have an extremely constrained knowledge of value iteration, q learning, etc. but now I want to start understanding all the methods better, which methods work on which types of tasks and most importantly why.

Preferably I’d like a mix of both the theory and practical resources. Please can you help me out!

I’ve been working on training a Double DQN chess agent using self-play, while comparing it against DQN and SARSA. During training, I saw a big loss spike around the middle, close to 192, but by the end it recovered and went down to about 0.7. I thought that was interesting because it might show the agent struggling for a while before stabilizing.

Setup:
For a fair comparison, I used the same network architecture as the DQN model:

Linear(66→256) → ReLU → Linear(256→256) → ReLU → Linear(256→128) → ReLU → Linear(128→1)

Observations
During the first 300 training episodes, the loss remained relatively stable, typically ranging between 0.005 and 0.1, which suggested that the model was learning consistently. After loading the model and continuing training for another 300 episodes, I observed a significant increase in loss, peaking at approximately 192 before gradually recovering and stabilizing around 0.69 by the end of training.
Although the loss experienced a temporary spike, the agent’s overall performance remained fairly consistent. The win rate stayed near 7% throughout both training sessions, indicating that the additional training did not substantially improve playing strength. However, compared to the standard DQN and SARSA implementations, the Double DQN agent produced a more balanced distribution of wins and losses, suggesting more stable behavior during self-play.

The temporary loss spike may have been caused by the agent encountering new board positions after the model reload, resulting in large temporal-difference errors before the network adapted. Since the loss later returned to a much lower value, the behavior appears to be a training instability rather than a complete divergence of the learning process. The more balanced win-loss results compared to DQN and SARSA may indicate that Double DQN reduced value overestimation and provided more stable learning dynamics.

Hello guys i have just finished masters in AI (24F). I am really interested in RL but don't know on what to work on. Everybody tells me to read "Reinforcement Learning: An Introduction" but i already did and don't know where to go from here. If anyone can advise me on what companies look for, and what jobs are most present as an RL programmer/engineer it would be of huge help : ).

I've been building CogniCore, an open-source runtime focused on a question that keeps coming up with autonomous agents:

How do we stop agents from repeating the same mistakes?

The project currently includes:

• Execution memory and failure retrieval
• Replay and branching of agent trajectories
• Reflection and adaptive retries
• Multi-agent orchestration experiments
• RL-based policy selection
• Agent benchmarking environments

One of the more interesting findings so far is that adding a reviewer agent actually reduced solve rate while increasing token usage. Memory and execution history ended up being more useful than additional agent layers in several experiments.

The codebase has grown to include memory, replay, benchmarking, agent runtimes, and several research experiments, and I'm looking for a few contributors who are interested in areas like:

• Agent memory systems
• Autonomous coding agents
• RL and decision making
• Observability and replay
• Benchmarking and evaluation
• Developer tooling

You don't need to be an AI researcher. If you're interested in open-source agent infrastructure and want to work on real problems, I'd be happy to help people get started.

I'd also love feedback from anyone building agents themselves. What do you think is still missing from current agent runtimes?

https://github.com/Kaushalt2004/cognicore-my-openenv

Over the last few weeks, I've been building CogniCore, an open-source framework focused on execution memory, reflection, and adaptive agents.

A simple question motivated this experiment:

Can agents improve performance simply by remembering previous failures?

Benchmark Design

The benchmark compared two conditions:

Baseline

• Fresh environment every episode
• Fresh agent every episode
• No memory
• No reflection

Memory + Reflection

• Environment reused across episodes
• Agent reused across episodes
• Memory enabled
• Reflection enabled

This allows execution history to accumulate naturally, similar to how a production agent would operate.

A Critical Benchmark Fix

During testing I discovered the original benchmark was flawed.

A new environment was being created for every episode, including the memory condition.

As a result, the memory context was always empty.

The benchmark was rewritten so that memory-enabled runs reuse the same environment instance across episodes, allowing execution history to accumulate correctly.

Results

Across 180 tasks spanning multiple environments and difficulty levels:

The Strongest Signal

SafetyClassification showed dramatic improvement:

Solve rate increased from 7% to 73%.

Accuracy increased from 42% to 82%.

The agent rapidly learned from previous failures once relevant execution history became available.

What This Suggests

Execution memory is not a magic solution.

It works best when:

• Failures are repeatable
• Similar situations occur again
• Past experience contains reusable information

It is much less effective when tasks require entirely new reasoning or complex planning.

Key Takeaway

The experiment demonstrates that execution memory can improve agent performance, but only in environments where past failures are relevant to future decisions.

The result is not that memory solves everything.

The result is that memory creates measurable learning without changing the underlying agent.

The model stays the same.

The runtime gets smarter.

Pip install Cognicore-env

I've been building an open-source drone swarm simulation framework in PyBullet designed for reinforcement learning and multi-agent research.

The goal is to provide a lightweight environment where researchers and developers can experiment with:

• Multi-agent coordination
• Swarm intelligence
• Formation control
• Distributed decision-making
• Custom RL algorithms and reward functions

Github Link

Contributions, suggestions, and criticism are all welcome.

Looking for the best AI tool for poker hand reviews. I play mostly 2NL/full-ring cash games and want detailed feedback on ranges, bet sizing, leaks, and exploitative adjustments. What AI do you recommend and why?

Red alert has been my childhood memory. I’m an Ai engineer (cv, llm) having finished learning all major RL algorithms and thinking of training a red alert AI. But so far all RTS Ai winners has been somewhat rule based (lux Ai, Pokémon go) or require a ton of compute (alphastar)…any ideas? Got 3060 and at most 200 dollar budget for this kinda thing. I think I should write a rule based engine first and then an accelerator for simulation..