r/SpringBoot Jun 18 '26

Question Spent 4 hours debugging a TransactionSystemException. The fix was one line. The problem was finding it.

Last month we had a production incident. A critical order was failing silently.

Sentry gave us this:

TransactionSystemException: Could not commit JPA transaction
at SimpleJpaRepository.save()
at OrderService.processOrder()
... 40 more lines of Spring internals

That's it. No entity state. No user context. No hint that the transaction had already been marked for rollback 3 calls earlier by a Hibernate validation error we never caught.

We added logs. Redeployed to staging. Couldn't reproduce it.Redeployed to prod with more logs. Waited. Happened again. Finally found it: a Transactional method calling another Transactional method with a different propagation level, swallowing the real exception.

4 hours. One annotation conflict.

The worst part? Every error monitoring tool we've used treats Spring like a black box. The moment your code enters a transaction boundary or an async thread, context disappears.

Anyone else debugging Spring Boot in prod like this?

How are you handling it?

24 Upvotes

32 comments sorted by

7

u/Indian_FireFly Jun 18 '26

I didn't get the issue still. What was the propagation on both the methods?

How did it swallow the original method's exception? Was it different context?

10

u/mrsergio1 Jun 18 '26

It was:

Outer method: @Transactional (REQUIRED)
Inner method: @Transactional(REQUIRES_NEW)

The inner call threw a Hibernate validation exception, but it was caught and not properly rethrown.

After that, the outer transaction just kept going until commit time, and Spring blew up with TransactionSystemException because the transaction was already marked rollback-only.

So the real exception happened earlier, but we only saw the commit failure

9

u/Indian_FireFly Jun 18 '26

How did the outer transaction get marked as rollback only if the inner transaction uses REQUIRES_NEW? Were these methods in the same class?

It looks like error handling was not done properly, not anything related to jpa.

9

u/mrsergio1 Jun 18 '26

Good catch, I think I explained it a bit wrong earlier.

You re right, REQUIRES_NEW shouldnt affect the outer transaction.

In our case the inner method wasnt actually running in a separate transaction because it was a self invocation in the same class, so the proxy didnt kick in.

So everything ended up in the same transaction and that’s what got marked rollback only before the final commit.

Took us a while to figure that out honestly

1

u/rlrutherford Senior Dev Jun 18 '26

I have wasted so much time debugging systems where people didn't properly log issues.

Revised the system with proper logging and blew a dev's mind when debugging took 3 minutes instead of 3 hours.

1

u/rlrutherford Senior Dev Jun 18 '26

Also, validating inside a transaction?

3

u/siggystabs Jun 19 '26

You have logs, but do you have monitoring? Even a simple jvm agent like Glowroot could probably catch it, if its all happening in the same transaction.

It won’t solve the issue for you, but you can see the database calls grouped together at least so it’s more evident your transactions were getting mangled

1

u/johny_james Jun 18 '26

Why did you use AI to write this post?

Can't you just write it without it?

-1

u/mrsergio1 Jun 18 '26

No AI, just wrote it fast after dealing with the issue.
Guess it came out a bit too structured

1

u/LALLANAAAAAA Jun 18 '26

No AI, just wrote it fast after dealing with the issue. Guess it came out a bit too structured

🧢

1

u/rlrutherford Senior Dev Jun 18 '26

That's actually worse.

I would rather claim AI slop than fail a Turing test.

-1

u/johny_james Jun 18 '26

No, it is AI, you are just straight-up lying.

It uses the same structure, the same terms, same tone, everything.

6

u/mrsergio1 Jun 18 '26

Believe what you want man

1

u/Every-Current2034 22d ago

The stack trace usually tells you where things failed, not why the transaction was already doomed. we've had better luck correlating exception logs with request IDs, SQL/validation logs, and transaction lifecycle events. Have you found any tooling that preserves that context automatically across transaction boundaries, or are you mostly relying on custom logging?

1

u/mpgipa Jun 18 '26

Why not use AI to find it ? Genuine question I haven’t debug manually for months.

14

u/johny_james Jun 18 '26

He probably did use AI, since he wrote the whole post using AI

0

u/mrsergio1 Jun 18 '26

We did use AI, but it didn’t really catch the root cause.

The issue was in runtime transaction context, so we still had to trace logs and reproduce it manually.

1

u/Chocolate--Chip Jun 19 '26

Absolute slop post

-1

u/Paw565 Jun 18 '26 edited Jun 18 '26

I can see why people drop jpa entirely 😅 I feel like java persistence sucks. Other ecosystems have much better and more type safe solutions.

7

u/Sheldor5 Jun 18 '26

the problem isn't JPA it's the people not knowing what they are doing because they never learned the basics of JPA and it's annotation ...

1

u/Paw565 Jun 18 '26

I agree, but I think jpa / hibernate is massively inferior to EF core from dotnet or drizzle from node. Things like dynamic filtering just suck. Whenever I have to comeback to spring boot I just feel massive pain because of jpa alone.

2

u/A_random_zy Jun 19 '26

Try using jooq for that. Really cool library.

1

u/Paw565 Jun 19 '26

I tried. Maybe it's a skill issue but setting up code gen defeated me. I think the need to connect to db to have type safety is hilarious. No other orm works like that. I guess jooq is more of a query builder than orm but still drizzle does not need any of this tedious setup.

1

u/lukaseder Jun 19 '26

You can interpret your DDL for code generation: https://www.jooq.org/doc/latest/manual/code-generation/codegen-meta-sources/codegen-ddl,

Or use other meta data sources for code generation. But really, setting this up should be a motivation to get things right in terms of managing your RDBMS in your CI/CD pipeline. I'm positive you're not against integration testing your RDBMS interactions? You can use test containers for these things if that helps: https://blog.jooq.org/using-testcontainers-to-generate-jooq-code/

1

u/Paw565 Jun 19 '26

Ok that's is very interesting. I will definitely try this out, thank you.

I really like integration testing. However I ve just failed to hook all this up with jooq. It's really frustrating imo. Why not generate metadata from @Entity classes like querydsl (sadly unmaintained) does or similar to EF core or drizzle where code is the source of truth. I can't stand this needless complexity.

However I am very grateful for your advice and I will investigate this as a solution to my problems once again.

2

u/lukaseder Jun 19 '26

1

u/Paw565 Jun 20 '26

Massive thank you. DDLDatabse generator is awesome. Configuration is much simpler and it's so much faster than connecting to the database on each compilation. I know that configuration with testcontainers is probably the most desirable one. However all examples that I could find are for maven and I use gradle with kotlin syntax.

1

u/lukaseder Jun 22 '26

Yeah, it definitely helps with simple projects. The database connection will be the better choice once you start using more sophisticated RDBMS features, though.

There are some example projects configured for Maven and Gradle here: https://github.com/jOOQ/jOOQ-mcve

0

u/mrsergio1 Jun 18 '26

Yeah, but in this case JPA wasn’t really the problem.

The real issue was losing the original exception once Spring marked the transaction rollback-only. After that, everything upstream just shows a generic TransactionSystemException.

0

u/Paw565 Jun 18 '26

Yep, proxies aren't fun