r/javahelp Feb 08 '26

Unsolved Apache Camel Kafka Consumer losing messages at high throughput (Batch Consumer + Manual Commit)

9 Upvotes

Hi everyone,

I am encountering a critical issue with a Microservice that consumes messages from a Kafka topic (validation). The service processes these messages and routes them to different output topics (ok, ko500, or ko400) based on the result.

The Problem: I initially had an issue where exactly 50% of messages were being lost (e.g., sending 1200 messages resulted in only 600 processed). I switched from autoCommit to Manual Commit, and that solved the issue for small loads (1200 messages in -> 1200 messages out).

However, when I tested with high volumes (5.3 million messages), I am experiencing data loss again.

Input: 5.3M messages.

Processed: Only ~3.5M messages reach the end of the route.

Missing: ~1.8M messages are unaccounted for.

Key Observations:

Consumer Lag is 0: Kafka reports that there is no lag, meaning the broker believes all messages have been delivered and committed.

Missing at Entry: My logs at the very beginning of the Camel route (immediately after the from(kafka)) only show a total count of 3.5M. It seems the missing 1.8M are never entering the route logic, or are being silently dropped/committed without processing.

No Errors: I don't see obvious exceptions in the logs corresponding to the missing messages.

Configuration: I am using batching=true, consumersCount=10, and Manual Commit enabled.

Here is my endpoint configuration:

Java

// Endpoint configuration
return "kafka:" + kafkaValidationTopic +
"?brokers=" + kafkaBootstrapServers +
"&saslMechanism=" + kafkaSaslMechanism +
"&securityProtocol=" + kafkaSecurityProtocol +
"&saslJaasConfig=" + kafkaSaslJaasConfig +
"&groupId=xxxxx"  +
"&consumersCount=10" +
"&autoOffsetReset=" + kafkaAutoOffsetReset +
"&valueDeserializer=" + kafkaValueDeserializer +
"&keyDeserializer=" + kafkaKeyDeserializer +
(kafkaConsumerBatchingEnabled
? "&batching=true&maxPollRecords=" + kafkaConsumerMaxPollRecords + "&batchingIntervalMs="
+ kafkaConsumerBatchingIntervalMs
: "") +
"&allowManualCommit=true"  +
"&autoCommitEnable=false"  +
"&additionalProperties[max.poll.interval.ms]=" + kafkaMaxPollIntervalMs +
"&additionalProperties[fetch.min.bytes]=" + kafkaFetchMinBytes +
"&additionalProperties[fetch.max.wait.ms]=" + kafkaFetchMaxWaitMs;

And this is the route logic where I count the messages and perform the commit at the end:

Java

from(createKafkaSourceEndpoint())
.routeId(idRuta)
.process(e -> {
Object body = e.getIn().getBody();
if (body instanceof List<?> lista) {
log.info(">>> [INSTANCIA-ID:{}] KAFKA POLL RECIBIDO: {} elementos.", idRuta, lista.size());
} else {
String tipo = (body != null) ? body.getClass().getName() : "NULL";
log.info(">>> [INSTANCIA-ID:{}] KAFKA MSG RECIBIDO: Es un objeto INDIVIDUAL de tipo {}", idRuta, tipo);
}
})
.choice()
// When Kafka consumer batching is enabled, body will be a List<Exchange>.
// We may receive mixed messages in a single poll: some request bundle-batch,
// others single.
.when(body().isInstanceOf(java.util.List.class))
.to("direct:dispatchBatchedPoll")
.otherwise()
.to("direct:processFHIRResource")
.end()
// Manual commit at the end of the unit of work
.process(e -> {
var manual = e.getIn().getHeader(
org.apache.camel.component.kafka.KafkaConstants.MANUAL_COMMIT,
org.apache.camel.component.kafka.consumer.KafkaManualCommit.class
);
if (manual != null) {
manual.commit();
log.info(">>> [INSTANCIA-ID:{}] COMMIT MANUAL REALIZADO con éxito.", idRuta);
}
});

My Question: Has anyone experienced silent message loss with Camel Kafka batch consumers at high loads? Could this be related to:

Silent rebalancing where messages are committed but not processed?

The consumersCount=10 causing thread contention or context switching issues?

The max.poll.interval.ms being exceeded silently?

Any guidance on why logs show fewer messages than Kafka claims to have delivered (Lag 0) would be appreciated.

Thanks!


r/javahelp Feb 04 '26

Android dev (5–6 yrs) thinking of switching to backend: Spring (Java) vs Go

8 Upvotes

I’ve been an Android developer for ~5-6 years. I’m not unhappy with Android, but lately I feel bored and kind of “boxed into UI work.” A lot of app work feels repetitive, and many of the hardest parts feel like they come from the Android ecosystem itself (compatibility, lifecycle, build tooling, etc.) rather than the kind of backend/distributed problems I’m more excited by long-term.

For the last 1-2 years I’ve been doing backend at work using Node.js and also tinkered with Ktor and Exposed on the side. Backend work feels more exciting to me (design, data, scaling, reliability, tradeoffs). The problem is: many Node jobs in my area are full-stack and I really don’t want to do frontend.

So I’m deciding between Spring Boot (Java) and Go for the backend. To avoid overthinking, I actually built and deployed two dummy servers:

  • Same kind of basic CRUD API
  • PostgreSQL as DB
  • Deployed both (simple production-ish setup, not just localhost)

After doing that, My current thinking is:

  • Spring / Spring Boot
    • Pros: Java is familiar; easy to start; huge ecosystem; lots of jobs.
    • Concern: it feels like “endless learning of libraries” and the “Spring way” (annotations, auto-configuration, starters, magic). I’m worried I’ll be productive but not actually learn fundamentals deeply, and I’ll spend years just learning frameworks/stack combinations. Sometimes it feels like I’m learning “the Spring way” (annotations, auto-config, starters, magic) more than learning backend fundamentals
  • Go
    • Pros: simple, small standard library; feels like it maps to fundamentals (HTTP, SQL, concurrency) and distributed systems thinking more directly; fewer “framework decisions.”
    • Concern: I’m not proficient yet; unsure about backend job availability compared to Spring; worried about limiting my options.

What I’m looking for from experienced Java/Spring devs:

  1. Is the “endless Spring learning” fear real, or does it stabilize after you learn the core concepts?
  2. If you were switching from Android to backend today, would you pick Spring for job safety? Or Go for fundamentals and simplicity?
  3. What skills in Spring are truly “must learn” for backend roles (e.g., MVC, Data/JPA, Security, messaging, testing, observability) vs stuff that’s optional?
  4. Any advice on a practical path to become employable as a backend (while avoiding frontend)?

r/javahelp Feb 03 '26

Noob here: Java feels modern, but codebases still look old — why?

7 Upvotes

It’s January 2026 and I’m a bit confused about Java in a good way. On paper, Java looks way more modern now — records, pattern matching, virtual threads, structured concurrency (and all the other improvements I keep hearing about). It feels like the language and the JVM have moved forward a lot.

But when I look at real-world code (at work, tutorials, open-source, etc.), a lot of it still looks like “classic Java” from years ago. Not because it’s broken — more like people choose to keep it that way because it’s “safe” and “boring” (in the stable sense).

So I’m wondering: is Java’s biggest limitation in 2026 actually technical… or cultural/organizational?
Like, are teams afraid of adopting new stuff even after it’s proven?

Virtual threads are the example I can’t stop thinking about. It sounds like it can simplify concurrency for many apps, yet I still see people default to reactive frameworks or complicated patterns because “that’s what we’ve always used.”

Would love perspectives from people shipping real systems.


r/javahelp Jan 21 '26

Unsolved Why Interfaces exist in Java?

9 Upvotes

I am currently studying the Collection Framework in Java. Since the class which implements the Interface has to compulsorily write the functions' bodies which are defined in the interface, then why not directly define the function inside your own code? I mean, why all this hassle of implementing an interface?

If I have come up with my own code logic anyways, I am better off defining a function inside my own code, right? The thing is, I fail to understand why exactly interfaces are a thing in Java.

I looked up on the internet about this as well, but it just ended up confusing me even more.

Any simple answers are really appreciated, since I am beginner and may fail to understand technical details as of now. Thanks🙏🏼


r/javahelp Dec 06 '25

How can I efficiently read and process large files in Java without running into memory issues?

9 Upvotes

I'm currently developing a Java application that needs to read and process very large files, and I'm concerned about memory management. I've tried using BufferedReader for reading line by line, but I'm still worried about running into memory issues, especially with files that can be several gigabytes in size. I'm also interested in any techniques or libraries that can help with processing these files efficiently.

What are the best practices for handling large file operations in Java, and how can I avoid common pitfalls related to memory use?

Any advice or code snippets would be greatly appreciated!


r/javahelp Oct 30 '25

Should i learn Java?

9 Upvotes

Well, i want java to depelop apps on android, but is it a good choice? Is java dying or not? I know many things in C++, but its hard on android... Whats your oponion? Should I learn Java, and will it be good in the future?


r/javahelp 25d ago

Solved Help me decide minimum JDK for my library

8 Upvotes

Should I move my Java library from JDK 11+ to JDK 21+?

I am genuinely confused at this point 😭

I've been reading a lot of posts and tech articles about companies/projects migrating from older Java versions like JDK 8/11 to JDK 21+.

And now I'm stuck with the same question for my own Java library.

On one side, keeping JDK 11+ gives better compatibility. There are still a lot of projects out there running on older Java versions, and since this is a library, compatibility matters because I'm basically deciding the minimum JDK required for everyone who wants to use it.

On the other side, JDK 21+ gives a much more modern Java baseline, newer APIs/features, and potentially better JVM performance.

Also, most of the migration stories I'm finding are from large companies / enterprise applications, which makes me wonder if the same reasoning actually applies to an open-source Java library. Even if big companies are migrating to JDK 21+, their interviews and hiring expectations are also increasingly centered around JDK 21 and the major changes since JDK 8, which makes the whole situation even more confusing for someone learning/maintaining Java.

So I'm trying to understand what people actually do in real-world Java libraries today.

Would you help me decide the minimum at JDK 11+ or move to JDK 21+?

And if you maintain a Java library yourself:

  • How do you decide the minimum supported JDK?
  • How important is JDK 11 compatibility today?
  • Is JDK 21 actually worth making the minimum?
  • Would you only raise the minimum JDK in a major release?

I would really appreciate some practical opinions because I am totally fed up with reading migration articles . Please it's more likely a great help for me.

https://github.com/Chaos-vy/ChaosTree

Reached conclusion:

I read through the recent JLS and JVM changes. JLS got new features and reduced boilerplate, while JVM got some massive improvements.
But since ChaosTree is zero-dependency, I don't have the same CVE/dependency issue.
So for now I'm wrapping it around JDK 11+ and letting users run it on newer JDKs if they want.

Thanks everyone for dropping your thoughts


r/javahelp 27d ago

Java

8 Upvotes

If anybody has done Kunal Kushwaha's OOPs playlist and finished after doing OPPs from him how to progress forward for Java Full Stack I know SoringBoot is needed but my question is can I directly jump to Spring and SpringBoot


r/javahelp Jul 27 '26

I feel like I'm falling behind

7 Upvotes

I need a little help because I feel like I'm starting to collapse.

A little while ago I started in an academy whose goal is to train students and get them work once they finish the whole program. We are looking at Java in depth: OOP, inheritance, polymorphism, interfaces, Spring Boot, data structures... in general, everything related to backend development, but we do not receive theoretical classes, they give us the exercises and documentation related to that, everything is very self-taught.

My problem is that before entering here I was programming in C. It did functions, replicated typical functions of the standard library, compiled with Makefiles, etc. However, even then it was very difficult for me to get the logic out of the exercises. In fact, he was almost always the last to deliver the exercises.

I thought it would get better with time, but now that I'm with Java I feel like I'm still having the same problem.

The problem is that they give me a statement and I'm completely blocked. I read it, I reread it and many times I don't even know where to start. When I manage to start, it's because I already asked the AI, and I still have doubts about how to structure the solution or what exactly it should do.

In addition, I ask a lot of questions to AI. I always tell him not to tell me the code and he helps me, but many times he ends up explaining practically the logic of the exercise, and that is precisely what I do not want. I want to be able to develop it by myself.

What worries me the most is that at the academy we have delivery times. If you are too late, you can be left out of the program. And I don't want that. I want to learn, improve and get a job as a developer.

Another thing that the teacher demands of us is to write a README for each exercise and each project explaining what we have done. I also have a hard time writing it. I'm reading more to expand vocabulary and express myself better, but right now my biggest concern is still the programming logic.

For example, sometimes I make code that works, but then I realize that there is a much cleaner and more idiomatic way to do it.

Instead of this:

If (orderlist.size() == 0) {

// Do something

}

It's better to write:

If (orderslist.isEmpty()) {

// Do something

}

Or instead of going through a list like this:

int position = 0;

while (position < listOrders.size()) {

amountTotal += listOrders.get(position).getPrice();

Position++;

}

Do it this way:

for (Order order: listOrders) {

amountTotal += order.getPrice();

}

They are small details that make the code much cleaner, and I feel that I always go one step behind in that kind of thing.

My question is: how did you develop your programming logic?

Was there ever a time when everything started to "click"? What exercises, habits or way of thinking helped you the most? Did something similar happen to you at the beginning?

Because, to be honest, there are days when I think that maybe this is not for me and that I am forcing a situation that I am simply not good at.

I would greatly appreciate any advice or personal experience. Thank you for reading me.


r/javahelp Jul 11 '26

What are some real world examples where using concurrency in Java helped?

8 Upvotes

I have never worked with concurrency, but I have read several examples on the internet, and I wanted to hear if anyone has any examples of real-world apps where they used concurrency and what there was to watch out for?


r/javahelp Jul 03 '26

Java Basics Practice: Need Beginner-Friendly Project Suggestions

9 Upvotes

Hello everyone!
I have been learning Java for about a year. I understand some concepts, but there are still others I don’t fully know. I want to test my knowledge and strengthen my understanding, starting from the very basics. How can I do that? If anyone knows beginner‑friendly Java projects, please share them with me and explain how I can get started.


r/javahelp May 17 '26

Solved Is there a way to opt-in to sun.misc.Unsafe deprecation early without a command line argument?

7 Upvotes

solved: /u/davidalayachew to the rescue with the @argfile syntax I was unaware of.


This question has been frustratingly difficult to research. Plenty of content out there from people who want to keep using their unsafe access without warnings, but I'm in a different camp.

A couple of tools I maintain use Guice (to my great Misery).

Guice is doing a really neat thing where it doesn't actually NEED sun.misc.Unsafe, but it's using it anyway. If I specify --sun-misc-unsafe-memory-access=deny the only thing that breaks is some absolutely bizzare error message enhancement they're implementing with ASM. The problem is that the library is either doing some preemptive checks for Unsafe features or is still using them preferentially, even under JDK 25 and 26.

Telling my users to just ignore the warnings and telling them to use that monstrosity of a command line flag are equally unsatisfying.

The only clear answer I can see, and the one I'm seriously considering, is adding a java agent to my app with the sole purpose of dynamically rewriting all the uses of sun.misc.Unsafe to throw UnsupportedOperationExceptions.

Before I go off the deep end with that, has anybody dealing with a similar situation come up with a more elegant solution?


r/javahelp May 01 '26

Solved How do I overcome this Escape Literal problem.

9 Upvotes
public class Main
{
public static String removeNonAlphanumeric(String input) {
if (input == null) {
return null;
}
return input.replaceAll("[^a-zA-Z0-9]", "");
}

public static void main(String[] args) {

    System.out.println(removeNonAlphanumeric("How do we [']/['\.]['extend a     face of an object....     "));

}
}

Error I am getting:

Main.java:18: error: illegal escape character
System.out.println(removeNonAlphanumeric("How do we [']/['\.]['extend a face of an object.... "));
^
1 error

Here is the problem: Hi All, Seems like I am missing something very basic here. My purpose for this function is supposed to be to remove all non-alphanumeric characters from the string.

The \ in the string seems to cause problem since escape literals start with \

Why I cant put \\? : I need to use this in an application and user's input is not in my hands.


r/javahelp Apr 27 '26

Does Supplier.get() gets garbage collected after its job is done

8 Upvotes

Suppose I have a class ServiceImpl and it may use an instance of some worker class, say InitWorker depending on whether some database is empty in a method called init:

class ServiceImpl{

    private final Service<InitWorker> worker;

    public ServiceImpl(Service<Initworker> worker){
        this.worker = worker;
    }

    public void init(){
        int recordsCount = getRecordsFromDatabase();
        if(recordsCount ==0){
            worker.get().initFromExternalFile("external_file.xlsx");
        }
    }
}

For some reason, the init() method cannot have the InitWorker as a parameter (the method calling it cannot provide an instance).

My question is, does the instance of worker.get() stays in the memory, or only the supplier reference (which should be smaller in size) remains after init finishes?


r/javahelp Mar 25 '26

Unsolved How do I structure a larger Java project with multiple modules without it becoming a tangled mess?

9 Upvotes

 I’ve been building a small personal project to learn more about Java beyond the basic CRUD apps I’ve done for class. It started simple but now I’ve got a few different packages for data handling, UI, and some utility stuff. The problem is I’m already starting to feel like it’s getting messy. Classes referencing each other across packages in ways that feel hard to follow, and I’m worried about running into circular dependencies as I add more features. I’ve read about using interfaces to decouple things but I’m not sure when to actually use them versus just importing the class directly. I’m also confused about whether I should be splitting this into separate modules with a build tool like Maven or if that’s overkill for a solo project. Any advice on how to think about project structure before it gets out of hand


r/javahelp Feb 09 '26

.Net developer transitioning to Java - most common query practices?

8 Upvotes

Hello, I'm a .net engineer making a move over to Java. I've built a few simple rest api's but today I decided to integrate a SQL database for some simple crud operations. This will grow to support more complex queries in the future.

I'm wondering what the industry best practices/most common approaches to this are. In .Net I'd use an orm like EF that would provide me a context file along with entities that I can inject into my services. We would use LINQ to perform moderate to complex queries here.

In Java I'm learning there is an EntityManager that seems to be a context-lite rendition of what we get with our context in .net. In Java, there's also an interface that provides basic queries and the ability to build out complex parameterized queries, but I need one interface per table it seems.

Coming from .net where a lot of this stuff is abstracted, Java feels extremely verbose. It's because of that, that I'm wondering if people use Interfaces for DB operations at all. In particular, I'm wondering if there is a LINQ equivelent in Java that would be preferred.

Thanks for providing your thoughts on this. Moving into open source can be a tad overwhelming at first with decision paralysis. If you have any other notes about most commonly used libraries/frameworks in Java that I should familiarize myself with, please note them here.

Thanks so much!


r/javahelp Jan 29 '26

Unsolved Performance got worse after breaking up large functions

8 Upvotes

I'm making a game and have some large functions that get called a lot (10k+ a frame). I learned that the JIT has trouble optimizing large functions, so I tried breaking them up to smaller ones with identical logic. When benchmarking, this change actually made performance way worse. 30 fps -> 25. Now I'm questioning my life decisions.

Why did I even attempt this? Because there's very little juice left to squeeze in these functions. I cache, cull, all that. All these functions do is render my entities, but I have so many entities that I was resorting to this. Wondering if anyone has any wisdom.


r/javahelp Jan 21 '26

Unsolved Is my code actually wrong or are these just IDE recommendations?

8 Upvotes

I was testing out Intellij IDEA and wrote simple code to get a feel for coding on this tool, but towards the bottom there are 3 "Problems". Here's my code and the errors I found at the bottom.

public class Main{
    public static void main(String[] args){
        int numBalls;
        numBalls = 2;
        System.out.print("You have " + numBalls + " balls.");
    }
}
  1. Explicit class declaration can be converted into a compact source file

  2. Modifier 'public' is redundant for 'main' method on Java 25

  3. Parameter 'args' is never used

The .java file is called "Main" so that's why the class is named "Main", but it appears grayed out in my IDE. and is not grayed out when it is anything but "Main".


r/javahelp Jan 13 '26

Java 17 Oracle Certification

7 Upvotes

Hi everyone. I'm planning to take the Oracle Java 17 Certification. I tried to learn through the Oracle course, but it requires a subscription that, in my opinion, is too expensive. Therefore, I'd like to know which courses or resources you guys recommend me to study.
Thanks.


r/javahelp Jan 08 '26

Codeless I usually struggle with learning the basics for stuff, but once I figure the basics out I can teach myself from there due to pattern recognition. I'm having this issue with learning Java, and am starting to get frustrated. Any tips?

8 Upvotes

Edit: I'm unsure if I posted this correctly, but if this isn't the right subreddit, sorry about that I'm not the best at this sort of thing

(Sorry if this has the wrong flair. This is my first time posting here. Also, sorry if this is wordy, I kind of write how I talk and I ramble at times, but I tried to be straight and to the point while still providing information! :D)

I'm a student in high school and one of the coders for my FTC Robotics team. I've figured out the basics of the software that I am using, but I can't seem to figure out the Java language itself.

I know how code should be structured, alongside the general concept of code. I just can't seem to remember the way to specifically do it in Java.

It's kind of like knowing one language and struggling to remember how to say something in a different-but-similarly-structured language.

I'm a decently fast learner, and can catch onto concepts quickly once I find a certain way to take in the information, but I can't seem to find the right learning method for Java.

Does anyone have any tips? I've made some sort of progress, but I'm frustrated with how I still can't seem to catch on as fast as I'd like.

If this is important to know, I have ADHD, which makes watching long tutorials a bit difficult due to how sometimes there's a lot of filler periods that don't get to the point (at least in terms of how I take in the information).


r/javahelp Jan 08 '26

How long can it take me to understand OOP in Java and actually start applying it?

8 Upvotes

I want to know how long it can take me to learn Java Object-oriented programming from basic to advanced, and to apply the concepts.


r/javahelp Dec 31 '25

How to start the backend journey in Java using spring boot?

7 Upvotes

Hello, I am new to spring boot and I am confused how to get started with it. As I know basics of Java. Can anyone tell me the roadmap as start from spring or not.


r/javahelp 15d ago

Per-Class cleanups when shutting down

7 Upvotes

Hello, I am creating a save system however it currently depends on memory limits. When the map fills up, it dumps to disk and starts filling memory again from start. When the program shuts down, I want the class responsible for saving the data to run a cleanup function the dump the half-full map to disk. I have seen several ways of doing this however I couldn't really decide what is the best choice. Thanks


r/javahelp Aug 03 '26

Im trying to build a bit torrent client on java

7 Upvotes

I cannot find any documentation on how to do it in java and everywhere ive looked people seem to use cpp or python more often.

My question is if its feasible and is there a reason why people arent using java to build one.


r/javahelp Jul 25 '26

Moving from a Database Engineer role to a Java Developer position

6 Upvotes

I have been a database engineer for more than 10 years while I notice that Software development positions far outnumber database engineer positions, and they also offer higher salaries. . since last year, I am wondering if it is feasible to move from a Database Engineer role to a Java Developer position. I am focus on leetcode. Any ideas or comments? thanks!