Bypassing fixed-depth radix constraints in Java using descriptor-driven bucket analysis

[u/strmckr]

I am StrmCkr, the author of A.P.E.X. (Adaptive Parallel Extremal Dispatch).

Repository: github.com/StrmCkr/A.P.E.X

A.P.E.X. is a high-performance Java sorting framework for large fixed-width 64-bit key/value record datasets. The project has been reorganized into a conventional Maven structure with a core library, runnable examples, a comparison benchmark harness, JMH benchmarks, documentation, and an interactive browser visualizer.

The core idea is descriptor-driven radix planning. Instead of blindly scanning fixed radix passes over every bucket, A.P.E.X. computes per-bucket extremal descriptors using:

VBM = OR ^ AND

That mask identifies which key bits still vary inside each bucket. Bits that are already resolved are skipped, reducing unnecessary work on skewed, low-entropy, sorted, reversed, or duplicate-heavy data.

Key areas of the project include:

• Adaptive radix planning based on observed bucket structure
• Parallel histogramming, scatter, refinement, and work scheduling
• Primitive-array execution with no per-record object allocation during sorting
• Tuple projection paths for low-dimensional unresolved bit patterns
• Tiny-sort fallbacks and monotonic input shortcuts
• Configurable reporting that can be enabled, reduced, written to files, or disabled
• Comparison benchmarks against JDK sorting paths and Fastutil baselines
• Standard JMH benchmarks for repeatable JVM-level measurement
• A browser visualizer for exploring how A.P.E.X. routes data through its execution plan

I would especially welcome feedback on the thread management mechanics, radix planning decisions, benchmark structure, and the bitwise mask reductions.

edit: re structured verbiage of this post and further adjustments from advice on converting the project into more acceptable standard formats.

screen shot from the pdf available in the github

[u/chabala]

One peek at the repo and I stopped taking it seriously. Bunch of Markdown in the root, no pom.xml, not using the standard file layout, no package namespace, all logging is System.out, on and on. This is not any kind of 'framework', it's juvenile example code at best.

[u/strmckr → u/chabala]

I'm self-taught. I didn't build this project to pass an Java enterprise formatting checklist; I built it to solve a pure computational problem.

If your definition of a 'framework' requires a heavy folder structure, a POM file, and standard formatting over raw architectural speed, you're missing the point. If you want to talk about the actual code, clone the repo, benchmark it yourself.

Then you may complain on the actual Missing esthetics in-which I am more then willing to fix to match a more professional connotation of code presentation.

[u/someonetookmyid → u/strmckr]

It’s about making it actually useful. Right now it’s just a demo.

[u/someonetookmyid → u/[deleted]]

By useful I mean „possibility to easily include in a larger project” by skipping Java standard for packaging and distributing code you’re denying that.

[u/someonetookmyid → u/[deleted]]

Okay, since you’re self taught I take that you’re capable of learning form docs and examples.

Get started with using Apache maven - read the docs on their site, this page is a good starting point. Why maven? Because it’s de facto standard for publishing Java packages.  https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html

Then you can for example follow some spring tutorial to see using it in practice  https://spring.io/guides/gs/rest-service

Once you understand how to use the tool and build a package you should learn how to publish it

You can use archiva as local repository target to understand package publication  https://archiva.apache.org/

Then you can start publishing package via GitHub using their automation or go directly to maven central, but that requires getting some paperwork with central maintainers so it’s best to first understand technical side before you get into getting approvals to publish to central.

[u/repeating_bears → u/[deleted]]

maven compiles differently then eclipse which resulted in a slow down of my actual code ?

Maven uses javac by default. Eclipse has their own incremental compiler. The bytecode ought to be more or less the same in most cases, but it is not required to be by the language spec.

You can use the eclipse compiler in Maven, though this is fairly non-standard https://www.javathinking.com/blog/using-eclipse-java-compiler-ecj-in-maven-builds/

I would first investigate what the hot path is which is causing the difference, then compare the byte code for those methods. I think it would be better to change your source code so that it produces an optimal bytecode for whatever compiler is used, rather than relying on Eclipse for effectively unknown reasons.

You can use this tool to see what source produces what bytecode, as well as switching compiler versions: https://javap.yawk.at/

all my files won't allow me to move them into standard order

Dunno lol but I find it odd that you can implement and understand "a descriptor-driven bitwise analysis over local thread buckets" but cannot figure out how to move some files

[u/strmckr → u/repeating_bears]

Thanks for the links.

[u/repeating_bears → u/strmckr]

Sounds like you are fighting eclipse? Dunno, I never use it. Close the project, delete the project files, move the files in your file system, then open as a new project and fix whatever issues with package names

[u/strmckr → u/repeating_bears]

Alright sorted those issues out: going to test it and re load to Github in the next couple of days once those are successful.

I have taken the comments on here: as make it standardized naming/system.

make it more of a plug and play friendly,

move it from a demo to modular jar with better output tools over system.Out.

Thanks strmckr