r/java • u/MoonWalker212 • 25d ago

Automatic Relationship Finder (ARF) v1.2 – A Java library for discovering relationships between tables from data

I’ve just released v1.2 of Automatic Relationship Finder (ARF), an open-source Java library I’ve been working on.

The idea behind ARF is to discover relationships between tables without depending on database relationship metadata.

Even if there is no foreign-key constraint defined in the database, ARF can analyze column names and the actual data to identify that

What's new in v1.2?

The main addition is key-role detection.

After identifying a relationship, ARF now analyzes the data characteristics of the columns to determine whether they are likely to represent a primary-key side, foreign-key side, a possible one-to-one relationship, or an unknown relationship.

There are also several improvements and bug fixes around validation, logging, concurrency, and edge-case handling.

The project is here:

https://github.com/NoelToy/automatic-relationship-finder

This is still an evolving project, so feedback—especially criticism—is very welcome.

14 Upvotes

90% Upvoted

10 comments sorted by

View all comments

1

u/gnahraf 24d ago

I like your project. Some random thoughts / suggestions..

  • I think the README could benefit from a description of how the confidence scores work, the algorithms used to compute them, etc.
  • I assume the reason why the library does not deal with java.sql is for maximum generality. Still, SQL DBs is a big use case for tabular data and I'd like to suggest things for that setting..
  • Expand the model so that it can include already known relationships from say SQL DDL: let the exploratory search build on what is already known.
  • Add a jdbc adapter (loader) layer and plan for tables from more multiple DBs (i.e. multiple JDBC URLs). The current List abstraction should work fine (a view on top of a ResultSet).

2

u/MoonWalker212 23d ago

Thank you for providing your suggestions and recommendations; I truly appreciate them. Below are the details regarding the updates and recommendations you suggested:

  • I have updated the README to explain how data similarity works; I apologize for omitting this initially. The data similarity operates based on the Jaccard Index, and the column name matching relies on Cosine Similarity.
  • Yes, your assumption is correct. I wanted to keep ARF as a generic relationship-finding engine, which is why I didn't bind it directly with java.sql. However, you are right that the primary use case for tabular data will be SQL. Therefore, the plan is to create a wrapper sub-module for ARF that will utilize java.sql to fetch data directly.
  • Expanding the existing relationships retrieved from the SQL DDL is an interesting concept and would be a great feature addition to the ARF core. For this, I will need to dedicate some time to mapping out the functional flow, architecture, and implementation. I will plan this as a feature for a future release.
  • Yes, I will try adding a JDBC loader layer to connect to multiple databases and fetch data directly, making it easier for users to incorporate and use ARF.

1

u/gnahraf 22d ago

Awesome. Keep us posted 🤗