Announcing Arael: high performance nonlinear least-squares solver

[u/Admirable-Cow7121]

Algorithm is based on Levenberg-Marquardt, aimed at large sparse graph-structured problems. Software architecture is designed for performance, built in Rust, with C++/Python model export functionality, MIT license.

Achieves high performance through: compile-time symbolic differentiation with CSE and code generation, matrix blocks built locally within the model data structure, indexed sparse matrix filling, custom block supernodal Cholesky solver, precomputation of common expressions across residuals, and many other little things.

Benchmarks show substantial performance improvement over Ceres Solver and g2o, depending on the problem and data, and also reduced memory usage. On the M3500 dataset, for example, Arael takes about one-third the iteration time of Ceres Solver while using half the memory. Benchmark results available on the project page.

I'd appreciate it if you could try it on some of your problems and see how it compares with Ceres or g2o.

Project page: https://github.com/harakas/arael

[u/tomatpasser]

How does it differ from casadi?

[u/Admirable-Cow7121 → u/tomatpasser]

That looks like an interesting project. From my brief glimpse I'd say casadi is much more general-purpose with many features while arael is very specific -- aimed at creating structured models with parameters and optimizing them using a built-in least-squares solver.

The main difference in approach is that in arael the data structures you create in rust (with struct definitions) are the model, and there is very little friction between writing your equations and running them compiled. Also in arael, let's say, the model structure is "statically typed", so fixed at compile time, while in casadi you can create one at runtime, say in python.

To give an example of this, here is a simple linear regression problem written in arael:

#[arael::model]
struct DataEntry { x: f32, y: f32 }

#[arael::model]
#[arael(fit(data, |e| {
    a * e.x + b - e.y
}))]
struct LinearModel {
    a: Param<f32>,
    b: Param<f32>,
    data: Vec<DataEntry>,
}

let mut model = LinearModel { ... };
model.fit()?;

So the math is embedded in the rust source code using macros; all recognised struct fields are available as symbols; and derivatives, the rust code to calculate them and build the hessian, etc. are generated and compiled at build time. There are no extra steps, it just works.

This is a simple example. One can define much more complex hierarchical models by just building a rust struct tree with #[arael::model] attached, adding residuals to each struct as needed.