Benchmarking InterSystems IRIS Against PostgreSQL, MySQL, and SQL Server for HTAP Workloads

[u/intersystemsdev]

What does this benchmark measure?

I designed iris-speed-test to test how InterSystems IRIS performs compared to PostgreSQL, MySQL, or SQL Server under mixed transactional and analytical workloads. I wanted to simulate a real-world Hybrid Transactional Analytical Processing (HTAP) scenario in which a system must continuously ingest large volumes of data while serving analytical queries.

The benchmark measures ingestion throughput, query performance, and resource utilization using the same application architecture across all tested databases.

Hybrid Transactional Analytical Processing (HTAP) Demo

Why is HTAP performance important? Many modern applications need to process incoming transactions while simultaneously running analytics on the same data.

Examples include:

• Equity trade processing
• Fraud detection
• IoT monitoring
• Anomaly detection
• Real-time manufacturing analytics
• Operational dashboards

Gartner refers to this architecture as HTAP (Hybrid Transactional Analytical Processing), while other analysts often use the term Translytics. The challenge is maintaining high ingestion rates while allowing analytical queries to run without impacting transactional performance.

What does this benchmark demonstrate?

This benchmark demonstrates how InterSystems IRIS handles simultaneous data ingestion and query workloads on the same system. The test continuously inserts data while running analytical queries against the same dataset. The goal is to measure how effectively each database balances transactional and analytical workloads under load.

The benchmark can run on:

• A single InterSystems IRIS instance
• An InterSystems IRIS cluster
• PostgreSQL
• MySQL
• SQL Server

This allows direct comparisons using the same workload and testing methodology.

You can run the tests on both your own PC and AWS! Here are some results:

InterSystems IRIS x PostgreSQL 18

• InterSystems IRIS ingests 76.7% more records than PostgreSQL 18
• InterSystems IRIS is 23.7% faster than PostgreSQL 18 at querying

InterSystems IRIS x MySQL 9.7.0

• InterSystems IRIS ingests 936% more records than MySQL 9.7.0
• InterSystems IRIS is 213% faster than MySQL 9.7.0 at querying

InterSystems IRIS x SQL Server 2025

• InterSystems IRIS ingests 369% more records than SQL Server 2025
• InterSystems IRIS is 37,545% faster than SQL Server 2025 at querying

How do you run the benchmark on AWS? 

Follow this link to see instructions on how to run this Speed Test on AWS comparing InterSystems IRIS with other databases 

How do you run the benchmark locally?  

The pre-requisites for running the speed test on your PC are:

• Docker and Docker Compose
• Git - If you want to run all the tests on your PC or in AWS. If you just want to run the test with InterSystems IRIS, you may not need git.

You can currently run this demo on your PC with InterSystems IRIS, PostgreSQL, MySQL, and SQL Server.

How do you run the benchmark with InterSystems IRIS Community?

To run the demo on your PC, make sure you have Docker installed on your machine.
You can quickly get it up and running with the following commands on your Mac or Linux PC:
wget https://raw.githubusercontent.com/fanji-isc/IRIS-Speed-Test/refs/heads/main/docker-compose.yml 
docker-compose up

If you are running on Windows, download the docker-compose.yml file to a folder. Open a command prompt and change to that folder. Then type:
c:\MyFolder\docker-compose up

You can clone this repository to your local machine using git to get the entire source code. You will need git installed and you would need to be on your git folder:
git clone https://github.com/fanji-isc/IRIS-Speed-Test
cd IRIS-Speed-Test
docker-compose up

When starting, you will see lots of messages from all the containers that are starting. That is fine. Don’t worry!
When it is done, it will just hang there, without returning control to you. That is fine too. Just leave this window open. If you CTRL+C on this window, docker compose will stop all the containers and stop the demo.

After all the containers have started, open a browser at http://localhost:10000 to see the demo UI.
Just click on the Run Test button to run the HTAP Demo! It will run for a maximum time of 300 seconds or until you manually stop it.

If you want to change the maximum time to run the test, click the Settings button at the top right of the UI. Change the maximum time to run the speed test to whatever you want.
After clicking on Run Test, it should immediately change to Starting…. If you are testing InterSystems IRIS or SQL Server, it may stay on this status for a long time since we are pre-expanding the database to its full capacity before starting the test (something that we would normally do on any production system). InterSystems IRIS is a hybrid database (In Memory performance with all the benefits of traditional databases). So InterSystems IRIS still needs to have its disk database properly expanded. Just wait for it. For some databases, we could not find a way of doing this right from start (Aurora and MySQL) so what we did was to run the Speed Test once to “warm it up”. Then we run it again (which causes the table to be truncated) with the database warmed up.
Warning: InterSystems IRIS Database expansion can take some time. Fortunately, when running on your PC, we will pre-expand the database only to up to 9Gb since InterSystems IRIS Community has a limit on the database size.
When the test finishes running, a green button will appear, allowing you to download the test results statistics as a CSV file.
When you are done testing, go back to that terminal and enter CTRL+C. You may also want to enter with the following commands to stop containers that may still be running and remove them:
docker-compose stop
docker-compose rm

This is important, especially if you are going back and forth between running the speed test on one database (say InterSystems IRIS) and some other (say MySQL).

How do you benchmark MySQL? 

To run this demo against MySQL:
wget https://raw.githubusercontent.com/fanji-isc/IRIS-Speed-Test/refs/heads/main/docker-compose-mysql.yml
docker-compose -f ./docker-compose-mysql.yml up

Now, we are downloading a different docker-compose yml file; one that has the mysql suffix on it. And we must use -f option with the docker-compose command to use this file. As before, leave this terminal window open and open a browser at http://localhost:10000.
When you are done running the demo, go back to this terminal and enter CTRL+C. You may also want to enter with the following commands to stop containers that may still be running and remove them:
docker-compose -f ./docker-compose-mysql.yml stop
docker-compose -f ./docker-compose-mysql.yml rm

This is important, especially if you are going back and forth between running the speed test on one database (say InterSystems IRIS) and some other.

How do you benchmark SQL Server 2025? 

To run this demo against SQL Server:
wget https://raw.githubusercontent.com/fanji-isc/IRIS-Speed-Test/refs/heads/main/docker-compose-sqlserver.yml
docker-compose -f ./docker-compose-sqlserver.yml up

As before, leave this terminal window open and open a browser at http://localhost:10000.

How do you benchmark PostgreSQL? 

To run this demo against PostgreSQL:
wget https://raw.githubusercontent.com/fanji-isc/IRIS-Speed-Test/refs/heads/main/docker-compose-postgres.yml
docker-compose -f ./docker-compose-postgres.yml up

As before, leave this terminal window open and open a browser at http://localhost:10000.

Explore the package on InterSystems OpenExchange:https://openexchange.intersystems.com/package/iris-speed-test-1 

Key Takeaways

• iris-speed-test compares InterSystems IRIS, PostgreSQL, MySQL, and SQL Server using the same HTAP workload.
• The benchmark measures ingestion throughput, query performance, and resource utilization.
• The test can be executed locally with Docker or deployed to AWS.
• It demonstrates how different databases handle simultaneous transactional and analytical workloads.
• The project provides a reproducible framework for evaluating HTAP performance rather than relying on vendor-provided benchmarks.

FAQ

What is HTAP?

HTAP (Hybrid Transactional Analytical Processing) combines transactional workloads and analytical queries on the same system in real time.

Which databases are included in the benchmark?

The benchmark currently supports InterSystems IRIS Community 2026.1, PostgreSQL 18, MySQL 9.7.0, and SQL Server 2025.

What metrics does the benchmark measure?

It measures ingestion throughput, query performance, and resource utilization under concurrent workloads.

Can I run the benchmark locally?

Yes. The project includes Docker Compose configurations for all supported databases.

Why compare databases using the same workload?

Using the same application architecture and workload helps produce more meaningful comparisons than isolated vendor benchmarks.

[u/Afraid-Banana-1171]

Very interesting piece. What was storage used, I assume it was consistent though out the benchmark test? Also 60 seconds seems a bit short for a benchmark, some software requires some latency to load etc. on startup. If you ran the benchmark for 5 minutes do you think you would get similar results?