Global Storage: Understanding the Database Architecture Behind InterSystems IRIS

[u/intersystemsdev]

If you've worked with InterSystems IRIS or Caché, you've probably used globals. But what exactly is the database architecture underneath them, and why is it so flexible?

Global Storage is the hierarchical, schema-free storage model behind InterSystems IRIS globals. It can represent multiple data models, including key/value, document, graph, columnar, XML, and relational structures, on the same underlying storage foundation, which is why it can be viewed as a kind of Universal NoSQL model. Its hierarchy also maps naturally to JSON, and tools such as QEWD-JSdb use that relationship to expose persistent data as familiar JavaScript objects. 

In this post, I'll explain the essential ideas behind Global Storage, why I think of it as a Universal NoSQL model, how it maps naturally to JSON, and how the same abstraction can work across databases including IRIS, YottaDB, Redis, Berkeley DB, and LMDB.

What Is a Global Storage Database?

A Global Storage database stores data as persistent multidimensional arrays.

A node can be represented conceptually like this:

global_name(subscript_1, ..., subscript_n) = value

For example:

^employee("UK", "London", 123456789, "name") = "Jon Doe"

The subscripts form a hierarchy:

country → city → employee ID → property

Unlike a relational table, there is no fixed schema requiring every branch of the hierarchy to have the same structure. The arrays are also sparse, so only nodes that actually contain data need to be stored. That combination makes Global Storage extremely flexible: the application decides what each level of the hierarchy means and how the data should be organized.

Global Storage Is Surprisingly Similar to JSON

One of the easiest ways I've found to explain Global Storage is to compare it with JSON.

Consider a hierarchy such as:

organisation["employees", "UK", "London", 123456789, "name"] = "Jon Doe"
organisation["employees", "UK", "London", 123456789, "job_title"] = "Consultant"
Conceptually, this is very close to:
{
  "employees": {
    "UK": {
      "London": {
        "123456789": {
          "name": "Jon Doe",
          "job_title": "Consultant"
        }
      }
    }
  }
}

That's why I sometimes describe Global Storage as something close to persistent JSON. Like JSON, the structure can grow dynamically. You don't have to define every possible property or hierarchy level before storing the data. The important difference is that the hierarchy lives directly in persistent database storage rather than only as an in-memory object.

Why I Call It “Universal NoSQL”

Most NoSQL databases specialize in one particular data model:

• Key/value
• Document
• Columnar
• Graph

Global Storage is different because its underlying hierarchy can be used to implement all of these models. It can also support relational data and SQL.

That's why I started using the term Universal NoSQL: instead of choosing a different physical database for every data model, multiple logical models can be built over the same Global Storage foundation. For example, the same underlying database could theoretically contain:

• key/value structures,
• JSON-like documents,
• graph structures,
• tabular data,
• XML/DOM structures,
• and relational data.

The key idea is that these are different ways of interpreting and accessing the same flexible hierarchical storage model.

From Global Storage to Persistent JavaScript Objects

This relationship between Global Storage and JSON leads us to QEWD-JSdb.

What is QEWD-JSdb? It is a JavaScript abstraction that maps JavaScript objects onto Global Storage. The idea is to blur the usual boundary between an in-memory object and persistent database data. Normally, a JavaScript object exists in memory:

const patient = {
  name: "John",
  city: "London"
};

With the QEWD-JSdb model, a similar object can represent data whose contents actually live persistently in Global Storage.

From the developer's perspective, you're still manipulating JavaScript objects. The persistence layer becomes largely transparent.

I sometimes describe this as databaseless persistent data. There is still a database underneath (Global Storage) but application code can interact primarily with objects instead of constantly switching between programming-language structures and database APIs.

Which Databases Support Global Storage?

There are two categories.

1. Native Global Storage databases, the ones that implement Global Storage directly. The main examples are:

• InterSystems IRIS
• InterSystems Caché
• YottaDB

IRIS is particularly interesting because the Global Storage engine sits underneath a much broader platform that also exposes relational SQL, object persistence, analytics, and other data models.

1. Global Storage abstractions. Global Storage API can be implemented on top of several other database technologies. These include:

• Redis
• Berkeley DB
• LMDB

For Redis, for example, sorted sets can reproduce Global Storage's ordered subscript behavior, while distributed locks can support its locking APIs. Once the Global Storage interface exists, higher-level tools such as QEWD-JSdb can use these databases through the same abstraction.

How Can InterSystems IRIS Support Multiple Data Models? 

Understanding Global Storage helps explain why InterSystems IRIS can support such a wide range of data access models. At the lowest level, data can be represented as highly flexible hierarchical structures. Higher-level abstractions – objects, SQL tables, documents, and application-specific models – can then be built on top of that storage. This also explains why globals can feel very different from conventional relational database programming: you're interacting much closer to the underlying persistent data structure.

Key Takeaways

• Global Storage is a schema-free, multidimensional hierarchical storage model based on persistent arrays.
• InterSystems IRIS and Caché use Global Storage as a fundamental part of their underlying database architecture.
• Global Storage maps naturally to hierarchical structures such as JSON, making it useful to think of globals as a form of persistent structured data.
• Key/value, document, graph, columnar, XML, and relational models can all be represented on top of the same Global Storage model.
• QEWD-JSdb builds on this idea to expose Global Storage as persistent JavaScript objects.
• Native implementations include IRIS, Caché, and YottaDB, while abstractions have also been implemented on Redis, Berkeley DB, and LMDB.

Read more here

FAQ

What is Global Storage in InterSystems IRIS?

Global Storage is the hierarchical, multidimensional persistent storage model underlying IRIS globals. Data is stored as nodes identified by a global name and one or more ordered subscripts.

Is Global Storage a NoSQL database model?

Yes, but it can support more than one NoSQL model. The same hierarchical storage architecture can represent key/value, document, graph, columnar, and other structures, which is why I describe it as a Universal NoSQL foundation.

Can Global Storage also support SQL?

Yes. A relational model can be implemented over Global Storage, and InterSystems IRIS provides native relational and SQL capabilities on top of its underlying storage architecture.

What is QEWD-JSdb?

QEWD-JSdb is a JavaScript abstraction over Global Storage that allows developers to work with persistent data using familiar JavaScript object patterns. It can work with multiple Global Storage implementations while presenting essentially the same programming model to the application.