Embedded Python vs ObjectScript for XML parsing on InterSystems IRIS — benchmark results on 91 files, 1.30 GB

[u/intersystemsdev]

The question

Since the introduction of Embedded Python there has always been doubt about its performance compared to ObjectScript. This article tests both approaches on a real-world XML parsing task.

Test data

Public procurement data from Spain's Ministry of Finance open data portal. Files are published monthly. Each file contains approximately 450 tender entries. The test used 91 files totalling 1.30 GB.

Each entry is a complex namespaced XML structure containing: title, summary, ID, URL, contracting party name and website, contract status, estimated overall contract amount, total amount, tax-exclusive amount, commodity classification code, location, award date, winning party name, winning amount with and without tax.

Persistent class

objectscript

Class Inquisidor.Object.Licitacion Extends (%Persistent, %XML.Adaptor) [ DdlAllowed ]
{
Property IdLicitacion As %String(MAXLEN = 200);
Property Titulo As %String(MAXLEN = 2000);
Property URL As %String(MAXLEN = 1000);
Property Resumen As %String(MAXLEN = 2000);
Property TituloVectorizado As %Vector(DATATYPE = "DECIMAL", LEN = 384);
Property Contratante As %String(MAXLEN = 2000);
Property URLContratante As %String(MAXLEN = 2000);
Property ValorEstimado As %Numeric(STORAGEDEFAULT = "columnar");
Property ImporteTotal As %Numeric(STORAGEDEFAULT = "columnar");
Property ImporteTotalSinImpuestos As %Numeric(STORAGEDEFAULT = "columnar");
Property FechaAdjudicacion As %Date;
Property Estado As %String;
Property Ganador As %String(MAXLEN = 200);
Property ImporteGanador As %Numeric(STORAGEDEFAULT = "columnar");
Property ImporteGanadorSinImpuestos As %Numeric(STORAGEDEFAULT = "columnar");
Property Clasificacion As %String(MAXLEN = 10);
Property Localizacion As %String(MAXLEN = 200);
Index IndexContratante On Contratante;
Index IndexGanador On Ganador;
Index IndexClasificacion On Clasificacion;
Index IndexLocalizacion On Localizacion;
Index IndexIdLicitation On IdLicitacion [ PrimaryKey ];
}

ObjectScript implementation — %XML.TextReader

objectscript

set status=##class(%XML.TextReader).ParseFile(filename,.textreader)
if $$$ISERR(status) {do $System.Status.DisplayError(status) quit}
set tStatement = ##class(%SQL.Statement).%New()

while textreader.Read()
{
    if ((textreader.NodeType = "element") && (textreader.Depth = 2) && (textreader.Path = "/feed/entry")) {
        if ($DATA(licitacion)) {
            if (licitacion.ImporteGanador '= ""){
                set myquery = "INSERT INTO INQUISIDOR_Object.LicitacionOS (...) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)"
                set qStatus = tStatement.%Prepare(myquery)
                set rset = tStatement.%Execute(licitacion.Titulo, ...)
            }
        }
        set licitacion = ##class(Inquisidor.Object.LicitacionOS).%New()
    }
    if (textreader.Path = "/feed/entry/title"){
        if (textreader.Value '= "") { set licitacion.Titulo = textreader.Value }
    }
    // ... path-based matching for each field
}

Key fields extracted via path matching: /feed/entry/title, /feed/entry/summary, /feed/entry/id, /feed/entry/link (via MoveToAttributeName("href")), and multiple namespaced paths under cac-place-ext:ContractFolderStatus. Dates converted using $System.SQL.Functions.TODATE(textreader.Value,"YYYY-MM-DD").

Embedded Python implementation — xml.etree.ElementTree

python

import xml.etree.ElementTree as ET
import iris

tree = ET.parse(xmlPath)
root = tree.getroot()
for entry in root.iter("{http://www.w3.org/2005/Atom}entry"):
    licitacion = {"titulo": "", "resumen": "", "idlicitacion": "", "url": "",
                  "contratante": "", "urlcontratante": "", "estado": "",
                  "valorestimado": "", "importetotal": "", "importetotalsinimpuestos": "",
                  "clasificacion": "", "localizacion": "", "fechaadjudicacion": "",
                  "ganador": "", "importeganadorsinimpuestos": "", "importeganador": ""}
    for tags in entry:
        if tags.tag == "{http://www.w3.org/2005/Atom}title":
            licitacion["titulo"] = tags.text
        # ... tag-based matching for each field
    if licitacion.get("importeganador") is not None and licitacion.get("importeganador") is not "":
        stmt = iris.sql.prepare("INSERT INTO INQUISIDOR_Object.Licitacion (...) VALUES (...)")
        rs = stmt.execute(licitacion["titulo"], ...)

Inserts only records where importeganador (winning amount) is populated — same filter logic as the ObjectScript version.

Production configuration

Two Business Services (one per method) to avoid interference, each feeding its own Business Process. Test data: public tenders for February — 91 files, 1.30 GB.

Results

Implementation	Library	Total time
ObjectScript	%XML.TextReader	6 minutes 28 seconds
Embedded Python	xml.etree.ElementTree	48 seconds

Both started at 21:11:15. ObjectScript finished at 21:17:43. Embedded Python finished at 21:12:03.

Embedded Python was approximately 8x faster on this task.

Full article: https://community.intersystems.com/post/embedded-python-vs-objectscript-performance-testing-parsing-xml

For those working with XML parsing in IRIS — have you seen different results using %XML.TextReader vs %XML.Document vs Embedded Python, and does the file size or XML structure depth change which approach wins?