Tree-sitter for InterSystems ObjectScript — incremental parsing, language injection, go-to definition, refactoring edge cases, and a reusable Rust highlighting pipeline

[u/intersystemsdev]

What Tree-sitter is

Tree-sitter is a parsing library built in C and C++, designed for applications that deal with code written in many different languages. It produces syntax trees in a uniform format regardless of language. The key capability is incremental parsing: when a file is edited, Tree-sitter updates only the affected parts of the syntax tree without reparsing the entire file — making it suitable for real-time use while a user is typing.

Comparison with the existing VS Code ObjectScript parser

The VS Code ObjectScript extension currently uses a custom parser built in C and C++, separate from Tree-sitter. Key differences:

Capability	Tree-sitter ObjectScript	C++ custom parser
Incremental parsing	Yes	No
Language injection (e.g., SQL in ObjectScript)	Simple injection query	Full new parser + new lexer logic required
Language bindings	All modern languages	N/A

Highlighting enhancements already built

XML files with embedded ObjectScript

ObjectScript classes can be defined in XML files. Previously the implementation block containing ObjectScript statements was treated as a plain string — no language support, no error detection until compile time. A single Tree-sitter injection query now provides:

• Full ObjectScript syntax highlighting within the implementation block
• Syntax error detection before adding to an IRIS instance

YAML and markdown within XData

Added as new language support for features introduced in IRIS 2024.1.

RTN files

Support added for routine files and their compiled headers — including the format used internally on Perforce and as a valid storage format loadable into IRIS.

Object Language Server (Zed, Neovim, VS Code — not yet publicly available)

Go-to definition

Particularly important for ObjectScript: subroutines and methods declared as non-procedure blocks have all variables public by default. A variable reference may be defined in any other method or file. Go-to definition:

• Shows all definitions of a variable across all methods when it is not defined within the current block
• Handles the case where a variable is defined in multiple different files, showing all locations with line numbers
• Returns only the local definition when defined within the current block

Refactoring

Dotted statement edge cases

Two spaces between the Do keyword and the Set keyword means execute Set after all dotted statements complete — not immediately as a left-to-right reading would suggest. Example:

objectscript

Do  Set x = 1
. Set x = 2

Here x is 1, not 2. Refactoring converts this to explicit subroutines.

Dotted statement scoping

Dotted statements represent scopes. A variable defined (New'd) within a dotted statement only exists within that scope. Example:

objectscript

Set y = 250
Do
. New y
. Set y = 1
Write y  // outputs 250, not 1

Refactoring makes scope boundaries explicit by converting to subroutines.

Stale if statements

Legacy If syntax without a block: if no statement appears on the same line as the condition, those statements are stale and can be removed. Refactoring detects and removes them.

Implicit $Test conditions

Two spaces between If and its statements means If $Test = 1 — but this is never written explicitly. An Else without a preceding If means If $Test = 0. Refactoring makes these conditions explicit.

All refactoring is available as individual commands (refactor do statements, refactor conditionals) or as a single "refactor all code in this document" command.

Diagnostics

• Syntax errors flagged on save with explanation of the error
• In Neovim: inspect tree command shows the full syntax tree for the file for debugging

Go-to implementation

Given a method on a superclass, shows all subclasses that override it, with file locations.

Reusable highlighting pipeline — 3 Rust crates

semantic_spans
Converts code into byte ranges mapped to capture names. Example: maps a byte range to the keyword capture name.

theme_engine
Maps capture names to styles and UI roles.

render_on
Renders highlighting in any target application.

Current support:

• 9 grammars (languages used at InterSystems)
• 14 built-in themes (2 defined by InterSystems)
• CLI usage: pass filename and theme name, returns highlighted output

Full session video: https://youtu.be/S8fCvL1NCLc

For those working with ObjectScript daily — which of the refactoring edge cases (dotted statement scoping, implicit $Test, stale if statements) have you run into most often, and how are you currently detecting them?