Skip to content

Add InterSystems IRIS Document Store integration#485

Open
pedrohenrique-sc wants to merge 1 commit into
deepset-ai:mainfrom
s-c-ai:feat/intersystems-iris-document-store
Open

Add InterSystems IRIS Document Store integration#485
pedrohenrique-sc wants to merge 1 commit into
deepset-ai:mainfrom
s-c-ai:feat/intersystems-iris-document-store

Conversation

@pedrohenrique-sc
Copy link
Copy Markdown

This PR adds InterSystems IRIS to the Haystack integrations

InterSystems IRIS is an enterprise-grade, high-performance multimodel data platform that features a native VECTOR SQL type and VECTOR_COSINE similarity function. It is widely used in healthcare, finance, and logistics for handling massive, mission-critical workloads.

This integration includes the following components for Haystack 2.x:

  • IRISDocumentStore: A production-ready DocumentStore backed by InterSystems IRIS.
  • IRISEmbeddingRetriever: Retrieves documents from the IRISDocumentStore using vector embedding similarity.
  • IRISBm25Retriever: Retrieves documents from the IRISDocumentStore using Okapi BM25 keyword-based search.

Use case:
Enterprise data architectures often require keeping generative AI workloads close to the transactional or legacy data without duplicating it into standalone vector databases.

By adding this integration, developers can:

  • Use InterSystems IRIS as a unified database for both relational metadata and vector embeddings.
  • Build advanced Hybrid Search RAG pipelines by combining IRISEmbeddingRetriever (semantic search) and IRISBm25Retriever (lexical search) to improve retrieval accuracy.
  • Deploy secure and scalable AI agents leveraging existing InterSystems infrastructure.

Documentation:
Full documentation and usage examples can be found here: InterSystems Iris DocumentStore

@pedrohenrique-sc pedrohenrique-sc requested a review from a team as a code owner May 22, 2026 17:38
@kacperlukawski kacperlukawski self-requested a review May 25, 2026 15:13
- [License](#license)

## Overview
An integration of **InterSystems IRIS** database with [Haystack 2](https://haystack.deepset.ai/) by deepset. In IRIS, the native `VECTOR(DOUBLE, N)` type is used for storing document embeddings, and the `VECTOR_COSINE` function enables high-performance dense retrievals using SIMD operations.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it's planned to be published on Haystack's website, I don't think we need a link to Haystack2, although our users would appreciate a link to InterSystems IRIS.

Comment on lines +33 to +37
In addition to the IRISDocumentStore, the library includes the following Haystack components which can be used in a pipeline:

- IRISEmbeddingRetriever - A component used to query the vector store and find semantically related Documents. It uses VECTOR_COSINE natively in the database.

- IRISBm25Retriever - A keyword-based retriever that implements Okapi BM25 over the stored documents.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In addition to the IRISDocumentStore, the library includes the following Haystack components which can be used in a pipeline:
- IRISEmbeddingRetriever - A component used to query the vector store and find semantically related Documents. It uses VECTOR_COSINE natively in the database.
- IRISBm25Retriever - A keyword-based retriever that implements Okapi BM25 over the stored documents.
In addition to the `IRISDocumentStore`, the library includes the following Haystack components which can be used in a pipeline:
- `IRISEmbeddingRetriever` - A component used to query the vector store and find semantically related Documents. It uses `VECTOR_COSINE` natively in the database.
- `IRISBm25Retriever` - A keyword-based retriever that implements Okapi BM25 over the stored documents.


Note: For the examples below, you will also need an embedder like sentence-transformers.

**Requires:** Python 3.10+ (Recommended/Tested on 3.12) and a running InterSystems IRIS instance.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would advice running it against all the supported python versions.


## Usage

Once installed, you can start using ``IRISDocumentStore`` as any other document stores that support embeddings.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Once installed, you can start using ``IRISDocumentStore`` as any other document stores that support embeddings.
Once installed, you can start using `IRISDocumentStore` as any other document stores that support embeddings.


Quick Start

Create a ```.env``` file using ```.env.example``` template and import the default config credentials for IntersystemsIris.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Create a ```.env``` file using ```.env.example``` template and import the default config credentials for IntersystemsIris.
Create a `.env` file using `.env.example` template and import the default config credentials for IntersystemsIris.

Wouldn't that create a block of code, instead of inline one?

Comment on lines +185 to +190
This integration introduces:

- `IRISDocumentStore` A DocumentStore backed by InterSystems IRIS
- `IRISEmbeddingRetriever`Retrieve documents from `IRISDocumentStore` by embedding similarity.

- `IRISBm25Retriever ` Retrieve documents from `IRISDocumentStore` using Okapi BM25.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This integration introduces:
- `IRISDocumentStore` A DocumentStore backed by InterSystems IRIS
- `IRISEmbeddingRetriever`Retrieve documents from `IRISDocumentStore` by embedding similarity.
- `IRISBm25Retriever ` Retrieve documents from `IRISDocumentStore` using Okapi BM25.
This integration introduces:
- `IRISDocumentStore` A DocumentStore backed by InterSystems IRIS
- `IRISEmbeddingRetriever`Retrieve documents from `IRISDocumentStore` by embedding similarity.
- `IRISBm25Retriever ` Retrieve documents from `IRISDocumentStore` using Okapi BM25.

IRIS_PASSWORD="SYS"
```
### Example (RAG)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please describe the goal of this example? I would also advice splitting it into indexing and search.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants