Skip to content

In-Memory Persistence

In-memory persistence stores search indexes entirely in RAM using Lucene's ByteBuffersDirectory. This approach provides extremely fast search performance but comes with trade-offs around data persistence and memory usage.

For an overview of all storage options, see the persistence overview. For production deployments, consider S3 persistence or local disk storage.

When to Use In-Memory Storage

In-memory storage is perfect for development environments where you need fast iteration cycles and don't care about data persistence across restarts. It's ideal for performance testing when benchmarking search performance, as it eliminates disk I/O bottlenecks and provides pure computational performance metrics.

A common pattern is to use in-memory storage for searcher nodes in distributed deployments, where searchers download indexes from S3 into memory for ultra-fast query processing, while indexers persist data to disk. It's also useful for short-lived search tasks or batch processing jobs where indexes don't need to survive beyond the process lifetime.

Configuration Example

Basic in-memory storage configuration:

core:
  cache:
    dir: /tmp/cache

inference:
  embedding:
    e5-small:
      model: intfloat/e5-small-v2

schema:
  movies:
    store:
      local:
        memory:
    fields:
      title:
        type: text
        search:
          lexical:
            analyze: en
          semantic:
            model: e5-small
      overview:
        type: text
        search:
          lexical:
            analyze: en

For more field configuration options, see the schema mapping guide. For embedding model configuration, see the embeddings guide.

Kubernetes Deployment

When using in-memory storage in Kubernetes, ensure adequate memory allocation:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nixiesearch
spec:
  template:
    spec:
      containers:
      - name: nixiesearch
        image: nixiesearch/nixiesearch:latest
        resources:
          requests:
            memory: "2Gi"
          limits:
            memory: "4Gi"

For complete Kubernetes setup instructions, see the Kubernetes deployment guide.

Further Reading