Spill to Disk

How to avoid out of memory with Spill to Disk

When queries require a huge amount of memory to be processed, Indexima can offload intermediate computation results on disk. Memory is used as its best, without exceeding the cluster maximum available memory. This way, your cluster can avoid running out of memory and is still able to answer queries.

How to use Spill to Disk

Spill to disk can be enabled with the following command in the Developer Console:

CODE

SET_ spill.enable = true

If you want to resize the maximum memory used for results :

CODE

SET_ spill.memory.size.mb = 1024

All configuration parameters for the spill to disk function can be set on-the-fly, or be written in the galactica.conf

How to know if a query has spilled to disk

In the monitor, on the queries page, a query with the icon

has spilled to disk.

Configuration

Go to SpillToDisk configuration