Details
-
Improvement
-
Resolution: Unresolved
-
High
-
None
-
5.1
-
None
-
None
Description
For import/batch scripts, it would be useful to be able to clear the in-memory stash cache every N loop passes, to avoid consuming all the memory on the server.
Right now the only way to make import scripts work (eg 1M contents) is to:
- either disable completely stash during import, which means changing configuration files twice (not too nice)
- or run import scripts by making them work piecewise on small batches of work, and manage a master-process forking out some workers
a workaround is to have the longrunning script clear the SPI cache every now and then, but this only works if the on-disk SPI cache is almost empty. When it contains 1M objects, clearing it takes basically forever (ie not doable). It also has a negative impact on server operations.
best results would be with scripts which can still keep the on-disk stash cache while clearing the in-memory one