AW: [Isolate-interest] Shared Memory ?

[Ernst, Matthias]

Pete, Curt,

thanks for your answers. Basically my question comes from the
circumstance that
our software builds caches in-memory that are rather expensive to
recreate
(very slow backend). A crash or restart due to software update makes us
lose this cache.
Customer customizations often introduce memory leaks and I'm looking for
a way to isolate
the cache from that and us from the blame :-)

Furthermore, the cache ends up in the old gen and may lead to long GC
pauses; so 
I wonder if a number of small VMs attached to a shared, non-gc'ed memory
"blob"
may provide a model that scales better and is more crash resilient.
(Disregarding the fact
that I have to manage that blob of memory myself).

> Looking at the current Link API, it would seem pretty trivial 
> to create Input/OutputStream wrappers for it.  From there, it 
> is easy to create Data/Object/Input/OutputStreamS and 
> XMLEncoder/DecoderS. 

If the Link API provided throughput anywhere close to DirectByteBuffer
access that would
be ok for me. One isolate could play the role of the shared memory area,
only executing
our code; other isolates could be workers, hosting "dangerous"
application code.
 
> Do you have more info and a link for "something like that" 
> from Gemstone?

Gemstone's "GemFire" product, in its core, is similar to what I
describe:
- a shared memory area with an own memory manager that even does garbage
collection
- a native library to attach a JVM to this area
- utilities for object serialization to and from this area
- facilities for synchronization of attached VMs

I've never actually used this beyond toying around with an evaluation
but they claim to build 
very scalable and resilient systems around it.

> What are the limitations involved in using MappedByteBufferS 
> in multiple isolates to share memory.  Is there a problem 
> with this technique, or you just want a mechanism that 
> doesn't depend on NIO? 

I was unsure whether values written in one process would be
instantaneously available in all others (modulo memory barriers). The
API doc says something about that:

"The content of a mapped byte buffer can change at any time, for example
if the content of the corresponding region of the mapped file is changed
by this program or another. Whether or not such changes occur, and when
they occur, is operating-system dependent and therefore unspecified. "

Does this uncertainty only apply to the File<->Memory transfer ?


Matthias