[Building Sakai] Clustering a HashMap

[Ian Boston]

On 18 Jan 2011, at 10:22, Adrian Fish wrote:

> Thanks for the replies Ian and Lance. From the ehCache docs I gained the impression that the replication was more fine grained than you describe. What you are saying, if I understand correctly, is that entire strata within the main map that ehcache uses are replaced if even one element at that level is changed.

No,  its on a per element level.
Its possible I miss read you email but I thought you said you were going to put a hashmap into the cache if not already there.
eg
Map<?,?> myPayload = new HashMap<?,?>();

....

String cacheKey = "akey";

Cache cache = memoryService.newCache(....);
if ( ! cache.containsKey(cacheKey) ) {
   cache.put("akey", myPayload);
}


myPayload.put("key", "value");
cache.put("akey", myPayload);


-----------------------------
as opposed to use the Cache as a Map eg

Cache cache = memoryService.newCache(....);
cache.put("key", "value");

if you meant the latter, then you might be ok, it all depends on the size of your values.
Ian



> That's obviously a bit heavy, so I'll think again.
> 
> I'll have a look at the tools you've suggested, thanks !
> 
> Cheers,
> 
> Adrian.
> 
> On 17/01/2011 18:00, Ian Boston wrote:
>> On 14 Jan 2011, at 12:45, Adrian Fish wrote:
>> 
>>> At the moment I'm playing with creating an EhCacheFactory bean in my
>>> spring config, using the MemoryService to create a Cache instance and
>>> then putting the hashmaps in that if they are not already there.
>> IIRC ehCache replicates on element levels. So if you put a hashmap into a Cache as a single Cache element, then every time you modifiy the map it will replicate, which will soon kill your server with replication traffic.
>> 
>> Try and put the contents of the map at the lowest level in the map, then when you change one item, only that will be replicated.
>> Also think hard about if you really want replication, or just invalidation. In general relying on replication as a mechanism to distribute messages is soon going to overload a production system. I would look at things like node.js or cometD and communicate a chat end point to the each user with messages going to that destination rather than rely on a replicated, in memory structure to achieve this.
>> 
>> If you really must use replication and ehCache, then configure it with JGroups underneath as thats probably the most efficient transport if you servers are on the same subnet,
>> 
>> If you want an alternative, take a look at infinspan which does a better job of replication in larger clusters, and then there is memcached (but you will have to OS level things on install)
>> 
>> HTH
>> Ian
>> 
>> 
>> 
> 
> -- 
> ==================================
> Adrian Fish
> Software Engineer
> Centre for e-Science
> Bowland Tower South C Floor
> Lancaster University
> Lancaster
> LA1 4YW
> email: a.fish at lancaster.ac.uk
> 
> http://confluence.sakaiproject.org/display/YAFT/Yaft
> http://confluence.sakaiproject.org/display/CLOG/Home
> http://confluence.sakaiproject.org/display/BBB/Home
>