[Deploying Sakai] [sakai-nakamura] OAE clustering

[Clay Fenlason]

Joining the production list to the conversation, as I know it's of
interest among pilot deployers. My impression is that some among us
were going to start doing some of that missing testing in the next
couple weeks ... or at least talking about doing so.

~Clay

On Fri, Aug 19, 2011 at 5:14 AM, Ian Boston <ieb at tfd.co.uk> wrote:
> Dave,
>
> There are 2 issues with clustering.
> 1. Its not be tested at all, and there are somethings that we are
> doing that are beyond what CLE did in this area. We need replication
> of small amounts of data, not just invalidation. It should be a simple
> ehcache config.... (Q: how many times have I said "... should....
> simple... " in the same sentence before?)
>
> 2. Solr bundle was originally designed to support 1 Solr query per
> http request. In that mode a central Solr server would have worked,
> with remote connections. Unfortunately, the UI requires many Solr
> requests per http requests and I dont think that the added latency of
> a remote Solr box is not going to make the UI happy. The UI is happy
> now, because there is no latency. Solr is embedded in the same JVM. I
> have been working on a solution, but its not ready, and I dont know if
> it will make it into V1.1
>
>
> There are going to be other issues surrounding the UI tolerance for
> information propagation (ie, how soon it must see the data appear in
> Solr after an update is made). I dont think we have done a good job of
> the internal information architecture of the data within the server,
> evidenced by just how often this issue keeps coming up. If we had done
> a good job, we would not have any SQL type queries, and we would be
> able to rely on eventually consistent (sub 10s) update of data. At it
> stands we need ACID updates on much of the data, which means, at
> present you need to bind sessions to app server nodes and have a big
> iron database behind the cluster. Not good.
>
> Storing content streams in Blobs.
> You can, but a config switch, but it was not designed for a RDBMS
> environment. It was designed for a Column environment where the
> content cant be streamed or seeked. Data is stored as 1MB chunks, with
> 64 1MB chunks per conceptual Row (64 columns per row in Cassandra).
> That allows us to seek in 1MB increments without loading all the data.
>
> Oracle blobs are already seek-able, and the DB handles any storage
> chunking. The simples solution would be to write a new stream storage
> handler just for Oracle.... if thats what you want... and your DBA's
> are happy. Most DBAs dont like big blobs in the DB. Migrating from
> file to Blob should be relatively trivial.
>
> HTH
>
> Ian
>
>
>
> On 19 August 2011 07:38, David Roma <david.roma.csu at gmail.com> wrote:
>> Hey all..
>> I remember there being some issues around clustering.. I just want to
>> clarify where we are for V1, and where we will be for V1.1. Just after a
>> high level summary.
>> I presume the clustering issues are in relation to solr? where are indexes
>> stored, will they automatically recreate if we add a new node in the system?
>> v1... v1.1
>> Are there any other issues with clustering that we know about (other than it
>> being untested)
>> Cheers,
>> Dave.
>> p.s. There is an option to store all sparse content in the
>> database isn't there? other than storage usage would there be performance
>> implications here? (this would make for a simpler production architecture)
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Sakai Nakamura" group.
>> To post to this group, send email to sakai-kernel at googlegroups.com.
>> To unsubscribe from this group, send email to
>> sakai-kernel+unsubscribe at googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/sakai-kernel?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups "Sakai Nakamura" group.
> To post to this group, send email to sakai-kernel at googlegroups.com.
> To unsubscribe from this group, send email to sakai-kernel+unsubscribe at googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sakai-kernel?hl=en.
>
>