[Building Sakai] . Re: Elastic Search (SRCH-111) (John Bush)

[Colin Hebert]

Solr < 4 does not support support soft commits altogether, so this is not
an issue (somewhat less efficient due to the need of constant hard commits
but there is no way to lose anything).

Solr4 supports now soft commits (useful for NRT), and added a transaction
log (SOLR-2700) for the new replication system (SolrCloud). So there is no
need to add anything.

If lucid was providing some kind of transaction log, I'll assume that it is
what is now available directly within solrcloud.

Cheers,
Colin
 On 28 Jan 2013 06:22, "Ian Boston" <ieb at tfd.co.uk> wrote:

> On 27 January 2013 16:13, Colin Hebert <colin.hebert at it.ox.ac.uk> wrote:
> > I'm not sure why you would _need_ to implement a WAL.
>
> To avoid having to commit segments to disk on every update ?
>
> If you soft commit you risk loosing data. If you commit on every
> update you cause a reader reload on every update which flushes all
> term vectors and cached information. Having a WAL you can replay in
> the event of failure means you can do a soft commit to memory avoiding
> having to perform a hard commit. At least, thats the way ES avoids
> hard commits on every update.  Unfortunately, Solr master/slave
> replication requires hard commits to replicate. If you buy a
> commercial Solr distribution from Lucid then IIUC you get replication
> of soft commit. ES replicates by replicating the write operation
> avoiding the hard commit issue completely.
>
> HTH
> Ian
>
> >
> > On 27 January 2013 16:34, Ian Boston <ieb at tfd.co.uk> wrote:
> >>  elasticsearch.index.number_of_replicas=1
> >>>
> >>>
> >>
> >>
> >> Make sure you have enough replicas to make the index resilient. Iiuc
> that
> >> means >= 3.
> >>
> >> Also, if using sol4 you will need to manage replicas yourself, and might
> >> also need to implement your own write ahead log. I did some blg posts on
> >> this a while back.
> >> Hth
> >> Ian
> >>
> >> _______________________________________________
> >> sakai-dev mailing list
> >> sakai-dev at collab.sakaiproject.org
> >> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >>
> >> TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org
> >> with a subject of "unsubscribe"
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20130128/c31401c0/attachment.html