[Building Sakai] Search index troubles

[Warwick Chapman]

Speaking of which, on Debian (and probably Ubuntu as well), setting ulimit
-n 4096 to increase from default of 1024 seems to only work for that session
for that user.

Further, while this may be obvious to some, the ulimit should be increased
for the user which runs tomcat - it is not a systemwide setting.

Setting the limit for the user in /etc/security/limits.conf does not seem to
work in Debian as the figure remains unchanged at 1024.  I'm still not
certain that I've resolved this problem, but I have placed a ulimit -n 4096
in the ~/.profile of the user that runs tomcat as well as at the stop of the
script which starts Sakai on boot in the hope that will work.

-- Warwick Bruce Chapman | +27 83 7797 094 | http://warwickchapman.com


On Sun, Nov 28, 2010 at 6:42 PM, David Horwitz <david.horwitz at uct.ac.za>wrote:

>  Hi,
>
> I helped Warwick trouble shoot his problem - it looked like his server had
> hit a file handle limit while doing a local optimization that left  the
> local index in an inconsistent state. Stopping the server, deleting the
> *local* index and restarting it fixed it.
>
> The local index is set by:
>
>
> localIndexBase at org.sakaiproject.search.api.JournalSettings=/usr/local/searchindex/
>
> and defaults to {tomcat_home}sakai/indexwork/
>
> Tip when trouble shooting search errors - go back to the first error in
> your catalina.log this will indicate what caused the error rather than the
> last one which indicates that the index is corrupt
>
> D
>
>
>
>
>
> On 11/28/2010 05:40 PM, Berg, Alan wrote:
>
> Hi all,
>
> I am seeing this on qa1-nl as well with the newest 2.8 alpha tag.
>
>  Alan
>
> Alan Berg
> QA Director - The Sakai Foundation
>
> Senior Developer / Quality Assurance
> Group Education and Research Services
> Central Computer Services
> University of Amsterdam
>
> http://home.uva.nl/a.m.berg
>   ------------------------------
> *From:* sakai-dev-bounces at collab.sakaiproject.org [
> sakai-dev-bounces at collab.sakaiproject.org] on behalf of Warwick Chapman [
> warwickchapman at gmail.com]
> *Sent:* 28 November 2010 07:51
> *To:* Tom Hall
> *Cc:* sakai-dev
> *Subject:* Re: [Building Sakai] Search index troubles
>
>  When you say searchjournal directory, do you
> mean: /opt/sakai/index/index-import/
>
> -- Warwick Bruce Chapman | +27 83 7797 094 | http://warwickchapman.com
>
>
> On Fri, Nov 26, 2010 at 5:15 PM, Tom Hall <thall at brocku.ca> wrote:
>
>> HI Warwick
>>
>> This looks similar to something we have seen.  Can you please tell me how
>> many files are in your searchjournal directory, so that I can be sure.
>>
>> Tom
>>
>> On 11/26/2010 1:57 AM, Warwick Chapman wrote:
>>
>>  Hi All
>>
>>  Any pointers on this issue?
>>
>>  2010-11-26 08:55:49,283  WARN IndexManager
>> org.sakaiproject.search.optimize.impl.OptimizeIndexOperation - Failed to
>> compete optimize
>> org.sakaiproject.search.transaction.api.IndexTransactionException: Failed
>> to prepare
>> at
>> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.prepare(IndexTransactionImpl.java:157)
>> at
>> org.sakaiproject.search.optimize.impl.OptimizeIndexOperation.runOnce(OptimizeIndexOperation.java:90)
>> at
>> org.sakaiproject.search.journal.impl.IndexManagementTimerTask.run(IndexManagementTimerTask.java:139)
>> at java.util.TimerThread.mainLoop(Timer.java:512)
>> at java.util.TimerThread.run(Timer.java:462)
>> Caused by: java.lang.IllegalArgumentException: Directory
>> org.apache.lucene.store.SimpleFSDirectory@/opt/sakai/index/index-import/1106
>> appears more than once
>> at org.apache.lucene.index.IndexWriter.noDupDirs(IndexWriter.java:3607)
>> at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3544)
>> at
>> org.sakaiproject.search.optimize.impl.OptimizeTransactionListenerImpl.prepare(OptimizeTransactionListenerImpl.java:171)
>> at
>> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.firePrepare(IndexTransactionImpl.java:313)
>> at
>> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.prepare(IndexTransactionImpl.java:147)
>>
>>  -- Warwick Bruce Chapman | +27 83 7797 094 | http://warwickchapman.com
>>
>>
>> _______________________________________________
>> sakai-dev mailing listsakai-dev at collab.sakaiproject.orghttp://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>
>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>>
>>
>>
>
> _______________________________________________
> sakai-dev mailing listsakai-dev at collab.sakaiproject.orghttp://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>
>
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
> "unsubscribe"
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20101128/3e5a60cd/attachment.html