[Deploying Sakai] "Too many open files" and search indexing

[Colin Tatham]

14/11/11 10:50 PM, Matthew Buckett:
> On 14 November 2011 06:42, Colin Tatham<colin.tatham at unimelb.edu.au>  wrote:
>> We increased the value set in /etc/security/limits.conf:
...
>> which seems to have changed the limit:
>> [sakai at crete ~]$ ulimit -n
>> 65535
>>
>> but I still get the errors?
>
> How are you starting tomcat? as /etc/security/limits.conf only applies
> to process started after a login, if you have an upstart script then
> they won't bet getting picked up.


I'm still not having much joy with this, any clues in this bit of the stack trace?

2011-11-11 17:44:34,467  INFO IndexManager org.sakaiproject.search.journal.impl.SharedFilesystemJournalStorage - ++++++ Saving /servers/sakai_home/search/indexwork/indexer-work/indextx-1320991326766 to shared
2011-11-11 17:44:43,847  INFO IndexManager org.sakaiproject.search.indexer.impl.TransactionalIndexWorker - Indexed 998 documents in 131961 ms 7.562840536219034 documents/second into save point 9
2011-11-11 17:44:46,470  WARN IndexManager org.sakaiproject.search.journal.impl.MergeUpdateOperation - Failed to complete merge of 8
org.sakaiproject.search.transaction.api.IndexTransactionException: Failed to commit 
	at org.sakaiproject.search.transaction.impl.IndexTransactionImpl.commit(IndexTransactionImpl.java:195)
	at org.sakaiproject.search.journal.impl.MergeUpdateOperation.runOnce(MergeUpdateOperation.java:95)
	at org.sakaiproject.search.journal.impl.IndexManagementTimerTask.run(IndexManagementTimerTask.java:137)
	at java.util.TimerThread.mainLoop(Timer.java:512)
	at java.util.TimerThread.run(Timer.java:462)
Caused by: org.sakaiproject.search.transaction.api.IndexTransactionException: Failed to close index with deletions
	at org.sakaiproject.search.journal.impl.JournaledFSIndexStorageUpdateTransactionListener.commit(JournaledFSIndexStorageUpdateTransactionListener.java:245)
	at org.sakaiproject.search.transaction.impl.IndexTransactionImpl.fireCommit(IndexTransactionImpl.java:324)
	at org.sakaiproject.search.transaction.impl.IndexTransactionImpl.commit(IndexTransactionImpl.java:188)
	... 4 more
Caused by: java.io.FileNotFoundException: /servers/sakai_home/search/indexwork/index-import/8/_i.cfs (Too many open files)
	at java.io.RandomAccessFile.open(Native Method)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
	at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput$Descriptor.<init>(SimpleFSDirectory.java:78)
	at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.<init>(SimpleFSDirectory.java:108)
	at org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.<init>(NIOFSDirectory.java:106)
	at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:82)
	at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70)
	at org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:115)
	at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:667)
	at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:628)
	at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:108)
	at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:80)
	at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683)
	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:73)
	at org.apache.lucene.index.IndexReader.open(IndexReader.java:476)


Thanks!

Colin

--
Colin Tatham
Educational Technologist | Learning Environments
University Library | The University of Melbourne
http://www.trs.unimelb.edu.au