[Deploying Sakai] "Too many open files" and search indexing

[Majeed]

We had a problem last weekend where our dev environment kept running 
into a similar "Too many files" error. Tomcat would crash and not be 
listening on 8080 anymore. We are on Solaris and used the pfiles command 
to determine what files a specific pid had opened. Should be able to use 
lsof on linux. Anyhow the main culprit seemed to be files in the 
"indexwork" directory for us. After 3 weeks of running there were over 
65K files in there. Adding some code to the setenv.sh to delete the 
contents of indexwork dir upon a tomcat restart resolved the issue for 
us. Not to sure of the side effects yet, as we are pretty new to the app.

local.properties:localIndexBase at org.sakaiproject.search.api.JournalSettings=${sakai.home}/indexwork

Good Luck,
Majeed

-- 
Majeed Abu-Qulbain
Lead Unix Admin - CISS
Illinois State University



On 11/17/2011 10:00 AM, Matthew Buckett wrote:
> On 17 November 2011 12:00, Colin Tatham<colin.tatham at unimelb.edu.au>  wrote:
>> 14/11/11 10:50 PM, Matthew Buckett:
>>> On 14 November 2011 06:42, Colin Tatham<colin.tatham at unimelb.edu.au>    wrote:
>>>> We increased the value set in /etc/security/limits.conf:
>> ...
>>>> which seems to have changed the limit:
>>>> [sakai at crete ~]$ ulimit -n
>>>> 65535
>>>>
>>>> but I still get the errors?
>>> How are you starting tomcat? as /etc/security/limits.conf only applies
>>> to process started after a login, if you have an upstart script then
>>> they won't bet getting picked up.
>>
>> I'm still not having much joy with this, any clues in this bit of the stack trace?
> Attached is a test class. Download it, compile it and run it on a
> machine to see how many files the JVM can open:
>
> javac OpenFiles.java
> java OpenFiles
>
> On my MacBook I get:
>
> Too many open files in system
> Failed to create tmp file. Already got: 10014
>
> On my Ubuntu system:
>
> Too many open files in system
> Failed to create tmp file. Already got: 4087
>
>
>
> _______________________________________________
> production mailing list
> production at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/production
>
> TO UNSUBSCRIBE: send email to production-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/production/attachments/20111117/9f14f2cb/attachment.html