[Deploying Sakai] "Too many open files" and search indexing

[Colin Tatham]

18/11/11 3:00 AM, Matthew Buckett:
> On 17 November 2011 12:00, Colin Tatham<colin.tatham at unimelb.edu.au>  wrote:
>> 14/11/11 10:50 PM, Matthew Buckett:
>>> On 14 November 2011 06:42, Colin Tatham<colin.tatham at unimelb.edu.au>    wrote:
>>>> We increased the value set in /etc/security/limits.conf:
>>>> which seems to have changed the limit:
>>>> [sakai at crete ~]$ ulimit -n
>>>> 65535

> Attached is a test class. Download it, compile it and run it on a
> machine to see how many files the JVM can open:

Thanks. On the server:

[sakai at crete ~]$ java OpenFiles
Too many open files
Failed to create tmp file. Already got: 65529

Which is pretty similar to the limit that's set in limits.conf, and far 
more than the number of open files reported by lsof -p $java_pid when 
the sakai indexer runs into trouble:

Number of open files: 2649  Max: 2655

Some background info:
In production, our binary content is stored in the DB, but we're 
planning to extract that when migrating to the new servers.

I've run Sakai with the settings for the extraction which completes 
successfully, although it does report that about 800 out of 40,000 files 
have zero content. (Those files also have zero content when accessing 
them via the web interface before the extraction process, so I don't 
think they get messed up by the extraction.)

Rebuilding the search indexes on an almost identical server running 
Sakai with binary content still in the DB completes without the errors. 
I think that's probably just because it doesn't have to access the files 
on the file system as part of the digesting phase, but I wonder whether 
the files with zero content are a factor in the digesting phase? It does 
report 'file not found' errors, but nothing more.

Thanks

Colin

> On my MacBook I get:
>
> Too many open files in system
> Failed to create tmp file. Already got: 10014
>
> On my Ubuntu system:
>
> Too many open files in system
> Failed to create tmp file. Already got: 4087
>



-- 
Colin Tatham
Educational Technologist | Learning Environments
University Library | The University of Melbourne
http://www.trs.unimelb.edu.au