[Deploying Sakai] Sakai is inaccessible - Failed to compete merge....

Leon Kolchinsky lkolchin at gmail.com
Thu Feb 24 19:27:23 PST 2011 

Hello David,

Thank you.
I've cleared the search index and deleted metadata from Oracle DB:


delete from search_journal;
delete from search_node_status;
delete from search_segments;
delete from search_transaction;
delete from searchbuilderitem;
delete from searchwriterlock;

Now, there are no nasty messages in the logs ;)

Cheers,
Leon Kolchinsky



On Tue, Feb 22, 2011 at 17:25, David Horwitz <david.horwitz at uct.ac.za>wrote:

>  Hi Leon,
>
> This indicates your local search index on this node is corrupted. To fix
> this:
>
> 1) Stop the node
> 2) Delete the files in
> localIndexBase at org.sakaiproject.search.api.JournalSettings (defaults to
> {tomcat}/sakai/indexwork)
> 3) Restart
>
>
> When running in a cluster this will reload the index from shared storage (
> sharedJournalBase at org.sakaiproject.search.api.JournalSettings). We actualy
> have our start up script do this before starting the node.
>
> To completly clear the index with all nodes stopped (or search disabled):
>
> 1) do steps 1 & 2 above
> 2) delete the contents of the shared index storage
> 3) clear the metadata from the database:
>
> truncate table search_journal;
> truncate table search_node_status;
> truncate table search_segments;
> truncate table search_transaction;
> truncate table searchbuilderitem;
> truncate table searchwriterlock;
>
> (mysql syntax oracle may differ)
>
>
> Hope this helps
>
> D
>
>
>
> On 02/22/2011 07:02 AM, Leon Kolchinsky wrote:
>
>  Hello Guys,
>
> We're running 2.5.4 version of Sakai.
> Today we found out that the application is inaccessible and had to restart
> the server.
> On the startup I can see those errors:
>
> INFO: Server startup in 129663 ms (2011-02-22 15:16:02,112
> main_org.apache.catalina.startup.Catalina)
> WARN: Failed to compete merge of 11923 (2011-02-22 15:17:06,180
> Timer-1_org.sakaiproject.search.journal.impl.MergeUpdateOperation)
> org.sakaiproject.search.transaction.api.IndexTransactionException: Failed
> to delete documents
>         at
> org.sakaiproject.search.journal.impl.JournaledFSIndexStorageUpdateTransactionListener.prepare(JournaledFSIndexStorageUpdateTransactionListener.java:161)
>         at
> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.firePrepare(IndexTransactionImpl.java:312)
>         at
> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.prepare(IndexTransactionImpl.java:146)
>         at
> org.sakaiproject.search.journal.impl.MergeUpdateOperation.runOnce(MergeUpdateOperation.java:94)
>         at
> org.sakaiproject.search.journal.impl.IndexManagementTimerTask.run(IndexManagementTimerTask.java:135)
>         at java.util.TimerThread.mainLoop(Timer.java:512)
>         at java.util.TimerThread.run(Timer.java:462)
> Caused by: java.io.IOException:
> /srv/apache-tomcat-5.5.23/sakai/indexwork/index-import/11872 not a directory
>         at org.apache.lucene.store.FSDirectory.init(FSDirectory.java:180)
>         at
> org.apache.lucene.store.FSDirectory.getDirectory(FSDirectory.java:141)
>         at
> org.sakaiproject.search.journal.impl.JournaledFSIndexStorage.getIndexReaderInternal(JournaledFSIndexStorage.java:802)
>         at
> org.sakaiproject.search.journal.impl.JournaledFSIndexStorage.getIndexReader(JournaledFSIndexStorage.java:712)
>         at
> org.sakaiproject.search.journal.impl.JournaledFSIndexStorage.getDeletionIndexReader(JournaledFSIndexStorage.java:545)
>         at
> org.sakaiproject.search.journal.impl.JournaledFSIndexStorageUpdateTransactionListener.prepare(JournaledFSIndexStorageUpdateTransactionListener.java:124)
>         ... 6 more
> WARN: Failed to start merge operation  (2011-02-22 15:17:06,205
> Timer-1_org.sakaiproject.search.journal.impl.MergeUpdateOperation)
> org.sakaiproject.search.journal.api.JournalErrorException: Journal is
> stalled at ID 11923
>         at
> org.sakaiproject.search.journal.impl.JournaledFSIndexStorageUpdateTransactionListener.open(JournaledFSIndexStorageUpdateTransactionListener.java:88)
>         at
> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.fireOpen(IndexTransactionImpl.java:359)
>         at
> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.open(IndexTransactionImpl.java:75)
>         at
> org.sakaiproject.search.journal.impl.MergeUpdateManager.openTransaction(MergeUpdateManager.java:48)
>         at
> org.sakaiproject.search.journal.impl.MergeUpdateOperation.runOnce(MergeUpdateOperation.java:92)
>         at
> org.sakaiproject.search.journal.impl.IndexManagementTimerTask.run(IndexManagementTimerTask.java:135)
>         at java.util.TimerThread.mainLoop(Timer.java:512)
>         at java.util.TimerThread.run(Timer.java:462)
> INFO: Local Merge Operation
>  (2011-02-22 15:17:06,205
> Timer-1_org.sakaiproject.search.journal.impl.MergeUpdateOperation)
> WARN: Failed to compete optimize  (2011-02-22 15:17:06,237
> Timer-1_org.sakaiproject.search.optimize.impl.OptimizeIndexOperation)
> org.sakaiproject.search.optimize.api.OptimizedFailedIndexTransactionException:
> Failed to Optimize indexes
>         at
> org.sakaiproject.search.optimize.impl.OptimizeTransactionListenerImpl.prepare(OptimizeTransactionListenerImpl.java:184)
>         at
> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.firePrepare(IndexTransactionImpl.java:312)
>         at
> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.prepare(IndexTransactionImpl.java:146)
>         at
> org.sakaiproject.search.optimize.impl.OptimizeIndexOperation.runOnce(OptimizeIndexOperation.java:90)
>         at
> org.sakaiproject.search.journal.impl.IndexManagementTimerTask.run(IndexManagementTimerTask.java:135)
>         at java.util.TimerThread.mainLoop(Timer.java:512)
>         at java.util.TimerThread.run(Timer.java:462)
> Caused by: java.io.IOException:
> /srv/apache-tomcat-5.5.23/sakai/indexwork/index-import/11872 not a directory
>         at org.apache.lucene.store.FSDirectory.init(FSDirectory.java:180)
>         at
> org.apache.lucene.store.FSDirectory.getDirectory(FSDirectory.java:141)
>         at
> org.sakaiproject.search.optimize.impl.OptimizeTransactionListenerImpl.prepare(OptimizeTransactionListenerImpl.java:168)
>         ... 6 more
>
>
> What does it mean?
> Any resolution?
>
> Thanks,
>  Leon
>
>
>
>
> _______________________________________________
> production mailing listproduction at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/production
>
>
> TO UNSUBSCRIBE: send email to production-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>
>
>
> _______________________________________________
> production mailing list
> production at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/production
>
> TO UNSUBSCRIBE: send email to
> production-unsubscribe at collab.sakaiproject.org with a subject of
> "unsubscribe"
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/production/attachments/20110225/95c5f20e/attachment.html

More information about the production mailing list