[Deploying Sakai] Sakai is inaccessible - Failed to compete merge....

Leon Kolchinsky lkolchin at gmail.com
Tue Feb 22 15:42:27 PST 2011 

Thanks David,

This is a one node system ;(
We'll move soon to the 2 node system (CISCO CSM integrated).

1) So basically no harm done if I'd delete {tomcat}/sakai/indexwork contents
before every restart?
2) Regarding 2 node system:
We're testing 2 instances of Sakai running on the same Oracle DB.

Every server has it's own {tomcat}/sakai/indexwork content.

Can we leave it like that or should we use NFS share to
host {tomcat}/sakai/indexwork indexes?

Cheers,
Leon Kolchinsky



On Tue, Feb 22, 2011 at 17:25, David Horwitz <david.horwitz at uct.ac.za>wrote:

>  Hi Leon,
>
> This indicates your local search index on this node is corrupted. To fix
> this:
>
> 1) Stop the node
> 2) Delete the files in
> localIndexBase at org.sakaiproject.search.api.JournalSettings (defaults to
> {tomcat}/sakai/indexwork)
> 3) Restart
>
>
> When running in a cluster this will reload the index from shared storage (
> sharedJournalBase at org.sakaiproject.search.api.JournalSettings). We actualy
> have our start up script do this before starting the node.
>
> To completly clear the index with all nodes stopped (or search disabled):
>
> 1) do steps 1 & 2 above
> 2) delete the contents of the shared index storage
> 3) clear the metadata from the database:
>
> truncate table search_journal;
> truncate table search_node_status;
> truncate table search_segments;
> truncate table search_transaction;
> truncate table searchbuilderitem;
> truncate table searchwriterlock;
>
> (mysql syntax oracle may differ)
>
>
> Hope this helps
>
> D
>
>
>
> On 02/22/2011 07:02 AM, Leon Kolchinsky wrote:
>
>  Hello Guys,
>
> We're running 2.5.4 version of Sakai.
> Today we found out that the application is inaccessible and had to restart
> the server.
> On the startup I can see those errors:
>
> INFO: Server startup in 129663 ms (2011-02-22 15:16:02,112
> main_org.apache.catalina.startup.Catalina)
> WARN: Failed to compete merge of 11923 (2011-02-22 15:17:06,180
> Timer-1_org.sakaiproject.search.journal.impl.MergeUpdateOperation)
> org.sakaiproject.search.transaction.api.IndexTransactionException: Failed
> to delete documents
>         at
> org.sakaiproject.search.journal.impl.JournaledFSIndexStorageUpdateTransactionListener.prepare(JournaledFSIndexStorageUpdateTransactionListener.java:161)
>         at
> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.firePrepare(IndexTransactionImpl.java:312)
>         at
> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.prepare(IndexTransactionImpl.java:146)
>         at
> org.sakaiproject.search.journal.impl.MergeUpdateOperation.runOnce(MergeUpdateOperation.java:94)
>         at
> org.sakaiproject.search.journal.impl.IndexManagementTimerTask.run(IndexManagementTimerTask.java:135)
>         at java.util.TimerThread.mainLoop(Timer.java:512)
>         at java.util.TimerThread.run(Timer.java:462)
> Caused by: java.io.IOException:
> /srv/apache-tomcat-5.5.23/sakai/indexwork/index-import/11872 not a directory
>         at org.apache.lucene.store.FSDirectory.init(FSDirectory.java:180)
>         at
> org.apache.lucene.store.FSDirectory.getDirectory(FSDirectory.java:141)
>         at
> org.sakaiproject.search.journal.impl.JournaledFSIndexStorage.getIndexReaderInternal(JournaledFSIndexStorage.java:802)
>         at
> org.sakaiproject.search.journal.impl.JournaledFSIndexStorage.getIndexReader(JournaledFSIndexStorage.java:712)
>         at
> org.sakaiproject.search.journal.impl.JournaledFSIndexStorage.getDeletionIndexReader(JournaledFSIndexStorage.java:545)
>         at
> org.sakaiproject.search.journal.impl.JournaledFSIndexStorageUpdateTransactionListener.prepare(JournaledFSIndexStorageUpdateTransactionListener.java:124)
>         ... 6 more
> WARN: Failed to start merge operation  (2011-02-22 15:17:06,205
> Timer-1_org.sakaiproject.search.journal.impl.MergeUpdateOperation)
> org.sakaiproject.search.journal.api.JournalErrorException: Journal is
> stalled at ID 11923
>         at
> org.sakaiproject.search.journal.impl.JournaledFSIndexStorageUpdateTransactionListener.open(JournaledFSIndexStorageUpdateTransactionListener.java:88)
>         at
> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.fireOpen(IndexTransactionImpl.java:359)
>         at
> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.open(IndexTransactionImpl.java:75)
>         at
> org.sakaiproject.search.journal.impl.MergeUpdateManager.openTransaction(MergeUpdateManager.java:48)
>         at
> org.sakaiproject.search.journal.impl.MergeUpdateOperation.runOnce(MergeUpdateOperation.java:92)
>         at
> org.sakaiproject.search.journal.impl.IndexManagementTimerTask.run(IndexManagementTimerTask.java:135)
>         at java.util.TimerThread.mainLoop(Timer.java:512)
>         at java.util.TimerThread.run(Timer.java:462)
> INFO: Local Merge Operation
>  (2011-02-22 15:17:06,205
> Timer-1_org.sakaiproject.search.journal.impl.MergeUpdateOperation)
> WARN: Failed to compete optimize  (2011-02-22 15:17:06,237
> Timer-1_org.sakaiproject.search.optimize.impl.OptimizeIndexOperation)
> org.sakaiproject.search.optimize.api.OptimizedFailedIndexTransactionException:
> Failed to Optimize indexes
>         at
> org.sakaiproject.search.optimize.impl.OptimizeTransactionListenerImpl.prepare(OptimizeTransactionListenerImpl.java:184)
>         at
> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.firePrepare(IndexTransactionImpl.java:312)
>         at
> org.sakaiproject.search.transaction.impl.IndexTransactionImpl.prepare(IndexTransactionImpl.java:146)
>         at
> org.sakaiproject.search.optimize.impl.OptimizeIndexOperation.runOnce(OptimizeIndexOperation.java:90)
>         at
> org.sakaiproject.search.journal.impl.IndexManagementTimerTask.run(IndexManagementTimerTask.java:135)
>         at java.util.TimerThread.mainLoop(Timer.java:512)
>         at java.util.TimerThread.run(Timer.java:462)
> Caused by: java.io.IOException:
> /srv/apache-tomcat-5.5.23/sakai/indexwork/index-import/11872 not a directory
>         at org.apache.lucene.store.FSDirectory.init(FSDirectory.java:180)
>         at
> org.apache.lucene.store.FSDirectory.getDirectory(FSDirectory.java:141)
>         at
> org.sakaiproject.search.optimize.impl.OptimizeTransactionListenerImpl.prepare(OptimizeTransactionListenerImpl.java:168)
>         ... 6 more
>
>
> What does it mean?
> Any resolution?
>
> Thanks,
>  Leon
>
>
>
>
> _______________________________________________
> production mailing listproduction at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/production
>
>
> TO UNSUBSCRIBE: send email to production-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>
>
>
> _______________________________________________
> production mailing list
> production at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/production
>
> TO UNSUBSCRIBE: send email to
> production-unsubscribe at collab.sakaiproject.org with a subject of
> "unsubscribe"
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/production/attachments/20110223/e9011f9f/attachment.html

More information about the production mailing list