[Deploying Sakai] Issue with 1 of our production servers.

[Tom Hall]

Hello again everyone

Mystery solved.  There has been a change in the way DNS is behaving in 
our environment.  All of our Sakai servers are in the 10.x address space 
and for some reason that our DNS folks haven't been able to explain yet 
the behavior of reverse lookups for 10.x addresses are now behaving 
differently (should have immediately failed but was taking 30 seconds to 
a minute).

Solution to the problem within our Sakai environment was to put 
/etc/hosts entries fore all of our application servers onto the database 
servers.

Tom


On 4/4/2011 10:39 AM, Tom Hall wrote:
> Hi Everyone
>
> One of the 5 application servers in our production environment has
> started to have performance problems, starting at about 8:59 EDT
> yesterday morning according to our monitoring tools.  The problem was
> actually reported via an php web site that is running on the same server
> that makes web service calls to the local instance of Sakai and also
> accesses the same database server.  According to the catalina.out and
> apache logs, there was no unusual activity at that time.  Our network
> monitors don't show any unusual traffic levels either.
>
> I have done a reboot of the system and restarted Sakai (without letting
> any users near it).  During the start process, there appears to be a
> very long pause with the last line in catalina.out being
>
> 2011-04-04 10:17:33,310  INFO main
> org.sakaiproject.site.impl.DbSiteService - init(): site table:
> SAKAI_SITE external locks: true
>
> Normal start up times are around 75000 ms, this mornings was 332045 ms.
>
> The SAKAI_LOCKS table is empty when this is pause occurs.
>
> Does anyone have any suggestions.
>
> Thanks
>
> Tom
> _______________________________________________
> production mailing list
> production at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/production
>
> TO UNSUBSCRIBE: send email to production-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"