[Building Sakai] Email archive - some emails not sending

Mon Sep 9 16:18:47 PDT 2013

Thanks for the background on maxObjectsToQueryFor. We missed it when we
were reviewing changes to jldap-beans.xml, and also would have thought it
was irrelevant as we had thought the LDAP bind user was exempt from this
restriction. It obviously wasn't. :/

So prior to this change, it was a single request per object? Just want to
confirm that all our assumptions are correct, including that it worked fine
in 2.6 because it was well under the 50 object max (which 1 request for
every object would be.)

This all makes good sense from a performance standpoint. One connection
beats multiple connections when looking up users in a directory.

To Sam's question, I believe it was aggressively-limited based on a
historic precedent set by our public ldap server. I've since cranked it up
to 2000, which I believe to be the default (found that to be the setting in
another OU in the same config file).

As far as I can tell, we haven't had a single error since the change on the
LDAP side.

Back to the disappearing email behavior, the only thing that still concerns
me is there are no Sizelimit exceptions happening concurrent to those
emails that never got sent. The exceptions are generated, however, when
something like Site Info triggers it. Is it possible that the exception was
suppressed by the emailservice's error handling?

Also, is this enough of an edge case where it doesn't make sense to add
additional logging or a note in the readme file? As Sam said, 200 should be
well above the default for most. I want to save someone else beating their
head against this one, yet not sure it's worth anyone taking to time to
document.

-Mike

On Mon, Sep 9, 2013 at 6:30 AM, Aaron Zeckoski <azeckoski at unicon.net> wrote:

> What Sam is getting at here is that if you want decent performance you
> will want to crank the max limit in your LDAP way up to at least 1000
> records. The default in openldap is about 500 (set by sizelimit). I
> believe the openldap manual actually suggests a setting as high as 10000.
>
> Depending on the LDAP server you are using you will see much better
> performance by ramping the number up on the LDAP side as opposed to using
> the paging functionality supported by maxObjectsToQueryFor (multiple
> smaller requests are generally slower than 1 big one). Tailor that to the
> load on your LDAP server of course. If it is like most LDAP servers and is
> generally idle then cranking this up should not have any noticeable impact.
>
> -AZ
>
>
>
> On Mon, Sep 9, 2013 at 9:15 AM, Sam Ottenhoff <ottenhoff at longsight.com>wrote:
>
>> Right, I saw a "Size limit exceeded" error from another email of yours
>> over the weekend. This is an error from your LDAP server telling Sakai that
>> it has requested too many records from LDAP.  We tried to set the default
>> for maxObjectsToQueryFor low enough that it wouldn't be over any default
>> LDAP config while still being performant.  Is your limit of 50 a default
>> from the vendor or is it an aggressively-limited default set by your LDAP
>> admin?  The defaults I see are usually 1000 or 2000.
>>
>> The maxObjectsToQueryFor was implemented in 2.9+ and changes the getUsers
>> method into a batch request method instead of iterating through single
>> requests.  So if you have a class roster of 500 students, the LDAP will
>> only receive 500/maxObjectsToQueryFor requests instead of the previous
>> behavior of 500 individual LDAP requests.
>>
>> --Sam
>>
>>
>> On Mon, Sep 9, 2013 at 1:32 AM, Michael Osterman <
>> ostermmg.sakai at gmail.com> wrote:
>>
>>> Hi Steve,
>>>
>>> I think I figured it out (way too late) last night.
>>>
>>> It seems to be an issue with our LDAP server configuration and the jldap
>>> provider's maxObjectsToQueryFor not getting along as documented in this
>>> comment:
>>>
>>> https://jira.sakaiproject.org/browse/SAK-15634?focusedCommentId=181037&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-181037
>>>
>>> I was able to reproduce this behavior in Site Info for a class with more
>>> than 50 students prior to adjusting the nsslapd-sizelimit on our LDAP;
>>> basically, the list of enrolled people is missing on a restart of Tomcat
>>> until you refresh Site Info to trigger getUsers() enough times to fill up
>>> the UserDirectory cache, 50 at a time. Once it got all the class cached,
>>> then the LDAP Exceptions went away. I could also see myself generating the
>>> errors on the LDAP server.
>>>
>>> I'm now 99% convinced this is why the professor's emails were failing (a
>>> subsequent test message today went through). The getUsers() was triggering
>>> an uncaught exception when the mail service was getting the list via
>>> getUsers(), and the outgoing email simply never happened.
>>>
>>> I also checked our mail server logs for all the prior failing mails for
>>> this professor, and I can see the email coming in to our @
>>> cleo.whitman.edu list address, and ZERO connections back from our Sakai
>>> server attempting to send the message.
>>>
>>> Keeping my fingers crossed that this fixed it.
>>>
>>> I'm still interested in debugging this more on our test instance to see
>>> where the actual exception is happening. Can your recommend the appropriate
>>> class to reference for increasing logging to DEBUG? Can't recall how to
>>> reference these.
>>>
>>> Cheers,
>>> Mike
>>>
>>>
>>> On Sun, Sep 8, 2013 at 4:03 PM, Steve Swinsburg <
>>> steve.swinsburg at gmail.com> wrote:
>>>
>>>> I've had an issue where certain email addresses were stopping the email
>>>> from going out to the entire list, but was never able to figure out why.
>>>> Can you try debug logging the emailservice to see exactly what it's doing?
>>>>
>>>> Cheers,
>>>> Steve
>>>>
>>>>
>>>> Sent from my iPad
>>>>
>>>> On 08/09/2013, at 12:35, Michael Osterman <ostermmg.sakai at gmail.com>
>>>> wrote:
>>>>
>>>> A professor just let us know that their students aren't received emails
>>>> sent to their class listserv (a.k.a. Email Archive address).
>>>>
>>>> The messages were indeed received by Sakai (we're on 2.9.x) as they are
>>>> listed in the Email Archive interface, but there is no log entry in
>>>> catalina.out that matches the messages.
>>>>
>>>> I've just sent a test message to confirm, that email does work to a
>>>> separate list, and the message is delivered as expected, and I have a log
>>>> entry like this as expected:
>>>>
>>>> 2013-09-07 16:14:51,622  INFO Thread-358570
>>>> org.sakaiproject.email.impl.BasicEmailService - sendToUsers
>>>> [rest removed]
>>>>
>>>> I checked the settings for the class, and everything seems to be
>>>> correct:
>>>>
>>>> We've been getting intermittent reports of "email not working" since
>>>> our upgrade from 2.6.x to 2.9.x, sometimes related to Email Archive,
>>>> sometimes related to Announcements.
>>>>
>>>>  Has anyone experienced this problem, or does anyone have pointers for
>>>> getting more information on what's going on with the email service?
>>>>
>>>> Cheers,
>>>> Mike
>>>>
>>>> _______________________________________________
>>>> sakai-dev mailing list
>>>> sakai-dev at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>>
>>>> TO UNSUBSCRIBE: send email to
>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>>> "unsubscribe"
>>>>
>>>>
>>>
>>> _______________________________________________
>>> sakai-dev mailing list
>>> sakai-dev at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>
>>> TO UNSUBSCRIBE: send email to
>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>> "unsubscribe"
>>>
>>
>>
>> _______________________________________________
>> sakai-dev mailing list
>> sakai-dev at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>
>> TO UNSUBSCRIBE: send email to
>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>> "unsubscribe"
>>
>
>
>
> --
> Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20130909/7deddf5b/attachment.html 