[Building Sakai] junk in email messages

[Sean Baker]

Charles,

Thank goodness there's someone else out there experiencing this - I was 
beginning to think that we were the only ones!

Here's what we've found locally with what I believe is an identical 
problem (and hopefully identical cause):

  * Our Sakai 2.7.1 instances did not have these issues; sadly it
    slipped through our pre-2.8 testing
  * We upgraded Tomcat to 5.5.34 when we upgraded Sakai to 2.8.0
    (self-hosted rSmart instance)
  * We upgraded our JDK to 1.6.0.26 when we upgraded Sakai to 2.8.0
  * As you describe, characters are not correctly converted into the
    destination character set by the mail utility on our production
    systems (I think it may actually be occurring at the MIME Part level
    but am not sure on that as I was forced to cut testing short)
  * For the time being, I've simply overridden all mail character
    encoding to be set to UTF-8 in order to keep users from the problem
    while we sort it out internally
  * And now for the weird piece - the rSmart 2.8 demonstration server
    does not have these problems

Given the last bit, my own thinking has been that the issue is likely to 
do with the Tomcat libraries (less likely), the JDK version (more likely 
- there have been bugs like this in the JDKs previously), or the server 
configuration itself (allowed character sets, etc.).

Looking for commonality of our problematic environments, can I ask which 
Sakai, Tomcat, and JDK versions you're on at your end?

Thanks for making me feel a little more sane :).

Sean



On 1/31/12 16:21 PM, Charles Hedrick wrote:
> We had a complain that announcements containing special characters, e.g. , show up with garbage characters in the plaintext version.
>
> Looking at it,&nbsp turns into the correct NS character. However MyMessage in BasicEmailService checks to see whether all the characters can be handled with ISO_8859_1. They can, because it has NS. We then call
>
>                                                      bodyPart.setContent(partText.toString(), mimeType);
>
> using a Mime type of text/plain; charset=iso-8859-1. Unfortunately it appears that the Java mail mechanism doesn't actually use the charset here (and even using setText didn't help). It outputs UTF-8. However since the header claims ISO-8859-1, the mail program displays it incorrectly.
>
> In theory it makes sense to use 8859 rather than UTF_8 where possible. But the Java mail code seems almost intentionally designed to make it impossible to get this right. I get buried in layers of objects with ill-defined properties. Does anyone know the BasicEmailService well enough to see what's going on?
>
>
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"

-- 
Ne Desit Virtus,

Sean R. Baker
1LT, MS
United States Army
Office #: (301) 319-0712
Email: sean.baker at usuhs.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20120201/e7f16a14/attachment.html