[Building Sakai] SAK-17171/KNL-66 (plain text handling)

[Lydia Li]

Anthony,

    Sorry for misleading you.  Our fix is actually not trying to resolve 
SAK-17171/KNL-66.  Samigo does not encounter the problem in 
SAK-17171/KNL-66.
   
    The 22 December thread made us revisit our fixes for 2 other related 
bugs:  SAK-14153 and SAK-17199.  Our existing code checks if the input 
is plain text or rich text ( see Stephen Marguard's case #3 below) , and 
if it's plain text, we are currently calling the 
convertPlaintextToFormattedText method.  However we found out that this 
method also converts foreign languages to the &# format and saves that 
in DB.  It makes it impossible to query for a foreign language 
assessment directly in DB.  We would like to change that behavior. 

     We are looking to add a new method similar to 
FormattedText.convertPlaintextToFormattedText(), except it would not 
convert foreign language characters.   Why does 
convertPlaintextToFormattedText convert high bit unicode anyway?   What 
is that block of code for?

     We can provide a new method (Karen and I are thinking about calling 
it convertPlaintextToFormattedTextNoHighUnicode()  ) .  We would like to 
have it added to FormattedText.java.

     We should finish this and have it ready for QA by end of this week. 

thanks,
Lydia



Stephen Marquard wrote:
> IMO, we should follow this set of rules:
>
> 1. For fields that are always plain text, store as-is in the database, escape on output 
> 2. For fields that are entered in rich text, sanitize on input, store formatted, display as-is
> 3. For fields that may be entered in plain text or rich text, escape or sanitize on input depending on input type, store formatted, display as-is.
>
> escape = convert for example < to &lt;
>
> sanitize = check for an acceptable set of tags, escape the rest (e.g. <script> gets escaped to &lt;script&gt; <strong> is unchanged) OR input not matching the validation rules is rejected and not stored (with some appropriate UI behaviour).
>
> Cheers
> Stephen
>
>   
>>>> Anthony Whyte <arwhyte at umich.edu> 1/12/2010 9:32 PM >>> 
>>>>         
> Lydia--There is no definitive "do this/don't do this" statement  
> regarding plain text handling in Sakai other than what you might  
> surmise from reading the Javadocs.  However, see the 22 December  
> thread.  What do you plan to do regarding SAK-17171 and how long do  
> you expect your rework to take?
>
> http://n2.nabble.com/Building-Sakai-2-6-2-SAK-17171-Botimer-vs-Botimer-td4205175.html
>
> My view is that in certain/many cases we should be storing plain text  
> input raw without escaping it and then sanitizing it when responding  
> to a request.  However, there are numerous fields where escaping plain  
> text input appears to me warranted (names, addresses, etc.). This I  
> believe is also the view of Botimer, Zeckoski and Marquard.  Bear in  
> mind that escaping text when responding to a request may tax web  
> servers differently than when sanitizing text as part of a form  
> submission.
>
> The question of plain text input/output handling is an important one  
> and we have yet to proceed beyond the pre-Holiday discussion to  
> articulate (and enforce) a general practice.
>
> This issue raises concerns relative to the 2.5.6/2.6.2 releases (2.7.0  
> as well).  Do we block the impending maintenance releases until we  
> have resolved the questions raised by SAK-17171/KNL-66?
>
> Cheers,
>
> Anth
>
>
> On Jan 12, 2010, at 1:55 PM, Lydia Li wrote:
>
>   
>> Anthony,
>>   In light of the discussion for SAK-17171, we are looking into  
>> doing some rework on our earlier fixes and we'd like to get the  
>> fixes into 2.6.2. Just wanted to give you a heads up.
>>
>>   Btw, was there a solution to SAK-17171 or KNL-66?  I followed the  
>> thread but didn't see a definitive solution for it.  I thought you  
>> might have more intimate knowledge on the progress of those bugs.
>>
>> thanks,
>> Lydia
>>
>>
>>
>>
>>     
>
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>
>
>