[Using Sakai] Short URLs for files with non ASCII characters
Steve Swinsburg
steve.swinsburg at gmail.com
Mon Aug 19 07:45:05 PDT 2013
Actually I think this is highlighting a problem in Sakai related to the
storage of urls to resources. If you upload a file that contains a special
character, and Sakai subsequently creates a URL to it that contains the
special character, the URL is invalid. Most browsers translate it ok so its
not a big deal, but its still invalid according to the RFC.
http://tools.ietf.org/html/rfc3986
This is what is breaking the url shortener since a servlet serves out a
redirect to the URL and Tomcat cannot handle the invalid characters.
So this resource Id created by uploading a file into Sakai is invalid:
tttp://localhost:8081/access/content/group/aa5385a0-e2fd-457f-8824-c1d5f6e344f4/Test%20file%20ó%20â%20ë.pdf
On Mon, Aug 19, 2013 at 11:07 PM, Steve Swinsburg <steve.swinsburg at gmail.com
> wrote:
> I can reproduce this. Jira raised here:
> https://jira.sakaiproject.org/browse/SHORTURL-39
> Its late for me right now so if you have time to review and submit a patch
> I can commit it. Otherwise I'll look at this as soon as I can.
>
> cheers,
> Steve
>
>
> On Mon, Aug 19, 2013 at 8:34 PM, Steve Swinsburg <
> steve.swinsburg at gmail.com> wrote:
>
>> It's totally possible there is a weird encoding happening in the URL
>> shortener, I wrote it ;)
>> If that's the case i'll fix it, I just want to be sure. I'll upload a
>> file with accented characters to my instance and test.
>>
>> Cheers,
>> S
>>
>> Sent from my iPad
>>
>> On 19/08/2013, at 14:31, Rafael Morales Gamboa <rmorales at suv.udg.mx>
>> wrote:
>>
>> It looks like the database was setup as UTF-8 from the beginning, because
>> that is said in the instructions to install Sakai CLE 2.9. I do not
>> remember changing it to UTF-8 after installation. The fact is that the full
>> URL uses UTF-8, but the shortener encodes it as Latin 1.
>>
>> Would there be any way to test if the database was not created as UTF-8?
>> Could someone who is totally sure its database was created as UTF-8 try to
>> reply my case? It would require to upload a document from Windows with an
>> 'ó' (accute o) in its name, and then generate a short URL for it.
>>
>>
>> 2013/8/18 Steve Swinsburg <steve.swinsburg at gmail.com>
>>
>>> If your database was not setup as UTF-8 to begin with, then you may need
>>> to export it all out and re-import it with the correct character set. I
>>> haven't done this though.
>>>
>>>
>>> On Mon, Aug 19, 2013 at 9:56 AM, Rafael Morales Gamboa <
>>> rmorales at suv.udg.mx> wrote:
>>>
>>>> I searched the url_randomised_mappings_t, ran the results through od,
>>>> and found that
>>>>
>>>> 0000000 1 1 \t H a q X O C \t h t t p : /
>>>> 0000020 / v i r t u a l . c u d i . e d
>>>> 0000040 u . m x : 8 0 8 0 / a c c e s s
>>>> 0000060 / c o n t e n t / g r o u p / 8
>>>> 0000100 b f 7 3 c 8 c - 0 9 6 8 - 4 9 1
>>>> 0000120 4 - 8 c 4 4 - 9 1 8 d 7 6 f a 3
>>>> 0000140 8 9 6 / I n t r o d u c c i 363 n
>>>> 0000160 a S a k a i - P e r s p
>>>> 0000200 e c t i v a d e l U s u a r
>>>> 0000220 i o . p d f \n
>>>>
>>>> So the URL is stored using #f3 (octal 363) for the 'ó' (accute o). In
>>>> other words, Sakai/MySQL is using the ISO 8859-1 encoding instead of UTF-8
>>>> to store shortened the URL.
>>>>
>>>> Regards,
>>>> Rafael
>>>>
>>>>
>>>> 2013/8/18 Steve Swinsburg <steve.swinsburg at gmail.com>
>>>>
>>>>> Go to the database and see what the full url is for this shortened
>>>>> URL. Does it match the actual resource URL?
>>>>>
>>>>> Cheers,
>>>>> Steve
>>>>>
>>>>> Sent from my iPad
>>>>>
>>>>> On 17/08/2013, at 7:23, Rafael Morales Gamboa <rmorales at suv.udg.mx>
>>>>> wrote:
>>>>>
>>>>> Actually, I was wrong. Tomcat is configured with URIEncoding="UTF-8"
>>>>> and the database seems to be OK (despite de fact that there is no
>>>>> character_set_server=utf8 in my.conf:
>>>>>
>>>>> mysql> SELECT default_character_set_name FROM
>>>>> information_schema.SCHEMATA S
>>>>> -> WHERE schema_name = "sakai";
>>>>> +----------------------------+
>>>>> | default_character_set_name |
>>>>> +----------------------------+
>>>>> | utf8 |
>>>>> +----------------------------+
>>>>> 1 row in set (0.00 sec)
>>>>>
>>>>> The question is whether I should add that configuration line to
>>>>> my.conf, anayway.
>>>>>
>>>>> Regards,
>>>>> Rafael
>>>>>
>>>>> El 15/08/2013 06:50 p.m., Mike DeSimone escribió:
>>>>>
>>>>> Hi Rafael,
>>>>>
>>>>> Sometimes things like this are due to database settings and tomcat
>>>>> connector settings. Can you confirm the 8080 connector
>>>>> has URIEncoding="UTF-8"? Also, if this is MySQL, then ensure this is in
>>>>> the my.cnf: character_set_server=utf8 and the database was created with
>>>>> utf8 encoding.
>>>>>
>>>>> more info can be found here:
>>>>> https://confluence.sakaiproject.org/display/DOC/Sakai+CLE+2.9+Release+Notes#SakaiCLE29releasenotes-Databasesupport
>>>>> and:
>>>>> https://confluence.sakaiproject.org/pages/viewpage.action?pageId=82249313(section 3 for tomcat setup)
>>>>>
>>>>> I also have found that files cannot easily be renamed in the Sakai
>>>>> UI. Not sure if there's any other solution than your suggestion.
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> *********************************************
>>>>> Mike DeSimone
>>>>> Director of Enterprise Operations
>>>>> Asahi Net International, Inc. (*ANI*)
>>>>> O: +1 (602) 490-0473
>>>>> W: anisakai.com
>>>>>
>>>>> [image: Inline image 1]
>>>>> **********************************************
>>>>>
>>>>>
>>>>> On Wed, Aug 14, 2013 at 4:25 PM, Rafael Morales Gamboa <
>>>>> rmorales at suv.udg.mx> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> On Sakai 2.9.1 I uploaded a file to Resources that was called
>>>>>> 'Introducción a Sakai - Perspectiva del Usuario.pdf' in my computer, and I
>>>>>> renamed it to 'Introducción a Sakai.pdf' in Resources and made it public.
>>>>>> Under properties I can see now its URL:
>>>>>>
>>>>>>
>>>>>> http://virtual.cudi.edu.mx:8080/access/content/group/8bf73c8c-0968-4914-8c44-918d76fa3896/Introducci%C3%B3n%20a%20Sakai%20-%20Perspectiva%20del%20Usuario.pdf
>>>>>>
>>>>>> So, the URL is composed from the original name of the file in my
>>>>>> computer, and I cannot see a way to change it unless renaming the file in
>>>>>> my computer before uploading it to Resources.
>>>>>>
>>>>>> Now, if I choose the Short URL option I get the URL
>>>>>> http://virtual.cudi.edu.mx:8080/x/HaqXOC, and at least three
>>>>>> distinct behaviours in browser (latest public releases of them):
>>>>>>
>>>>>> 1. In Firefox, I get error 404.
>>>>>> 2. In Chrome, it is translated to
>>>>>> http://virtual.cudi.edu.mx:8080/access/content/group/8bf73c8c-0968-4914-8c44-918d76fa3896/Introducci%F3n%20a%20Sakai%20-%20Perspectiva%20del%20Usuario.pdf('ó' is encoded as %F3 instead of %C%B3) and I get error 404.
>>>>>> 3. In Internet Explorer it works! ('ó' is just 'ó').
>>>>>>
>>>>>> The questions are (1) why do I get all this different behaviours? and
>>>>>> (2) how can I get it to work on all these browsers?
>>>>>>
>>>>>> Regards,
>>>>>> Rafael
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> sakai-user mailing list
>>>>>> sakai-user at collab.sakaiproject.org
>>>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-user
>>>>>>
>>>>>> TO UNSUBSCRIBE: send email to
>>>>>> sakai-user-unsubscribe at collab.sakaiproject.org with a subject of
>>>>>> "unsubscribe"
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> sakai-user mailing list
>>>>> sakai-user at collab.sakaiproject.org
>>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-user
>>>>>
>>>>> TO UNSUBSCRIBE: send email to
>>>>> sakai-user-unsubscribe at collab.sakaiproject.org with a subject of
>>>>> "unsubscribe"
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> sakai-user mailing list
>>>> sakai-user at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-user
>>>>
>>>> TO UNSUBSCRIBE: send email to
>>>> sakai-user-unsubscribe at collab.sakaiproject.org with a subject of
>>>> "unsubscribe"
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-user/attachments/20130820/31eab437/attachment-0001.html
More information about the sakai-user
mailing list