[Building Sakai] Help needed converting resource storage to filesystem

[Sam Ottenhoff]

Hi Josh,

As my comment on KNL-164 noted, I don't think there has been much 
testing with Oracle.  This process should run smoothly for MySQL, but I 
believe there may be problems in DbContentService fetching the blob.  I 
am attaching a patch that took care of the issue for another Oracle 
institution.  If it works for you, I will go ahead and refactor the 
patch and submit it as a new KNL ticket.

The important chunk of code is in DbContentService around line 2620:

                                                 Object clob = 
result.getObject(3);
                                                 if(clob != null && clob 
instanceof byte[])
                                                 {
+                                                       
System.out.println("received a valid binary object");
                                                         edit = new 
BaseResourceEdit();
                                                         
resourceSerializer.parse(edit, (byte[]) clob);
                                                 }
+                                               else if(clob != null) {
+                                                       
System.out.println("About to get a blob from Oracle");
+                                                       Blob blob = 
result.getBlob(3);
+                                                       long blobLength 
= blob.length();
+                                                       edit = new 
BaseResourceEdit();
+                                                       
resourceSerializer.parse(edit, (byte[]) blob.getBytes(1, (int) blobLength));
+                                                       
System.out.println("Finished parsing the blob");
+                                               }
+                                               else
+                                               {
+                                                       
System.out.println("received a null clob:" + clob);
+                                               }


--Sam


On 12/10/2009 4:16 PM, Joshua Swink wrote:
> I'm embarking on moving resources out of the database and into the
> filesystem. I've read related messages in this list and
> http://jira.sakaiproject.org/browse/KNL-164, but I'd like to form a
> more complete picture of the process.
>
> I ran a conversion script to deal with the binary/xml entity body
> issue, and then set the convertToFile, bodyPath, and bodyVolumes
> properties. The results were disappointing so I'd like to get a deeper
> understanding of what needs to be done and why.
>
> What happened was this: When running the conversion script, a lot of
> messages like the following were printed:
>
> WARN [main] (SchemaConversionController.java:208) -   -->  Source is
> null for id: /attachment/200910-3317-WRI-116-001/Assignments/....
>
> 219106 of those messages were printed. For comparison, the
> content_resource table has 182655 rows.
>
> Then I set the three properties in sakai.properties and started Sakai.
> It worked for quite a while, printing messages like the following:
>
> 2009-12-09 21:23:55,918  WARN main
> org.sakaiproject.content.impl.DbContentService - convertToFile(): null
> xml :
>
> 547947 lines like that appeared in the log. No files or subdirectories
> were created in the directory designated in bodyPath.
>
> The command to run the conversion script:
>
> bash content-runconversion.sh.COLUMBIA -j
> /local/pkg/sak/sakai/tomcat/common/lib/ojdbc14.jar -p
> sakai/tomcat/sakai/local.properties -k 1.0.11 -s 2.6.0
> ucmcrops/kernel/tools/content-conversion/upgradeschema-oracle.config>
> conversion_export.log 2>&1&
>
> I attached the conversion script.
>
> Hopefully I haven't wiped out the resources... it will be a hassle to
> export another copy to test this on.
>
> Josh
>    
>
>
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20091211/7f1572a6/attachment.html 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: dbcontentservice-oracle-clob.patch
Url: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20091211/7f1572a6/attachment.pl