[Building Sakai] serious blocker probelm with convertToFile: "null xml"? (2.6.x)

Tue Apr 21 19:07:51 PDT 2009

The method convertToFile() in DbContentService has an SqlReader to  
deserialize resources during the conversion from database storage to  
filesystem storage of resource bodies.  The SqlReader was never  
rewritten to deal with BINARY_ENTITY serialization.  It attempts to  
read the XML and gets nulls, so it fails.

Jim

On Apr 21, 2009, at 9:48 PM, Jim Eng wrote:

> Hi Will,
>
> Do you know from your logs or from queries of the CONTENT_RESOURCE
> table in your database whether ContentHostingService is using XML or
> BINARY_ENTITY serialization before the conversion?
>
> I am guessing that nobody has done db to filesystem conversion in
> 2.6.x.  Were you running 2.5 or 2.4 and converted to 2.6?
>
> How many rows are in the CONTENT_RESOURCE table?
>
> How many rows are in the CONTENT_RESOURCE_BODY_BINARY table?
>
> Thanks.
>
> Jim
>
>
>
> On Apr 21, 2009, at 9:25 PM, will trillich wrote:
>
>> We're stuck without some pointers here... We're having one heck of a
>> time finding the problem converting from resources-in-sql to
>> resources-in-filesystem -- any suggestions are welcome!
>>
>>
>>
>> Short version:
>>
>> We're trying to migrate 2.6.x resources from SQL storage to  
>> filesystem
>> storage -- and three XSL files (formFiew-related files) get  
>> converted,
>> nothing else does. Lots of "DbContentService - convertToFile: null
>> xml" log lines in catalina.out.
>>
>> Once Sakai is up, when trying to access any of the items from
>> resources, the item isn't found of course (error reported in the
>> browser). We can then upload new files and they're stored in the
>> filesystem just fine; these newly uploaded items can be
>> viewed/downloaded just fine.
>>
>> We can comment out bodyPath and bodyVolumes, restart sakai, and the
>> original resources are healthy (viewable/downloadable) again.
>>
>> What are we missing?
>>
>>
>>
>> Long version:
>>
>> http://bugs.sakaiproject.org/confluence/display/DOC/Configuring+Content+Hosting
>> , right?
>>
>> # in sakai.properties:
>> bodyPath at org.sakaiproject.content.api.ContentHostingService=/Volumes/
>> mongo/sakai
>> bodyVolumes
>> @org
>> .sakaiproject
>> .content
>> .api
>> .ContentHostingService
>> =a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z
>> convertToFile at org.sakaiproject.content.api.ContentHostingService=true
>>
>> After starting sakai with convertToFile=true, there are exactly three
>> items in the bodyPath tree (there should be dozens even for a fresh
>> sakai install):
>>
>> bash-3.2$ cd /Volumes/mongo/sakai/
>> bash-3.2$ find . -type f
>> ./c/2009/112/00/b51c365b-f67d-4ff1-9f13-6a7cfb07fa06
>> ./d/2009/112/00/ad68e45b-ef6b-4d4e-9a01-1575260c2c9c
>> ./s/2009/112/00/4a935949-0437-4024-a486-7a108072393f
>>
>> The three resources above are formView XSL files. Those are the only
>> ones that get converted.
>>
>> ./x/2009/111/22/124ef7d4-36c2-4170-b027-853e704c5d07
>>
>> Thereafter, uploaded files (one shown above), as a test, worked just
>> fine. (That is, conversion doesn't seem to be working at all, but new
>> uploads are just fine.)
>>
>>
>> # from catalina.out:
>> <snip>
>> 2009-04-21 20:41:16,502  INFO main
>> org.sakaiproject.event.impl.BaseNotificationService -
>> org.sakaiproject.event.impl.DbNotificationService$$EnhancerByCGLIB$$346b4059 at ed5990.init
>> ()
>> 2009-04-21 20:41:16,503  INFO main
>> org.sakaiproject.event.impl.DbNotificationService - .init(): table:
>> SAKAI_NOTIFICATION locks-in-db: false
>> 2009-04-21 20:41:16,546  INFO main
>> org.sakaiproject.content.impl.DbResourceTypeRegistry - init()
>> 2009-04-21 20:41:16,550  INFO main
>> org.sakaiproject.content.impl.ResourceTypeRegistryImpl - init()
>> 2009-04-21 20:41:16,719  INFO main
>> org.sakaiproject.content.impl.DbContentService -
>> ====================================================
>> ====================================================
>> Conversion of the ContentHostingService database tables is complete.
>> Using new filesize column
>> ====================================================
>> ====================================================
>>
>> 2009-04-21 20:41:16,747  INFO main
>> org.sakaiproject.content.impl.DbContentService - Migration of data to
>> the Binary format will be performed by this node
>> 2009-04-21 20:41:16,814  INFO main
>> org.sakaiproject.content.impl.BaseContentService - Loaded Storage as
>> org.sakaiproject.content.impl.DbContentService$DbStorage at 9f750 for
>> org.sakaiproject.content.impl.DbContentService at 4ba63b
>> 2009-04-21 20:41:16,849  INFO main
>> org.sakaiproject.content.impl.BaseContentService - init(): site  
>> quota:
>> 1048576 body path: /Volumes/wikiblogs3/sakai volumes: a, b, c, d, e,
>> f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z,
>> 2009-04-21 20:41:16,977  INFO main
>> org.sakaiproject.content.impl.DbContentService - convertToFile
>> 2009-04-21 20:41:16,995  WARN main
>> org.sakaiproject.content.impl.DbContentService - convertToFile: null
>> xml : /user/ce1a4481-91d8-455b-a2b9-33f5a26a3a77/Principios y
>> reflexiones/Rubricas/rubricasola_porta_e_revisin_p3.pdf
>> 2009-04-21 20:41:16,995  WARN main
>> org.sakaiproject.content.impl.DbContentService - convertToFile: null
>> xml : /group/PortfolioAdmin/system/contentOverText.xml
>> 2009-04-21 20:41:16,995  WARN main
>> org.sakaiproject.content.impl.DbContentService - convertToFile: null
>> xml : /group/PortfolioAdmin/system/simpleRichText.xml
>> 2009-04-21 20:41:16,995  WARN main
>> org.sakaiproject.content.impl.DbContentService - convertToFile: null
>> xml : /user/ce1a4481-91d8-455b-a2b9-33f5a26a3a77/Principios y
>> reflexiones/Rubricas/rubricasola_porta_e_revisin_p2.pdf
>> 2009-04-21 20:41:16,995  WARN main
>> org.sakaiproject.content.impl.DbContentService - convertToFile: null
>> xml : /group/PortfolioAdmin/system/freeFormRenderer.xml
>>
>> </snip>
>>
>> 2009-04-21 20:41:17,178  WARN main
>> org.sakaiproject.content.impl.DbContentService - convertToFile: null
>> xml : /attachment/fdfb4c6e-64f5-4ea1-ac16-e1cb99109263/Matrices/
>> 77b08a12-efef-42ac-93ad-5ebfbb8aa03c/Reflexion
>> Principio 2.docx
>> 2009-04-21 20:41:17,178  WARN main
>> org.sakaiproject.content.impl.DbContentService - convertToFile: null
>> xml : /group/PortfolioAdmin/system/2column.jpg
>> 2009-04-21 20:41:17,179  INFO main
>> org.sakaiproject.content.impl.DbContentService - convertToFile:
>> converted resources: 0
>> 2009-04-21 20:41:17,179  INFO main
>> org.sakaiproject.content.impl.DbContentService - convertToFile: done
>> 2009-04-21 20:41:17,180  INFO main
>> org.sakaiproject.content.impl.DbContentService - init(): tables:
>> CONTENT_COLLECTION CONTENT_RESOURCE CONTENT_RESOURCE_BODY_BINARY
>> CONTENT_ENTITY_GROUPS locks-in-db: false bodyPath:
>> /Volumes/wikiblogs3/sakai storage:
>> org.sakaiproject.content.impl.DbContentService$DbStorage at 9f750
>> 2009-04-21 20:41:17,181  INFO main
>> org
>> .sakaiproject
>> .component.app.messageforums.MessageForumsMessageManagerImpl
>> - init()
>>
>> </snip>
>>
>>
>>
>> # uname -a
>> Darwin vstream.uprrp.edu 9.6.0 Darwin Kernel Version 9.6.0: Mon Nov  
>> 24
>> 17:39:01 PST 2008; root:xnu-1228.9.59~1/RELEASE_PPC Power Macintosh
>>
>> # sakai source code version (from svn info):
>> URL: https://source.sakaiproject.org/svn/sakai/branches/sakai_2-6-x
>> Repository Root: https://source.sakaiproject.org/svn
>> Repository UUID: 66ffb92e-73f9-0310-93c1-f5514f145a0a
>> Revision: 60519
>> Node Kind: directory
>> Schedule: normal
>> Last Changed Author: arwhyte at umich.edu
>> Last Changed Rev: 60121
>> Last Changed Date: 2009-04-15 18:01:01 -0400 (Wed, 15 Apr 2009)
>>
>> $ $JAVA_HOME/bin/java -version
>> Shared archive: sharing disabled for server vm
>> java version "1.5.0_16"
>> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_16-
>> b06-284)
>> Java HotSpot(TM) Client VM (build 1.5.0_16-133, mixed mode)
>>
>> $ tomcat/bin/catalina.sh version
>> Using CATALINA_BASE:   /opt/tomcat
>> Using CATALINA_HOME:   /opt/tomcat
>> Using CATALINA_TMPDIR: /opt/tomcat/temp
>> Using JRE_HOME:       /Library/Java/Home
>> Shared archive: sharing disabled for server vm
>> Server version: Apache Tomcat/5.5.26
>> Server built:   Jan 28 2008 01:35:23
>> Server number:  5.5.26.0
>> OS Name:        Mac OS X
>> OS Version:     10.5.6
>> Architecture:   ppc
>> JVM Version:    1.5.0_16-b06-284
>> JVM Vendor:     Apple Inc.
>>
>> -- 
>> will trillich
>> "If you do not change direction, you may end up where you are
>> heading." -- Lao Tzu
>> _______________________________________________
>> sakai-dev mailing list
>> sakai-dev at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>
>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org
>> with a subject of "unsubscribe"
>>
>>
>
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org 
>  with a subject of "unsubscribe"
>
>