[Building Sakai] Course Management : CmSynchronizer JOB : File Size

[Kevin P. Foote]

Wow.. 3G xml .. bet that takes a while to load in a browser ha.. ;-)

I load 3k+ courses with the CmSynchronizer but my xml is only ~ 20-30M

I create the xml file in about 40 sec with perl but the Synchronizer
takes about 1 hour. 

If your dealing with 3G .. you'd probabbly be better off writing
something customized like others do as Casey mentioned. 

------
thanks
  kevin.foote

On Wed, 6 May 2009, Casey Dunn wrote:

-> On Wed, May 6, 2009 at 10:57 AM, Thomas Amsler <tpamsler at ucdavis.edu> wrote:
-> 
-> > I am experimenting with loading CM data via the CmSynchronizer Job. Is
-> > anybody using this appraoch to load their CM data into the Sakai CM
-> > tables? How are folks handling large data set that result int large
-> > CMData.xml file. I am dealing with about ~3GB of data.
-> >
-> > Source Code:
-> >
-> > https://source.sakaiproject.org/svn/course-management/branches/sakai_2-5-x/cm-impl/hibernate-impl/impl/src/java/org/sakaiproject/coursemanagement/impl/job/
-> >
-> >
-> > Best,
-> > -- Thomas
-> 
-> 
-> Stanford avoids that problem - the XML is split down to the Course level,
-> containing Offerings and their Sections. ( the example CmSynchronizer served
-> as an example; there are other Stanford specific tweaks to the feed )
-> 
-> A 3Gig XML file must really be expensive to deal with.
-> 
-> why not knock it apart, Thomas?
->