[Contrib: Evaluation System] A little progress on import/export

Jim Eng jimeng at umich.edu
Thu Jan 28 08:20:59 PST 2010


Hi Stephen,

Our requirements were for efficient import of large amounts of data to  
sync CTools (our Sakai instance) with an external system.  Entity  
broker did not seem to be the best way for us to accomplish that, but  
at the moment I can't put my hands on the exchange of email in which  
that was discussed.

I am implementing the import mechanism as a separate servlet that can  
be run on one machine in the cluster (or even outside the cluster).   
The servlet will depend on EVALSYS, but nothing in EVALSYS or any  
other part of Sakai will depend on this servlet at all.  That will  
make it easier for institutions (including Michigan) to customize it  
to meet institution-specific needs without needing to branch off the  
released versions of EVALSYS.

Because of the way this is implemented, it will be easy to implement a  
REST-like export capability, but it will not be a full RESTful  
interface ala the Direct Servlet.  It's possible that some of the work  
done on this servlet could be leveraged in the future when someone has  
cycles to complete the RESTful impl in EVALSYS, but we do not expect  
to be doing that at this time.

Thanks.

Jim



On Jan 23, 2010, at 12:48 AM, Stephen Marquard wrote:

> Hi Jim,
>
> If you're creating support for import/export via REST, can I suggest  
> that you wire this up using entitybroker? It's a mechanism that many  
> are familiar with and has the advantage of being self-documenting  
> (via the /direct/.../describe ... URLs).
>
> Regards
> Stephen
>
>>>> Jim Eng <jimeng at umich.edu> 1/23/2010 2:23 AM >>>
> I have been working on import of various EVALSYS entities, because
> that's a pretty big priority for us.  I know a number of other schools
> are more interested in export than import, so I wanted to give a
> glimpse of what's coming.
>
> The approach we're taking has the side benefit that we get export
> almost for free.  Today I took a few minutes to write about 30 lines
> of code to retrieve a bunch of entities from the database and output
> them to the log in the XML format we will be using for import.  The
> results are here:
>
> http://www-personal.umich.edu/~jimeng/eval_import/first_sample_output.xml
>
> These are just toy examples, and there are no EvalResponse entities
> because I haven't quite gotten there yet.  But my point is that export
> is going to be very easy once we get import working.  I am writing a
> little servlet to import via HTTP POST of a file containing an XML
> document in this format.  Export would be handled with an HTTP GET to
> the same servlet with some request parameters to indicate what to
> export. We don't expect to use the export very much at Michigan, so
> we'd be glad to get some help spec'ing out how that should work.
>
> This format is quite a bit different from the last version of the
> schema you saw.  There will be a new version of the schema in a week
> or so to reflect these changes.
>
> The output example above includes some internal fields (like
> "lastModified" and "id") that would not be present in imports. We'll
> probably be able suppress any of those that are not useful in the
> export.
>
> Thanks.
>
> Jim
>
>
>
> _______________________________________________
> evaluation mailing list
> evaluation at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>
> TO UNSUBSCRIBE: send email to evaluation-unsubscribe at collab.sakaiproject.org 
>  with a subject of "unsubscribe"
>
>
>
>



More information about the evaluation mailing list