[Portfolio] Slow-downs

Tue May 19 12:37:00 PDT 2009

Yes, this is correct. Our CONTENT_RESOURCE table has the  
RESOURCE_TYPE_ID column populated.

There are two kinds of content conversion: 1) breaking various  
attributes out into columns and populating them, and 2) converting  
the XML column to the BINARY_ENTITY serialization. If I recall  
correctly, there were batch and online modes for some of this.

All of the details are not clear to me including what happens  
automatically, what is required to run on 2.5 and beyond, and the  
impacts on the form searching in a 2.6.x environment  
(getResourcesOfType vs. findResources) "without conversion" (if  
that's possible).

Thanks,
-Noah

On May 19, 2009, at 1:59 PM, Maurer, Christopher Wayne wrote:

> Noah,
> You guys have run the content conversion already, right?  So all  
> your forms should have the type pulled out into it’s own column?   
> Well, not the form type, but the fact that the resource item is a  
> form?
>
> Chris
>
>
> On 5/19/09 1:56 PM, "Noah Botimer" <botimer at umich.edu> wrote:
>
> Marc,
>
> You've done a masterful job. I will only add that we are looking  
> into this issue very seriously at Michigan. It's somewhat likely  
> that we will end up engineering a new form finding mechanism for  
> our Fall term. We will certainly keep the group posted.
>
> Thanks,
> -Noah
>
> On May 19, 2009, at 12:37 PM, Marc Zaldivar wrote:
>
> Sean,
>
>  Teggin said that yesterday in the call you were discussing the  
> “form searching slow-down” issue.  Yes, VT hit this one pretty hard  
> this semester, and I know Michigan has also been faced with a  
> growing issue from this.  We spent a couple of hours on the phone  
> with Noah talking over some solutions, and though I didn’t talk to  
> him, I know our dev group also talked with Chuck Hedrick too.
>
>  The breakdown of the issue came for us because we are using so  
> many of the “UROP”-type portfolios for our templates, which are  
> basically using form pages for each tab of a final display  
> portfolio, in our case sometimes as many as 17 or 18 different  
> forms for each complete portfolio (sometimes they were limitlessly  
> repeatable, so some students may have had upwards of 30 forms to  
> put into different slots in the template).  Each time the “Save  
> Changes” button is hit, it would search the database 5 times per  
> slot per form instance, making that sometimes 1000s of searches per  
> creation attempt.  When you do this in a demonstration with 25-30  
> kids in the room (ok, now everyone push “Save Changes...”), it’s  
> fatal.  Even without that, given the activity ramp-up we are  
> seeing, we were seeing an average of 60+ seconds for new users, and  
> for folks like me that own a lot of forms, it was often beyond our  
> threshold for database link-up at 10 minutes or something  
> ridiculous.  So, big big problem that is only going to get bigger  
> as we expand.  (We had at that time about 4k forms out of 250k  
> items in the database.)
>
>  Ultimately, we tested several solutions, with the big two being  
> (1) a patch that Noah/rSmart came up with that (I think) refined  
> the search parameters without limiting it to My Workspace, creating  
> a cache that was searched repeatedly rather than having to keep  
> hitting the database time and again, and (2) Chuck Hedrick’s  
> solution to limit all searching to the My Workspace first.   
> Ultimately, #2 was faster on our test speeds.  We felt that maybe  
> #1 would be a better long-term solution, but that it would take  
> some real time to refine that search protocol and build the cache  
> effectively.  Hopefully, we’ll still work toward that ultimate  
> solution.  However, we deployed #2 and saw my admin account’s wait  
> time drop from 10+minutes to about 18 seconds.  Average users are  
> now 2-5 seconds tops.  Much, much better so far.
>
>  Do you see any potential for problems if presentations are being  
> created from forms created through a matrix?  Mostly, I’d assume  
> that these forms would be located in the various Portfolio  
> Interaction folders, in the MW.  The one potential is for non- 
> matrix portfolios, you’d have to specify the need to store forms in  
> a folder in the MW rather than in a course drop box or resource area.
>
>  Just thought I’d share our perspective on the issue.  Noah, please  
> feel free to detail out my all to0 non-tech-savvy description of  
> your patch and what it does!
>
>  Marc Z
>  --
>  Marc Zaldivar,  
> Ph.D.                                                                  
> 2210A Torgersen Hall
>  Director, Virginia Tech Electronic Portfolio Initiatives         
> Blacksburg, VA 24061-0292
>  Learning  
> Technologies                                                           
>    540.231.8994
>  Virginia  
> Tech                                                                   
>             marcz at vt.edu
>
>
>
> _______________________________________________
> portfolio mailing list
> portfolio at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/portfolio
>
> TO UNSUBSCRIBE: send email to portfolio- 
> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/portfolio/attachments/20090519/6de00064/attachment.html 