[Building Sakai] Production issue with Lesson Builder

Mon Jan 13 09:25:29 PST 2014

Our testing didn't show that LSNBLDR-304<https://jira.sakaiproject.org/browse/LSNBLDR-304> was the solution.

And thanks Bryan.    We also tracked down the faculty member to ask them to stop.  Thank goodness they were a good sport about it!     When did Longsight introduce the code (and have any schools been through the first week or so?

Megan

From: sakai-dev-bounces at collab.sakaiproject.org [mailto:sakai-dev-bounces at collab.sakaiproject.org] On Behalf Of Neal Caidin
Sent: Monday, January 13, 2014 10:51 AM
To: Kirschner, Beth
Cc: sakai-dev (sakai-dev at collab.sakaiproject.org)
Subject: Re: [Building Sakai] Production issue with Lesson Builder

Is it related to this one

https://jira.sakaiproject.org/browse/LSNBLDR-304   ?

-- Neal

[cid:image001.jpg at 01CF105A.8D6FA900]
Kirschner, Beth<mailto:bkirschn at umich.edu>
January 13, 2014 at 10:45 AM
Is this written up in JIRA anywhere?

- Beth

_______________________________________________
sakai-dev mailing list
sakai-dev at collab.sakaiproject.org<mailto:sakai-dev at collab.sakaiproject.org>
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev

TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org<mailto:sakai-dev-unsubscribe at collab.sakaiproject.org> with a subject of "unsubscribe"
[cid:image002.jpg at 01CF105A.8D6FA900]
Bryan Holladay<mailto:holladay at longsight.com>
January 13, 2014 at 10:39 AM
>Does that help with the excessive number of items?

Yes and no.  It prevents the excessive number of orphaned pages by the instructor re-importing and deleting the half imported material.  I've seen cases where it takes 10+ minutes for the import to complete, which is why instructor's keep trying the import and causing millions of pages to be created.  We actually had to track the instructor down and tell them to stop importing until we get the fix in since every time we restarted the instructor went back in to re-import.  However, this patch doesn't address the issue inside of LB's import code.  So I'm sure there is room for more improvement, but it seemed like it was the combination of multiple imports + non-performant import code.  The good/bad thing about this issue is that it only comes up in the first week or so of the semester.  So it won't be an issue again until next semester and any fix you add you won't really be certain until then as well.  I'm not going to jinx us by saying everything is going fine so far with that patch with this start of this semester but you can infer that if you wish ;)

Thanks,
Bryan

_______________________________________________
sakai-dev mailing list
sakai-dev at collab.sakaiproject.org<mailto:sakai-dev at collab.sakaiproject.org>
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev

TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org<mailto:sakai-dev-unsubscribe at collab.sakaiproject.org> with a subject of "unsubscribe"
[cid:image002.jpg at 01CF105A.8D6FA900]
May, Megan Marie<mailto:mmmay at iu.edu>
January 13, 2014 at 9:16 AM
Thanks Bryan!     We started looking at that patch last night.

Does that help with the excessive number of items?  Even with 1 import we've seen a high number of duplicate entries (over 1000).
Megan

From: Bryan Holladay [mailto:holladay at longsight.com]
Sent: Monday, January 13, 2014 8:38 AM
To: May, Megan Marie
Cc: sakai-dev (sakai-dev at collab.sakaiproject.org<mailto:sakai-dev at collab.sakaiproject.org>)
Subject: Re: [Building Sakai] Production issue with Lesson Builder

My first assumption would be that it is from importing previous course LB's into the new courses.  We've seen a lot of issues with performance, excessive number of items and server stress with this process.  We created/provided this jira to help alleviate some of the issue:

SAK-23897 Site Info -> Import from site needs to run in a seperate thread and email user when finished

We were seeing instructors importing several times b/c it would either time out or they'd give up on the process and try again (or click several times).  If I wanted to bring down a Sakai server, that is how I would do it (site import with LB)

-Bryan

On Sun, Jan 12, 2014 at 1:22 PM, May, Megan Marie <mmmay at iu.edu<mailto:mmmay at iu.edu>> wrote:
On Friday the following SQL began running for excessive periods of time

update lesson_builder_items set pageId=:1 , sequence=:2 , type=:3 , sakaiId=:4 , name=:5 , html=:6 , description=:7 , height=:8 , width=:9 , alt=:10 , nextPage=:11 , format=:12 , required=:13 , alternate=:14 , prerequisite=:15 , subrequirement=:16 , requirementText=:17 , sameWindow=:18 , groups=:19 , anonymous=:20 , showComments=:21 , forcedCommentsAnonymous=:22 , gradebookId=:23 , gradebookPoints=:24 , gradebookTitle=:25 , altGradebook=:26 , altPoints=:27 , altGradebookTitle=:28 where id=:29

As a stop gap we've bounced the app servers these are occurring on.    We've noted HUGE increases in the number of rows in LESSON_BUILDER_ITEMS  & the increases pertain to  rows in lesson_builder_items show a SAKAIID of "/dummy"  (Import issues?)   The behavior seems like a loop - and we've looked at LSNBLDR-304 although the cause of it is ambiguous so we're hesitant to move forward with it.

Has anyone encountered this?

Megan

Megan May
Manager, Learning Management Systems / Kuali Student Development
UITS Enterprise Student Systems
Enterprise Software Division
Indiana University
Office:  317-274-4528<tel:317-274-4528>

_______________________________________________
sakai-dev mailing list
sakai-dev at collab.sakaiproject.org<mailto:sakai-dev at collab.sakaiproject.org>
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev

TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org<mailto:sakai-dev-unsubscribe at collab.sakaiproject.org> with a subject of "unsubscribe"

_______________________________________________
sakai-dev mailing list
sakai-dev at collab.sakaiproject.org<mailto:sakai-dev at collab.sakaiproject.org>
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev

TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org<mailto:sakai-dev-unsubscribe at collab.sakaiproject.org> with a subject of "unsubscribe"
[cid:image002.jpg at 01CF105A.8D6FA900]
Bryan Holladay<mailto:holladay at longsight.com>
January 13, 2014 at 8:37 AM
My first assumption would be that it is from importing previous course LB's into the new courses.  We've seen a lot of issues with performance, excessive number of items and server stress with this process.  We created/provided this jira to help alleviate some of the issue:

SAK-23897 Site Info -> Import from site needs to run in a seperate thread and email user when finished

We were seeing instructors importing several times b/c it would either time out or they'd give up on the process and try again (or click several times).  If I wanted to bring down a Sakai server, that is how I would do it (site import with LB)

-Bryan

_______________________________________________
sakai-dev mailing list
sakai-dev at collab.sakaiproject.org<mailto:sakai-dev at collab.sakaiproject.org>
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev

TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org<mailto:sakai-dev-unsubscribe at collab.sakaiproject.org> with a subject of "unsubscribe"
[cid:image002.jpg at 01CF105A.8D6FA900]
May, Megan Marie<mailto:mmmay at iu.edu>
January 12, 2014 at 1:22 PM
On Friday the following SQL began running for excessive periods of time

update lesson_builder_items set pageId=:1 , sequence=:2 , type=:3 , sakaiId=:4 , name=:5 , html=:6 , description=:7 , height=:8 , width=:9 , alt=:10 , nextPage=:11 , format=:12 , required=:13 , alternate=:14 , prerequisite=:15 , subrequirement=:16 , requirementText=:17 , sameWindow=:18 , groups=:19 , anonymous=:20 , showComments=:21 , forcedCommentsAnonymous=:22 , gradebookId=:23 , gradebookPoints=:24 , gradebookTitle=:25 , altGradebook=:26 , altPoints=:27 , altGradebookTitle=:28 where id=:29

As a stop gap we've bounced the app servers these are occurring on.    We've noted HUGE increases in the number of rows in LESSON_BUILDER_ITEMS  & the increases pertain to  rows in lesson_builder_items show a SAKAIID of "/dummy"  (Import issues?)   The behavior seems like a loop - and we've looked at LSNBLDR-304 although the cause of it is ambiguous so we're hesitant to move forward with it.

Has anyone encountered this?

Megan

Megan May
Manager, Learning Management Systems / Kuali Student Development
UITS Enterprise Student Systems
Enterprise Software Division
Indiana University
Office:  317-274-4528

_______________________________________________
sakai-dev mailing list
sakai-dev at collab.sakaiproject.org<mailto:sakai-dev at collab.sakaiproject.org>
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev

TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org<mailto:sakai-dev-unsubscribe at collab.sakaiproject.org> with a subject of "unsubscribe"

--
Neal Caidin
Sakai Community Coordinator
Apereo Foundation
neal.caidin at apereo.org<mailto:neal.caidin at apereo.org>
Skype me! (but let me know in advance for the first interaction) - nealkdin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20140113/fbe6644c/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 1196 bytes
Desc: image001.jpg
Url : http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20140113/fbe6644c/attachment.jpg 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 770 bytes
Desc: image002.jpg
Url : http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20140113/fbe6644c/attachment-0001.jpg 