botimer at umich.edu
Tue May 18 13:19:40 PDT 2010
You may be interested in http://jira.sakaiproject.org/browse/SAK-18459.
It is now merged to 2.7, so you should see some better performance in things like viewing a matrix for a given user and viewing a given student form as an instructor.
I am also looking at employing http://jira.sakaiproject.org/browse/SAK-18458 in portfolio viewing. It offers a signature to eliminate an unused locking check that causes enormous synchronization delays. On a dense portfolio or under concurrency, this could be causing significant trouble. It was enough to cripple a report we need to run (taking ~2.5 mins for 600 forms on high-end hardware).
For the record, we found these by profiling with YourKit. We had to find the scaling variables (number of sites for an instructor viewing a form and number of forms when reporting) and load up enough dummy data to cause the issues.
I don't mean to ignore your larger concern; just offering some very recent, significant improvement that may have slipped your radar.
To your point of the difficulty tracking and resolving these, they took upwards of a week to find, fix, test, and commit to trunk/kernel/2.7, requisite expertise and tools in hand. It is certainly difficult to fly in and pick this kind of thing up.
On May 18, 2010, at 12:14 PM, Charles Hedrick wrote:
>> Charles...and others...is our pace on things like OSP performance and functionality mostly a resource gap? A skills gap in sufficient levels of the right skills on the project? A functional requirements gap in clarity of what needs to be done next?
> To be fair, it's a problem with the initial implementation of OSP. The current OSP developers, who as far as I can tell are not the people who created the problems, are trying to simultaneously add the functionality that their institutions need, work on a very clunky UI, and work on performance. There probably isn't enough manpower assigned to OSP to do all of that well.
> We have a couple of developers, but not enough manpower to do sustained work on OSP, although I do things as necessary. Unfortunately there's a sharp learning curve to deal with the kinds of issues I'm concerned with.
> From a technical point of view, we're seeing a typical problem with hibernate: people do implementations, and stop when they work, rather than investigating the actual database queries being made, and considering whether they will work out when the application is used with a lot of users active. I realize that hibernate can perform quite reasonably, but it requires either serious expertise in hibernate, or careful investigate of the actual queries generated. Of course any problem don't show up for real until 2 years later, and at that point a different set of developers have to deal with it.
> This problem almost killed Samigo in the early days. It required a significant commitment to performance work. We came very close to abandoning Sakai because of those problems.
> portfolio mailing list
> portfolio at collab.sakaiproject.org
> TO UNSUBSCRIBE: send email to portfolio-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
More information about the portfolio