[Contrib: Evaluation System] provider not being called
Jim Eng
jimeng at umich.edu
Wed Sep 29 07:39:19 PDT 2010
Aaron Z,
I asked some questions yesterday, and I think most of us still don't have a good overtview of how evalsys uses the group provider to populate its user tables. Here are my questions from yesterdasy, but mostly could you just give a little narrative about how this works?
Could you describe in a little more detail about how those tables get populated? Is there something in evalsys that makes calls to the provider periodically, gets info from there and updates the info in those tables? Or is the provider itself responsible for adding data to the new tables? If this sync-up is a timed operation within evalsys, is the timing configurable?
Thanks.
Jim
On Sep 29, 2010, at 10:11 AM, Aaron Zeckoski wrote:
> Chuck,
> You are definitely welcome to create a branch and then do anything you
> like in that branch. This is community open source after all and we
> would welcome some contributions from rutgers to this project. As you
> are new to the project, the way we tend to handle contributions is via
> patches to JIRA issues first and then later if you are interested and
> demonstrate ability, then we can discuss commit privileges.
>
> As for the changes you mention below, this is my understanding of the
> current situation:
> 1) We will not be returning to direct use of providers, as I explained
> before, this change was made to address performance issues (mostly
> around calculating percentages and completion numbers)
> 2) The snapshots are an important part of the performance improvements
> that were made, it is quite possible to update the data and in fact,
> this happens each time the assignments are modified for an evaluation,
> you probably just need to add a way to force the update to happen via
> REST or something
> The snapshots also provide a way to ensure the data which is needed to
> construct an evaluation, does not depend on external data which may
> change or simply become unavailable
> 3) We will not be removing hibernate usage as that change is simply
> too far reaching, though I would certainly like to know what
> performance issues you are seeing and where the slowdowns occur (which
> methods for example), it may be possible to address them
>
> Hope this helps
> -AZ
>
>
> On Wed, Sep 29, 2010 at 8:48 AM, Charles Hedrick <hedrick at rutgers.edu> wrote:
>> We're going to do further testing to see whether we have a problem. I'm the performance guy, but I'm not an expert on this particular application. I saw things missing that I expected to see, but this may just have been due to the state those particular assessments were in. Now that I know what's going on, we can fix any problems with the providers.
>>
>> However there are two things that I consider disabling. We've probably going to go back to 1.2 for this semester and work on 1.3 with a target of next semester:
>>
>> * We consider the use of snapshots as a problem. I see your issue, but from the user point of view it's broken the application. If you continue using a snapshot we're going to need a way to update it.
>>
>> * I'm bothered by the extreme slowness of "my surveys." It looks like the code that deals with the snapshot is significantly slower than our provider code. We've haven't done enough testing to know whether that page is the only place with this problem.
>>
>> I can fix both of these things locally, but I'd rather not do it with only 3 weeks left until we need to start using evaluations. I'd also like to agree on a design with others in the community if possible. My suggestions are:
>>
>> * If possible return to using the providers.
>> * If this isn't possible, provide a way to do a resync. Although we'd do it in the early morning, performance would still be a concern.
>> * If I have to fix "my surveys" I'll probably do it by rewriting the database access in the main loop. My first thought is to bypass genericDAO and hibernate, but it may be that a more specific query through hibernate would work. I suspect the whole loop could be done with a single query, which would probably make the hibernate overhead insignificant (assuming it can be made to issue the right query).
>>
>>
>> On Sep 28, 2010, at 10:02 AM, Aaron Zeckoski wrote:
>>
>>> Jim,
>>> That is correct. The group provider is only consulted when setting up
>>> the evals or updating them. Otherwise, the internal DB groups records
>>> are used. This was done to remove the bottlenecks and performance
>>> issues related to providers which were not written as efficiently. It
>>> also simplified a lot of the logic by removing complex merges of data
>>> from various sources.
>>>
>>> For the rutgers situation, it may be beneficial to examine the groups
>>> tables and see if data is missing. If the groups tables have the data
>>> but the users still are unable to access things then there might be
>>> something wrong.
>>>
>>> -AZ
>>>
>>>
>>> On Tue, Sep 28, 2010 at 8:40 AM, Jim Eng <jimeng at umich.edu> wrote:
>>>> I am following this discussion closely because we are hoping to move to a
>>>> newer version of evalsys. It seems to me that Aaron Z told us a year or so
>>>> ago that the way the group provider works had changed at some point.
>>>> If I recall correctly, in early versions of evalsys, the provider was
>>>> consulted frequently while handling user requests. In trunk (or 1.3.x), I
>>>> think he said that the group information is stored internally and the group
>>>> provider is consulted occasionally to update it. That was done to
>>>> streamline queries that matched users with evals by group membership, IIRC.
>>>>
>>>> Is this a correct description? If so, your question might be why the group
>>>> info is not being persisted adequately within sakai, rather than why the
>>>> group provider is not being consulted in handling particular requests.
>>>> Jim
>>>>
>>>>
>>>> On Sep 28, 2010, at 9:21 AM, Aaron Watters wrote:
>>>>
>>>> Hi Richard,
>>>>
>>>> I work with Chuck at Rutgers. I will try to answer your questions and Chuck
>>>> can
>>>> correct me if he feels the need.
>>>>
>>>> On Mon, Sep 27, 2010 at 4:20 PM, Richard C. Moyer II <rmoyer at umd.edu> wrote:
>>>>>
>>>>> Charles,
>>>>>
>>>>> Here at Maryland we also use the eval group provider, but
>>>>> unfortunately, I cannot reproduce your error. Currently I'm in the middle of
>>>>> testing evaluation trunk, revision 70243, against our config and providers
>>>>> with the hope of migrating to sakai 2.7.
>>>>>
>>>>> To help trouble shoot your situation, we'll need a little more
>>>>> details.
>>>>>
>>>>> 1. 'We're not seeing people in our evaluations' - are you signing on as a
>>>>> student and there are no evaluations or are signing on as an instructor or
>>>>> admin?
>>>>>
>>>>
>>>> We tried both. The students see no evaluations and the admins see 0
>>>> students in the evaluations.
>>>>
>>>>> 2. 'Our evaluations come entirely from the Rutgers provider.' - are you
>>>>> building your evaluations outside of the sakai evaluation tool and importing
>>>>> in or are you building the evaluation using this tool and your groups,
>>>>> students, instructors, and TA come from your provider?
>>>>>
>>>>
>>>> We import the evaluation data directly into the database.
>>>>
>>>>>
>>>>> 3. 'old code' do you have a date or revision number?
>>>>>
>>>> I believe it was the 1.2 stable release.
>>>>
>>>>>
>>>>> 4. 'new code' do you have a date or revision number?
>>>>>
>>>>
>>>> 1.3 RC 2.
>>>>
>>>>>
>>>>> 5. what are the results from the Test EvalGroupProvider from the
>>>>> Administrate screen? As an admin, student & groupId.
>>>>> Are you getting back Rutger's IDs or sakai IDs?
>>>>>
>>>>
>>>> "Unexpected error" with the following (abbreviated) traceback in
>>>> catalina.out
>>>>
>>>> 2010-09-28 09:05:03,200 WARN (RenderHandlerBracketer.java:107) - <Exception
>>>> rendering view: >
>>>> java.lang.IllegalArgumentException: Cannot add leaf component with ID
>>>> summary-link of class uk.org.ponder.rsf.components.UIInternalLink as a child
>>>> of component with ID viewroot of class uk.org.ponder.rsf.view.ViewRoot
>>>> since it would displace an existing child of the same name.
>>>> Please remove the existing component first.
>>>> at
>>>> uk.org.ponder.rsf.components.UIContainer.addComponent(UIContainer.java:142)
>>>> at
>>>> uk.org.ponder.rsf.components.UIInternalLink.make(UIInternalLink.java:40)
>>>> at
>>>> org.sakaiproject.evaluation.tool.producers.AdminTestEGProviderProducer.fillComponents(AdminTestEGProviderProducer.java:133)
>>>> at
>>>> uk.org.ponder.rsf.view.support.ViewCollector.fillComponents(ViewCollector.java:56)....
>>>>
>>>>
>>>>>
>>>>> 6. are you using a UserDirectoryProvider?
>>>>
>>>> It doesn't look like it. Should we be?
>>>> The Rutgers interface implements the following signature
>>>>
>>>> public class RutgersEvalGroupsProvider implements
>>>> org.sakaiproject.evaluation.providers.EvalGroupsProvider {...}
>>>>
>>>>>
>>>>> The summaryProducer, and supporting code has gone through several
>>>>> enhancements and re-writes over the years. It's possible that depending on
>>>>> how old your code is that your caught in a void that has since been fixed.
>>>>>
>>>>> Please let us know this info and perhaps we can dig a little deeper into
>>>>> the code.
>>>>
>>>> Thanks for your attention. -- Aaron Watters (aaron at rutgers.edu)
>>>>
>>>> _______________________________________________
>>>> evaluation mailing list
>>>> evaluation at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>>>
>>>> TO UNSUBSCRIBE: send email to evaluation-unsubscribe at collab.sakaiproject.org
>>>> with a subject of "unsubscribe"
>>>>
>>>> _______________________________________________
>>>> evaluation mailing list
>>>> evaluation at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>>>
>>>> TO UNSUBSCRIBE: send email to evaluation-unsubscribe at collab.sakaiproject.org
>>>> with a subject of "unsubscribe"
>>>>
>>>
>>>
>>>
>>> --
>>> Aaron Zeckoski - Software Engineer - http://tinyurl.com/azprofile
>>> _______________________________________________
>>> evaluation mailing list
>>> evaluation at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>>
>>> TO UNSUBSCRIBE: send email to evaluation-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>>
>>
>
>
>
> --
> Aaron Zeckoski - Software Engineer - http://tinyurl.com/azprofile
>
>
More information about the evaluation
mailing list