[Contrib: Evaluation System] provider not being called

Aaron Zeckoski azeckoski at unicon.net
Wed Sep 29 07:53:54 PDT 2010 

In short, the provider is consulted when the evaluation is created
(or, more accurately, when the assignments are adjusted). First, the
provider will determine which groups are viable to be selected. Then
the provider answers the questions about who is allowed to take the
eval and who is being evaluated in those groups. Both of these types
of data are mapped into the tables for the evaluation. The groups data
was always mapped into the tables since version 0.1 (maybe 5 years
ago). The memberships change is fairly recent by comparison (as in, it
happened about 2 years ago).

Providers don't do anything with tables. They only are able to answer
questions and do not really know anything about the internals of the
system (nor should they). It is not timed. It is action driven. There
is no syncing unless the assignments are changed.

My memory on the precise reasoning behind these changes is somewhat
fuzzy at this point since it has been quite awhile since the changes
were made but I do recall it was related to performance and relevance
of the data. I think UCT was at least partially involved so Marquard
might have some comments.

-AZ


On Wed, Sep 29, 2010 at 9:39 AM, Jim Eng <jimeng at umich.edu> wrote:
> Aaron Z,
>
> I asked some questions yesterday, and I think most of us still don't have a good overtview of how evalsys uses the group provider to populate its user tables.  Here are my questions from yesterdasy, but mostly could you just give a little narrative about how this works?
>
> Could you describe in a little more detail about how those tables get populated?  Is there something in evalsys that makes calls to the provider periodically, gets info from there and updates the info in those tables?  Or is the provider itself responsible for adding data to the new tables?  If this sync-up is a timed operation within evalsys, is the timing configurable?
>
> Thanks.
>
> Jim
>
>
> On Sep 29, 2010, at 10:11 AM, Aaron Zeckoski wrote:
>
>> Chuck,
>> You are definitely welcome to create a branch and then do anything you
>> like in that branch. This is community open source after all and we
>> would welcome some contributions from rutgers to this project. As you
>> are new to the project, the way we tend to handle contributions is via
>> patches to JIRA issues first and then later if you are interested and
>> demonstrate ability, then we can discuss commit privileges.
>>
>> As for the changes you mention below, this is my understanding of the
>> current situation:
>> 1) We will not be returning to direct use of providers, as I explained
>> before, this change was made to address performance issues (mostly
>> around calculating percentages and completion numbers)
>> 2) The snapshots are an important part of the performance improvements
>> that were made, it is quite possible to update the data and in fact,
>> this happens each time the assignments are modified for an evaluation,
>> you probably just need to add a way to force the update to happen via
>> REST or something
>> The snapshots also provide a way to ensure the data which is needed to
>> construct an evaluation, does not depend on external data which may
>> change or simply become unavailable
>> 3) We will not be removing hibernate usage as that change is simply
>> too far reaching, though I would certainly like to know what
>> performance issues you are seeing and where the slowdowns occur (which
>> methods for example), it may be possible to address them
>>
>> Hope this helps
>> -AZ
>>
>>
>> On Wed, Sep 29, 2010 at 8:48 AM, Charles Hedrick <hedrick at rutgers.edu> wrote:
>>> We're going to do further testing to see whether we have a problem. I'm the performance guy, but I'm not an expert on this particular application. I saw things missing that I expected to see, but this may just have been due to the state those particular assessments were in. Now that I know what's going on, we can fix any problems with the providers.
>>>
>>> However there are two things that I consider disabling. We've probably going to go back to 1.2 for this semester and work on 1.3 with a target of next semester:
>>>
>>> * We consider the use of snapshots as a problem. I see your issue, but from the user point of view it's broken the application. If you continue using a snapshot we're going to need a way to update it.
>>>
>>> * I'm bothered by the extreme slowness of "my surveys." It looks like the code that deals with the snapshot is significantly slower than our provider code. We've haven't done enough testing to know whether that page is the only place with this problem.
>>>
>>> I can fix both of these things locally, but I'd rather not do it with only 3 weeks left until we need to start using evaluations. I'd also like to agree on a design with others in the community if possible. My suggestions are:
>>>
>>> * If possible return to using the providers.
>>> * If this isn't possible, provide a way to do a resync. Although we'd do it in the early morning, performance would still be a concern.
>>> * If I have to fix "my surveys" I'll probably do it by rewriting the database access in the main loop. My first thought is to bypass genericDAO and hibernate, but it may be that a more specific query through hibernate would work. I suspect the whole loop could be done with a single query, which would probably make the hibernate overhead insignificant (assuming it can be made to issue the right query).
>>>
>>>
>>> On Sep 28, 2010, at 10:02 AM, Aaron Zeckoski wrote:
>>>
>>>> Jim,
>>>> That is correct. The group provider is only consulted when setting up
>>>> the evals or updating them. Otherwise, the internal DB groups records
>>>> are used. This was done to remove the bottlenecks and performance
>>>> issues related to providers which were not written as efficiently. It
>>>> also simplified a lot of the logic by removing complex merges of data
>>>> from various sources.
>>>>
>>>> For the rutgers situation, it may be beneficial to examine the groups
>>>> tables and see if data is missing. If the groups tables have the data
>>>> but the users still are unable to access things then there might be
>>>> something wrong.
>>>>
>>>> -AZ
>>>>
>>>>
>>>> On Tue, Sep 28, 2010 at 8:40 AM, Jim Eng <jimeng at umich.edu> wrote:
>>>>> I am following this discussion closely because we are hoping to move to a
>>>>> newer version of evalsys.  It seems to me that Aaron Z told us a year or so
>>>>> ago that the way the group provider works had changed at some point.
>>>>> If I recall correctly, in early versions of evalsys, the provider was
>>>>> consulted frequently while handling user requests.  In trunk (or 1.3.x), I
>>>>> think he said that the group information is stored internally and the group
>>>>> provider is consulted occasionally to update it.  That was done to
>>>>> streamline queries that matched users with evals by group membership, IIRC.
>>>>>
>>>>> Is this a correct description?  If so, your question might be why the group
>>>>> info is not being persisted adequately within sakai, rather than why the
>>>>> group provider is not being consulted in handling particular requests.
>>>>> Jim
>>>>>
>>>>>
>>>>> On Sep 28, 2010, at 9:21 AM, Aaron Watters wrote:
>>>>>
>>>>> Hi Richard,
>>>>>
>>>>> I work with Chuck at Rutgers.  I will try to answer your questions and Chuck
>>>>> can
>>>>> correct me if he feels the need.
>>>>>
>>>>> On Mon, Sep 27, 2010 at 4:20 PM, Richard C. Moyer II <rmoyer at umd.edu> wrote:
>>>>>>
>>>>>> Charles,
>>>>>>
>>>>>>        Here at Maryland we also use the eval group provider, but
>>>>>> unfortunately, I cannot reproduce your error. Currently I'm in the middle of
>>>>>> testing evaluation trunk, revision 70243, against our config and providers
>>>>>> with the hope of migrating to sakai 2.7.
>>>>>>
>>>>>>        To help trouble shoot your situation, we'll need a little more
>>>>>> details.
>>>>>>
>>>>>> 1. 'We're not seeing people in our evaluations' - are you signing on as a
>>>>>> student and there are no evaluations or are signing on as an instructor or
>>>>>> admin?
>>>>>>
>>>>>
>>>>> We tried both.  The students see no evaluations and the admins see 0
>>>>> students in the evaluations.
>>>>>
>>>>>> 2. 'Our evaluations come entirely from the Rutgers provider.' - are you
>>>>>> building your evaluations outside of the sakai evaluation tool and importing
>>>>>> in or are you building the evaluation using this tool and your groups,
>>>>>> students, instructors, and TA come from your provider?
>>>>>>
>>>>>
>>>>> We import the evaluation data directly into the database.
>>>>>
>>>>>>
>>>>>> 3. 'old code' do you have a date or revision number?
>>>>>>
>>>>> I believe it was the 1.2 stable release.
>>>>>
>>>>>>
>>>>>> 4. 'new code' do you have a date or revision number?
>>>>>>
>>>>>
>>>>> 1.3 RC 2.
>>>>>
>>>>>>
>>>>>> 5. what are the results from the Test EvalGroupProvider from the
>>>>>> Administrate screen? As an admin, student & groupId.
>>>>>> Are you getting  back Rutger's IDs or sakai IDs?
>>>>>>
>>>>>
>>>>> "Unexpected error" with the following (abbreviated) traceback in
>>>>> catalina.out
>>>>>
>>>>> 2010-09-28 09:05:03,200 WARN (RenderHandlerBracketer.java:107) - <Exception
>>>>> rendering view: >
>>>>> java.lang.IllegalArgumentException: Cannot add leaf component with ID
>>>>> summary-link of class uk.org.ponder.rsf.components.UIInternalLink as a child
>>>>> of component with ID   viewroot   of class uk.org.ponder.rsf.view.ViewRoot
>>>>> since it would displace an existing child of the same name.
>>>>>    Please remove the existing component first.
>>>>>         at
>>>>> uk.org.ponder.rsf.components.UIContainer.addComponent(UIContainer.java:142)
>>>>>         at
>>>>> uk.org.ponder.rsf.components.UIInternalLink.make(UIInternalLink.java:40)
>>>>>         at
>>>>> org.sakaiproject.evaluation.tool.producers.AdminTestEGProviderProducer.fillComponents(AdminTestEGProviderProducer.java:133)
>>>>>         at
>>>>> uk.org.ponder.rsf.view.support.ViewCollector.fillComponents(ViewCollector.java:56)....
>>>>>
>>>>>
>>>>>>
>>>>>> 6. are you using a UserDirectoryProvider?
>>>>>
>>>>> It doesn't look like it.  Should we be?
>>>>> The Rutgers interface implements the following signature
>>>>>
>>>>> public class RutgersEvalGroupsProvider implements
>>>>> org.sakaiproject.evaluation.providers.EvalGroupsProvider {...}
>>>>>
>>>>>>
>>>>>> The summaryProducer, and supporting code has gone through several
>>>>>> enhancements and re-writes over the years. It's possible that depending on
>>>>>> how old your code is that your caught in a void that has since been fixed.
>>>>>>
>>>>>> Please let us know this info and perhaps we can dig a little deeper into
>>>>>> the code.
>>>>>
>>>>> Thanks for your attention.  -- Aaron Watters (aaron at rutgers.edu)
>>>>>
>>>>> _______________________________________________
>>>>> evaluation mailing list
>>>>> evaluation at collab.sakaiproject.org
>>>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>>>>
>>>>> TO UNSUBSCRIBE: send email to evaluation-unsubscribe at collab.sakaiproject.org
>>>>> with a subject of "unsubscribe"
>>>>>
>>>>> _______________________________________________
>>>>> evaluation mailing list
>>>>> evaluation at collab.sakaiproject.org
>>>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>>>>
>>>>> TO UNSUBSCRIBE: send email to evaluation-unsubscribe at collab.sakaiproject.org
>>>>> with a subject of "unsubscribe"
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Aaron Zeckoski - Software Engineer - http://tinyurl.com/azprofile
>>>> _______________________________________________
>>>> evaluation mailing list
>>>> evaluation at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>>>
>>>> TO UNSUBSCRIBE: send email to evaluation-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>>>
>>>
>>
>>
>>
>> --
>> Aaron Zeckoski - Software Engineer - http://tinyurl.com/azprofile
>>
>>
>
>



-- 
Aaron Zeckoski - Software Engineer - http://tinyurl.com/azprofile

More information about the evaluation mailing list