[Contrib: Evaluation System] consolidated email woes

Thu Jan 31 11:50:50 PST 2013

I would agree that there was a gap in testing & QA that missed the problem with MySql. Jim has checked in a fix for this in a branch for https://jira.sakaiproject.org/browse/EVALSYS-1352. UM is planning on load testing this fix in March and if it proves viable, we will merge to trunk and work with the community to test and tag another version of evaluations.

- Beth

On Jan 25, 2013, at 12:44 PM, Adam Marshall wrote:

> I would say that code shouldn't be committed if it doesn't work with both databases. As we use MySQL, we cannot move to this version of Evaluations.
> 
> Is there a plan to fix this?
> 
> adam
> 
> -----Original Message-----
> From: John Bush [mailto:john.bush at rsmart.com] 
> Sent: 25 January 2013 17:20
> To: Adam Marshall
> Cc: Jim Eng; evaluation at collab.sakaiproject.org
> Subject: Re: [Contrib: Evaluation System] consolidated email woes
> 
> Adam:  in my experience its not working for MySQL. I haven't tried it in Oracle.  We are on version 1.4 I believe.
> 
> On Fri, Jan 25, 2013 at 10:03 AM, Adam Marshall <adam.marshall at it.ox.ac.uk> wrote:
>> So the current state of play is that the feature works on both MySQL and Oracle but isn't optimal on either platform?
>> 
>> Adam
>> 
>> -----Original Message-----
>> From: Jim Eng [mailto:jimeng at umich.edu]
>> Sent: 25 January 2013 16:54
>> To: Adam Marshall
>> Cc: John Bush; evaluation at collab.sakaiproject.org
>> Subject: Re: [Contrib: Evaluation System] consolidated email woes
>> 
>> Just the consolidated email processing in particular versions of evalsys (not sure which ones, though).
>> 
>> Consolidated email processing could be very efficient if it selects all the information it needs to generate a single announcement or reminder email to each evaluator each day, instead of the usual process of generating an email to each evaluator for each evaluation. The real cost of the individual emails is retrieving data and construction java objects for each evaluator. Someone from Michigan had written code a few years ago that selected the needed info into a temporary table, but that work had problems that kept it from being merged to trunk of evalsys.  Later I worked on rewriting it to get it into trunk, but we found that the HQL that generated efficient queries in MySQL resulted in queries that were not at all efficient in Oracle. When the HQL was rewritten to perform well in Oracle, it didn't work in MySQL.  We tried various other approaches, but none of them worked for both MySQL and Oracle.  Last year, Aaron suggested rewriting it to assemble the info in memory ra
> ther than in temporary database tables. That works well for both MySQL and Oracle, though it is a bit less zippy than the other approach.
>> 
>> Jim
>> 
>> 
>> 
>> 
>> On Jan 25, 2013, at 11:01 AM, Adam Marshall wrote:
>> 
>>> Are you saying that some features of Evalsys only work with Oracle?
>>> 
>>> adam
>>> 
>>> -----Original Message-----
>>> From: evaluation-bounces at collab.sakaiproject.org
>>> [mailto:evaluation-bounces at collab.sakaiproject.org] On Behalf Of Jim 
>>> Eng
>>> Sent: 25 January 2013 14:57
>>> To: John Bush
>>> Cc: evaluation at collab.sakaiproject.org
>>> Subject: Re: [Contrib: Evaluation System] consolidated email woes
>>> 
>>> We made a couple attempts to get a version of that code that works for both Oracle and MySQL, and we eventually gave up trying to do that.  I'm not sure the state of the code in trunk at this point.  As I recall, the code in 1.4.x was working for Oracle but not MySQL, but it's *possible* that differences in the way people use evalsys could result in data differences that would cause it to fail for Oracle as well.
>>> 
>>> We have a new version of this code in a branch that assembles info to generate consolidated notifications in memory instead of in a database table.  That needs to be merged to trunk eventually.  But a few months ago it came out that some of the assumptions that were used to make that approach possible were incorrect. In particular, we had agreed a couple years ago that people would either use consolidated notifications or individual emails, but would not mix them on a single system.  It turns out, that is incorrect.  Daphne has proposed changes to email configuration that would make it possible to fix that.  It involves selecting an approach for each type of message (e.g. reminders, available-announcements, etc).  I think that needs to be fixed in conjunction with merging changes to the consolidated email processing code.
>>> 
>>> I will defer to Beth from there.  I'm no longer working on evalsys, and I'm not sure who from Michigan has been assigned to it.
>>> 
>>> Jim
>>> 
>>> 
>>> 
>>> On Jan 25, 2013, at 3:12 AM, John Bush wrote:
>>> 
>>>> Well when I manually put the template ids in there, things started 
>>>> working. Based on the hibernate query I see when I step through it, 
>>>> its seems like nulls are bad.
>>>> 
>>>> insert into EvalEmailProcessingData
>>>> (eauId,userId,groupId,emailTemplateId,evalId,evalDueDate) select 
>>>> user.id as eauId,user.userId as userId,user.evalGroupId as groupId, 
>>>> eval.availableEmailTemplate.id as emailTemplateId,eval.id as evalId, 
>>>> eval.dueDate as evalDueDate from EvalAssignUser as user inner join 
>>>> user.evaluation as eval where user.type = "evaluator" and 
>>>> eval.startDate <= current_timestamp() and user.completedDate is null 
>>>> and eval.availableEmailTemplate.type = "ConsolidatedAvailable" and 
>>>> user.availableEmailSent is null and user.reminderEmailSent is null
>>>> 
>>>> eval.availableEmailTemplate.type = "ConsolidatedAvailable" is not 
>>>> going to match b/c availableEmailTemplate is null.  Maybe someone UM 
>>>> could comment.  I'm guessing this is a front end problem, maybe when 
>>>> the consolidated options is selected its not somehow getting stored 
>>>> correctly.  Although I would expect more people to have the problem 
>>>> if that was the case.  But maybe not too many people use this feature.
>>>> 
>>>> On Thu, Jan 24, 2013 at 8:31 PM, Aaron Zeckoski <azeckoski at unicon.net> wrote:
>>>>> Null values in those evaluation fields should not be a problem. The 
>>>>> system should default over to using the default templates when 
>>>>> those are not set. I am not sure if the consolidated emails work 
>>>>> that way or not as that code is separate but someone from UM can 
>>>>> probably answer this.
>>>>> 
>>>>> -AZ
>>>>> 
>>>>> 
>>>>> On Thu, Jan 24, 2013 at 5:27 PM, John Bush <john.bush at rsmart.com> wrote:
>>>>>> we are running eval sys 1.4 and noticing evaluations are having 
>>>>>> trouble sending out consolidated emails.  I've traced the problem 
>>>>>> to the fact that the templates are not being sent for the evaluation.
>>>>>> The AVAILABLE_EMAIL_TEMPLATE_FK REMINDER_EMAIL_TEMPLATE_FK columns 
>>>>>> are NULL.  I see a few jira issue concerning some of this.  I'm 
>>>>>> wondering if anyone else has had this this problem and whether its 
>>>>>> been addressed.  We are using mysql.
>>>>>> 
>>>>>> Before I dig in too far looking to see if this is something that 
>>>>>> has already been fixed.
>>>>>> 
>>>>>> --
>>>>>> John Bush
>>>>>> 602-490-0470
>>>>>> _______________________________________________
>>>>>> evaluation mailing list
>>>>>> evaluation at collab.sakaiproject.org
>>>>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>>>>> 
>>>>>> TO UNSUBSCRIBE: send email to evaluation-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile
>>>> 
>>>> 
>>>> 
>>>> --
>>>> John Bush
>>>> 602-490-0470
>>>> _______________________________________________
>>>> evaluation mailing list
>>>> evaluation at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>>> 
>>>> TO UNSUBSCRIBE: send email to evaluation-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>>> 
>>> _______________________________________________
>>> evaluation mailing list
>>> evaluation at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>> 
>>> TO UNSUBSCRIBE: send email to evaluation-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>> 
> 
> 
> 
> --
> John Bush
> 602-490-0470
> _______________________________________________
> evaluation mailing list
> evaluation at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/evaluation
> 
> TO UNSUBSCRIBE: send email to evaluation-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"