[Contrib: Evaluation System] Errors/Failures in unit tests

Tue Jul 16 10:58:50 PDT 2013

Okay, thanks Aaron.

FYI, I just created https://jira.sakaiproject.org/browse/EVALSYS-1380

Brian Jones
Applications Development
Information Technology Services
Support Services Building, Room 4326
Western University
(519) 661-2111 x86969
bjones86 at uwo.ca

-----Original Message-----
From: azeckoski at gmail.com [mailto:azeckoski at gmail.com] On Behalf Of Aaron Zeckoski
Sent: Tuesday, July 16, 2013 1:54 PM
To: Brian Jones
Cc: Matthew Jones; Sakai Evaluations List
Subject: Re: [Contrib: Evaluation System] Errors/Failures in unit tests

Errors means the test is dying and cannot even run enough to succeed or fail. In other words, errors are like a worse version of a failure.

So yes, those need to be addressed and are in fact indicative of a much worse code problem (in general).

-AZ

On Tue, Jul 16, 2013 at 1:43 PM, Brian Jones <bjones86 at uwo.ca> wrote:
> Okay thanks, I can open a Jira for that.
>
> What's your opinion on tests that are reporting 'errors', but not 'failures' (in my modified branch that I'm trying to create a .patch for)? Is it worthwhile looking into these/fixing them?
>
> Thanks again
>
> Brian Jones
> Applications Development
> Information Technology Services
> Support Services Building, Room 4326
> Western University
> (519) 661-2111 x86969
> bjones86 at uwo.ca
>
>
> -----Original Message-----
> From: azeckoski at gmail.com [mailto:azeckoski at gmail.com] On Behalf Of 
> Aaron Zeckoski
> Sent: Tuesday, July 16, 2013 1:36 PM
> To: Brian Jones
> Cc: Matthew Jones; Sakai Evaluations List
> Subject: Re: [Contrib: Evaluation System] Errors/Failures in unit 
> tests
>
> Sounds like something related to java 7 I guess. I am still on java 6 myself and don't have a java 7 install on OSX where I can test this.
>
> Might be worth opening a JIRA with the details of the failures. I suppose something in java 7 has somehow broken those tests.
> -AZ
>
>
> On Tue, Jul 16, 2013 at 1:22 PM, Brian Jones <bjones86 at uwo.ca> wrote:
>> Hey Aaron & Matt , thanks for your responses.
>>
>> I'm getting the same results as Matt (below). I'm on Windows 7, 
>> compiling with Java 1.7.0_21
>>
>> Results :
>>
>> Failed tests:
>>
>> testGetEvaluationsForOwnerAndGroups(org.sakaiproject.evaluation.dao.E
>> v
>> aluati
>> onDaoImplTest)
>>
>> testGetViewableGroupsForEvalAndUserByRole(org.sakaiproject.evaluation.logic.
>> ReportingPermissionsImplTest)
>>
>> Tests run: 274, Failures: 2, Errors: 0, Skipped: 0
>>
>> Brian Jones
>> Applications Development
>> Information Technology Services
>> Support Services Building, Room 4326
>> Western University
>> (519) 661-2111 x86969
>> bjones86 at uwo.ca
>>
>> From: Matthew Jones [mailto:matthew at longsight.com]
>> Sent: Tuesday, July 16, 2013 1:20 PM
>> To: Aaron Zeckoski
>> Cc: Brian Jones; Sakai Evaluations List
>> Subject: Re: [Contrib: Evaluation System] Errors/Failures in unit 
>> tests
>>
>> Which version of java are you compiling with? With 1.7.0_21 it's 
>> giving 2 repeatable failures. I tried to upgrade the junit in impl to 
>> make it more predictable
>> (https://jira.sakaiproject.org/browse/KNL-1064) and it does help, but 
>> it's still causing failures with either the default and name_ascending modes.
>>
>> Results :
>>
>> Failed tests:
>>
>> testGetEvaluationsForOwnerAndGroups(org.sakaiproject.evaluation.dao.E
>> v
>> aluati
>> onDaoImplTest)
>>
>> testGetViewableGroupsForEvalAndUserByRole(org.sakaiproject.evaluation.logic.
>> ReportingPermissionsImplTest)
>>
>> Tests run: 274, Failures: 2, Errors: 0, Skipped: 0
>>
>>
>> On Tue, Jul 16, 2013 at 1:15 PM, Aaron Zeckoski 
>> <azeckoski at unicon.net>
>> wrote:
>> I don't see any failures in the evals tests on trunk myself but 
>> generally speaking, a patch which causes test failures would be 
>> rejected.
>>
>> Tests run: 274, Failures: 0, Errors: 0, Skipped: 0 ...
>> [INFO] Sakai Evaluation Project .......................... SUCCESS 
>> [6.854s] [INFO] Sakai Evaluation API ..............................
>> SUCCESS [2.958s] [INFO] Sakai Evaluation Impl 
>> ............................. SUCCESS [1:25.506s] [INFO] Sakai 
>> Evaluation Pack ............................. SUCCESS [3.177s] [INFO] 
>> Sakai Evaluation Tool ............................. SUCCESS [52.914s] 
>> [INFO]
>> ---------------------------------------------------------------------
>> -
>> --
>> [INFO] BUILD SUCCESS
>> [INFO]
>> ---------------------------------------------------------------------
>> -
>> --
>> [INFO] Total time: 2:31.940s
>> [INFO] Finished at: Tue Jul 16 13:15:12 EDT 2013
>>
>> -AZ
>>
>>
>> On Tue, Jul 16, 2013 at 11:54 AM, Brian Jones <bjones86 at uwo.ca> wrote:
>>> Hi,
>>>
>>> I'm currently preparing a patch to submit to the community for new 
>>> functionality in the evalsys, but I noticed that there are several 
>>> unit tests that report either errors or failures during the build.
>>>
>>> I'm developing the patch against trunk of evalsys, and checked to 
>>> see if
>> the
>>> same failures are reported on a stock check-out of trunk; which they are.
>> It
>>> reports the same two tests as failing, but in my modified branch 
>>> there are
>> a
>>> bunch more which report 'errors' rather than 'failures'.
>>>
>>> Should I be concerned about this with regards to submitting the 
>>> patch to
>> the
>>> community? Should I take the time to fix the tests that are in error?
>> Should
>>> I ignore the ones that are in 'failure' as they seem to also fail in 
>>> the stock trunk?
>>>
>>> Or maybe I should comment them out and submit the patch with that as well?
>>> Any input will be helpful!
>>>
>>> Thanks,
>>>
>>> Brian Jones
>>> Applications Development
>>> Information Technology Services
>>> Support Services Building, Room 4326 Western University
>>> (519) 661-2111 x86969
>>> bjones86 at uwo.ca
>>>
>>>
>>>
>>> _______________________________________________
>>> evaluation mailing list
>>> evaluation at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>>
>>> TO UNSUBSCRIBE: send email to
>> evaluation-unsubscribe at collab.sakaiproject.org with a subject of
>> "unsubscribe"
>>
>>
>> --
>> Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile
>> _______________________________________________
>> evaluation mailing list
>> evaluation at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>
>> TO UNSUBSCRIBE: send email to
>> evaluation-unsubscribe at collab.sakaiproject.org
>> with a subject of "unsubscribe"
>>
>>
>
>
>
> --
> Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile
>

-- 
Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile