[Building Sakai] [WG: Sakai QA] Fwd: Automated testing -- Tools discussion
Jean-Francois Leveque
jean-francois.leveque at upmc.fr
Thu May 6 01:59:59 PDT 2010
I hear you.
I don't think Marist tests for GB2 are needed from a 2.7.0 release point
of view.
Where are we regarding Marists tests for code in the 2.7.0 release that
are not duplicates of rSmart tests?
Are the rSmart or Marists documented from a
http://confluence.sakaiproject.org/display/QA/Test+Conditions+and+Scripts
point of view?
https://source.sakaiproject.org/contrib/qa/trunk/functional/test_suite/readme.txt
and
http://confluence.sakaiproject.org/display/QA/Selenium+Remote+Control+and+TestNG
document the current state of
https://source.sakaiproject.org/contrib/qa/trunk/functional/test_suite.
Looks like this documentation needs an update. cle-suite-nightly.xml in
readme.txt looks like a specific rSmart name and cannot be found in
subversion.
sakai-suite-regsuite-26.xml looks like a 2.6 suite.
sakai-suite-setup-27.xml looks like a 2.7 suite which includes 2.6
teardown first.
sakai-suite-setup.xml looks like another 2.6 suite.
sakai-suite-single.xml looks like a specific rSmart teadrown.
sakai-suite-teardown.xml looks like the 2.6 teardown included in
sakai-suite-setup-27.xml.
Which suite should be used against 2.6, 2.7 and trunk and how should
they be used (is sakai-suite-teardown.xml needed before or after or
should the suite be updated to include it)?
--
Jean-Francois Leveque
UPMC
Corey McGarrahan a écrit :
> The suite contains scripts from rSmart, but they are written for OOTB
> Sakai. I know Marist tests for GB2 are going to be added soon. A lot of
> the Marist tests are duplicates of what is already checked in. Anyone
> who is writing scripts that are designed to run against OOTB Sakai
> should be commiting those tests to the QA/trunk//functional/ area. What
> I am worried about is having multiple copies of the same tests and I'll
> be looking out for that.
>
> For now I have been running the suite locally and have not been uploaded
> the results anywhere as of yet, outside of reporting any bugs that I run
> into. The results will be stored in the /test_suite/target/reports
> directory as index.html once the Selenium RC/TestNG automated build is
> running. This will address your third question as well. I've had
> conversations with Alan about this and I know he has done some work on
> this already.
>
> The suite as it is now is rather thin. All of the existing test were
> written as Acceptance Tests (positive path) and are going to need to be
> updated as new fixes and functionality are added. This test suite can
> (and should) become the automated regression testing suite for new
> builds. My goal/hope/idea is that these scripts will be the foundation
> for the regression suite and will be maintained and updated by the QA
> community.
>
> I will be updating these existing scripts as we move through the betas
> and into the RC's as things will obviously be changing. Anyone who is
> interested in gettin involved can find information in the links of the
> original message.
>
>
>
> On Wed, May 5, 2010 at 5:46 AM, Jean-Francois Leveque
> <jean-francois.leveque at upmc.fr <mailto:jean-francois.leveque at upmc.fr>>
> wrote:
>
> Does this suite contain scripts from Marist or scripts from rSmart only?
>
> Are the results of the run against 2.7.0-b08 stored somewhere?
>
> Could this suite be adapted and automatically run on the automated
> builds of trunk and the 2.7.x, 2.6.x and 2.5.x branches?
>
> Jean-Francois Leveque
> UPMC
>
> Corey McGarrahan a écrit :
>
> Automated testing for Sakai with Selenium IDE, Selenium RC and
> TestNG
>
> All of the existing Selenium scripts have been updated for
> 2.7.0-b08 -
> https://source.sakaiproject.org/contrib/qa/trunk/functional/2.7/Selenium
> and have been run against the latest build on QA1-NL
> <http://qa1-nl.sakaiproject.org/>.
>
> The scripts are also updated in the Selenium RC/TestNG automated
> test suite here -
> https://source.sakaiproject.org/contrib/qa/trunk/functional/test_suite/
>
> Instructions for running the Selenium scripts in the browser can
> be found here
> <http://confluence.sakaiproject.org/display/QA/Selenium>, while
> the instructions for running the Selenium RC/TestNG automated
> test suite can be found here
> <http://confluence.sakaiproject.org/display/QA/Selenium+Remote+Control+and+TestNG>.
>
>
> If you have any questions about setup or how to edit the scripts
> so they run locally please don't hesitate to ask me. Some basic
> guidelines:
>
> 1. Make sure to checkout the TestingData
>
> <https://source.sakaiproject.org/contrib/qa/trunk/functional/2.7/Resources>
> folder.
>
> All resources referenced in the scripts are found in there.
> 2. All paths to uploads will need to be changed to point to the
>
> TestingData folder as they are not being hosted on a public
> server
> at this time. 3. All scripts are written with the
> assumption that the site to be
>
> tested is already open in the browser.
> 1. This does not apply to Selenium RC/TestNG
> 4. All scripts were written against QA1-NL. Any deviations
> from that
>
> build will need to be accounted for; ie. admin pw
>
> Corey McGarrahan
> rSmart
>
>
>
>
> --
> Corey Jack McGarrahan
> rSmart
More information about the sakai-dev
mailing list