[Building Sakai] Reminder about the CLE performance testing framework

Wed Jul 18 04:57:52 PDT 2012

Hi Juan,

If you could add a stress test which literally just logs in as a non-admin
user and sends the same message again and again to portal-chat/new from a
single thread that would be a great start. The next step would be to
parallelise the requests so we can get a handle on behaviour as we hit
Tomcat or Apache's thread/concurrent connections limit.

Any help greatly appreciated.

Cheers again,
Adrian.

On 18 July 2012 11:18, Juan Jose Meroño Sanchez <jjmerono at um.es> wrote:

>
> Hi Adrian,
>
>     This is a tricky problem !!
>     The test you are talking about performs a site browse with a thread
> group while some users are chatting in another thread group.
>     Is this the best way to test the chat? I 'm not sure, but when I wrote
> it I was thinking in detect performance decrease because of chat usage.
>     Can you get conclusions on how jgroup works in a cluster setting? I
> don't thing so.
>
>     What would I do?
>
>         Create an specific test for chat (just login chat and logout),
> trying to keep constant the messages/second value.
>         Run the test on a single node, and collect JVisualVM info.
>         Run the same test on a 2 node cluster, and collect JVisualVM info
> from the same node.
>
>     With this information you can get conclusions on requeriments to
> exchange X messages/second between 2 nodes versus requeriments on no
> cluster running.
> I think JVisualVM info is important because jgroups do a lot of work on
> server side specially in cluster environments.
>
> If you want I can help you !!  I can create the test and add to the
> framework, then you can run on your servers (with your users) and collect
> the information.
>
> Bye !!
>
> El 18/07/2012 11:31, Adrian Fish escribió:
>
> Hi again Juan,
>
> I see you've already set up some tests for the portal chat which is the
> area I'm interested in. I want to get an idea of raw throughput  on a
> single node and try and get some idea of how well jgroups is performing in
> replicating the messages across a cluster.
>
> If you've got any ideas on how to best achieve that I will up the offer to
> two cañas.
>
> Saludos,
> Adrian.
>
> On 18 July 2012 10:00, Adrian Fish <adrian.r.fish at gmail.com> wrote:
>
>> Thanks. I will have a play with it. If it is good, I will buy you that
>> elusive caña :)
>>
>>
>> On 17 July 2012 08:08, Juan Jose Meroño Sanchez <jjmerono at um.es> wrote:
>>
>>>
>>> I recomend the last version 1.3.1, I release it few days ago !!
>>> Versions prior to 1.2 are completely different, don't use it !!
>>>
>>> Version 1.3.1 uses version 1.3.1 of "testcases" (both are independent).
>>> The goal is to add more and more tests, and improve the existing ones,
>>> so version of "testcases" change independently.
>>> The tests in core package are not enough, specially for OAE, I hope
>>> community adopt the framework and help me to add tests, improve the
>>> existing ones, and the framework too.
>>>
>>> Bye !!
>>>
>>> El 16/07/2012 23:32, Adrian Fish escribió:
>>>
>>> Which version of the testing framework do you recommend for the CLE?
>>> There are a few flavours under the qa contrib area.
>>>
>>> Cheers,
>>> Adrian.
>>>
>>> On 16 July 2012 10:35, Alan Berg <bergsmooth at gmail.com> wrote:
>>>
>>>> The QA environment is hybrid mode with qa1-nl (older version of sakai
>>>> CLE) and qa20-us. This is where the metal hits the road.
>>>>
>>>> The stress testing is run through Maven, so you can start testing from
>>>> your desktop.
>>>>
>>>> Alan
>>>>
>>>>
>>>> On 16 July 2012 11:11, Adrian Fish <adrian.r.fish at gmail.com> wrote:
>>>>
>>>>> I'll make sure blackboard are aware of the work you've done. I'm
>>>>> presuming they will have infrastructure available and software to run on
>>>>> that infrastructure will be more than welcome.
>>>>>
>>>>> Cheers,
>>>>> Adrian.
>>>>>
>>>>>
>>>>> On 16 July 2012 09:30, Juan Jose Meroño Sanchez <jjmerono at um.es>wrote:
>>>>>
>>>>>>  Hi all,
>>>>>>
>>>>>>     Don't buy me a drink, checkout my framework and test, and the buy
>>>>>> me a drink :-)
>>>>>>     If something doesn't work or you need help, please write me a
>>>>>> comment, email me, raise your hand,... don't keep silent as me. :-)
>>>>>>
>>>>>> Bye !!
>>>>>>
>>>>>> El 16/07/2012 9:41, Alan Berg escribió:
>>>>>>
>>>>>>  Hi fellow hard workers,
>>>>>>
>>>>>> I noticed comments on performance testing. There is actually a
>>>>>> community framework available using SoapUI, Jmeter and Maven. This is
>>>>>> excellent for automation and standardization of data models. It is for CLE
>>>>>> and hybrid mode. Juan Jose Meroño Sanchez  wrote this. It is an evolution
>>>>>> of the work from the now defunct Performance WG and then some provisioning
>>>>>> work I did. As a third generation attempt Juan’s efforts are much more
>>>>>> polished, using the best of breed open source tools.
>>>>>>
>>>>>> It would be excellent if we improve this work rather than build
>>>>>> organizational bunkers with commercial software. We could share models.
>>>>>>
>>>>>> See:
>>>>>> https://confluence.sakaiproject.org/display/QA/CLE+Load+Test+Framework<https://webmail.uva.nl/owa/redir.aspx?C=e174c8a7450c42cda6261fcc5cc14f76&URL=https%3a%2f%2fconfluence.sakaiproject.org%2fdisplay%2fQA%2fCLE%2bLoad%2bTest%2bFramework>
>>>>>>
>>>>>> For a cheaper example, I have written a microbenchmark for sakai OAE.
>>>>>> (Work in Progress).
>>>>>>
>>>>>> https://github.com/AlanBerg/SakaiOAE-Open/tree/master/TESTS/jmeter_microbenchmark
>>>>>>
>>>>>> Juans work is more comprehensive and good place to rebuild a WG on
>>>>>> Performance.
>>>>>>
>>>>>> If you see Juan Jose Menono at a conference pat him on the back and
>>>>>> buy him a drink. I am sure after working quietly on this project he
>>>>>> deserves at least that.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Alan
>>>>>>
>>>>>> Alan Berg
>>>>>> University of Amsterdam
>>>>>>
>>>>>>
>>>>>>
>>>>>>  _______________________________________________
>>>>>> sakai-dev mailing listsakai-dev at collab.sakaiproject.orghttp://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>>>>
>>>>>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> sakai-dev mailing list
>>>>>> sakai-dev at collab.sakaiproject.org
>>>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>>>>
>>>>>> TO UNSUBSCRIBE: send email to
>>>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>>>>> "unsubscribe"
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> sakai-dev mailing list
>>>>> sakai-dev at collab.sakaiproject.org
>>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>>>
>>>>> TO UNSUBSCRIBE: send email to
>>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>>>> "unsubscribe"
>>>>>
>>>>
>>>>
>>>
>>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20120718/4b243189/attachment.html 