[Building Sakai] Experience with Oracle RAC

[Jon Gorrono]

Here is a collection of pages I could find about our performance
testing with RAC. I said 'cool-headed' but I didn't mean to imply well
organized.

In order to help makes some sense out of the order and some
fuzziness....we first noticed performance issue with the RAC in FALL
2007 and investigated this into the first half of the first quarter of
2008 IIRC..


meeting notes in which it is mentioned that adjusting oracle query
cost estimator index parameter seemed to have no effect :
https://confluence.ucdavis.edu/confluence/display/UCDSAKAI/Sakai+Hardware+meeting+Sept+19%2C+2007


Some load testing with Sylk Perfromer in which (from standup meeting
notes) 'Initial (jan 2008) load test results suggest that two RAC
nodes are slower than one':

https://confluence.ucdavis.edu/confluence/display/UCDSAKAI/RAC+Load+Test+Plan

Some 'notes from the field' ... the tone even suggests they were
written under fire.
https://confluence.ucdavis.edu/confluence/display/UCDSAKAI/Conversation+with+Drew+Zhu+Feb142008
(note:IIRC, I think we determined that turning off ASM with RAC was
not possible.)

We eventually (feb 2008) decided that we could use a single node RAC
... it tested well and we deployed it to our production environment
only to have it fail immediately. The failure manifested after 3 or 4
nodes were up... our test environment is only two nodes.. We rolled
back to the non-RAC config during that same maintenance window... here
is a note from the front lines:

'We attempted to move the SmartSite databases to the Oracle RAC again
this morning but it
appeared that we encountered an unexplained caching problem which led
to slow performance.
As a result we backed out again to the non RAC environment so the
system is running the
same as it was previously."


On Wed, Oct 7, 2009 at 4:30 PM, Jon Gorrono <jpgorrono at ucdavis.edu> wrote:
> Did you say Oracle RAC? Slowly I turned... step by step....
> (http://www.youtube.com/watch?v=9pQii1L8fGk)
>
> We have some cool headed doc's about our experiences with rac in out
> confluence, which I can dig out if you would like the details.
>
> In contrast I can give you a sense of our experience with RAC: it was
> not good. There were to issues that bubbled to the top of the heap and
> lingered long enough for us to decide to give ip on the RAC
>
> 1). a) the RAC insists on managing segment fragmentation and b) it sux at it
>  - we would setup our tables views and mviews and a bunch of nicely
> performing indexes and the RAC would split up the table and index
> segments until the performance tanked. And we coul dnot turn it off
> even though it appeared that we had declaratively
> 2) There was a lot of speculation that the number of small tables used
> in sakai for simple low data lookups would harm performance in a RAC
> environment due to the added overhead for synch'ing those tables over
> the RAC private network... but we never tested this: #1 killed the
> project.
>
> On Tue, Oct 6, 2009 at 11:12 AM, Mame-Awa Diop <mame-awa.diop at hec.ca> wrote:
>> Hello everybody,
>>
>> I was wondering if you guys using Oracle as a database for Sakai and using a
>> clustering tool like Oracle RAC has had any special behavior with the
>> clustering tool ? Is there something worth monitoring with the clustering
>> tool ?
>> If you use another clustering tool, your input would be appreciated also.
>>
>>
>> Cheers,
>>
>> --
>>
>> Mame Awa Diop
>> Analyste programmeur
>> Service de gestion des technologies et de l'information
>> HEC Montréal
>> 3000, chemin de la côte-Sainte-Catherine
>> Montréal  (Québec)   H3T 2A7
>>
>> GTI: 514.340.6000 #2029 - Bureau 3.755
>> Projet OpenSyllabus: 514.340.6776 - Bureau 4.521
>> 5255, avenue Decelles
>> Montréal (Québec) Canada
>> H3T 2B
>>
>> Télécopieur: (514)340-5637
>>
>> _______________________________________________
>> sakai-dev mailing list
>> sakai-dev at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>
>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org
>> with a subject of "unsubscribe"
>>
>
>
>
> --
> Jon Gorrono
> PGP Key: 0x5434509D -
> http{pgp.mit.edu:11371/pks/lookup?search=0x5434509D&op=index}
> Thawte Notary - https{www.thawte.com/cgi/personal/wot/directory.exe?node=312}
> GSWoT Introducer - {GSWoT:US75 5434509D Jon P. Gorrono <jpgorrono - gswot.org>}
> http{ats.ucdavis.edu}
>
> Sent from Davis, CA, United States
>



-- 
Jon Gorrono
PGP Key: 0x5434509D -
http{pgp.mit.edu:11371/pks/lookup?search=0x5434509D&op=index}
Thawte Notary - https{www.thawte.com/cgi/personal/wot/directory.exe?node=312}
GSWoT Introducer - {GSWoT:US75 5434509D Jon P. Gorrono <jpgorrono - gswot.org>}
http{ats.ucdavis.edu}

Sent from Davis, CA, United States