[Building Sakai] problem with BLOB table in Sakai DB

Tue May 11 06:36:42 PDT 2010

There is also this patch you can apply to the kernel to get the fix in your environment:
http://jira.sakaiproject.org/secure/attachment/20141/KNL-164-1.0.x.patch

You'll need to check out the source and build it though, since the kernel is a binary deploy by default.

cheers,
Steve

On 11/05/2010, at 11:31 PM, Steve Swinsburg wrote:

> As Jim mentioned, you should be able to set this in sakai.properties:
> 
> convertToFile at org.sakaiproject.content.api.ContentHostingService = true
> bodyPath at org.sakaiproject.content.api.ContentHostingService = /usr/local/sakai/kernel/bodyContent/
> bodyVolumes at org.sakaiproject.content.api.ContentHostingService = vol1,vol2,vol3,vol4,vol5
> 
> startup and be on your way.
> 
> However:
> 
> There is a bug that manifested itself under certain conditions, so resources didn't convert out properly, see comments in KNL-164 and SAK-16146. This is fixed in 2.5.7 and kernel 1.0.14 and later.
> 
> What release of Sakai 2.6 are you on? The same issue above affects 2.6 however the kernel version for previous 2.6 releases are less than 1.0.14 so won't contain the fix... You may need to manually adjust your SAKAI-SRC/master/pom.xml <sakai.kernel.version> to 1.0.14 to get the updated kernel.
> 
> cheers,
> Steve
> 
> 
> On 11/05/2010, at 11:17 PM, Jim Eng wrote:
> 
>> The most recent information I found about the conversion process is in this KIRA ticket:
>> 
>> http://jira.sakaiproject.org/browse/KNL-164?focusedCommentId=79042&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_79042
>> 
>> There's also documentation in Confluence.  The conversion from database to filesystem storage has been in place since version 1.5 or before.  The instructions have not changed, though the implementation has changed a bit, especially a couple years ago when the storage of metadata changed from XML to "binary entity" serialization.   
>> 
>> The reason I'm giving you a link to the JIRA ticket is that the ticket includes notes from three or four people who tested this process a year or so ago, after the conversion to binary-entity serialization. They may have additional suggestions.  
>> 
>> It's been a while since I have looked at it, but as I read the notes in this ticket, it looks like the method suggested there will work no matter which version of sakai you are using. 
>> 
>> When we have done big conversions like this, we usually start by making copies of the relevant tables and running a test off-line.  This is like the manufacturer of a harsh chemical cleaning solution telling you to test it on a sample of fabric before using it on your best suit.  It would be good if you could do that.  
>> 
>> HTH.
>> 
>> Jim
>> 
>> 
>> On May 11, 2010, at 8:51 AM, Patrick Coleman wrote:
>> 
>>> 
>>> Hi Jim,
>>> 
>>> That make sense so my next question is how do I "convert to filesystem
>>> storage for content bodies."
>>> 
>>> Thanks.
>>> 
>>> Pat.
>>> 
>>> 
>>> Jim Eng wrote:
>>>> 
>>>> Hi Pat,
>>>> 
>>>> I can't answer your specific questions about how MySQL handles that, but I
>>>> can caution that generally storing the content bodies in the database is
>>>> not expected to scale very well.  Storage of content bodies in the db
>>>> simplifies setup for pilots and demos, but the general recommendation is
>>>> to convert to storing content bodies in the filesystem before usage is
>>>> expected to put much load on that table. That said, the last time we
>>>> discussed this on the list, a number of people spoke up to say they are
>>>> running small instances in production and storing file bodies in the
>>>> database without problems.  Maybe somebody who is doing that can give more
>>>> focused advice.  My advice would be convert to filesystem storage for
>>>> content bodies.
>>>> 
>>>> Jim
>>>> 
>>>> 
>>>> On May 11, 2010, at 8:19 AM, Patrick Coleman wrote:
>>>> 
>>>>> 
>>>>> In my Sakai DB there is a blob table (type longblob) called
>>>>> content_resource_body_binary
>>>>> This table is about 2.8GB in size and I'm having problems backing it up
>>>>> or
>>>>> even looking at it's content.
>>>>> 
>>>>> If I do a "select * from content_resource_body_binary" I get an error
>>>>> saying..
>>>>> 
>>>>> ERROR 2020 (HY000): Got packet bigger than 'max_allowed_packet' bytes
>>>>> 
>>>>> I have the  'max_allowed_packet' set to 1GB which I think is the max it
>>>>> can
>>>>> be set to?
>>>>> 
>>>>> When I try and do a backup of the table (or the whole DB) using mysqldump
>>>>> I
>>>>> get this error..
>>>>> 
>>>>> mysqldump: Error 2013: Lost connection to MySQL server during query when
>>>>> dumping table `content_resource_body_binary` at row: 1582
>>>>> 
>>>>> I done check table etc and it always says ok?
>>>>> 
>>>>> Is there any limit to the size of a table like this? It just started to
>>>>> happen a few weeks ago.
>>>>> I'm running mysql 5.0.84 and Sakai 2.6
>>>>> 
>>>>> Thanks.
>>>>> 
>>>>> Pat.
>>>>> -- 
>>>>> View this message in context:
>>>>> http://old.nabble.com/problem-with-BLOB-table-in-Sakai-DB-tp28523391p28523391.html
>>>>> Sent from the Sakai - Development mailing list archive at Nabble.com.
>>>>> 
>>>>> _______________________________________________
>>>>> sakai-dev mailing list
>>>>> sakai-dev at collab.sakaiproject.org
>>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>>> 
>>>>> TO UNSUBSCRIBE: send email to
>>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>>>> "unsubscribe"
>>>>> 
>>>>> 
>>>> 
>>>> _______________________________________________
>>>> sakai-dev mailing list
>>>> sakai-dev at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>> 
>>>> TO UNSUBSCRIBE: send email to
>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>>> "unsubscribe"
>>>> 
>>>> 
>>> 
>>> -- 
>>> View this message in context: http://old.nabble.com/problem-with-BLOB-table-in-Sakai-DB-tp28523391p28523675.html
>>> Sent from the Sakai - Development mailing list archive at Nabble.com.
>>> 
>>> _______________________________________________
>>> sakai-dev mailing list
>>> sakai-dev at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>> 
>>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>>> 
>>> 
>> 
>> _______________________________________________
>> sakai-dev mailing list
>> sakai-dev at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>> 
>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20100511/b235d43e/attachment.html 