[sakai-pmc] Sakai 10 update: trunk branched to sakai-10.x (r306122); trunk now 11

Tue Feb 18 16:38:36 PST 2014

 Total Size of all files 309,034,668Total size of all files 1MB and
smaller116,666,774Total
size of all files 500 MB and smaller 86,657,618
Switching to Java 7 looks like it saves around 30MB. Cleaning up this
spring and castor (also mentioned today) issues should save around 5MB.

Doesn't seem like we need the 7MB aws-java-sdk-1.3.26.jar from
elasticsearch-cloud-aws by default so I'll create a new issue for that one.
Good stuff!

On Tue, Feb 18, 2014 at 7:18 PM, Matthew Jones <matthew at longsight.com>wrote:

> Yeah, then that just goes back to the discussion and potential for runtime
> exceptions either when things aren't updated for the newest version or if
> they have a version in shared and in their local. This is especially a
> problem for contrib projects that aren't as updated, and I'm not too sold
> on the overall gains for the smaller libs. The majority of our memory
> issues have been problems with bad caching, often which wasn't noticed for
> a long time after, rather than this initial known hit.
>
> Though I agree there might be some improvements to be made in this space
> too, and looking at the list would reveal some libraries in the core that
> should be upgraded, something we don't often do.
>
> Probably less of a problem for some libraries, more of a problem for most
> frameworks, even if there is a memory savings.
>
> Below is a list of the 300 megabytes of initial load that I have from
> Sakai 11. Some newer (to 11) components like the elasticsearch jar might
> have bumped it up.
>
> - Poi and fop seem like good candidate for shared. Looks like calendar is
> still using a really old fop?
> - spring-test doesn't seem like it should be on the list, looks like some
> projects have that scope wrong.
>
> Possibly some more interesting things from this list! (Just lsof -p
> <process id> | grep mem)
>
>
> https://docs.google.com/spreadsheet/pub?key=0AgADdxCIA4KKdFQ1V3VYSTNScVd3dkxlNVJZOC1RUFE&output=html
>
>
> On Tue, Feb 18, 2014 at 6:35 PM, Steve Swinsburg <
> steve.swinsburg at gmail.com> wrote:
>
>> Each webapp also bundles their own versions of some libraries that could
>> possibly be shared, increasing the memory footprint further. Might be
>> something to look at in the future since we have so many webapps.
>>
>> sent from my mobile device
>> On 19/02/2014 1:55 AM, "Aaron Zeckoski" <azeckoski at unicon.net> wrote:
>>
>>> FYI on the Sakai 10 and 11 trunks that I noticed this morning while
>>> profiling.
>>>
>>> Current permgen used (without OSP): 282 MB
>>> Current running trunk (unloaded and without primed caches): 281 MB
>>> In other words, we now use more memory storing classes than we use to
>>> minimally run Sakai. Roughly a 10% increase over 2.9.
>>>
>>> Largest jars in shared (not exactly related because of how
>>> classloading works but FYI):
>>> -rw-r--r--    1 azeckoski  admin  3123360 Jun  6  2013
>>> hibernate-core-3.6.10.Final.jar
>>> -rw-r--r--    1 azeckoski  admin  2012027 Dec 13  2012 castor-1.0.jar
>>> -rw-r--r--    1 azeckoski  admin  1347353 Feb 11 20:33
>>> ehcache-core-2.6.8.jar
>>> -rw-r--r--    1 azeckoski  admin   988514 Dec 13  2012
>>> commons-math-2.2.jar
>>> -rw-r--r--    1 azeckoski  admin   940732 Feb  4 06:53 ical4j-1.0.5.2.jar
>>> -rw-r--r--    1 azeckoski  admin   867800 Jun  6  2013
>>> spring-core-3.2.3.RELEASE.jar
>>> -rw-r--r--    1 azeckoski  admin   862874 Jun  6  2013
>>> spring-context-3.2.3.RELEASE.jar
>>> -rw-r--r--    1 azeckoski  admin   706710 Jun  6  2013
>>> hsqldb-1.8.0.10.jar
>>> -rw-r--r--    1 azeckoski  admin   659343 Jun  6  2013
>>> javassist-3.16.1-GA.jar
>>> -rw-r--r--    1 azeckoski  admin   629206 Feb 18 08:57
>>> samigo-services-11-SNAPSHOT.jar
>>>
>>> It is probably not ideal that we are using a few hundred MBs to
>>> maintain idle state on an unloaded unused system either.
>>> Probably not something we need to spend time on but just something to
>>> be aware of.
>>> -AZ
>>>
>>>
>>>
>>> On Mon, Feb 17, 2014 at 10:43 PM, Anthony Whyte <arwhyte at umich.edu>
>>> wrote:
>>> > I've branched Sakai trunk to sakai-10.x as of r306122.  Sakai trunk is
>>> now
>>> > Sakai 11.  All trunk commits  that are intended for Sakai 10 but
>>> committed
>>> > post- r306122 are in trunk only and will require merging to sakai-10.x
>>> if
>>> > appropriate.
>>> >
>>> > sakai-10.x maintenance/release branch
>>> > Get it: svn co
>>> > https://source.sakaiproject.org/svn/sakai/branches/sakai-10.x/sakai-10.x
>>> > Version: 10-SNAPSHOT (r306124)
>>> > Branch names: All 10.x branch names have reverted to pre-indie naming
>>> > conventions and are now standardized as
>>> >
>>> > https://source.sakaiproject.org/svn/[moduleName]/branches/sakai-10.x.
>>> >
>>> > trunk (the future starts now)
>>> > Get it: svn co https://source.sakaiproject.org/svn/sakai/trunksakai-trunk
>>> > Version: 11-SNAPSHOT (r306125)
>>> > Jenkins: master, kernel, entitybroker builds back to normal, other
>>> jobs to
>>> > follow.
>>> >
>>> > my tests
>>> > sakai-10.x: fresh check out, install, build, deploy, startup (Tomcat
>>> 7.0.50
>>> > and MySQL 5.6.15, MySQL connector J 5.1.28, Maven 3.0.4).
>>> > run 1: Success (no test failures)
>>> > trunk:  fresh check out, install, build, deploy, startup (Tomcat
>>> 7.0.50 and
>>> > MySQL 5.6.15, MySQL connector J 5.1.28, Maven 3.0.4).
>>> > run 1: Build failure: sitestats-impl test failure (reported earlier by
>>> > Aaron)
>>> > run 2 (no code change): Success (no test failures)
>>> >
>>> > Issues encountered
>>> > sakai-10.x/trunk local build: Samigo unavailable as a tool choice in
>>> > worksite setup; I was able to add it to a test site using the sites
>>> tool.
>>> >
>>> > Log warnings of note:
>>> >
>>> > 2014-02-17 21:01:54,364  WARN localhost-startStop-1
>>> >
>>> org.sakaiproject.springframework.orm.hibernate.AddableSessionFactoryBean -
>>> > Unsuccessful schema statement: create index URL_INDEX on
>>> > URL_RANDOMISED_MAPPINGS_T (URL)
>>> > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: BLOB/TEXT
>>> column
>>> > 'URL' used in key specification without a key length
>>> >
>>> > 2014-02-17 21:02:03,097  WARN localhost-startStop-1
>>> >
>>> org.sakaiproject.springframework.orm.hibernate.AddableSessionFactoryBean -
>>> > Unsuccessful schema statement: create index irwikihistory_ref on
>>> > rwikihistory (referenced)
>>> > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: BLOB/TEXT
>>> column
>>> > 'referenced' used in key specification without a key length
>>> >
>>> > 2014-02-17 21:02:04,101  WARN localhost-startStop-1
>>> >
>>> org.sakaiproject.springframework.orm.hibernate.AddableSessionFactoryBean -
>>> > Unsuccessful schema statement: create index irwikiobject_ref on
>>> rwikiobject
>>> > (referenced)
>>> > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: BLOB/TEXT
>>> column
>>> > 'referenced' used in key specification without a key length
>>> >
>>> > 2014-02-17 21:04:00,521  INFO localhost-startStop-1
>>> > org.sakaiproject.lessonbuildertool.service.ForumEntity - initDao
>>> template
>>> > org.springframework.orm.hibernate3.HibernateTemplate at 49d8ad8a
>>> > SamigoEntity thinks this is Sakai verison 2.6.0, defaulting Samigo
>>> edit link
>>> > to false  [Really?]
>>> >
>>> > TODOS
>>> > Update <scm> paths, sakai-10.x branch
>>> > Jenkins job(s) for sakai-10.x
>>> > Have a beer with csev
>>> >
>>> > We need a 10.x Nightly build server.
>>> >
>>> > Cheers,
>>> >
>>> > Anth
>>> >
>>> > anthony whyte | its and mlibrary | university of michigan |
>>> > arwhyte at umich.edu | 517-980-0228
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > sakai-pmc mailing list
>>> > sakai-pmc at collab.sakaiproject.org
>>> > http://collab.sakaiproject.org/mailman/listinfo/sakai-pmc
>>> >
>>>
>>>
>>>
>>> --
>>> Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile
>>> _______________________________________________
>>> sakai-pmc mailing list
>>> sakai-pmc at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-pmc
>>>
>>
>> _______________________________________________
>> sakai-pmc mailing list
>> sakai-pmc at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-pmc
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-pmc/attachments/20140218/649c79ae/attachment.html 