[Building Sakai] How to checkout the 2.9.0 tag source including all tagged indie source?

Thu Nov 29 06:54:43 PST 2012

It looks like this pom and externals doesn't include the pack module and
profile.

I'm still of the opinion that maintaining all of these extra branches is
going to be too much extra work in the future. If someone (like Steve) is
going to consistently volunteer every release to contribute the effort in
providing, testing and supporting this "release branch+all" then I'm for
it. Otherwise if this is a problem for the community we need to rethink the
way we do our releases as I don't have the extra time available to add in
these extra steps.

I kind of like the way Chris and I were doing the intermediate releases
back in the first few betas for 2.9 back in January. Mostly Monolithic
Sakai, just using a 2.9.x nightly and putting that out on the QA's calling
it 2.9.0-MM-DD-YYYY. It was something everyone could deploy and rely on.
Then for the final release we could actually do it properly but we wouldn't
actually need to release all of the artifacts and assemblies to the
repository (the artifacts for the stuff in core aren't there anyway), just
the api's and utils that contrib tools might depend on. The only real value
I can see for those is so someone could build a contrib tool without
downloading and building the entire source.

I feel that the value of the assemblies for core tools is low (since you
need maven anyway to get the assemblies right now), the value of the
artifacts in the repositories (other than apis and utils) is even lower.
Yet it eats up hours during every release cycle still preparing this stuff.
I'm skeptical if anyone actually takes (for instance) the 2.9.1 assembly of
something and drops it in to 2.9.0 without the rest of 2.9.1. They'd have
to manually do a lot of cleanup for that to work anyway.

On Thu, Nov 29, 2012 at 9:27 AM, Neal Caidin <nealcaidin at sakaifoundation.org
> wrote:

> I downloaded
> https://source.sakaiproject.org/svn/sakai/tags/sakai-2.9.0-all/  and ran
> "mvn install -Ppack-demo"  .  The build chugged along until the end, when I
> got "[WARNING] The requested profile "pack-demo" could not be activated
> because it does not exist."  There is a pack-demo directory with a
> build.xml and a pom.xml .
>
> Is that expected behavior? Maybe I'm doing something wrong?
>
> The same command does work for me for sakai-2.9.0, but not for
> sakai-2.9.0-all .
>
> (Please consider this part of my learning curve. Building Sakai CLE is
> still new to me ;-) )
>
> Thanks,
> Neal
>
>
>
> On Nov 29, 2012, at 8:10 AM, "Gast, Cynthia (cmw6s)" <
> cmw6s at eservices.virginia.edu> wrote:
>
> Thanks, Steve.
>
> I just tested the new tag, using 'svn export', and it ran to completion.
> I hope someone else can test building this for you, but I will, as soon as
> we get a machine configured for 2.9.0 :-)
>
> Thanks again to everyone who contributed to this discussion, and for all
> the help and attention to this matter -- its a HUGE time savings for
> checking out ALL the 2.9.0 source, including the indie source!
>
> Cheers,
> Cindy
>
> ------------------------------
> *From:* Steve Swinsburg [steve.swinsburg at gmail.com]
> *Sent:* Thursday, November 29, 2012 8:05 AM
> *To:* Gast, Cynthia (cmw6s)
> *Cc:* Cris J Holdorph; sakai-dev
> *Subject:* Re: [Building Sakai] How to checkout the 2.9.0 tag source
> including all tagged indie source?
>
> Hi Cindy,
>
> The API mod was just a test to see if my local source mods made it out to
> Tomcat after a full rebuild, and they did.
>
> I am using Java 6. I will probably move to Java 7 later next year. I have
> a bunch of other projects that need to be compatibility tested first.
>
> cheers,
> Steve
>
>
> On Thu, Nov 29, 2012 at 11:53 PM, Gast, Cynthia (cmw6s) <
> cmw6s at eservices.virginia.edu> wrote:
>
>> Thanks, Steve!
>>
>> I'll test the 'svn export' of this new tag.  Unfortunately, I'm not able
>> to test building yet, we need to get a machine set up and another group
>> here handles that for us.
>>
>> Was the API mod you mention required to build?
>>
>> Steve or anyone, are you building with Java 6 or 7?  That is an
>> outstanding question I have, and need to specify the Java version for our
>> machine setup request.  We'd love to know that its safe to go with Java 7
>> because of the Feb 2013 EOL announced for Java 6.
>>
>> Thanks,
>> Cindy
>>
>> ------------------------------
>> *From:* sakai-dev-bounces at collab.sakaiproject.org [
>> sakai-dev-bounces at collab.sakaiproject.org] on behalf of Steve Swinsburg [
>> steve.swinsburg at gmail.com]
>> *Sent:* Wednesday, November 28, 2012 7:30 PM
>> *To:* Cris J Holdorph
>> *Cc:* sakai-dev
>>
>> *Subject:* Re: [Building Sakai] How to checkout the 2.9.0 tag source
>> including all tagged indie source?
>>
>> I've created a 2.9.0-all tag here:
>> https://source.sakaiproject.org/svn/sakai/tags/sakai-2.9.0-all/
>>
>> It has all of the relevant source including indies and modules updated
>> etc, and seems to work fine. I modified one of the API's in one of the
>> projects (just added a string constant), did a full rebuild of the entire
>> lot, decompiled the deployed version of that jar that I modified and it
>> matched my local source.
>>
>> Could someone else please verify?
>>
>> cheers,
>> Steve
>>
>>
>>
>> On Thu, Nov 29, 2012 at 7:43 AM, Cris J Holdorph <holdorph at unicon.net>
>> wrote:
>>
>>> I had a similar experience to Aaron, but with slightly tweaked code.  I
>>> worked on it a while.  I didn't exactly give up, but the situation I
>>> finally ended up in was a bunch of custom 1-offs that I had to build
>>> locally like of specific versions of purepom RELEASES, plugins,
>>> sakai-rsf, etc.  Once I had done all of those *AND* had updated a few of
>>> the sakai 2.9.x-all sources to point to my 1-offs, then I could finally
>>> remove all of the ~/.m2/repository/org/sakaiproject/ directory tree and
>>> do an offline build.  It did require doing things in the right order
>>> though (like plugin, purepoms, sakai-rsf, etc.).
>>>
>>> ---- Cris J H
>>>
>>> On 11/28/2012 01:16 PM, Aaron Zeckoski wrote:
>>> > I think that it *should* be possible to do an offline build once an
>>> > "online" build has been completed successfully. However, if I delete
>>> > the Sakai artifacts from my M2 repo and leave all the others I cannot
>>> > do an offline build of the 2.9 code.
>>> >
>>> > I haven't had much luck convincing others that this is a real issue
>>> > for community users but hopefully this conversation will help with
>>> > that.
>>> >
>>> > -AZ
>>> >
>>> >
>>> > On Wed, Nov 28, 2012 at 3:07 PM, Matthew Jones <matthew at longsight.com>
>>> wrote:
>>> >> I believe what you were describing was being seen by Unicon, which
>>> was why I
>>> >> had to keep cleaning the SNAPSHOTS from the repository and fixing bugs
>>> >> awhile ago.
>>> >>
>>> >> Indies have a parent of master
>>> >> The core has a parent of base which has a parent of master
>>> >>
>>> >> I believe that if you just dump the indies into base (which is not a
>>> parent
>>> >> of any of the indies) then when maven builds up it's dependency tree,
>>> it
>>> >> doesn't necessarily see that assignment is a dependency of lessons
>>> (for
>>> >> example) since it can just go out to the repository and get all of the
>>> >> artifacts that lessons needs. This is *probably* okay for the most
>>> part
>>> >> since the dependencies are really just apis and most people are just
>>> >> modifying the impls. But if you're modifying impls I don't think that
>>> maven
>>> >> would necessarily order assignments (and everything else) before
>>> lessons
>>> >> unless the parent of lessons was base. If you dumped the indies
>>> modules into
>>> >> master, I'd probably be more confident.
>>> >>
>>> >> It's likely this didn't turn up in 2.7 because the only indies then
>>> were
>>> >> kernel and like 2-3 others. There were some in 2.8 but less and most
>>> of the
>>> >> time people were probably messed up (or partially saved) anyway with
>>> >> purepoms and getting the versions that were defined locally in the
>>> top level
>>> >> of each tool anyway. I don't know because Michigan never went to 2.8
>>> and
>>> >> Longsight has built off 2.8.x and also deploys tools individually.
>>> >>
>>> >> 2.9.0 (and maybe even the 2.9.x-all as far as indies are concerned)
>>> could
>>> >> potentially cause all new problems unless it's 100% guaranteed that
>>> maven
>>> >> does correct reactor ordering and never goes out to the repository for
>>> >> artifacts that it has available to build. Since it's not possibly to
>>> run in
>>> >> offline mode (because of artifacts that we don't have the source for)
>>> I'm
>>> >> not too sure I'd 100% believe it without seeing a few build logs for
>>> myself.
>>> >>
>>> >>
>>> >> On Wed, Nov 28, 2012 at 2:50 PM, Steve Swinsburg <
>>> steve.swinsburg at gmail.com>
>>> >> wrote:
>>> >>>
>>> >>> Changing the version numbers should not be required. If the source
>>> module
>>> >>> are listed in the base pom and there are no assembly deploy poms,
>>> then maven
>>> >>> will build a dependency graph and adjust the build order to suit. It
>>> does
>>> >>> that now, it's just that since there is no indie source it will get
>>> >>> artifacts solely from the remote repo. If you look at the build
>>> you'll see
>>> >>> individual jars being downloaded, like API etc, as well as the all
>>> in one
>>> >>> tomcat overlay. It's redundant but that's how Maven sees the
>>> dependency
>>> >>> graph.
>>> >>>
>>> >>> So adding the source modules as per the current -all branches, you
>>> should
>>> >>> not get any sakai app artifacts being downloaded, they should all be
>>> built.
>>> >>>
>>> >>> If this does still occur then there is a fundamental issue, and it
>>> would
>>> >>> have been seen in 2.7 and earlier, since as part of the tag/release
>>> process
>>> >>> ALL artifacts are deployed to the Sakai maven repo and you would
>>> have seen
>>> >>> the conflicts then.
>>> >>>
>>> >>>
>>> http://source.sakaiproject.org/maven2/org/sakaiproject/sakai-announcement-api/
>>> >>>
>>> >>> I'll have a look at the parts you posted Cynthia, and get this
>>> finalised
>>> >>> today.
>>> >>>
>>> >>> Cheers
>>> >>> Steve
>>> >>>
>>> >>> Sent from my iPhone
>>> >>>
>>> >>> On 29/11/2012, at 4:46, "Gast, Cynthia (cmw6s)"
>>> >>> <cmw6s at eservices.virginia.edu> wrote:
>>> >>>
>>> >>> Matthew - wow, so much to know about building with all the indie
>>> source --
>>> >>> thanks for sharing your knowledge.  I'm definitely still learning; I
>>> also
>>> >>> wish it were easy to build everything without having to make pom
>>> version
>>> >>> changes to avoid downloading community-built artifacts sneaking in.
>>> >>> Customizing all the version numbers in poms takes a chunk of
>>> start-up time,
>>> >>> and I found I really had to manually examine each pom for possible
>>> >>> dependencies that contain hard-coded version numbers for projects we
>>> build;
>>> >>> I didn't see how I could automate this sweeping version edit.  I'll
>>> look at
>>> >>> the 'groovy' work you did.  Maybe 2.9.x will be better without
>>> purepoms; and
>>> >>> some of this hard-coding of version numbers occurs in contrib tools.
>>>  The
>>> >>> use of version properties, and defining a version number in one
>>> place, is
>>> >>> extremely helpful, but I wasn't able to assume before that it was a
>>> practice
>>> >>> that was consistently followed.
>>> >>>
>>> >>> So, for the first build of a new release (after making the
>>> uva-version
>>> >>> updates), I closely examine the build logs for incorrect (not
>>> uva-built)
>>> >>> downloaded versions of artifacts and adjust until those we build are
>>> >>> referenced.
>>> >>>
>>> >>> We were considering not customizing the version numbers this time
>>> around,
>>> >>> to save some start up time, but I'm glad we've had this discussion
>>> today
>>> >>> because it now appears important that we continue to do so.
>>> >>>
>>> >>> Thanks,
>>> >>> Cindy @ UVa
>>> >>>
>>> >>>
>>> >>> ________________________________
>>> >>> From: Matthew Jones [matthew at longsight.com]
>>> >>> Sent: Wednesday, November 28, 2012 11:31 AM
>>> >>> To: Gast, Cynthia (cmw6s)
>>> >>> Cc: Steve Swinsburg; sakai-dev at collab.sakaiproject.org
>>> >>> Subject: Re: [Building Sakai] How to checkout the 2.9.0 tag source
>>> >>> including all tagged indie source?
>>> >>>
>>> >>> The "deploy" modules are not externals, they are actually *in* the
>>> >>> directory, see
>>> >>> https://source.sakaiproject.org/svn/sakai/tags/sakai-2.9.0/
>>> >>>
>>> >>> there is
>>> >>> core-deploy/
>>> >>> framework-shared-deploy/
>>> >>> jstl-shared-deploy/
>>> >>> kernel-deploy/
>>> >>>
>>> >>> In the 2.9.x-all there is only deploy (which deploys a few non-sakai
>>> >>> things into common and shared) and jstl-shared-deploy (which deploys
>>> a few
>>> >>> jstl things)
>>> >>>
>>> >>> With these 3 extra directory, the sakai:deploy could download the
>>> >>> assemblies in the repository and use those as part of the build.
>>> You'll
>>> >>> either need to remove those from the base pom or remove specific
>>> pieces you
>>> >>> don't want from their individual poms. It's possible that it could
>>> grab
>>> >>> assemblies that you build from your local repository but it still
>>> might
>>> >>> check for updates on Sonatype and use those instead. (Unless you have
>>> >>> different versions built and defined in master, then it would be
>>> forced to
>>> >>> use those of course)
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Wed, Nov 28, 2012 at 11:25 AM, Gast, Cynthia (cmw6s)
>>> >>> <cmw6s at eservices.virginia.edu> wrote:
>>> >>>>
>>> >>>> Hello Steve and Matthew:
>>> >>>> I'm attaching a file for your review, hopefully it will allow for
>>> the
>>> >>>> checkout of all source for the 2.9.0 tag release, including the
>>> source for
>>> >>>> all indie projects.
>>> >>>>
>>> >>>> It is a combination of the current 2.9.0-tag externals, plus all the
>>> >>>> indies 'tag' releases as also represented in the externals file from
>>> >>>> Matthew.
>>> >>>>
>>> >>>> I don't see the various "*-deploy" projects specifically listed, so
>>> I'm
>>> >>>> assuming these will come along as they normally do with the
>>> 2.9.0-tag
>>> >>>> checkout?
>>> >>>>
>>> >>>> I'm looking at this as a way to checkout (or export) the source for
>>> all
>>> >>>> projects in the release.  Building same will be the next step :-)
>>> >>>>
>>> >>>> Thanks for your willingness to help with this - I'd love to have a
>>> >>>> one-step source checkout.  Let me know when I should attempt a
>>> checkout for
>>> >>>> all tag release source.
>>> >>>>
>>> >>>> Cheers,
>>> >>>> Cindy @ UVa
>>> >>>>
>>> >>>>
>>> >>>> ________________________________
>>> >>>> From: sakai-dev-bounces at collab.sakaiproject.org
>>> >>>> [sakai-dev-bounces at collab.sakaiproject.org] on behalf of Gast,
>>> Cynthia
>>> >>>> (cmw6s) [cmw6s at eservices.virginia.edu]
>>> >>>> Sent: Wednesday, November 28, 2012 8:53 AM
>>> >>>> To: Steve Swinsburg; Matthew Jones
>>> >>>> Cc: sakai-dev at collab.sakaiproject.org
>>> >>>>
>>> >>>> Subject: Re: [Building Sakai] How to checkout the 2.9.0 tag source
>>> >>>> including all tagged indie source?
>>> >>>>
>>> >>>> Thanks Steve!
>>> >>>> I'll make a stab at the externals this morning, send it to you both
>>> --
>>> >>>> let you put it in place and I'll be happy to try it once you say so.
>>> >>>>
>>> >>>> Cindy
>>> >>>>
>>> >>>> ________________________________
>>> >>>> From: Steve Swinsburg [steve.swinsburg at gmail.com]
>>> >>>> Sent: Wednesday, November 28, 2012 6:40 AM
>>> >>>> To: Matthew Jones
>>> >>>> Cc: Gast, Cynthia (cmw6s); sakai-dev at collab.sakaiproject.org
>>> >>>> Subject: Re: [Building Sakai] How to checkout the 2.9.0 tag source
>>> >>>> including all tagged indie source?
>>> >>>>
>>> >>>> It should be pretty easy to just create a .externals that points to
>>> the
>>> >>>> various tags instead of the branch, using the tagged 2.9.0 one as a
>>> base,
>>> >>>> then adding in the indie tags. Then combine that with the tagged
>>> version of
>>> >>>> core-deploy and the main pom and its done.
>>> >>>>
>>> >>>> I'd imagine that is what Chris was suggesting, rather than specific
>>> >>>> revisions from the branch.
>>> >>>> If it wasn't late in the eve I'd do it. Chris/Cynthia if you dont
>>> get to
>>> >>>> it before Friday, I'll get onto this.
>>> >>>>
>>> >>>> cheers,
>>> >>>> Steve
>>> >>>>
>>> >>>>
>>> >>>> On Wed, Nov 28, 2012 at 12:19 PM, Matthew Jones <
>>> matthew at longsight.com>
>>> >>>> wrote:
>>> >>>>>
>>> >>>>> I'd just use the 2.9.x-all branch. If you used 2.9.0 you'd have to
>>> mess
>>> >>>>> around too much with the core-deploy and change too many other
>>> versions.
>>> >>>>>
>>> >>>>> If you wanted to lock it down to specific revisions, you'd want to
>>> have
>>> >>>>> an msub in svn and make a copy of the 2.9.x-all branch, then
>>> either manually
>>> >>>>> or with some script go down each external and set it to the
>>> revision which
>>> >>>>> was tagged for 2.9.0
>>> >>>>>
>>> >>>>> #Pseudocode
>>> >>>>> foreach external in 2.9.x-all {
>>> >>>>> do a regex of 2.9.x to 2.9.0
>>> >>>>> svn info on the 2.9.0 revision
>>> >>>>> print back revision number in the external
>>> >>>>> }
>>> >>>>> svn propset external
>>> >>>>> svn commit
>>> >>>>>
>>> >>>>> I had a script that *kind of* did this for Michigan, but it set the
>>> >>>>> latest 2.9.x revision, and I had to manually bump it back if we
>>> didn't want
>>> >>>>> that version.
>>> >>>>>
>>> >>>>>
>>> https://source.sakaiproject.org/svn/msub/umich.edu/ctools/builds/trunk/tools/versionSetter.pl
>>> >>>>>
>>> >>>>> I didn't think it would take more than an hour to write this
>>> script, it
>>> >>>>> ended up taking a little more than 2 hours because the poms are
>>> pretty
>>> >>>>> inconsistent. The names of the artifactId don't always match the
>>> names of
>>> >>>>> the directory they're in, not all poms have version tags in them
>>> (so they
>>> >>>>> inherit from the master, this is mostly true for the ones in core)
>>> and the
>>> >>>>> core is still on a 2 digit version.
>>> >>>>>
>>> >>>>> However the versions for core and indies for 2.9.0 should be
>>> correct in
>>> >>>>> this externals
>>> >>>>>
>>> https://source.sakaiproject.org/contrib/cle-release/scripts/externals/
>>> >>>>>
>>> >>>>> Then if you did want some newer version you could just update it.
>>> >>>>>
>>> >>>>> What was your suggestion, Chris?
>>> >>>>>
>>> >>>>>
>>> >>>>> On Tue, Nov 27, 2012 at 2:45 PM, Gast, Cynthia (cmw6s)
>>> >>>>> <cmw6s at eservices.virginia.edu> wrote:
>>> >>>>>>
>>> >>>>>> Hello:
>>> >>>>>>
>>> >>>>>> We are setting up for our 2.9.0 upgrade.  We need source for all
>>> the
>>> >>>>>> projects.  First, I exported the 2.9.0 tag, but this only
>>> provides source
>>> >>>>>> for some projects (as has been discussed); all indie project
>>> tagged source
>>> >>>>>> would have to be acquired still.  I took a look at the checkout
>>> using the
>>> >>>>>> 2.9.x-all branches (svn export
>>> >>>>>>
>>> https://source.sakaiproject.org/svn/sakai/branches/sakai-2.9.x-all/
>>> >>>>>> ./2.9.x-all).  I see the externals is set to the trunk branches.
>>>  So while
>>> >>>>>> it does allow me to acquire source for all the core + indie
>>> projects, it is
>>> >>>>>> source from trunk (with pom files referring to SNAPSHOT versions).
>>> >>>>>>
>>> >>>>>> Has anyone created an externals that allows one to export the
>>> source
>>> >>>>>> for the 2.9.0 tag plus the tag versions of the indies that make
>>> it up? ... a
>>> >>>>>> 2.9.0-all ?  This would be a huge help to those of us who do
>>> build all
>>> >>>>>> source.
>>> >>>>>>
>>> >>>>>> If not, would it help if I created the externals file to allow us
>>> to do
>>> >>>>>> this?  Otherwise, I'll go get each indie's source one-by-one
>>> according to
>>> >>>>>> the version tag.
>>> >>>>>>
>>> >>>>>> Thanks for considering this,
>>> >>>>>> Cynthia Gast
>>> >>>>>> UVa Sakai development team
>>> >>>>>> cmw6s at virginia.edu
>>> >>>>>>
>>> >>>>>> _______________________________________________
>>> >>>>>> sakai-dev mailing list
>>> >>>>>> sakai-dev at collab.sakaiproject.org
>>> >>>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>> >>>>>>
>>> >>>>>> TO UNSUBSCRIBE: send email to
>>> >>>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>> >>>>>> "unsubscribe"
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> _______________________________________________
>>> >>>>> sakai-dev mailing list
>>> >>>>> sakai-dev at collab.sakaiproject.org
>>> >>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>> >>>>>
>>> >>>>> TO UNSUBSCRIBE: send email to
>>> >>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>> >>>>> "unsubscribe"
>>> >>>>
>>> >>>>
>>> >>>
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> sakai-dev mailing list
>>> >> sakai-dev at collab.sakaiproject.org
>>> >> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>> >>
>>> >> TO UNSUBSCRIBE: send email to
>>> sakai-dev-unsubscribe at collab.sakaiproject.org
>>> >> with a subject of "unsubscribe"
>>> >
>>> >
>>> >
>>> _______________________________________________
>>> sakai-dev mailing list
>>> sakai-dev at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>
>>> TO UNSUBSCRIBE: send email to
>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>> "unsubscribe"
>>>
>>
>>
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
> "unsubscribe"
>
>
>
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
> "unsubscribe"
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20121129/4d913295/attachment.html 