[Building Sakai] How to checkout the 2.9.0 tag source including all tagged indie source?

Wed Nov 28 16:30:10 PST 2012

I've created a 2.9.0-all tag here:
https://source.sakaiproject.org/svn/sakai/tags/sakai-2.9.0-all/

It has all of the relevant source including indies and modules updated etc,
and seems to work fine. I modified one of the API's in one of the projects
(just added a string constant), did a full rebuild of the entire lot,
decompiled the deployed version of that jar that I modified and it matched
my local source.

Could someone else please verify?

cheers,
Steve

On Thu, Nov 29, 2012 at 7:43 AM, Cris J Holdorph <holdorph at unicon.net>wrote:

> I had a similar experience to Aaron, but with slightly tweaked code.  I
> worked on it a while.  I didn't exactly give up, but the situation I
> finally ended up in was a bunch of custom 1-offs that I had to build
> locally like of specific versions of purepom RELEASES, plugins,
> sakai-rsf, etc.  Once I had done all of those *AND* had updated a few of
> the sakai 2.9.x-all sources to point to my 1-offs, then I could finally
> remove all of the ~/.m2/repository/org/sakaiproject/ directory tree and
> do an offline build.  It did require doing things in the right order
> though (like plugin, purepoms, sakai-rsf, etc.).
>
> ---- Cris J H
>
> On 11/28/2012 01:16 PM, Aaron Zeckoski wrote:
> > I think that it *should* be possible to do an offline build once an
> > "online" build has been completed successfully. However, if I delete
> > the Sakai artifacts from my M2 repo and leave all the others I cannot
> > do an offline build of the 2.9 code.
> >
> > I haven't had much luck convincing others that this is a real issue
> > for community users but hopefully this conversation will help with
> > that.
> >
> > -AZ
> >
> >
> > On Wed, Nov 28, 2012 at 3:07 PM, Matthew Jones <matthew at longsight.com>
> wrote:
> >> I believe what you were describing was being seen by Unicon, which was
> why I
> >> had to keep cleaning the SNAPSHOTS from the repository and fixing bugs
> >> awhile ago.
> >>
> >> Indies have a parent of master
> >> The core has a parent of base which has a parent of master
> >>
> >> I believe that if you just dump the indies into base (which is not a
> parent
> >> of any of the indies) then when maven builds up it's dependency tree, it
> >> doesn't necessarily see that assignment is a dependency of lessons (for
> >> example) since it can just go out to the repository and get all of the
> >> artifacts that lessons needs. This is *probably* okay for the most part
> >> since the dependencies are really just apis and most people are just
> >> modifying the impls. But if you're modifying impls I don't think that
> maven
> >> would necessarily order assignments (and everything else) before lessons
> >> unless the parent of lessons was base. If you dumped the indies modules
> into
> >> master, I'd probably be more confident.
> >>
> >> It's likely this didn't turn up in 2.7 because the only indies then were
> >> kernel and like 2-3 others. There were some in 2.8 but less and most of
> the
> >> time people were probably messed up (or partially saved) anyway with
> >> purepoms and getting the versions that were defined locally in the top
> level
> >> of each tool anyway. I don't know because Michigan never went to 2.8 and
> >> Longsight has built off 2.8.x and also deploys tools individually.
> >>
> >> 2.9.0 (and maybe even the 2.9.x-all as far as indies are concerned)
> could
> >> potentially cause all new problems unless it's 100% guaranteed that
> maven
> >> does correct reactor ordering and never goes out to the repository for
> >> artifacts that it has available to build. Since it's not possibly to
> run in
> >> offline mode (because of artifacts that we don't have the source for)
> I'm
> >> not too sure I'd 100% believe it without seeing a few build logs for
> myself.
> >>
> >>
> >> On Wed, Nov 28, 2012 at 2:50 PM, Steve Swinsburg <
> steve.swinsburg at gmail.com>
> >> wrote:
> >>>
> >>> Changing the version numbers should not be required. If the source
> module
> >>> are listed in the base pom and there are no assembly deploy poms, then
> maven
> >>> will build a dependency graph and adjust the build order to suit. It
> does
> >>> that now, it's just that since there is no indie source it will get
> >>> artifacts solely from the remote repo. If you look at the build you'll
> see
> >>> individual jars being downloaded, like API etc, as well as the all in
> one
> >>> tomcat overlay. It's redundant but that's how Maven sees the dependency
> >>> graph.
> >>>
> >>> So adding the source modules as per the current -all branches, you
> should
> >>> not get any sakai app artifacts being downloaded, they should all be
> built.
> >>>
> >>> If this does still occur then there is a fundamental issue, and it
> would
> >>> have been seen in 2.7 and earlier, since as part of the tag/release
> process
> >>> ALL artifacts are deployed to the Sakai maven repo and you would have
> seen
> >>> the conflicts then.
> >>>
> >>>
> http://source.sakaiproject.org/maven2/org/sakaiproject/sakai-announcement-api/
> >>>
> >>> I'll have a look at the parts you posted Cynthia, and get this
> finalised
> >>> today.
> >>>
> >>> Cheers
> >>> Steve
> >>>
> >>> Sent from my iPhone
> >>>
> >>> On 29/11/2012, at 4:46, "Gast, Cynthia (cmw6s)"
> >>> <cmw6s at eservices.virginia.edu> wrote:
> >>>
> >>> Matthew - wow, so much to know about building with all the indie
> source --
> >>> thanks for sharing your knowledge.  I'm definitely still learning; I
> also
> >>> wish it were easy to build everything without having to make pom
> version
> >>> changes to avoid downloading community-built artifacts sneaking in.
> >>> Customizing all the version numbers in poms takes a chunk of start-up
> time,
> >>> and I found I really had to manually examine each pom for possible
> >>> dependencies that contain hard-coded version numbers for projects we
> build;
> >>> I didn't see how I could automate this sweeping version edit.  I'll
> look at
> >>> the 'groovy' work you did.  Maybe 2.9.x will be better without
> purepoms; and
> >>> some of this hard-coding of version numbers occurs in contrib tools.
>  The
> >>> use of version properties, and defining a version number in one place,
> is
> >>> extremely helpful, but I wasn't able to assume before that it was a
> practice
> >>> that was consistently followed.
> >>>
> >>> So, for the first build of a new release (after making the uva-version
> >>> updates), I closely examine the build logs for incorrect (not
> uva-built)
> >>> downloaded versions of artifacts and adjust until those we build are
> >>> referenced.
> >>>
> >>> We were considering not customizing the version numbers this time
> around,
> >>> to save some start up time, but I'm glad we've had this discussion
> today
> >>> because it now appears important that we continue to do so.
> >>>
> >>> Thanks,
> >>> Cindy @ UVa
> >>>
> >>>
> >>> ________________________________
> >>> From: Matthew Jones [matthew at longsight.com]
> >>> Sent: Wednesday, November 28, 2012 11:31 AM
> >>> To: Gast, Cynthia (cmw6s)
> >>> Cc: Steve Swinsburg; sakai-dev at collab.sakaiproject.org
> >>> Subject: Re: [Building Sakai] How to checkout the 2.9.0 tag source
> >>> including all tagged indie source?
> >>>
> >>> The "deploy" modules are not externals, they are actually *in* the
> >>> directory, see
> >>> https://source.sakaiproject.org/svn/sakai/tags/sakai-2.9.0/
> >>>
> >>> there is
> >>> core-deploy/
> >>> framework-shared-deploy/
> >>> jstl-shared-deploy/
> >>> kernel-deploy/
> >>>
> >>> In the 2.9.x-all there is only deploy (which deploys a few non-sakai
> >>> things into common and shared) and jstl-shared-deploy (which deploys a
> few
> >>> jstl things)
> >>>
> >>> With these 3 extra directory, the sakai:deploy could download the
> >>> assemblies in the repository and use those as part of the build. You'll
> >>> either need to remove those from the base pom or remove specific
> pieces you
> >>> don't want from their individual poms. It's possible that it could grab
> >>> assemblies that you build from your local repository but it still might
> >>> check for updates on Sonatype and use those instead. (Unless you have
> >>> different versions built and defined in master, then it would be
> forced to
> >>> use those of course)
> >>>
> >>>
> >>>
> >>>
> >>> On Wed, Nov 28, 2012 at 11:25 AM, Gast, Cynthia (cmw6s)
> >>> <cmw6s at eservices.virginia.edu> wrote:
> >>>>
> >>>> Hello Steve and Matthew:
> >>>> I'm attaching a file for your review, hopefully it will allow for the
> >>>> checkout of all source for the 2.9.0 tag release, including the
> source for
> >>>> all indie projects.
> >>>>
> >>>> It is a combination of the current 2.9.0-tag externals, plus all the
> >>>> indies 'tag' releases as also represented in the externals file from
> >>>> Matthew.
> >>>>
> >>>> I don't see the various "*-deploy" projects specifically listed, so
> I'm
> >>>> assuming these will come along as they normally do with the 2.9.0-tag
> >>>> checkout?
> >>>>
> >>>> I'm looking at this as a way to checkout (or export) the source for
> all
> >>>> projects in the release.  Building same will be the next step :-)
> >>>>
> >>>> Thanks for your willingness to help with this - I'd love to have a
> >>>> one-step source checkout.  Let me know when I should attempt a
> checkout for
> >>>> all tag release source.
> >>>>
> >>>> Cheers,
> >>>> Cindy @ UVa
> >>>>
> >>>>
> >>>> ________________________________
> >>>> From: sakai-dev-bounces at collab.sakaiproject.org
> >>>> [sakai-dev-bounces at collab.sakaiproject.org] on behalf of Gast,
> Cynthia
> >>>> (cmw6s) [cmw6s at eservices.virginia.edu]
> >>>> Sent: Wednesday, November 28, 2012 8:53 AM
> >>>> To: Steve Swinsburg; Matthew Jones
> >>>> Cc: sakai-dev at collab.sakaiproject.org
> >>>>
> >>>> Subject: Re: [Building Sakai] How to checkout the 2.9.0 tag source
> >>>> including all tagged indie source?
> >>>>
> >>>> Thanks Steve!
> >>>> I'll make a stab at the externals this morning, send it to you both --
> >>>> let you put it in place and I'll be happy to try it once you say so.
> >>>>
> >>>> Cindy
> >>>>
> >>>> ________________________________
> >>>> From: Steve Swinsburg [steve.swinsburg at gmail.com]
> >>>> Sent: Wednesday, November 28, 2012 6:40 AM
> >>>> To: Matthew Jones
> >>>> Cc: Gast, Cynthia (cmw6s); sakai-dev at collab.sakaiproject.org
> >>>> Subject: Re: [Building Sakai] How to checkout the 2.9.0 tag source
> >>>> including all tagged indie source?
> >>>>
> >>>> It should be pretty easy to just create a .externals that points to
> the
> >>>> various tags instead of the branch, using the tagged 2.9.0 one as a
> base,
> >>>> then adding in the indie tags. Then combine that with the tagged
> version of
> >>>> core-deploy and the main pom and its done.
> >>>>
> >>>> I'd imagine that is what Chris was suggesting, rather than specific
> >>>> revisions from the branch.
> >>>> If it wasn't late in the eve I'd do it. Chris/Cynthia if you dont get
> to
> >>>> it before Friday, I'll get onto this.
> >>>>
> >>>> cheers,
> >>>> Steve
> >>>>
> >>>>
> >>>> On Wed, Nov 28, 2012 at 12:19 PM, Matthew Jones <
> matthew at longsight.com>
> >>>> wrote:
> >>>>>
> >>>>> I'd just use the 2.9.x-all branch. If you used 2.9.0 you'd have to
> mess
> >>>>> around too much with the core-deploy and change too many other
> versions.
> >>>>>
> >>>>> If you wanted to lock it down to specific revisions, you'd want to
> have
> >>>>> an msub in svn and make a copy of the 2.9.x-all branch, then either
> manually
> >>>>> or with some script go down each external and set it to the revision
> which
> >>>>> was tagged for 2.9.0
> >>>>>
> >>>>> #Pseudocode
> >>>>> foreach external in 2.9.x-all {
> >>>>> do a regex of 2.9.x to 2.9.0
> >>>>> svn info on the 2.9.0 revision
> >>>>> print back revision number in the external
> >>>>> }
> >>>>> svn propset external
> >>>>> svn commit
> >>>>>
> >>>>> I had a script that *kind of* did this for Michigan, but it set the
> >>>>> latest 2.9.x revision, and I had to manually bump it back if we
> didn't want
> >>>>> that version.
> >>>>>
> >>>>>
> https://source.sakaiproject.org/svn/msub/umich.edu/ctools/builds/trunk/tools/versionSetter.pl
> >>>>>
> >>>>> I didn't think it would take more than an hour to write this script,
> it
> >>>>> ended up taking a little more than 2 hours because the poms are
> pretty
> >>>>> inconsistent. The names of the artifactId don't always match the
> names of
> >>>>> the directory they're in, not all poms have version tags in them (so
> they
> >>>>> inherit from the master, this is mostly true for the ones in core)
> and the
> >>>>> core is still on a 2 digit version.
> >>>>>
> >>>>> However the versions for core and indies for 2.9.0 should be correct
> in
> >>>>> this externals
> >>>>>
> https://source.sakaiproject.org/contrib/cle-release/scripts/externals/
> >>>>>
> >>>>> Then if you did want some newer version you could just update it.
> >>>>>
> >>>>> What was your suggestion, Chris?
> >>>>>
> >>>>>
> >>>>> On Tue, Nov 27, 2012 at 2:45 PM, Gast, Cynthia (cmw6s)
> >>>>> <cmw6s at eservices.virginia.edu> wrote:
> >>>>>>
> >>>>>> Hello:
> >>>>>>
> >>>>>> We are setting up for our 2.9.0 upgrade.  We need source for all the
> >>>>>> projects.  First, I exported the 2.9.0 tag, but this only provides
> source
> >>>>>> for some projects (as has been discussed); all indie project tagged
> source
> >>>>>> would have to be acquired still.  I took a look at the checkout
> using the
> >>>>>> 2.9.x-all branches (svn export
> >>>>>> https://source.sakaiproject.org/svn/sakai/branches/sakai-2.9.x-all/
> >>>>>> ./2.9.x-all).  I see the externals is set to the trunk branches.
>  So while
> >>>>>> it does allow me to acquire source for all the core + indie
> projects, it is
> >>>>>> source from trunk (with pom files referring to SNAPSHOT versions).
> >>>>>>
> >>>>>> Has anyone created an externals that allows one to export the source
> >>>>>> for the 2.9.0 tag plus the tag versions of the indies that make it
> up? ... a
> >>>>>> 2.9.0-all ?  This would be a huge help to those of us who do build
> all
> >>>>>> source.
> >>>>>>
> >>>>>> If not, would it help if I created the externals file to allow us
> to do
> >>>>>> this?  Otherwise, I'll go get each indie's source one-by-one
> according to
> >>>>>> the version tag.
> >>>>>>
> >>>>>> Thanks for considering this,
> >>>>>> Cynthia Gast
> >>>>>> UVa Sakai development team
> >>>>>> cmw6s at virginia.edu
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> sakai-dev mailing list
> >>>>>> sakai-dev at collab.sakaiproject.org
> >>>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >>>>>>
> >>>>>> TO UNSUBSCRIBE: send email to
> >>>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
> >>>>>> "unsubscribe"
> >>>>>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> sakai-dev mailing list
> >>>>> sakai-dev at collab.sakaiproject.org
> >>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >>>>>
> >>>>> TO UNSUBSCRIBE: send email to
> >>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
> >>>>> "unsubscribe"
> >>>>
> >>>>
> >>>
> >>
> >>
> >> _______________________________________________
> >> sakai-dev mailing list
> >> sakai-dev at collab.sakaiproject.org
> >> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >>
> >> TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org
> >> with a subject of "unsubscribe"
> >
> >
> >
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
> "unsubscribe"
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20121129/0f9dbb9d/attachment.html 