[Building Sakai] How to checkout the 2.9.0 tag source including all tagged indie source?

Wed Nov 28 12:16:08 PST 2012

I think that it *should* be possible to do an offline build once an
"online" build has been completed successfully. However, if I delete
the Sakai artifacts from my M2 repo and leave all the others I cannot
do an offline build of the 2.9 code.

I haven't had much luck convincing others that this is a real issue
for community users but hopefully this conversation will help with
that.

-AZ

On Wed, Nov 28, 2012 at 3:07 PM, Matthew Jones <matthew at longsight.com> wrote:
> I believe what you were describing was being seen by Unicon, which was why I
> had to keep cleaning the SNAPSHOTS from the repository and fixing bugs
> awhile ago.
>
> Indies have a parent of master
> The core has a parent of base which has a parent of master
>
> I believe that if you just dump the indies into base (which is not a parent
> of any of the indies) then when maven builds up it's dependency tree, it
> doesn't necessarily see that assignment is a dependency of lessons (for
> example) since it can just go out to the repository and get all of the
> artifacts that lessons needs. This is *probably* okay for the most part
> since the dependencies are really just apis and most people are just
> modifying the impls. But if you're modifying impls I don't think that maven
> would necessarily order assignments (and everything else) before lessons
> unless the parent of lessons was base. If you dumped the indies modules into
> master, I'd probably be more confident.
>
> It's likely this didn't turn up in 2.7 because the only indies then were
> kernel and like 2-3 others. There were some in 2.8 but less and most of the
> time people were probably messed up (or partially saved) anyway with
> purepoms and getting the versions that were defined locally in the top level
> of each tool anyway. I don't know because Michigan never went to 2.8 and
> Longsight has built off 2.8.x and also deploys tools individually.
>
> 2.9.0 (and maybe even the 2.9.x-all as far as indies are concerned) could
> potentially cause all new problems unless it's 100% guaranteed that maven
> does correct reactor ordering and never goes out to the repository for
> artifacts that it has available to build. Since it's not possibly to run in
> offline mode (because of artifacts that we don't have the source for) I'm
> not too sure I'd 100% believe it without seeing a few build logs for myself.
>
>
> On Wed, Nov 28, 2012 at 2:50 PM, Steve Swinsburg <steve.swinsburg at gmail.com>
> wrote:
>>
>> Changing the version numbers should not be required. If the source module
>> are listed in the base pom and there are no assembly deploy poms, then maven
>> will build a dependency graph and adjust the build order to suit. It does
>> that now, it's just that since there is no indie source it will get
>> artifacts solely from the remote repo. If you look at the build you'll see
>> individual jars being downloaded, like API etc, as well as the all in one
>> tomcat overlay. It's redundant but that's how Maven sees the dependency
>> graph.
>>
>> So adding the source modules as per the current -all branches, you should
>> not get any sakai app artifacts being downloaded, they should all be built.
>>
>> If this does still occur then there is a fundamental issue, and it would
>> have been seen in 2.7 and earlier, since as part of the tag/release process
>> ALL artifacts are deployed to the Sakai maven repo and you would have seen
>> the conflicts then.
>>
>> http://source.sakaiproject.org/maven2/org/sakaiproject/sakai-announcement-api/
>>
>> I'll have a look at the parts you posted Cynthia, and get this finalised
>> today.
>>
>> Cheers
>> Steve
>>
>> Sent from my iPhone
>>
>> On 29/11/2012, at 4:46, "Gast, Cynthia (cmw6s)"
>> <cmw6s at eservices.virginia.edu> wrote:
>>
>> Matthew - wow, so much to know about building with all the indie source --
>> thanks for sharing your knowledge.  I'm definitely still learning; I also
>> wish it were easy to build everything without having to make pom version
>> changes to avoid downloading community-built artifacts sneaking in.
>> Customizing all the version numbers in poms takes a chunk of start-up time,
>> and I found I really had to manually examine each pom for possible
>> dependencies that contain hard-coded version numbers for projects we build;
>> I didn't see how I could automate this sweeping version edit.  I'll look at
>> the 'groovy' work you did.  Maybe 2.9.x will be better without purepoms; and
>> some of this hard-coding of version numbers occurs in contrib tools.  The
>> use of version properties, and defining a version number in one place, is
>> extremely helpful, but I wasn't able to assume before that it was a practice
>> that was consistently followed.
>>
>> So, for the first build of a new release (after making the uva-version
>> updates), I closely examine the build logs for incorrect (not uva-built)
>> downloaded versions of artifacts and adjust until those we build are
>> referenced.
>>
>> We were considering not customizing the version numbers this time around,
>> to save some start up time, but I'm glad we've had this discussion today
>> because it now appears important that we continue to do so.
>>
>> Thanks,
>> Cindy @ UVa
>>
>>
>> ________________________________
>> From: Matthew Jones [matthew at longsight.com]
>> Sent: Wednesday, November 28, 2012 11:31 AM
>> To: Gast, Cynthia (cmw6s)
>> Cc: Steve Swinsburg; sakai-dev at collab.sakaiproject.org
>> Subject: Re: [Building Sakai] How to checkout the 2.9.0 tag source
>> including all tagged indie source?
>>
>> The "deploy" modules are not externals, they are actually *in* the
>> directory, see
>> https://source.sakaiproject.org/svn/sakai/tags/sakai-2.9.0/
>>
>> there is
>> core-deploy/
>> framework-shared-deploy/
>> jstl-shared-deploy/
>> kernel-deploy/
>>
>> In the 2.9.x-all there is only deploy (which deploys a few non-sakai
>> things into common and shared) and jstl-shared-deploy (which deploys a few
>> jstl things)
>>
>> With these 3 extra directory, the sakai:deploy could download the
>> assemblies in the repository and use those as part of the build. You'll
>> either need to remove those from the base pom or remove specific pieces you
>> don't want from their individual poms. It's possible that it could grab
>> assemblies that you build from your local repository but it still might
>> check for updates on Sonatype and use those instead. (Unless you have
>> different versions built and defined in master, then it would be forced to
>> use those of course)
>>
>>
>>
>>
>> On Wed, Nov 28, 2012 at 11:25 AM, Gast, Cynthia (cmw6s)
>> <cmw6s at eservices.virginia.edu> wrote:
>>>
>>> Hello Steve and Matthew:
>>> I'm attaching a file for your review, hopefully it will allow for the
>>> checkout of all source for the 2.9.0 tag release, including the source for
>>> all indie projects.
>>>
>>> It is a combination of the current 2.9.0-tag externals, plus all the
>>> indies 'tag' releases as also represented in the externals file from
>>> Matthew.
>>>
>>> I don't see the various "*-deploy" projects specifically listed, so I'm
>>> assuming these will come along as they normally do with the 2.9.0-tag
>>> checkout?
>>>
>>> I'm looking at this as a way to checkout (or export) the source for all
>>> projects in the release.  Building same will be the next step :-)
>>>
>>> Thanks for your willingness to help with this - I'd love to have a
>>> one-step source checkout.  Let me know when I should attempt a checkout for
>>> all tag release source.
>>>
>>> Cheers,
>>> Cindy @ UVa
>>>
>>>
>>> ________________________________
>>> From: sakai-dev-bounces at collab.sakaiproject.org
>>> [sakai-dev-bounces at collab.sakaiproject.org] on behalf of Gast, Cynthia
>>> (cmw6s) [cmw6s at eservices.virginia.edu]
>>> Sent: Wednesday, November 28, 2012 8:53 AM
>>> To: Steve Swinsburg; Matthew Jones
>>> Cc: sakai-dev at collab.sakaiproject.org
>>>
>>> Subject: Re: [Building Sakai] How to checkout the 2.9.0 tag source
>>> including all tagged indie source?
>>>
>>> Thanks Steve!
>>> I'll make a stab at the externals this morning, send it to you both --
>>> let you put it in place and I'll be happy to try it once you say so.
>>>
>>> Cindy
>>>
>>> ________________________________
>>> From: Steve Swinsburg [steve.swinsburg at gmail.com]
>>> Sent: Wednesday, November 28, 2012 6:40 AM
>>> To: Matthew Jones
>>> Cc: Gast, Cynthia (cmw6s); sakai-dev at collab.sakaiproject.org
>>> Subject: Re: [Building Sakai] How to checkout the 2.9.0 tag source
>>> including all tagged indie source?
>>>
>>> It should be pretty easy to just create a .externals that points to the
>>> various tags instead of the branch, using the tagged 2.9.0 one as a base,
>>> then adding in the indie tags. Then combine that with the tagged version of
>>> core-deploy and the main pom and its done.
>>>
>>> I'd imagine that is what Chris was suggesting, rather than specific
>>> revisions from the branch.
>>> If it wasn't late in the eve I'd do it. Chris/Cynthia if you dont get to
>>> it before Friday, I'll get onto this.
>>>
>>> cheers,
>>> Steve
>>>
>>>
>>> On Wed, Nov 28, 2012 at 12:19 PM, Matthew Jones <matthew at longsight.com>
>>> wrote:
>>>>
>>>> I'd just use the 2.9.x-all branch. If you used 2.9.0 you'd have to mess
>>>> around too much with the core-deploy and change too many other versions.
>>>>
>>>> If you wanted to lock it down to specific revisions, you'd want to have
>>>> an msub in svn and make a copy of the 2.9.x-all branch, then either manually
>>>> or with some script go down each external and set it to the revision which
>>>> was tagged for 2.9.0
>>>>
>>>> #Pseudocode
>>>> foreach external in 2.9.x-all {
>>>> do a regex of 2.9.x to 2.9.0
>>>> svn info on the 2.9.0 revision
>>>> print back revision number in the external
>>>> }
>>>> svn propset external
>>>> svn commit
>>>>
>>>> I had a script that *kind of* did this for Michigan, but it set the
>>>> latest 2.9.x revision, and I had to manually bump it back if we didn't want
>>>> that version.
>>>>
>>>> https://source.sakaiproject.org/svn/msub/umich.edu/ctools/builds/trunk/tools/versionSetter.pl
>>>>
>>>> I didn't think it would take more than an hour to write this script, it
>>>> ended up taking a little more than 2 hours because the poms are pretty
>>>> inconsistent. The names of the artifactId don't always match the names of
>>>> the directory they're in, not all poms have version tags in them (so they
>>>> inherit from the master, this is mostly true for the ones in core) and the
>>>> core is still on a 2 digit version.
>>>>
>>>> However the versions for core and indies for 2.9.0 should be correct in
>>>> this externals
>>>> https://source.sakaiproject.org/contrib/cle-release/scripts/externals/
>>>>
>>>> Then if you did want some newer version you could just update it.
>>>>
>>>> What was your suggestion, Chris?
>>>>
>>>>
>>>> On Tue, Nov 27, 2012 at 2:45 PM, Gast, Cynthia (cmw6s)
>>>> <cmw6s at eservices.virginia.edu> wrote:
>>>>>
>>>>> Hello:
>>>>>
>>>>> We are setting up for our 2.9.0 upgrade.  We need source for all the
>>>>> projects.  First, I exported the 2.9.0 tag, but this only provides source
>>>>> for some projects (as has been discussed); all indie project tagged source
>>>>> would have to be acquired still.  I took a look at the checkout using the
>>>>> 2.9.x-all branches (svn export
>>>>> https://source.sakaiproject.org/svn/sakai/branches/sakai-2.9.x-all/
>>>>> ./2.9.x-all).  I see the externals is set to the trunk branches.  So while
>>>>> it does allow me to acquire source for all the core + indie projects, it is
>>>>> source from trunk (with pom files referring to SNAPSHOT versions).
>>>>>
>>>>> Has anyone created an externals that allows one to export the source
>>>>> for the 2.9.0 tag plus the tag versions of the indies that make it up? ... a
>>>>> 2.9.0-all ?  This would be a huge help to those of us who do build all
>>>>> source.
>>>>>
>>>>> If not, would it help if I created the externals file to allow us to do
>>>>> this?  Otherwise, I'll go get each indie's source one-by-one according to
>>>>> the version tag.
>>>>>
>>>>> Thanks for considering this,
>>>>> Cynthia Gast
>>>>> UVa Sakai development team
>>>>> cmw6s at virginia.edu
>>>>>
>>>>> _______________________________________________
>>>>> sakai-dev mailing list
>>>>> sakai-dev at collab.sakaiproject.org
>>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>>>
>>>>> TO UNSUBSCRIBE: send email to
>>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>>>> "unsubscribe"
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> sakai-dev mailing list
>>>> sakai-dev at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>>
>>>> TO UNSUBSCRIBE: send email to
>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>>> "unsubscribe"
>>>
>>>
>>
>
>
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org
> with a subject of "unsubscribe"

-- 
Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile