[sakai-core-team] Work on migrating to github

Mon Dec 22 09:22:44 PST 2014

Mathew B. you should add these files to an empty git repo first before you
begin the migration (maybe your already doing that, I didn't check)

.gitattributes and .gitignore

I would also add the following to your .gitattributes (
http://git-scm.com/docs/gitattributes) file since performing deltas on
binary files is useless overhead (.

*.jpg  binary -delta
*.png  binary -delta
*.gif  binary -delta
*.gz   binary -delta
*.bz2  binary -delta
*.swf  binary -delta
*.zip  binary -delta
*.pdf  binary -delta

Also a .gitignore should be added as well (copied the defaults from sakai's
subversion config file...

**.project
**.classpath
**.settings
**.studio
**/target

Yeah git has some newer features to help handle large repositories that
include binary data.. The following ideas may help:

Shallow clone:
git clone --depth depth remote-url

Anyone not interested in a lot of history could use this you can still
perform push and pull. This is probably ideal for build servers as well...

On Sun, Dec 21, 2014 at 2:38 PM, Steve Swinsburg <steve.swinsburg at gmail.com>
wrote:
>
> Splitting it based on size just sounds like a potential for confusion; if
> you checkout the code you want the code. Osp I am indifferent about since
> it is no longer included, but reference and help should be in the main
> repo. Otherwise we get into a situation where some size barrier means the
> repo gets split, ie other tolls could just as easily grow to some threshold
> size and cause more work for someone.
>
> In terms of the skins that are not used much, do they need to be in the
> source at all? Providing example skins in the source is a bit old fashioned
> nowadays, providing a skinning guide with examples would be better.
>
> Cheers,
> Steve
>
> sent from my mobile device
> On 21/12/2014 3:49 AM, "Matthew Jones" <matthew at longsight.com> wrote:
>
>> Thinking about your conversation on git sizes and what was included in
>> your test repository I had a few thoughts.
>>
>> - OSP
>>
>>   I think we should just either leave this in SVN, possibly just migrate
>> to contrib or create a separate git repository. This is no longer included
>> by default in 11 and it's one of the biggest pieces of code in Sakai (13MB
>> in the git repository). Also excluding metaobj and warehouse would save 20
>> MB. I believe that all 3 of these projects should either be in svn contrib
>> or in a completely separate github repository from the start.
>>
>> - Reference
>>   By far the largest chunk of space in a checkout is the reference
>> folder, occupying over 100MB alone on disk. Large portions of this are
>> documentation, optional tools and optional skins not required to actually
>> get Sakai to function. At Longsight, we keep reference in a separate
>> repository (in github) and I believe it would be a good idea to also keep
>> reference separate. I don't love the idea of submodules but I think it does
>> make sense for reference since it would save a lot of time and space in a
>> typical checkout.  Additionally help may also be useful to have a a
>> submodule along with so it's easier to update and it seems like help is
>> tied into the reference documentation. I could go either way with help.
>>
>> Still it feels like it's highly more efficient than subversion in either
>> case. Excited about the move, happy about the plan that was discussed on
>> the call.
>>
>> On Thu, Dec 18, 2014 at 9:08 AM, Matthew Buckett <
>> matthew.buckett at it.ox.ac.uk> wrote:
>>>
>>> Initial I started with the git repositories we used at oxford to make
>>> our local build. In this setup each top level folder is a seprate git
>>> repository. I wrote some scripts that would merge these repositories
>>> together and re-write the paths. There are some problems with this
>>> approach:
>>> - If misses some history. There has been a reasonable amount of
>>> re-organisation of the Sakai SVN repositories and files have moved
>>> between top level folders. Using the multiple git repositories as a
>>> starting point means that moves between folders are lost.
>>> - Big ugly merged. I have only been successful in merging the multiple
>>> repositories into one by creating a merge commit for each one. This
>>> results in a root like structure for all the previous history. I have
>>> attempted to replay history into one continuous branch but I couldn't
>>> get it to work.
>>> You can see an old copy of this approach on github[1].
>>>
>>> To attempt to get around these problems I looked at reposurgeon[2],
>>> this first attempts to mirror the Sakai repository locally using
>>> svnsync, however this would eventually get to a revision (~60000) and
>>> fail complaining that the repository wasn't good. I wasn't able to
>>> work around this so I got a full SVN dump through Neal and used this
>>> dump with reposurgeon, however I then started getting problems with
>>> reposurgeon processing the dump which looked to be bugs in
>>> reposurgeon.
>>>
>>> I also attempted to get a complete mirror of of /svn into git
>>> (starting with a single repository) and this completed in a few days.
>>> I then attempted to first drop all the empty commits and this took
>>> days (using git-filter-branch).
>>>
>>> Recently I've had lots of local work which has meant that I haven't
>>> been able to look at this for a while.
>>>
>>> [1] - https://github.com/sakaiproject/test-svn-convert
>>> [2] - http://www.catb.org/esr/reposurgeon/
>>>
>>> --
>>>   Matthew Buckett, VLE Developer, IT Services, University of Oxford
>>> _______________________________________________
>>> sakai-core-team mailing list
>>> sakai-core-team at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-core-team
>>>
>>
>> _______________________________________________
>> sakai-core-team mailing list
>> sakai-core-team at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-core-team
>>
>>
> _______________________________________________
> sakai-core-team mailing list
> sakai-core-team at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-core-team
>
>

-- 
earle,
asahi net int.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-core-team/attachments/20141222/91c8dcb8/attachment.html 