[sakai-pmc] [sakai-forum] Re: What do we think of this so far?

[Charles Severance]

I have a gmane crawler that I wrote in Python. 

www.py4inf.com/code/gmane.zip

I use the sakai mailing list as the example data to crawl and have done this for a number of years.   It is described in Chapter 15 (page 198) of my Python book.

http://do1.dr-chuck.com/py4inf/EN-us/book.pdf

I would guess that my crawler is likely not exactly what we want in terms of pages, but it is a nice start.   It turns out that it takes a long time to crawl a gmane list due to their requests w.r.t. rate limiting - so it needs a database so it does not have to start over all the time.

I don't know if this helps at all - just figured I would share.

/Chuck

On Jan 6, 2015, at 8:52 AM, John A. Lewis - Unicon <jlewis at unicon.net> wrote:

> There is an API for importing individual messages. Maybe someone wants to look at scripting a migration? On a quick search I didn't find much else that is ready to use.
> 
> https://developers.google.com/admin-sdk/groups-migration/v1/guides/manage-email-migrations
> 
> On Tue, Jan 6, 2015 at 8:41 AM, Neal Caidin <neal.caidin at apereo.org> wrote:
> Yeah, I think we should. That's one of the things I need to investigate (or use help in investigating). Maybe we can move the archives to a simple web server?
> 
> 
> On Tue, Jan 6, 2015 at 8:34 AM, Seth Theriault <slt at columbia.edu> wrote:
> On Tue, Jan 6, 2015 at 8:17 AM, Neal Caidin <neal.caidin at apereo.org> wrote:
> > FYI, I would like to decommission all Mailman mailing lists this year in
> > favor of other options. I was leaning towards using Apereo Google Groups as
> > the most viable option.
> 
> Do we plan to maintain the Mailman list archives in a specific manner?
> 
> Seth
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-pmc/attachments/20150106/db608c58/attachment.html