Postby IAHM-COL » Wed May 04, 2016 2:39 pm
This is getting quite interesting.
First off, naturally http protocol does not have any indication on pre-existing data. The Hypertext transfer protocol was not developed with that in mind. I can see how you could use some sha1sum or the sort to decide if redownload changed packages, using some scripting mechanism, so that is an interesting idea. Again -- some kind of re-invention of the wheel when you compare to similar capabilities offered by either Git/subversion or alikes.
Keep in mind that once a sha1sum, or alike, determines a package has changed, with http you are due for a full download of the package itself, as opposed to a download of the diffs. Big difference, with a major impact on bandwith required to maintain
In any case, I also wonder how this works at a larger scale -- as in downloading the whole planet beforehand: which is certainly not in the mind of Torsten, whom I'd argue, likes the case scenery of not distributing the whole package completely.
Maintaining SVN along with http seems to me the correct approach. In fact terrafs does use http from libcurl already and works great for its purpose. In that sense, the only reasonable time to maintain a parallel operation, I think, is "indefinite".
Having a fallback is an awesome idea. That is reason #211 of having a git system is preferable ove a monolithic and unique svn repository. A monolithic repository of 100+GB as you have is some sort of a unconfortable joke -- which I am sure Sourceforge pointed you at when you got the space to load that thing.
Again, just to clarify, I am not suggesting that having http mirrors is a bad idea for snapshots. I am suggesting it is a rather primary and innocent idea for a development sandbox. And certainly not a great idea to massively distribute changing material (as terrasync): due to the inherent disabilties of tracking what's available and what's changing. In that sense, maintaining your svn is the second best alternative you have. In my opinion, the best one is migrating the development all-together to terraGIT and mirroring terraGIT content via http mirrors as you plan --an use those http mirrors to feed running FG.