The Future of Terrasync ...

Talking about the core development, vent steam ... censoring free but no guarantee, "they" will listen.
User avatar
IAHM-COL
Posts: 6409
Joined: Sat Sep 12, 2015 3:43 pm
Location: Homey, NV (KXTA) - U.S.A
Contact:

The Future of Terrasync ...

Postby IAHM-COL » Tue May 03, 2016 11:20 pm

...spells death

I had been reading up Torsten new ideas about terrasync, and if I get this right he is killing SVN.
So....

The scenery (OFFICIAL) will have no development sandbox of any kind. Keep updated means downloading the whole 100GB again via http. terramaster will stop being functional.

TerraGIT seems, to me, the only viable function that will be left. And that is -- I must say--- in part, thanks to it not being part of the core developers at the mercy of Torsten not needing to question anyone about his ideas.

:? :shock: :o Outrageous.
https://raw.githubusercontent.com/IAHM-COL/gpg-pubkey/master/pubkey.asc

R.M.S.
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it?

User avatar
jwocky
Site Admin
Posts: 1833
Joined: Sat Sep 12, 2015 12:04 pm
Contact:

Re: The Future of Terrasync ...

Postby jwocky » Wed May 04, 2016 12:19 am

So, well, we can't always stop them from killing themselves, all we can do is keeping stuff alive on our own ways.
Free speech can never be achieved by dictatorial measures!

User avatar
legoboyvdlp
Posts: 1757
Joined: Mon Sep 14, 2015 9:49 pm
Location: Venezuela

Re: The Future of Terrasync ...

Postby legoboyvdlp » Wed May 04, 2016 1:40 am

No!

I have a 100 KB/S internet -- I cannot download 100GB.
~~Legoboyvdlp~~
Maiquetia / Venezuela Custom Scenery
Hallo! Ich bin Jonathan.
Hey!
Avatar created by InSapphoWeTrust CC BY-SA 2.0, https://commons.wikimedia.org/w/index.p ... d=27409879

User avatar
legoboyvdlp
Posts: 1757
Joined: Mon Sep 14, 2015 9:49 pm
Location: Venezuela

Re: The Future of Terrasync ...

Postby legoboyvdlp » Wed May 04, 2016 1:45 am

It looks bad. I will test, and if it is as bad as I thought, I'll switch to TerraGit. I was too scared before, due to tile sizes and internet.
~~Legoboyvdlp~~
Maiquetia / Venezuela Custom Scenery
Hallo! Ich bin Jonathan.
Hey!
Avatar created by InSapphoWeTrust CC BY-SA 2.0, https://commons.wikimedia.org/w/index.p ... d=27409879

User avatar
legoboyvdlp
Posts: 1757
Joined: Mon Sep 14, 2015 9:49 pm
Location: Venezuela

Re: The Future of Terrasync ...

Postby legoboyvdlp » Wed May 04, 2016 3:17 am

I should let Torsten answer, but he's likely fast asleep right now (Europe), unless he has to get up to let out the dog ... in which case I would recommend he stays away from his computer so he can get back to sleep. :-)

My understanding is that the http download method will function very much as before where only the changed files (if any) are downloaded.

The svn method requires 2x the actual space because of the way svn works under the hood. Git typically requires that you download every version ever created to get the latest copy (there are work arounds if you just want the most recent version.)

So the end result for the average user that just wants to download and use the scenery is that the http method will be lighter weight overall. It also simplifies the code inside FlightGear quite a bit and should make the synchronization more robust. And as mentioned before, it should simplify the task of setting up a scenery mirror for those that wish to do that.

As before, power users will have multiple options for updating scenery or mixing and matching from different sources if they choose.

Best regards,

Curt.
~~Legoboyvdlp~~
Maiquetia / Venezuela Custom Scenery
Hallo! Ich bin Jonathan.
Hey!
Avatar created by InSapphoWeTrust CC BY-SA 2.0, https://commons.wikimedia.org/w/index.p ... d=27409879

User avatar
J Maverick 16
Posts: 757
Joined: Sun Nov 08, 2015 3:16 pm
Location: Northern-Italy
Contact:

Re: The Future of Terrasync ...

Postby J Maverick 16 » Wed May 04, 2016 6:48 am

I love TerraSync, I can't live without it... :cry:
Why they keep destroying things that mostly all the FG pilots use?
Regards, Mav
Breakin' the sound barrier every day!

Scenery designer, basic livery maker, aircraft developer (current project: F-16).
Using Thrustmaster FCS Flight Pack.
Follow me also on Instagram & Twitter @j_maverick16, Google+ and YouTube.

User avatar
IAHM-COL
Posts: 6409
Joined: Sat Sep 12, 2015 3:43 pm
Location: Homey, NV (KXTA) - U.S.A
Contact:

Re: The Future of Terrasync ...

Postby IAHM-COL » Wed May 04, 2016 2:29 pm

Torsten in the Devel.List wrote:
Let me try to calm down emotions here.
The technique of the http repository is much simpler than that of the svn repository and it will be pretty simple to create a local mirror or to download chunks of data.
With a bit of to-be-developed scripting (python e.g.) there is no need to download existing data again. I don't know how terramaster works but I am sure it is easy to adapt to the new http access method.

I apologize to not have everything ready at once and only implement the transition step-by-step.
The first step was to provide the necessary infrastructure. This is complete, we have one public master hosted at sourceforge and to privately donated mirrors.
The second step was to implement the client to access the infrastructure. This is complete, thanks to James for all the coding.
The third step was to invent and implement a service resolver. This is also complete.
The fourth step is to test all this and collect feedback. This is what we currently do.
The next steps depend on the test-results and the feedback.
Those steps will of course contain
- documentation of the service
- creation of the necessary access tools
- a reasonable time of parallel operation of SVN and HTTP

The goal is to completely remove SVN and only use the HTTP repository. I'd like to reach that goal better sooner than later as we currently have two single-points of failure for the SVN-service.
The first is the http://scenery.flightgear.org/svn-service web service (runs on a private web server) and the second is the one-and-only svn-server available for public access, also hosted on a private server. If either on of those fails, the SVN-terrasync is dead. I have learned the hard way that not having a fallback is a bad idea.

I hope this clarifies the process a bit.
https://raw.githubusercontent.com/IAHM-COL/gpg-pubkey/master/pubkey.asc

R.M.S.
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it?

User avatar
SHM
Posts: 1960
Joined: Mon Sep 14, 2015 3:32 pm
Location: India

Re: The Future of Terrasync ...

Postby SHM » Wed May 04, 2016 2:35 pm

Haha they won't stand a chance against Terragit.
FG Pilot (2011-2018)
Prepar3d (2015 - 2023)
MSFS2020 (2020 - )
Image

User avatar
IAHM-COL
Posts: 6409
Joined: Sat Sep 12, 2015 3:43 pm
Location: Homey, NV (KXTA) - U.S.A
Contact:

Re: The Future of Terrasync ...

Postby IAHM-COL » Wed May 04, 2016 2:39 pm

This is getting quite interesting.

First off, naturally http protocol does not have any indication on pre-existing data. The Hypertext transfer protocol was not developed with that in mind. I can see how you could use some sha1sum or the sort to decide if redownload changed packages, using some scripting mechanism, so that is an interesting idea. Again -- some kind of re-invention of the wheel when you compare to similar capabilities offered by either Git/subversion or alikes.

Keep in mind that once a sha1sum, or alike, determines a package has changed, with http you are due for a full download of the package itself, as opposed to a download of the diffs. Big difference, with a major impact on bandwith required to maintain

In any case, I also wonder how this works at a larger scale -- as in downloading the whole planet beforehand: which is certainly not in the mind of Torsten, whom I'd argue, likes the case scenery of not distributing the whole package completely.

Maintaining SVN along with http seems to me the correct approach. In fact terrafs does use http from libcurl already and works great for its purpose. In that sense, the only reasonable time to maintain a parallel operation, I think, is "indefinite".

Having a fallback is an awesome idea. That is reason #211 of having a git system is preferable ove a monolithic and unique svn repository. A monolithic repository of 100+GB as you have is some sort of a unconfortable joke -- which I am sure Sourceforge pointed you at when you got the space to load that thing.

Again, just to clarify, I am not suggesting that having http mirrors is a bad idea for snapshots. I am suggesting it is a rather primary and innocent idea for a development sandbox. And certainly not a great idea to massively distribute changing material (as terrasync): due to the inherent disabilties of tracking what's available and what's changing. In that sense, maintaining your svn is the second best alternative you have. In my opinion, the best one is migrating the development all-together to terraGIT and mirroring terraGIT content via http mirrors as you plan --an use those http mirrors to feed running FG.
https://raw.githubusercontent.com/IAHM-COL/gpg-pubkey/master/pubkey.asc

R.M.S.
If we gave everybody in the World free software today, but we failed to teach them about the four freedoms, five years from now, would they still have it?

OPFOR77
Posts: 208
Joined: Wed Apr 27, 2016 7:30 pm

Re: The Future of Terrasync ...

Postby OPFOR77 » Wed May 04, 2016 4:31 pm

IAHM-COL wrote:First off, naturally http protocol does not have any indication on pre-existing data. The Hypertext transfer protocol was not developed with that in mind. I can see how you could use some sha1sum or the sort to decide if redownload changed packages, using some scripting mechanism, so that is an interesting idea. Again -- some kind of re-invention of the wheel when you compare to similar capabilities offered by either Git/subversion or alikes.


They were going to implement rsync (or something like it) to see if the data has been updated.
OPRF Fighter Jock and Dev


Return to “Core Development”

Who is online

Users browsing this forum: No registered users and 12 guests