| Re: [geogig-dev] mapdb storage module prototype |
Hi Sebastian,I'm very pleased that you feel comfortable doing geogig development and trying things out. Way to go!WRT MapDB, I'm still very hesitant to propose it as the replacement for the default storage backend.I did use it last year, on a couple geogig experimental branches. Not for an objectdb implementation though, but similar, for a temporary Node index when importing very large datasets (in the order of whole America osm exports). I decided against it and implemented FileNodeIndex, much less ambitious and to the point of my requirements.The rationale for my hesitation is something like:- it is a one-man project (nothing essentially wrong with that, but...)- off-heap memory usage (i.e. MappedByteBuffer) works like a gem on Linux/Unix'es but I'm afraid it can be quite problematic on Windows.- performance seemed to degrade quite "ungracefully" anyways- at least at that time, it was good enough for a non huge node index, but lacked the persistence guarantees a "proper" backend storage would require for storing the repo's objects. LMN was even making "switch-the-plug" kind of consistency testing on the bdb je backend to ensure repo consistency under catastrophe.Now, _all_ of that is off the top of my head, and I'm sure the mapdb developers would argue furiously. But the bottom line for me is we rather go with something which reliability can be trusted by being established and well known/used by a wider audience, and ideally that can be read by other software/programming languages.That said, SQLite is quite a pain in the neck too, as far as I can tell, but if I/we get the usage pattern right it might be that "consolidated" backend we need.Does that make sense?Cheers,Gabriel--On Fri, Sep 11, 2015 at 10:01 AM, <Schmidt.Sebastian2@xxxxxx> wrote:Hi all,
during the last days, I've been experimenting with mapdb (www.mapdb.org)
in order to find out if it would be suitable as a backend for geogig. So
I prototyped a mapdb storage plugin for geogig both for objects and graph
databases.
https://github.com/sebastian-r-schmidt/geogig/tree/mapdb-storage
On Linux Systems I am able to do import, log, status, branch, merge and
show commands from the commandline. But don't expect it to be production
ready, it's more a functional prototype and would need a bit of
polishing before using it in a production context. For example, on a
Windows machines I ran into some weird problem with background writing
processes and file deletion coming from mapdb storage system. I will
look into that further.
If you would like to try it out have a look at
https://github.com/sebastian-r-schmidt/geogig/tree/mapdb-storage/src/storage/mapdb
how to activate, current status and todos.
My Experiences with mapdb:
* Very easy to use interface
* Performance "feels" ok, but I didn't stress test it really yet.
* Currently Beta releases of 2.0 are happening about every four weeks.
There are quite a few issues reported on the issue tracker, the author
seems to work on them. I'm not so sure how to judge on that - no issues
could also mean that no-one is using it.
* I ran into a problem with multi key Maps and "submapping" (fixed in
current Snapshot Release).
* No so sure if "beta" means Feature Completeness in the case of mapdb.
From 2.0-beta5 to 2.0-beta6 the internal storage format changed. Well,
that's we're currently doing with geogig as well, so I should not
complain :-)
* Single Process Database. If one Process (or Thread) is accessing it,
others cannot reopen. I ran into this while trying to access a geogig
repo with the geoserver-geogig extension. The Code for UI integration
opens the DB to check which branches exist, and then it holds the
lock to the DB preventing the data store to be created.
Comments or even help on coding would be warmly welcomed.
Kind regards,
Sebastian
_______________________________________________
geogig-dev mailing list
geogig-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geogig-dev
Gabriel Roldán
Software Developer | Boundless
groldan@xxxxxxxxxxxxxxxx
@boundlessgeo
Gabriel Roldán
Software Developer | Boundless
groldan@xxxxxxxxxxxxxxxx
@boundlessgeo