Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geogig-dev] IP Review Update

Here is an actual bug report for BDB JE: https://www.locationtech.org/bugs/show_bug.cgi?id=60


On Wed, Sep 2, 2015 at 11:06 AM, Gabriel Roldan <groldan@xxxxxxxxxxxxxxxx> wrote:
As a replacement for BDB JE, I've been prototyping on a sqlite replacement, since the current one sucks so much performance wise.

The one big thing I found is the lack of a connection pool as the biggest perf killer.
It's not so bad for large queries, but we do a lot of one object requests, which means opening a connection to the db for each single object to fetch, which in turns means opening the db file, reading the db metadata, etc.

Added a connection pool and got a great perf improvement.

The other thing I'm experimenting with is using a int index (allowing duplicates) composed of the first 4 bytes of each object's sha1. Disambiguation could be either done in the database with an extra index on the whole sha string (as currently implemented), or at runtime. The later would result in a smaller and faster db (we avoid a 40 chars table column plus its index). Problem with it is we would have no primary key, hence no chance to do INSERT OR IGNORE... always playing this compromises game...

As for jline, we could use another argument parser, probably getting rid of the geogig-console (kind of git sh) at least for now.


On Thu, Jul 16, 2015 at 2:10 PM, Jody Garnett <jgarnett@xxxxxxxxxxxxxxxx> wrote:
We should also upgrade to geotools 14-m0 or 14-m1 since it is almost cleared. 



On Jul 16, 2015, at 12:35 AM, Tyler Battle <tbattle@xxxxxxxxxxxxxxxx> wrote:

Sorry folks, I got sucked away onto other projects.

From my understanding, the situation is now:

Requested version change:
  Protobuf 2.4.1 -> 2.5.0 [osmosis]
  hsqldb 2.2.8 -> 2.3.0 [geotools] RESOLVED
  woodstox-core-asl 4.1.4 - 4.2.0 [osmosis]
  jetty 6.1.5 -> ? [com.noelios.restlet]
  apache commons-codec 1.7 -> 1.6 or 1.8 [osmosis]

Rejected (blocker):
  jline2 2.11 [geogig]
  BerkeleyDB for Java [geogig]
  jsr305 [geogig] RESOLVED

The version changes don't scare me much, but they are transitive dependencies which complicates things. I'm going to take a look at the osmosis dependencies in the next few days. Could someone take a look at the restlet/jetty issue?
JSR has been resolved (thanks Sebastian!)
BerkeleyDB and jline are still our most problematic issues. Has anyone had a chance to further examine these?




On 21 May 2015 at 15:30, Jody Garnett <jgarnett@xxxxxxxxxxxxxxxx> wrote:
I think that makes sense, lets start up a thread for each one so we can work through them without being confused.

On Thu, May 21, 2015 at 2:13 PM, Tyler Battle <tbattle@xxxxxxxxxxxxxxxx> wrote:
As Jody mentioned, there's outstanding work to do for us to clear LocationTech's IP review.

Requested version change:
  Protobuf 2.4.1 -> 2.5.0 [osmosis]
  hsqldb 2.2.8 -> 2.3.0 [geotools]
  woodstox-core-asl 4.1.4 - 4.2.0 [osmosis]
  jetty 6.1.5 -> ? [com.noelios.restlet]
  apache commons-codec 1.7 -> 1.6 or 1.8 [osmosis]

Rejected (blocker):
  jline2 2.11 [geogig]
  BerkeleyDB for Java [geogig]
  jsr305 [geogig]

There will probably be more as the IP team progresses.

--

Tyler Battle

Software Engineer| Boundless

tbattle@xxxxxxxxxxxxxxxx

Victoria, BC, Canada



_______________________________________________
geogig-dev mailing list
geogig-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geogig-dev



--

_______________________________________________
geogig-dev mailing list
geogig-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geogig-dev



--

Tyler Battle

Software Engineer| Boundless

tbattle@xxxxxxxxxxxxxxxx

Victoria, BC, Canada


_______________________________________________
geogig-dev mailing list
geogig-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geogig-dev

_______________________________________________
geogig-dev mailing list
geogig-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geogig-dev



--

Gabriel Roldán
Software Developer | Boundless
groldan@xxxxxxxxxxxxxxxx
@boundlessgeo



_______________________________________________
geogig-dev mailing list
geogig-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geogig-dev



--

--
Jody Garnett
Technical Lead | Boundless
jgarnett@xxxxxxxxxxxxxxxx
877-673-6436
@jodygarnett

http://boundlessgeo.com/


Back to the top