Hi Tim, Kris,
We can definitely keep the conversation public.
On the topic of indexing, I'd note a few things: First, many
distributed multidimensional data storage and processing engines are
using space-filling curves. Because there is still some research
and development to optimize strategies and implementations, we
started the SFCurve project
(https://github.com/locationtech/sfcurve/) with the GeoTrellis
team. Currently, there are 2 and 3 dimensional implementations of a
Z-order curve. Cooking up a 4-dimensional Z-order curve would
straightforward. Alternatively, one could use the Uzaygezen
library (1) which implements arbitrary dimension compact Hilbert
curves.
I'd note that one doesn't necessarily require an index for all the
fields you might query against. As a concrete example, querying a
spatio-temporal index without a time bound requires a dramatic
increase in the number of ranges scanned. If there are common
access patterns for LIDAR where one queries by geo-extent only, then
a single 4d index would perform poorly.
There are two main ways to address this issue: first, one could
create multiple indices for the different query patterns. For
certain data sizes or query response constraints, this may make
sense. In fact, that's pretty typical for how GeoMesa handles
vector data. There are optimized tables which can address
spatio-temporal, spatio-only, attribute, and record queries. From
chatting with pointcloud folks, it sounds like they have so much
data that indexing it several times over is either prohibitive or
otherwise undesirable.
The second approach would be to be to craft an index which
compromises between various query types. This is where there's a
little research to sort out; a few of us have worked out the basic
details about composing space-filling curves. That technology
*should* be great for a case like this, but we need to find the time
(and funding) to work out an implementation, run some experiments,
write up the results, and advertise them...
Anyhow, I hope that helps give some idea of that state of things;
it's going to be exciting to see the pointcloud side of FOSS4G
software develop!
Cheers,
Jim
1. https://github.com/aioaneid/uzaygezen
On 7/18/2016 7:01 PM, EXI-Schier,
Timothy M wrote:
Hi Jim, Kris,
This is an area
of interest to likely more people as well, so can I ask if
your ongoing conversations could be in this forum please?
Does GeoMesa
have an efficient 4D index that would suit this?
Cheers,
Tim
Hi Kris,
It should be possible to use GeoMesa for pointcloud data. The
traditional way to get data into GeoMesa is via the GeoTools
datastore API; if you use that, each point would end up being
a SimpleFeature. (Alternatively, you could store a bundle of
clustered points together and do some fiddling in your
application.)
As an alternative, one could use the newer GeoMesa native API
to store collections of LIDAR points.
Either of these approaches would require a little elbow
grease. If you're interested, I'd be happy in discussing it
further.
Cheers,
Jim
On 07/18/2016 03:03 PM, Kris Nackaerts @
NoCrosstalk wrote:
Hi all,
According to the documentation Geomesa
focuses on a 3D index: X,Y,T. Is it possible to work with 3D
pointcloud data from Lidars? Preferably 4D. I’m looking for
an efficient storage and retrieval solution. To start with.
Thank’s,
Kris
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
|
