[geomesa-users] StatsCombiner

[Jeff Gold <jeff@xxxxxxxxxx>]

GeoMesa Team-
   Was the StatsCombiner issue from Nathan from July ever resolved? I
don't see a solution to that part of his question in the thread below
and I'm hitting the same issue in my upgrade from 1.2.1 to 1.2.6.

I'm using namespaces, I've done the Accumulo restart...




Thanks,
Jeff





From: Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx>
Date: Fri, 15 Jul 2016 16:04:51 -0400
Delivered-to: geomesa-users@xxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
Thunderbird/38.8.0

________________________________
Hi Nathan,

We've got a fix for the contains issue here:
https://github.com/locationtech/geomesa/pull/995

If you get a chance, could you try it out and let us know if it fixes
your problem? You'll have to re-ingest your polygons so that the fix
takes effect.

Thanks,

Emilio

On 07/14/2016 04:50 PM, Emilio Lahr-Vivaz wrote:

Hi Nathan,

For your contains issue: we've identified a bug in the way we index
polygons. Basically at very fine resolutions (e.g. a point), there are
some gaps in our index. We're working on a fix, but in the meantime
you might avoid the issue by querying a small polygon instead of a
point - e.g. 'contains(geom, POLYGON((-1 -1, -1 1, 1 1, 1 -1, -1
-1)))'. To apply the point filtering, you can manually filter the
results coming back - e.g.

Filter filter = ECQL.toFilter("contains(geom, POINT(0 0))");
 ...
SimpleFeature feature = reader.next();

if (filter.evaluate(feature)) {
...
}

Thanks,

Emilio

On 07/11/2016 03:43 PM, Nathan Mercer wrote:

Thanks again Jim,



Yes I have restarted Accumulo. Still getting the errors. Continuing to
try to figure it out.



And since you asked, I have another issue. When I upgraded to 1.2.3 it
appears that the contains method is not working. I have a MULTIPOLYGON
type in geomesa and I want to get back the polygons that contain a
point I pass in. I have an application that does this through a Spark
RDD and it used to work fine in 1.2.1, but not after the upgrade. Same
thing if I manually try to do an export using the geomesa tools with a
CQL query using contains. This proves to me that it can’t be my code
and there must be a problem with geomesa update. Also note that I am
positive my polygons contain the point. My polygons make up the
boundaries of the US. And I use a point that is right in the middle of
the US.



Cheers,

Nathan





From: geomesa-users-bounces@xxxxxxxxxxxxxxxx
[mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Jim
Hughes
Sent: Friday, July 8, 2016 4:44 PM
To: geomesa-users@xxxxxxxxxxxxxxxx
Subject: Re: [geomesa-users] StatsCombiner



Hi Nathan,

I'll respond inline since there are two different questions.

On 07/08/2016 05:18 PM, Nathan Mercer wrote:

Thank you Jim.



I followed the instructions for setting up Accumulo namespace to
install the distributed runtime jar within HDFS as per the
instructions herehttp://www.geomesa.org/documentation/user/installation_and_configuration.html#installing-the-accumulo-distributed-runtime-library



However, I am still getting these errors, although it doesn’t appear
on ingest any more, but it is happening on my querying it appears. But
I am sure I am querying using 1.2.3 so it should be compatible. Isn’t
installing this runtime library equal to deploying the iterators? Or
is there something I am missing. I have also removed all jar files
from the Accumulo lib/ext path so these should not be getting in the
way. But what do I do about old tables that were ingested using an
older version of geomesa? Will I need to re-ingest these? With
namespaces, in the future I should simply have to specify the table
with the namespace prefix and all should be well, but all my old
tables were ingested without the use of namespaces.


As a long shot, after you setup the namespace and removed the jars
from lib/ext, did you restart Accumulo?  If not, the old jars could
still be in the Accumulo tablet servers JVMs.


Finally, I am also seeing another strange thing with version 1.2.3. I
have an application that creates an AccumuloDataStore object and when
the application ends, it doesn’t shut down. It just stays open. I have
narrowed it down to the data store object. If I revert back to 1.2.1
this does not happen (I skipped from 1.2.1 to 1.2.3).


This is a bug, and we've since fixed it.  The issue is that the new
pre-computed stats writer was hanging on in a static context.  We've
moved the stats writing to be managed by each datastore, and we are
cleaning it up via the DataStore.dispose() method.

Good questions!  Keep them coming!

Jim


Cheers,

Nathan



From: geomesa-users-bounces@xxxxxxxxxxxxxxxx
[mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Jim
Hughes
Sent: Wednesday, July 6, 2016 4:00 PM
To: geomesa-users@xxxxxxxxxxxxxxxx
Subject: Re: [geomesa-users] StatsCombiner



Hi Nathan,

GeoMesa 1.2.3 has the ability to gather stats on the data during
ingest.  In order to do this, the StatsCombiner is configured on any
new tables (assuming you don't turn the collectStats flag off).

You should just have to deploy the GeoMesa 1.2.3 iterators on your
Accumulo classpath (or on a namespace's classpath).

Generally, during an update, the GeoMesa code for ingest clients,
Accumulo iterators, and query clients (like GeoServer) all should
match.  Since coordinating that may be complicated, I'd call attention
to Accumulo namespaces; I've used namespaces like geomesa120,
geomesa121, etc. to help keep things straight on our internal dev
cluster.

Cheers,

Jim

On 07/06/2016 05:38 PM, Nathan Mercer wrote:

Hi. I am trying to upgrade to Geomesa 1.2.3



I have a program which does some data ingest using a FeatureWriter.
When I try ingesting, I get the following error
java.lang.ClassNotFoundException:
org.locationtech.geomesa.accumulo.data.stats.StatsCombiner



Is there something new I need to be adding to my pom file or something?



I have been playing around with this for a while and can’t figure it out.



Thanks,

Nathan


  ­­


_______________________________________________

geomesa-users mailing list

geomesa-users@xxxxxxxxxxxxxxxx

To change your delivery options, retrieve your password, or
unsubscribe from this list, visit

https://www.locationtech.org/mailman/listinfo/geomesa-users