Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] StatsCombiner

Hi Nathan,

For your contains issue: we've identified a bug in the way we index polygons. Basically at very fine resolutions (e.g. a point), there are some gaps in our index. We're working on a fix, but in the meantime you might avoid the issue by querying a small polygon instead of a point - e.g. 'contains(geom, POLYGON((-1 -1, -1 1, 1 1, 1 -1, -1 -1)))'. To apply the point filtering, you can manually filter the results coming back - e.g.

Filter filter = ECQL.toFilter("contains(geom, POINT(0 0))");
 ...
SimpleFeature feature = reader.next();

if (filter.evaluate(feature)) {
...
}

Thanks,

Emilio

On 07/11/2016 03:43 PM, Nathan Mercer wrote:

Thanks again Jim,

 

Yes I have restarted Accumulo. Still getting the errors. Continuing to try to figure it out.

 

And since you asked, I have another issue. When I upgraded to 1.2.3 it appears that the contains method is not working. I have a MULTIPOLYGON type in geomesa and I want to get back the polygons that contain a point I pass in. I have an application that does this through a Spark RDD and it used to work fine in 1.2.1, but not after the upgrade. Same thing if I manually try to do an export using the geomesa tools with a CQL query using contains. This proves to me that it can’t be my code and there must be a problem with geomesa update. Also note that I am positive my polygons contain the point. My polygons make up the boundaries of the US. And I use a point that is right in the middle of the US.

 

Cheers,

Nathan

 

 

From: geomesa-users-bounces@xxxxxxxxxxxxxxxx [mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Jim Hughes
Sent: Friday, July 8, 2016 4:44 PM
To: geomesa-users@xxxxxxxxxxxxxxxx
Subject: Re: [geomesa-users] StatsCombiner

 

Hi Nathan,

I'll respond inline since there are two different questions.

On 07/08/2016 05:18 PM, Nathan Mercer wrote:

Thank you Jim.

 

I followed the instructions for setting up Accumulo namespace to install the distributed runtime jar within HDFS as per the instructions here http://www.geomesa.org/documentation/user/installation_and_configuration.html#installing-the-accumulo-distributed-runtime-library

 

However, I am still getting these errors, although it doesn’t appear on ingest any more, but it is happening on my querying it appears. But I am sure I am querying using 1.2.3 so it should be compatible. Isn’t installing this runtime library equal to deploying the iterators? Or is there something I am missing. I have also removed all jar files from the Accumulo lib/ext path so these should not be getting in the way. But what do I do about old tables that were ingested using an older version of geomesa? Will I need to re-ingest these? With namespaces, in the future I should simply have to specify the table with the namespace prefix and all should be well, but all my old tables were ingested without the use of namespaces.


As a long shot, after you setup the namespace and removed the jars from lib/ext, did you restart Accumulo?  If not, the old jars could still be in the Accumulo tablet servers JVMs. 


Finally, I am also seeing another strange thing with version 1.2.3. I have an application that creates an AccumuloDataStore object and when the application ends, it doesn’t shut down. It just stays open. I have narrowed it down to the data store object. If I revert back to 1.2.1 this does not happen (I skipped from 1.2.1 to 1.2.3).


This is a bug, and we've since fixed it.  The issue is that the new pre-computed stats writer was hanging on in a static context.  We've moved the stats writing to be managed by each datastore, and we are cleaning it up via the DataStore.dispose() method.
 
Good questions!  Keep them coming!

Jim


Cheers,

Nathan

 

From: geomesa-users-bounces@xxxxxxxxxxxxxxxx [mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Jim Hughes
Sent: Wednesday, July 6, 2016 4:00 PM
To: geomesa-users@xxxxxxxxxxxxxxxx
Subject: Re: [geomesa-users] StatsCombiner

 

Hi Nathan,

GeoMesa 1.2.3 has the ability to gather stats on the data during ingest.  In order to do this, the StatsCombiner is configured on any new tables (assuming you don't turn the collectStats flag off). 

You should just have to deploy the GeoMesa 1.2.3 iterators on your Accumulo classpath (or on a namespace's classpath).

Generally, during an update, the GeoMesa code for ingest clients, Accumulo iterators, and query clients (like GeoServer) all should match.  Since coordinating that may be complicated, I'd call attention to Accumulo namespaces; I've used namespaces like geomesa120, geomesa121, etc. to help keep things straight on our internal dev cluster.

Cheers,

Jim

On 07/06/2016 05:38 PM, Nathan Mercer wrote:

Hi. I am trying to upgrade to Geomesa 1.2.3

 

I have a program which does some data ingest using a FeatureWriter. When I try ingesting, I get the following error java.lang.ClassNotFoundException: org.locationtech.geomesa.accumulo.data.stats.StatsCombiner

 

Is there something new I need to be adding to my pom file or something?

 

I have been playing around with this for a while and can’t figure it out.

 

Thanks,

Nathan


  ­­  


_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users

 




_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users

 



_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users


Back to the top