Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] Query with coordinate 0.0

Thanks for circling back, and thanks again for bringing it to our attention in the first place!

-Emilio

On 06/23/2016 04:47 PM, Nathan Mercer wrote:

Well, after a couple months, I have finally been able to test this fix out. I see it has been implemented in Geomesa 1.2.2.

 

I can confirm that there is no longer an issue with contains at the 45 degree increments.

 

Cheers,

Nathan

 

From: geomesa-users-bounces@xxxxxxxxxxxxxxxx [mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Nathan Mercer
Sent: Friday, April 1, 2016 8:32 AM
To: Geomesa User discussions <geomesa-users@xxxxxxxxxxxxxxxx>
Subject: Re: [geomesa-users] Query with coordinate 0.0

 

Hi Emilio,

 

This is fantastic. Thank you all for the great support.

 

I will try this out and respond with my results.

 

Cheers,

Nathan

 

From: geomesa-users-bounces@xxxxxxxxxxxxxxxx [mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Emilio Lahr-Vivaz
Sent: Thursday, March 31, 2016 11:05 AM
To: Geomesa User discussions <geomesa-users@xxxxxxxxxxxxxxxx>
Subject: Re: [geomesa-users] Query with coordinate 0.0

 

Hi Nathan,

 

Chris Eichelberger put together a fix for the bug, and we merged it this afternoon. It's not available in any published artifacts yet,

but if you build from source it will be there. If you try it out, let us know how it goes.

 

Thanks,

 

Emilio

 

On Mon, 2016-03-28 at 15:22 -0600, Nathan Mercer wrote:

Thank you so much Emilio,

 

It certainly seems like a bug that needs to be fixed so I am glad you created the issue to track the problem.

 

Here is what I have implemented as the workaround:

 

val bufferdFilter = filterFactory.contains(filterFactory.property(geometryAttribute), filterFactory.literal(point.buffer(0.000001)))

val iswithin = GeoMesaSpark.rdd(new HadoopConfiguration, sparkContext, accumuloConfig, new Query(coverageFeature, bufferedFilter), None)

  .filter(feature => feature.getDefaultGeometry.asInstanceOf[MultiPolygon].contains(point).count > 0

 

This appears to work for me. It uses your idea of filtering to a small subset first. I do this by buffering a small amount around the point to create a polygon which works with the contains filter.

Next, I filter to check that the point is exactly within if my buffer was too large. It means doing two contains calls so it is not ideal, but it works for now.

 

Cheers,

Nathan

 

From: geomesa-users-bounces@xxxxxxxxxxxxxxxx [mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Emilio Lahr-Vivaz
Sent: Monday, March 28, 2016 2:02 PM
To: Geomesa User discussions <geomesa-users@xxxxxxxxxxxxxxxx>
Subject: Re: [geomesa-users] Query with coordinate 0.0

 

Hi Nathan,

 

I've been looking into your issue, and it seems that we don't handle literal points very well in our filter processing.

 

As a work-around, I would suggest using a broader filter to get your initial set of features, then applying your exact filter manually on that.

E.g. going back to your original example:

 

val contains = filterFactory.contains(filterFactory.property(geometryAttribute), filterFactory.literal(point))

val rdd = GeoMesaSpark.rdd(new HadoopConfiguration, sparkContext, accumuloConfig, new Query(coverageFeature, Filter.INCLUDE), None) // will select all your features

val isWithin = rdd.filter(contains.evaluate).count > 0 // will evaluate the contains filter on each feature

 

If you can select a subset of your features to evaluate, it will speed up the result - instead of using Filter.INCLUDE in the rdd creation, use a BBOX or time duration or something.

 

From a technical perspective, GeoMesa uses the bounding polygon (and time range) of a query to select a range of rows to scan in Accumulo. We then apply the exact filter to those rows - 

this saves us from scanning lots of records that we know don't match. However, we tend to assume that input queries will cover a polygon, not a point.

Surprisingly, this hasn't come up before now, so I guess it's a semi-reasonable assumption :)

 

I've created an issue to track the problem here: https://geomesa.atlassian.net/browse/GEOMESA-1150

 

Thanks,

 

Emilio

 

 

On Mon, 2016-03-28 at 13:14 -0600, Nathan Mercer wrote:

Thanks Emilio,

 

I have not tried your maven scala console yet, but I have tried some other things and have some more information.

 

First, I am using Accumulo 1.6.5 and GeoTools 14.2

I am now using a simple box for my MultiPolygon and it still fails. The box is just some random coordinates that appear over the United States.

I tried re-implementing the query using Java and without Spark and I am getting the same results. The code is as follows:

 

Point pnt = JTSFactoryFinder.getGeometryFactory().createPoint(new Coordinate(0.0,10.0));

FilterFactory2 ff = CommonFactoryFinder.getFilterFactory2();

Filter f = ff.contains(ff.property("the_geom"), ff.literal(pnt));

Query q = new Query(“FeatureName”, f);

AccumuloFeatureStore featureStore = (AccumuloFeatureStore)dataStore.getFeatureSource(“FeatureName”);

SimpleFeatureCollection results = featureStore.getFeatures(q);

System.out.println(results.isEmpty());

 

Here is my stack trace. Strangely, using this method I get the error even with non-zero coordinates if they are outside my polygon. Not sure why this wasn’t the case before.

 

Exception in thread "main" java.lang.IllegalStateException: getX called on empty Point

        at com.vividsolutions.jts.geom.Point.getX(Point.java:124)

        at org.locationtech.geomesa.utils.geohash.GeohashUtils$.considerCandidate$1(GeohashUtils.scala:1023)

        at org.locationtech.geomesa.utils.geohash.GeohashUtils$.getUniqueGeohashSubstringsInPolygon(GeohashUtils.scala:1043)

        at org.locationtech.geomesa.accumulo.index.GeoHashPlanner$class.geomToGeoHashes(QueryPlanners.scala:340)

        at org.locationtech.geomesa.accumulo.index.GeoHashKeyPlanner.geomToGeoHashes(QueryPlanners.scala:374)

        at org.locationtech.geomesa.accumulo.index.GeoHashPlanner$class.polyToPlan(QueryPlanners.scala:345)

        at org.locationtech.geomesa.accumulo.index.GeoHashKeyPlanner.polyToPlan(QueryPlanners.scala:374)

        at org.locationtech.geomesa.accumulo.index.GeoHashPlanner$class.getKeyPlan(QueryPlanners.scala:363)

        at org.locationtech.geomesa.accumulo.index.GeoHashKeyPlanner.getKeyPlan(QueryPlanners.scala:374)

        at org.locationtech.geomesa.accumulo.index.GeoHashKeyPlanner.getKeyPlan(QueryPlanners.scala:376)

        at org.locationtech.geomesa.accumulo.index.CompositePlanner$$anonfun$12.apply(QueryPlanners.scala:500)

        at org.locationtech.geomesa.accumulo.index.CompositePlanner$$anonfun$12.apply(QueryPlanners.scala:500)

        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)

        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)

        at scala.collection.immutable.List.foreach(List.scala:381)

        at scala.collection.TraversableLike$class.map(TraversableLike.scala:245)

        at scala.collection.immutable.List.map(List.scala:285)

        at org.locationtech.geomesa.accumulo.index.CompositePlanner.getKeyPlan(QueryPlanners.scala:500)

        at org.locationtech.geomesa.accumulo.index.STIdxStrategy.planQuery(STIdxStrategy.scala:225)

        at org.locationtech.geomesa.accumulo.index.STIdxStrategy.getQueryPlan(STIdxStrategy.scala:122)

        at org.locationtech.geomesa.accumulo.index.STIdxStrategy.getQueryPlan(STIdxStrategy.scala:33)

        at org.locationtech.geomesa.accumulo.index.QueryPlanner$$anonfun$getQueryPlans$6$$anonfun$2.apply(QueryPlanner.scala:141)

        at org.locationtech.geomesa.accumulo.index.QueryPlanner$$anonfun$getQueryPlans$6$$anonfun$2.apply(QueryPlanner.scala:141)

        at org.locationtech.geomesa.utils.stats.MethodProfiling$class.profile(MethodProfiling.scala:26)

        at org.locationtech.geomesa.accumulo.index.QueryPlanner.profile(QueryPlanner.scala:55)

        at org.locationtech.geomesa.accumulo.index.QueryPlanner$$anonfun$getQueryPlans$6.apply(QueryPlanner.scala:141)

        at org.locationtech.geomesa.accumulo.index.QueryPlanner$$anonfun$getQueryPlans$6.apply(QueryPlanner.scala:136)

        at scala.collection.Iterator$$anon$11.next(Iterator.scala:370)

        at org.locationtech.geomesa.accumulo.util.CloseableIterator$$anon$5.next(CloseableIterator.scala:34)

        at org.locationtech.geomesa.accumulo.util.SelfClosingIterator$$anon$1.next(CloseableIterator.scala:103)

        at org.locationtech.geomesa.accumulo.util.CloseableIterator$$anon$2.<init>(CloseableIterator.scala:65)

        at org.locationtech.geomesa.accumulo.util.CloseableIterator$class.ciFlatMap(CloseableIterator.scala:64)

        at org.locationtech.geomesa.accumulo.util.SelfClosingIterator$$anon$1.ciFlatMap(CloseableIterator.scala:97)

        at org.locationtech.geomesa.accumulo.index.QueryPlanner.scan$1(QueryPlanner.scala:92)

        at org.locationtech.geomesa.accumulo.index.QueryPlanner.executePlans(QueryPlanner.scala:113)

        at org.locationtech.geomesa.accumulo.index.QueryPlanner.runQuery(QueryPlanner.scala:83)

        at org.locationtech.geomesa.accumulo.data.AccumuloFeatureReaderImpl.<init>(AccumuloFeatureReader.scala:63)

        at org.locationtech.geomesa.accumulo.data.AccumuloFeatureReader$.apply(AccumuloFeatureReader.scala:54)

        at org.locationtech.geomesa.accumulo.data.AccumuloDataStore.getFeatureReader(AccumuloDataStore.scala:686)

        at org.locationtech.geomesa.accumulo.data.AccumuloFeatureCollection.reader(AccumuloFeatureSource.scala:150)

        at org.geotools.data.store.DataFeatureCollection.isEmpty(DataFeatureCollection.java:309)

        at com.nathan.geomesa.test.MyQuery.main(MyQuery.java:97)

 

 

 

 

From: geomesa-users-bounces@xxxxxxxxxxxxxxxx [mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Emilio Lahr-Vivaz
Sent: Monday, March 28, 2016 12:23 PM
To: Geomesa User discussions <geomesa-users@xxxxxxxxxxxxxxxx>
Subject: Re: [geomesa-users] Query with coordinate 0.0

 

I'm guessing it's a GeoMesa issue, but to rule out GeoTools, you could evaluate the filter directly against your multi-polygon using the maven scala console:

 

$ mvn scala:console -pl geomesa-compute

 

scala> import org.locationtech.geomesa.utils.geotools.SimpleFeatureTypes

scala> import org.locationtech.geomesa.features.ScalaSimpleFeature

scala> import org.locationtech.geomesa.utils.text.WKTUtils

 

scala> val sft = SimpleFeatureTypes.createType("mySft", "the_geom:MultiPolygon")

scala> val feature = new ScalaSimpleFeature("id", sft)

scala> feature.setAttribute("the_geom", WKTUtils.read("put your wkt here"))

scala> val filter = filterFactory.contains(filterFactory.property("the_geom"), filterFactory.literal(point))

scala> filter.evaluate(feature)

 

 

On Mon, 2016-03-28 at 12:11 -0600, Nathan Mercer wrote:

Hi Emilio,

 

I should have mentioned it. Sorry.

 

I am using Geomesa 1.2.0

 

I cannot provide one of the multipolygons, but just to make sure it is not the polygons I will try using a simple box and see if the result is the same and reply back to this thread with the results.

 

Cheers,

Nathan

 

From: geomesa-users-bounces@xxxxxxxxxxxxxxxx [mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Emilio Lahr-Vivaz
Sent: Monday, March 28, 2016 12:06 PM
To: Geomesa User discussions <geomesa-users@xxxxxxxxxxxxxxxx>
Subject: Re: [geomesa-users] Query with coordinate 0.0

 

Hi Nathan,

 

To help narrow it down, what version of GeoMesa are you using? If possible, could you provide one of the multipolygons that is failing?

 

Thanks,

 

Emilio

 

On Mon, 2016-03-28 at 11:53 -0600, Nathan Mercer wrote:

Hi,

 

I am trying to run a simple contains query. I want to simply check if my MultiPolygon feature contains a given Point. Or in other words that the Point is within my MultiPolygon.

 

I am using the following code in scala:

 

*Notes:

- filterFactory is an instance of FilterFactory2

- geometryAttribute is set to “the_geom” which is the geometry attribute for the MultiPolygon feature I created by ingesting from a shapefile

- point is a Point object of type com.vividsolutions.jts.geom.Point

- HadoopConfiguration is an alias for org.apache.hadoop.conf.Configuration

- sparkContext and accumuloConfig are all set up appropriately

- coverageFeature is the feature name for my MultiPolygon feature

- I have also tried ! .isEmpty instead of .count > 0

 

val filter = filterFactory.contains(filterFactory.property(geometryAttribute), filterFactory.literal(point))

val iswithin = GeoMesaSpark.rdd(new HadoopConfiguration, sparkContext, accumuloConfig, new Query(coverageFeature, filter), None).count > 0

 

This code actually works properly and gives me back the appropriate response except for one special case. This is when either the x or y (or both) coordinates of the point are 0.0

 

Does anybody have any ideas why this would fail? And also, any ideas of how to get around it? Is this a problem with GeoTools or GeoMesa?

 

I have been banging my head on this for a while now and cannot figure it out.

 

Thanks,

 

Nathan Mercer, P.Eng.

Software Developer | Intermap Technologies

#200-846 6th Ave. S.W., Calgary, Alberta, Canada, T2P 3E5

Tel. +1 403.266.0900 x217 or 403-539-2206 | Fax +1 403.265.0499 | Skype nathanmercer

nmercer@xxxxxxxxxxxx | www.intermap.com

 

LinkedIn-signature-for-office.jpg

 

CONFIDENTIALITY NOTICE - This e-mail transmission and any accompanying documents contain information belonging to the sender which may be confidential and legally privileged. This information is intended only for the use of the person to whom this e-mail transmission was sent as indicated above. If you are not the intended recipient, any disclosure, copying, distribution, or action taken in reliance on the contents of the information contained in this transmission is strictly prohibited.

 


  ­­  

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users


  ­­  

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users


  ­­  

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users


  ­­  

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users


  ­­  


  ­­  

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users


Back to the top