Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] Problem with Z3Idx queries (geomesa 1.1.0-rc2)
Hi Ben,

Thanks for the report.  I've got great news and some options for you.  First, the issue you are hitting has been reported as https://geomesa.atlassian.net/browse/GEOMESA-874 and is fixed in 1.1.0-rc4 and later. 

Upgrading should address this issue.  That said, we are pushing for a GeoMesa 1.2.0 release in the next month, and if I could get folks to do one update soon-ish, it'd be to that version.*

As a quick work around, since the bug deals with unbounded time ranges, we just need to put an upper temporal bound on the query.  It should be reasonable to say that we are querying for 2012 until now.  (I'm assuming you don't have future data or if you do that it could be bounded reasonably.)  If you try that with your filter, things should be up and working again.

Let us know if that doesn't do it for you!

Thanks,

Jim

* p.s.  For some small technical risk, you could try running client code with 1.1.0-rc4.  I *don't think* the GeoMesa Accumulo format changed, and if that's the case, you'd get the query planner fix without needing to worry with data re-ingest or upgrade.  Adding an upper bound to the query should be rock solid, but if that's impossible, this is another option which may work quickly.

On 1/27/2016 2:29 PM, Ben Southall wrote:

Hello,

 

I seem to be having some problems making spatio-temporal queries in geomesa 1.1.0-rc2; notably, it looks like something odd is happening to the temporal part of the query. Before I show the query planner output and the stack trace, a bit more info.  Here’s the table description from Accumulo:

 

~METADATA_ais_report attributes: []    mmsi:Long:index=full,latitude:Double,longitude:Double,observationTime:Date,currentHeading:Float,currentSpeed:Double,trueHeading:Integer,*geom:Point:srid=4326:index=full:index-value=true

~METADATA_ais_report dtgfield: []    observationTime

~METADATA_ais_report featureEncoding: []    kryo

~METADATA_ais_report schema: []    %~#s%2#r%#i%ais_report#cstr%0,3#gh%yyyyMMddHH#d::%~#s%3,2#gh::%~#s%#id

~METADATA_ais_report tables.idx.attr.name: []    nrl_global_ais_500k_attr_idx

~METADATA_ais_report tables.idx.st.name: []    nrl_global_ais_500k_st_idx

~METADATA_ais_report tables.queries.name: []    nrl_global_ais_500k_queries

~METADATA_ais_report tables.record.name: []    nrl_global_ais_500k_records

~METADATA_ais_report tables.sharing: []    true

~METADATA_ais_report tables.z3.name: []    nrl_global_ais_500k_ais_5freport_z3

~METADATA_ais_report version: []    5

~METADATA_ais_report visibilities: []

 

If I make just the spatial query, it works. If I make just the temporal query, it works. It looks like both of those queries are planned using the STIdxStrategy.

 

I’ve a small Accumulo cluster (3 nodes) with geomesa-distributed-runtime-1.1.0-rc.2.jar installed in the lib/ext directory of each node’s ACCUMULO_HOME.

 

It’s when I do a combined (“AND”) query that problems occur - here’s the output of the query planner, followed by the exception, which seems to indicate that the temporal part is causing problems:

 

--

 

Planning Query: feature type: ais_report, filter: [[ observationTime > Tue Jul 10 00:00:00 EDT 2012 ] AND [ geom intersects POLYGON ((-170 -80, -170 -79, -169 -79, -169 -80, -170 -80)) ]], [properties:  ALL ]

Original filter: [[ observationTime > Tue Jul 10 00:00:00 EDT 2012 ] AND [ geom intersects POLYGON ((-170 -80, -170 -79, -169 -79, -169 -80, -170 -80)) ]]

Rewritten filter: [[ observationTime > Tue Jul 10 00:00:00 EDT 2012 ] AND [ geom intersects POLYGON ((-170 -80, -170 -79, -169 -79, -169 -80, -170 -80)) ]]

Strategy: org.locationtech.geomesa.accumulo.index.Z3IdxStrategy

Transforms: None

Geometry filters: ArrayBuffer([ geom intersects POLYGON ((-170 -80, -170 -79, -169 -79, -169 -80, -170 -80)) ])

Temporal filters: ArrayBuffer([ observationTime > Tue Jul 10 00:00:00 EDT 2012 ])

Other filters: ArrayBuffer()

Tweaked geom filters are ArrayBuffer([ geom intersects POLYGON ((-170 -80, -170 -79, -169 -79, -169 -80, -170 -80)) ])

GeomsToCover: GEOMETRYCOLLECTION (POLYGON ((-170 -80, -170 -79, -169 -79, -169 -80, -170 -80)))

Interval:  2012-07-10T04:00:00.000Z/9999-12-31T23:59:59.000Z

Transforms: None

java.lang.ArithmeticException: Value cannot fit in an int: 253402300799

                at org.joda.time.field.FieldUtils.safeToInt(FieldUtils.java:190)

                at org.joda.time.field.BaseDurationField.getDifference(BaseDurationField.java:141)

                at org.joda.time.chrono.ZonedChronology$ZonedDurationField.getDifference(ZonedChronology.java:328)

                at org.joda.time.base.BaseSingleFieldPeriod.between(BaseSingleFieldPeriod.java:72)

                at org.joda.time.Seconds.secondsBetween(Seconds.java:101)

                at org.locationtech.geomesa.accumulo.data.tables.Z3Table$.secondsInCurrentWeek(Z3Table.scala:41)

                at org.locationtech.geomesa.accumulo.index.Z3IdxStrategy.getQueryPlans(Z3IdxStrategy.scala:134)

                at org.locationtech.geomesa.accumulo.index.QueryPlanner$$anonfun$4$$anonfun$5.apply(QueryPlanner.scala:130)

                at org.locationtech.geomesa.accumulo.index.QueryPlanner$$anonfun$4$$anonfun$5.apply(QueryPlanner.scala:126)

                at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)

                at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)

                at scala.collection.immutable.List.foreach(List.scala:318)

                at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)

                at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105)

                at org.locationtech.geomesa.accumulo.index.QueryPlanner$$anonfun$4.apply(QueryPlanner.scala:126)

                at org.locationtech.geomesa.accumulo.index.QueryPlanner$$anonfun$4.apply(QueryPlanner.scala:113)

                at org.locationtech.geomesa.utils.stats.MethodProfiling$class.profile(MethodProfiling.scala:26)

                at org.locationtech.geomesa.accumulo.index.QueryPlanner.profile(QueryPlanner.scala:50)

                at org.locationtech.geomesa.accumulo.index.QueryPlanner.getStrategyPlans(QueryPlanner.scala:113)

                at org.locationtech.geomesa.accumulo.index.QueryPlanner.planQuery(QueryPlanner.scala:69)

                at org.locationtech.geomesa.accumulo.data.AccumuloDataStore.planQuery(AccumuloDataStore.scala:844)

                at org.locationtech.geomesa.accumulo.data.AccumuloDataStore.explainQuery(AccumuloDataStore.scala:836)

                at sri.mast.MastQuery.main(MastQuery.java:185)

 

--

 

Any help gratefully received!

 

Thanks,

 

Ben



_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users

Back to the top