Jim and Emilio,
Thanks for the quick responses, very appreciated. We came up with short term work arounds (re-ingest without the indexes or blow the GeoServer layers out which I didn't want to do) but may not need them!
It takes me a little to move code to the real system that had data sets that showed the issues but I will try to get the snapshot moved this week to test it out.
Regards,
Diane
------------------------------
Hi Diane,
I merged Emilio's PR earlier today, and made sure the Hudson snapshot build on master and deployed updated 1.2.2-SNAPSHOST artifacts to LocationTech's Nexus. All that means as an alternative to build GeoMesa locally, you can depend on 1.2.2-SNAPSHOT in a Maven project to try out the fix.
If you use the GeoMesa distribution bundle, you can download a snapshot
here:
https://repo.locationtech.org/content/repositories/geomesa-snapshots/org/locationtech/geomesa/geomesa-dist/1.2.2-SNAPSHOT/geomesa-dist-1.2.2-20160516.220910-22-bin.tar.gz
Thanks for the clear bug report!
Cheers,
Jim
On 05/16/2016 02:26 PM, Emilio Lahr-Vivaz wrote:
Hi Diane,
I believe the issue has already been fixed in the latest code. There
were still some similar issues but they probably won't affect you -
I've put up a fix but it's not merged yet. If you are comfortable
building and deploying from source, the current master branch should
work for you. If not, then we should have a release for you soon.
Thanks,
Emilio
On 05/16/2016 11:45 AM, Emilio Lahr-Vivaz wrote:
Hi Diane,
I've recreated this issue with a unit test - as you say, OR'd
attribute predicates AND'd with a geometry query fail to return any
records. We're going to be releasing version 1.2.2 in the next few
days - I'll make sure a fix gets included.
Thanks,
Emilio
On 05/16/2016 10:29 AM, Diane Griffith wrote:
We are on GeoMesa 1.2.0 on top of accumulo 1.6 with GeoServer 2.8.1.
We started experiencing issues combining complex cql quleries
through GeoServer for sure once we started applying more attribute
indexes on our data.
What seems to break down was when we did multiple IN or = cql
queries through one geoserver call to the geomesa stack. Generic
examples of what I?m talking about that could fail would be against
a column defined as a String (I?ll call it Keyword) with index=full
with the cql looking something like the following 2 examples:
1.(Keyword IN (?test, ?foo?))
2.((Keyword = ?test?) OR (Keyword = ?foo?))
If each was queried separately through geoserver for the certain
bounding box it works. But as soon as we combine each working
IN/equal match together with the same bounding box it returns no
matches via the GeoServer GetMap call.
Also on the same dataset a query that could succeed was something like:
((Keyword LIKE ?foo%?) OR (Keyword IN (?test?))
I knew I did combine other queries so I backed it off to a simple
test dataset where I had not indexed any additional attribute
columns. So when I did similar string tests like the 2 that failed
above, they worked against a non-indexed string attribute column. I
had no issues combining cql on admittedly non-indexed Double
columns. Unclear if I indexed them if I would experience similar
combined CQL issues.
Is there a known issue doing a combined/multiple IN matches or =
matches on an indexed string attribute column?
Is there further things that anyone can suggest I look at to
determine why a multi IN clause or = clause fails on an indexed
string attribute column?
Thanks,
Diane Griffith
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.locationtech.org/mhonarc/lists/geomesa-users/attachments/20160516/0b0587d9/attachment.html>
------------------------------
Message: 3
Date: Tue, 17 May 2016 14:24:48 +0100
From: Jo Walsh <metazool@xxxxxxxxx>
To: geomesa-users@xxxxxxxxxxxxxxxx
Subject: [geomesa-users] CQLException on 'id' with Geomesa Geoserver
plugin
Message-ID:
<CAOdP5FyhnE=ZBbB-UZA44Y296O4ixPeSOyFK4+zKWLCvun9QaA@xxxxxxxxxxxxxx>
Content-Type: text/plain; charset="utf-8"
dear all,
I'm evaluating Geomesa/Accumulo as a backend for Geoserver and so far, so reasonable, however I have come across an issue after ingesting a shapefile with a column named 'id' and then trying to access that data source as a WFS layer in Geoserver - Geomesa causes Geotools raises a CQLException on the "id" field. I thought it worth raising here to see if others have come across the problem, what insight others might have into the root causes, and whether this is in fact the right forum for the issue.
I found a couple of what seem to be related tickets in Geotools/Geoserver but they are both resolved:
https://osgeo-org.atlassian.net/browse/GEOT-2574
https://osgeo-org.atlassian.net/browse/GEOS-6634
And there's an older, also fixed geomesa issue referencing the ID filter in
ECQL: https://locationtech.org/mhonarc/lists/geomesa-users/msg00195.html
Here's the contents of geoserver.log where geomesa is triggering the error:
org.geoserver.wfs.WFSException: Error occurred getting features ...
Caused by: java.lang.IllegalArgumentException: Unable to parse expression id=id org.geotools.filter.text.cql2.CQLException: Encountered "id" at line 1, column 1.
Was expecting one of:
<STRING_LITERAL> ...
"true" ...
"false" ...
"point" ...
"linestring" ...
"polygon" ...
"multipoint" ...
"multilinestring" ...
"multipolygon" ...
"geometrycollection" ...
"envelope" ...
"(" ...
"[" ...
"-" ...
<INTEGER_LITERAL> ...
<FLOATING_LITERAL> ...
<DATE> ...
<DATE_TIME> ...
<IDENTIFIER> ...
Parsing : id.
at
org.geotools.process.vector.TransformProcess.toDefinition(TransformProcess.java:185)
at
org.locationtech.geomesa.accumulo.index.QueryPlanner$.setQueryTransforms(QueryPlanner.scala:324)
at
org.locationtech.geomesa.accumulo.index.QueryPlanner$.configureQuery(QueryPlanner.scala:228)
at
org.locationtech.geomesa.accumulo.data.AccumuloFeatureCollection.getSchema(AccumuloFeatureSource.scala:115)
at
org.locationtech.geomesa.accumulo.data.AccumuloFeatureCollection.getSchema(AccumuloFeatureSource.scala:105)
So it's happening here:
https://github.com/locationtech/geomesa/blob/master/geomesa-accumulo/geomesa-accumulo-datastore/src/main/scala/org/locationtech/geomesa/accumulo/index/QueryPlanner.scala#L325
https://github.com/geotools/geotools/blob/master/modules/library/cql/ECQL.md
states that ID is a predicate / reserved word in ECQL.
This is only happening if I ingest a shapefile with a column named 'id' - i can replace the column name, re-ingest and the same data appears fine via WFS.
cheers,
Jo
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.locationtech.org/mhonarc/lists/geomesa-users/attachments/20160517/dcb0309f/attachment.html>
------------------------------
Message: 4
Date: Tue, 17 May 2016 10:50:24 -0400
From: Chris Eichelberger <cne1x@xxxxxxxx>
To: Geomesa User discussions <geomesa-users@xxxxxxxxxxxxxxxx>
Subject: Re: [geomesa-users] CQLException on 'id' with Geomesa
Geoserver plugin
Message-ID: <1463496624.29876.67.camel@xxxxxxxx>
Content-Type: text/plain; charset="UTF-8"
Jo,
Ah! ?This is an issue that's bitten us before, so 1) you're in good company; 2) this is exactly the right forum for your question.
It turns out that "id" is a reserved word in GeoTools: ?https://github.
com/geotools/geotools/blob/master/modules/library/cql/src/main/jjtree/E
CQLGrammar.jjt#L428
We do have an outstanding ticket for GeoMesa to warn users when ingesting using reserved words: ?https://geomesa.atlassian.net/browse/G
EOMESA-1028? We hope to make some headway on this as part of the upcoming 1.2.2 release of GeoMesa.
In the interim, there are good tools that can help you rename attributes in a Shapefile. ?For example, see this question about
ogr2ogr: ?http://gis.stackexchange.com/questions/58541/how-to-rename-fi
eld-names-in-a-shapefile-from-the-commandline??
Thanks!
Sincerely,
? -- Chris
On Tue, 2016-05-17 at 14:24 +0100, Jo Walsh wrote:
dear all,
I'm evaluating Geomesa/Accumulo as a backend for Geoserver and so far,
so reasonable, however I have come across an issue after ingesting a
shapefile with a column named 'id' and then trying to access that data
source as a WFS layer in Geoserver - Geomesa causes Geotools raises a
CQLException on the "id" field. I thought it worth raising here to see
if others have come across the problem, what insight others might have
into the root causes, and whether this is in fact the right forum for
the issue.
I found a couple of what seem to be related tickets in
Geotools/Geoserver but they are both resolved:
https://osgeo-org.atlassian.net/browse/GEOT-2574
https://osgeo-org.atlassian.net/browse/GEOS-6634
And there's an older, also fixed geomesa issue referencing the ID
filter in ECQL: https://locationtech.org/mhonarc/lists/geomesa-users/
msg00195.html?
Here's the contents of geoserver.log where geomesa is triggering the
error:
org.geoserver.wfs.WFSException: Error occurred getting features ...
Caused by: java.lang.IllegalArgumentException: Unable to parse
expression id=id org.geotools.filter.text.cql2.CQLException:
Encountered "id" at line 1, column 1.
Was expecting one of:
? ? <STRING_LITERAL> ...
? ? "true" ...
? ? "false" ...
? ? "point" ...
? ? "linestring" ...
? ? "polygon" ...
? ? "multipoint" ...
? ? "multilinestring" ...
? ? "multipolygon" ...
? ? "geometrycollection" ...
? ? "envelope" ...
? ? "(" ...
? ? "[" ...
? ? "-" ...
? ? <INTEGER_LITERAL> ...
? ? <FLOATING_LITERAL> ...
? ? <DATE> ...
? ? <DATE_TIME> ...
? ? <IDENTIFIER> ...
? ? ?Parsing : id.
? ? ? ? at
org.geotools.process.vector.TransformProcess.toDefinition(TransformPr
ocess.java:185)
? ? ? ? at
org.locationtech.geomesa.accumulo.index.QueryPlanner$.setQueryTransfo
rms(QueryPlanner.scala:324)
? ? ? ? at
org.locationtech.geomesa.accumulo.index.QueryPlanner$.configureQuery(
QueryPlanner.scala:228)
? ? ? ? at
org.locationtech.geomesa.accumulo.data.AccumuloFeatureCollection.getS
chema(AccumuloFeatureSource.scala:115)
? ? ? ? at
org.locationtech.geomesa.accumulo.data.AccumuloFeatureCollection.getS
chema(AccumuloFeatureSource.scala:105)
So it's happening here:
https://github.com/locationtech/geomesa/blob/master/geomesa-accumulo/
geomesa-accumulo-
datastore/src/main/scala/org/locationtech/geomesa/accumulo/index/Quer
yPlanner.scala#L325
https://github.com/geotools/geotools/blob/master/modules/library/cql/
ECQL.md
states that ID is a predicate / reserved word in ECQL.
This is only happening if I ingest a shapefile with a column named
'id' - i can replace the column name, re-ingest and the same data
appears fine via WFS.
cheers,
Jo
--
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
------------------------------
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://www.locationtech.org/mailman/listinfo/geomesa-users
End of geomesa-users Digest, Vol 27, Issue 3
********************************************
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
