Thanks Emilio for quick response.
Below are the required details.
Regarding update from 2.3.2 to 2.4.0, we
can upgrade it but it will require some
effort and time which I want to avoid
for now.
Exact filter which I
am using: BBOX(geometry,-180.0,-90.0,180.0,90.0) AND ingestionTimestamp <= '2019-12-23 06:00:00' AND nextTimestamp > '2019-12-23 06:00:00'
hbase(main):002:0> scan 'atlas'
ROW COLUMN+CELL
OSMNodes~attributes column=m:v, timestamp=1577234629875, value=*geometry:Point:srid=4326,ingestionTimestamp:Timestamp,nextTimestamp:Timestamp,serializerVersion:String,featurePayload:String;geomesa.index.dtg='ingestionTimestamp',geomesa.z.splits='60',geomesa.indices='z3:6:3:geometry:ingestionTimestamp,id:4:3:'
OSMNodes~stats-date column=m:v, timestamp=1577234629875, value=2019-12-25T00:43:49.836Z
OSMNodes~table.id.v4 column=m:v, timestamp=1577234646266, value=atlas_OSMNodes_id_v4
OSMNodes~table.z3.geometry.ingestionTimestamp.v6 column=m:v, timestamp=1577234629897, value=atlas_OSMNodes_z3_geometry_ingestionTimestamp_v6
OSMRelationMembers~attributes column=m:v, timestamp=1577234747359, value=ingestionTimestamp:Timestamp,relationId:String,featureTypeId:String,serializerVersion:String,featurePayload:String;geomesa.index.dtg='ingestionTimestamp',geomesa.indices='attr:8:3:relationId:ingestionTimestamp,attr:8:3:featureTypeId:ingestionTimestamp,id:4:3:'
OSMRelationMembers~stats-date column=m:v, timestamp=1577234747359, value=2019-12-25T00:45:47.320Z
OSMRelationMembers~table.attr.featureTypeId.ingestionTimestamp.v8 column=m:v, timestamp=1577234751575, value=atlas_OSMRelationMembers_attr_featureTypeId_ingestionTimestamp_v8
OSMRelationMembers~table.attr.relationId.ingestionTimestamp.v8 column=m:v, timestamp=1577234747380, value=atlas_OSMRelationMembers_attr_relationId_ingestionTimestamp_v8
OSMRelationMembers~table.id.v4 column=m:v, timestamp=1577234755743, value=atlas_OSMRelationMembers_id_v4
OSMRelations~attributes column=m:v, timestamp=1577234692949, value=*geometry:MultiPolygon:srid=4326,ingestionTimestamp:Timestamp,nextTimestamp:Timestamp,serializerVersion:String,featurePayload:String;geomesa.index.dtg='ingestionTimestamp',geomesa.z.splits='60',geomesa.indices='xz3:2:3:geometry:ingestionTimestamp,id:4:3:'
OSMRelations~stats-date column=m:v, timestamp=1577234692949, value=2019-12-25T00:44:52.909Z
OSMRelations~table.id.v4 column=m:v, timestamp=1577234710295, value=atlas_OSMRelations_id_v4
OSMRelations~table.xz3.geometry.ingestionTimestamp.v2 column=m:v, timestamp=1577234692970, value=atlas_OSMRelations_xz3_geometry_ingestionTimestamp_v2
OSMTestNodes~attributes column=m:v, timestamp=1577143864743, value=*geometry:Point:srid=4326,ingestionTimestamp:Timestamp,nextTimestamp:Timestamp,serializerVersion:String,featurePayload:String;geomesa.index.dtg='ingestionTimestamp',geomesa.z.splits='60',geomesa.indices='z3:6:3:geometry:ingestionTimestamp,id:4:3:'
OSMTestNodes~stats-date column=m:v, timestamp=1577143864743, value=2019-12-23T23:30:56.200Z
OSMTestNodes~table.id.v4 column=m:v, timestamp=1577143890005, value=atlas_OSMTestNodes_id_v4
OSMTestNodes~table.z3.geometry.ingestionTimestamp.v6 column=m:v, timestamp=1577143864809, value=atlas_OSMTestNodes_z3_geometry_ingestionTimestamp_v6
OSMWayNodes~attributes column=m:v, timestamp=1577234724952, value=ingestionTimestamp:Timestamp,wayId:String,nodeId:String,serializerVersion:String,featurePayload:String;geomesa.index.dtg='ingestionTimestamp',geomesa.indices='attr:8:3:wayId:ingestionTimestamp,attr:8:3:nodeId:ingestionTimestamp,id:4:3:'
OSMWayNodes~stats-date column=m:v, timestamp=1577234724952, value=2019-12-25T00:45:24.908Z
OSMWayNodes~table.attr.nodeId.ingestionTimestamp.v8 column=m:v, timestamp=1577234729162, value=atlas_OSMWayNodes_attr_nodeId_ingestionTimestamp_v8
OSMWayNodes~table.attr.wayId.ingestionTimestamp.v8 column=m:v, timestamp=1577234724973, value=atlas_OSMWayNodes_attr_wayId_ingestionTimestamp_v8
OSMWayNodes~table.id.v4 column=m:v, timestamp=1577234733300, value=atlas_OSMWayNodes_id_v4
OSMWays~attributes column=m:v, timestamp=1577234660315, value=*geometry:LineString:srid=4326,ingestionTimestamp:Timestamp,nextTimestamp:Timestamp,serializerVersion:String,featurePayload:String;geomesa.index.dtg='ingestionTimestamp',geomesa.z.splits='60',geomesa.indices='xz3:2:3:geometry:ingestionTimestamp,id:4:3:'
OSMWays~stats-date column=m:v, timestamp=1577234660315, value=2019-12-25T00:44:20.278Z
OSMWays~table.id.v4 column=m:v, timestamp=1577234677610, value=atlas_OSMWays_id_v4
OSMWays~table.xz3.geometry.ingestionTimestamp.v2 column=m:v, timestamp=1577234660337, value=atlas_OSMWays_xz3_geometry_ingestionTimestamp_v2
Hello,
If possible, can you upgrade to
2.3.2 or 2.4.0? There is at least
one bug that may cause that
behavior, that has been fixed in
those versions. Aside from that, it
would be helpful if you could
provide the exact filters you're
using, and the data from your
catalog table in hbase, as output by
scanning it through the hbase shell.
Thanks,
Emilio
On 1/15/20 2:58 PM, Amit
Srivastava wrote:
Hi All,
I am using Geomesa v2.3.0
with HBase. While running BBOX
export query on Geomesa the
results are inconsistent. I am
seeing few features are
missing in export. Can someone
help in debugging this?
What investigation I
have done so far?
- Compare the data ingested
vs exported. Found below
diff for it.
- Checked the Application
logs which is calling
Geomesa via GeoTools
interface. I am not seeing
any error in the application
log.
- Checked HBase related
metrics and logs. I am not
seeing any error during the
time when we performed the
execution.
- Re-Run the query on a
smaller BBOX which has
missing data, the reported
missing data in 2nd export
got returned by Geomesa in
3rd query. Hence I can't
find repreoducable steps for
this issue.
Missing Features In 2nd
Export Run:
Missing Features in 1st
Export Run:
--
Regards,
Amit Kumar
Srivastava
--
Regards,
Amit Kumar Srivastava