Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] TubeSelectProcess issue with "dtg" field

Hi Federica,

Thanks for mentioning that you observed the issue in the unit test.  I've pushed a fix to a branch off of GeoMesa 1.2.1 here: https://github.com/locationtech/geomesa/tree/f_fixTS.  If you build up to the Accumulo datastore and use those artifacts, you should be fine. 

I've also pushed up a fix to master as PR here (1) and the ticket is (2).

Thanks for reporting the issue; it should be fixed in GeoMesa 1.2.5.

Cheers,

Jim

1. PR: https://github.com/locationtech/geomesa/pull/1010
2. Ticket: https://geomesa.atlassian.net/browse/GEOMESA-1332

On 07/26/2016 05:31 AM, Federica Moscato wrote:

Hi Jim,

 

Unfortunately I don’t use a WPS request, but some custom code.

I attached the debugger to see the values of the variables.

 

What I saw is the following :

 

1-      features.getSchema.getUserData()(Constants.SF_PROPERTY_START_TIME) has the correct value for my Date field

2-      features.getSchema.getAttributeDescription has an empty value for the corresponding (Datefield).getUserData. The userData in features.getSchema.getAttributeDescriptions is only present for the geometry.

 

To check if it was my code to produce this error, I ran the test into my geomesa 1.2.1 local copy.

I was able to reproduce the same behaviour using the org/locationtech/geomesa/accumulo/process/tube/TubeSelectProcessTest.scala changing the name of the data field.

 

Hope this can help for further debugging.

 

Please let me know if I can provide more info.

 

Many thanks for your help,

Federica

 

 

 

From: geomesa-users-bounces@xxxxxxxxxxxxxxxx [mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Jim Hughes
Sent: 25 July 2016 19:35
To: geomesa-users@xxxxxxxxxxxxxxxx
Subject: Re: [geomesa-users] TubeSelectProcess issue with "dtg" field

 

Hi Federica,

Thanks for writing with this thorough report; I'd say this is a bug in how the date field is retrieved in the TubeSelectProcess.  I've filed a bug here (1). 

Honestly, this one is likely subtle.  I checked out the 1.2.1 code, and it looks like the 'dtgfield' entry should be picked up to populate the value in question.  It appears that the userData for the SimpleFeatureType is getting lost somewhere in the process.

Would you be able to provide the xml for the WPS request you are making?  Otherwise, if you are enable and interested in attaching a debugger, I could help walk you through what info would be interesting.  (The big idea would be to inspect the 'featureCollection' at line 109 in TubeSelectProcess to see if the SimpleFeatureType has any userData...

Cheers,

Jim

1.  https://geomesa.atlassian.net/browse/GEOMESA-1332

On 07/25/2016 01:16 PM, Federica Moscato wrote:

Dear geomesa developers,

 

First of all many thanks for the Geomesa project: I am an  happy  user, and it is very useful.

 

I am trying to use the TubeSelectProcess, but I am getting the following error: 

 

16/07/25 17:20:24 ERROR tube.NoGapFill: Unable to retrieve date field from input tubeFeatures...ensure there a field named dtg

java.lang.IllegalArgumentException: Unable to retrieve date field from input tubeFeatures...ensure there a field named "dtg"

        at org.locationtech.geomesa.accumulo.process.tube.TubeBuilder$$anonfun$transform$1.apply(TubeBuilder.scala:94)

        at org.locationtech.geomesa.accumulo.process.tube.TubeBuilder$$anonfun$transform$1.apply(TubeBuilder.scala:86)

        at scala.collection.Iterator$$anon$11.next(Iterator.scala:370)

        at scala.collection.Iterator$$anon$11.next(Iterator.scala:370)

        at scala.collection.Iterator$class.toStream(Iterator.scala:1180)

        at scala.collection.AbstractIterator.toStream(Iterator.scala:1194)

        at scala.collection.TraversableOnce$class.toSeq(TraversableOnce.scala:296)

        at scala.collection.AbstractIterator.toSeq(Iterator.scala:1194)

        at org.locationtech.geomesa.accumulo.process.tube.NoGapFill.createTube(TubeBuilder.scala:141)

        at org.locationtech.geomesa.accumulo.process.tube.TubeVisitor.tubeSelect(TubeSelectProcess.scala:161)

        at org.locationtech.geomesa.accumulo.data.AccumuloFeatureCollection.accepts(AccumuloFeatureSource.scala:135)

        at org.locationtech.geomesa.accumulo.process.tube.TubeSelectProcess.execute(TubeSelectProcess.scala:109)

 

It seems that it is looking in my catalog  for a default field that has to be called to “dtg”.

My catalog has his own data field with a different name (TS instead of dtg), and scanning the metadata table I can see that there is a link between the dtgfield and my own field

 

<snip>

 

~METADATA_positional dtgfield: []    TS

 

</snip>

 

Further debugging bring me to a piece of code that I suppose could be my problem

 

 

def extractDtgField(sft: SimpleFeatureType) =
  sft.getAttributeDescriptors
    .filter { _.getUserData.contains(Constants.SF_PROPERTY_START_TIME) }
    .headOption
    .map { _.getName.toString }
    .getOrElse(DEFAULT_DTG_FIELD)

 

 

This is seems to be always the default_dfg_field value.

 

The process that I am doing is to query with different filters my catalog, creating 2 different datasets: the input collection, and the dataset where to check matching features, and then run the TubeSelectionProcess over the two datasets.

For the context: I tried to run the same code on a catalog that contained the “dtg” field as the Date field, and the process was fine and I was getting the results that I wanted/expected.

 

The stack that I am using is Accumulo 1.7.0, Geomesa 1.2.1

 

GeoMesa Version 1.2.1 built on 2016-03-25 16:17.

Commit ID: e577807844341849afeab04f5d034c6ca1b663c4

Branch: geomesa-1.2.1

 

Any advice would be very much appreciate.

 

Many thanks,

Federica

This email (and any attachments) may contain confidential information and is intended solely for the recipient(s) to whom the email is addressed. If you received this email in error, please inform us immediately and delete the email and all attachments without further using, copying or disclosing the information. This email and any attachments are believed to be, but cannot be guaranteed to be, secure or virus-free. Satellite Applications Catapult Limited is registered in England & Wales. Company Number: 7964746. Registered office: Electron Building, Fermi Avenue, Harwell Oxford, Didcot, Oxfordshire OX11 0QR.


_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users

 

This email (and any attachments) may contain confidential information and is intended solely for the recipient(s) to whom the email is addressed. If you received this email in error, please inform us immediately and delete the email and all attachments without further using, copying or disclosing the information. This email and any attachments are believed to be, but cannot be guaranteed to be, secure or virus-free. Satellite Applications Catapult Limited is registered in England & Wales. Company Number: 7964746. Registered office: Electron Building, Fermi Avenue, Harwell Oxford, Didcot, Oxfordshire OX11 0QR.

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users


Back to the top