Re: [geomesa-users] Geomesa GDELT example crashes

[Jim Hughes <jnh5y@xxxxxxxx>]

Hi Manas,

At the minute, our Spark support is partial.  We have worked out turning a GeoMesa query into an AccumuloInputFormat; doing so allows us to use GeoMesa features as a source in Spark. 

Writing to Accumulo/GeoMesa in Spark will require a little more effort/thought.  It's on my list of things I'd love to do.  You and another user have both expressed interest, and I'd like to piece together something. 

As a concrete work-around, if you have the CSV at rest (i.e., before you start Spark), then I'd strongly suggest using our command line ingest tools.  If the data is being created in the Spark process, you could temporarily write it to file and ingest that.

Thanks,

Jim

On 01/27/2015 04:06 PM, Manas Kar wrote:

Hi Chris, 

 I got it working after adding the missing steps. It seems while relocating the classes I will also have to exclude the original classes for the Jar.

Any comments on my "spark example question"

Basically I am looking for a spark example that takes a CSV data and pushes to Accumulo using Geomesa with index and all.

It does not have to be fully functional, I am just looking for a direction.

Thanks

Manas

On Sun, Jan 25, 2015 at 8:59 PM, Manas Kar <manasdebashiskar@xxxxxxxxx> wrote:

Hi Chris, 

 I tried relocating joda-time library with out any positive result. It might be due to the fact I still don't know how the relocating classes work. 

I mean, even if my joda-time(V2.3) gets reallocated as org.shaded.joda.time the Accumulo Iterator's won't use that unless I modify Geomesa code to import org.shaded.joda.time. 

I was thinking why not bypass the problem of calling the example via hadoop jar.

Is there a spark example that pushes text data to Accumulo datastore using Geomesa(I know there are spark example of ingesting Accumulo datastore to render density Map) 

Thanks again for your help so far.

..Manas

On Fri, Jan 23, 2015 at 11:42 AM, Manas Kar <manasdebashiskar@xxxxxxxxx> wrote:

Hi Geomesa experts, 

 I have created a Accumulo1.6 + Geomesa(accumulo1.6 branch) on CDH5.3.

 The Quickstart program runs fine.

Now I am trying to run the https://github.com/geomesa/geomesa-gdelt.git example which fails with the "java.lang.NoSuchMethodError: org.joda.time.DateTime.<init>" error.

STACK TRACE

------------------------

Exception in thread "main" java.lang.NoSuchMethodError: org.joda.time.DateTime.<init>(IIIIIILorg/joda/time/DateTimeZone;)V

at org.locationtech.geomesa.core.index.IndexSchema$.<init>(IndexSchema.scala:92)

at org.locationtech.geomesa.core.index.IndexSchema$.<clinit>(IndexSchema.scala)

at org.locationtech.geomesa.core.index.IndexSchemaBuilder.randomNumber(IndexSchema.scala:370)

at org.locationtech.geomesa.core.data.AccumuloDataStore.buildDefaultSpatioTemporalSchema(AccumuloDataStore.scala:100)

at org.locationtech.geomesa.core.data.AccumuloDataStore.computeSpatioTemporalSchema(AccumuloDataStore.scala:370)

at org.locationtech.geomesa.core.data.AccumuloDataStore.createSchema(AccumuloDataStore.scala:389)

at org.locationtech.geomesa.core.data.AccumuloDataStore.createSchema(AccumuloDataStore.scala:484)

at org.locationtech.geomesa.core.data.AccumuloDataStore.createSchema(AccumuloDataStore.scala:70)

at geomesa.gdelt.GDELTIngest.main(GDELTIngest.java:115)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users