Re: [geomesa-users] GDELT MapReduce Ingest

[Jim Hughes <jnh5y@xxxxxxxx>]

Hi Jeff,

Awesome.  We haven't had any customers switch to Accumulo 1.7 quite yet, 
so we may have to work together a little in case you run into any errors.

For your question about rebuilding, our tutorials have been written over 
the course of two or so years.  There's lots of good stuff in there, and 
we do need to give that a run-through as we push for an official 
release.  If I recall, the MapReduce tutorial was written before the 
LocationTech Nexus repo was up and serving GeoMesa release candidates.  
As such, step 1 was indeed build GeoMesa;)  The GDELT jar should be a 
fat jar which bundles up the GeoMesa jars; as long as Maven can find 
them, you should be set.

In your Spark testing, have things been working?

Thanks,

Jim

On 11/17/2015 06:49 PM, Jeff Gold wrote:
> I'm using Accumulo 1.7 (default for HDP 2.3).
>
> I built GeoMesa and moved the libraries over to the Accumulo lib
> directory per the deployment/quick start guides
> (http://www.geomesa.org/geomesa-deployment/example). I was able to run
> the geomesa-accumulo example as well.
>
>
> Now, I'd like to try Spark so I followed this page:
> http://www.geomesa.org/spark/ and this page:
> http://www.geomesa.org/geomesa-gdelt-analysis/
>
> Actually, I have a question, other than the original build I did where
> I moved the geomesa jars into the Accumulo lib directory in the quick
> start guide, what is the purpose of rebuilding geomesa in the
> gdelt-analysis guide page at
> http://www.geomesa.org/geomesa-gdelt-analysis/  (that page has you
> building both, but the only step is to then use the gdelt jar, not
> sure what the purpose of the other geomesa build...
>
>
> Thanks,
> Jeff
>
>
> On Tue, Nov 17, 2015 at 6:36 PM, James Hughes <jnh5y@xxxxxxxx> wrote:
> > Hi Jeff,
> >
> > As two quick questions to help diagnosis things:
> >
> > Which version of Accumulo are you using?  And did you build GeoMesa
> > yourself?
> >
> > We've done some work to make GeoMesa compatible with Accumulo 1.5 and 1.6,
> > and this error is from that not working.  We should be able to help get you
> > sorted though.
> >
> > Cheers,
> >
> > Jim
> >
> > ----- Original Message -----
> > From:
> > "Geomesa User discussions" <geomesa-users@xxxxxxxxxxxxxxxx>
> >
> > To:
> > "Geomesa User discussions" <geomesa-users@xxxxxxxxxxxxxxxx>,
> > <jdk2pq@xxxxxxxx>
> > Cc:
> >
> > Sent:
> > Tue, 17 Nov 2015 18:19:12 -0500
> > Subject:
> > Re: [geomesa-users] GDELT MapReduce Ingest
> >
> >
> > Jake-
> > That did it, thank you! I was able to build and then started my
> > execution per the guide. Unfortunately, I hit an issue failing to find
> > method beginTableOperation in Accumulo.
> > I see that method in the Accumulo Javadocs for versions 1.4 and 1.5,
> > but not 1.6 (or 1.7).
> >
> > I'll see what I can figure out, but I thought you'd like to know.
> >
> > Here is part of the stack trace:
> > Exception in thread "main"
> > org.apache.accumulo.core.client.AccumuloException: Invalid method
> > name: 'beginTableOperation'
> > at
> > org.apache.accumulo.core.client.admin.TableOperationsImpl.doTableOperation(TableOperationsImpl.java:311)
> > at
> > org.apache.accumulo.core.client.admin.TableOperationsImpl.doTableOperation(TableOperationsImpl.java:279)
> > at
> > org.apache.accumulo.core.client.admin.TableOperationsImpl.create(TableOperationsImpl.java:207)
> > at
> > org.locationtech.geomesa.accumulo.data.AccumuloDataStore.org$locationtech$geomesa$accumulo$data$AccumuloDataStore$$ensureTableExists(AccumuloDataStore.scala:260)
> > at
> > orglocationtech.geomesa.accumulo.data.AccumuloDataStore.<init>(AccumuloDataStore.scala:126)
> > at
> > org.locationtech.geomesa.accumulodata.AccumuloDataStoreFactory$$anon$1.<init>(AccumuloDataStoreFactory.scala:78)
> >
> > at
> > org.locationtech.geomesa.accumulo.data.AccumuloDataStoreFactory.createDataStore(AccumuloDataStoreFactory.scala:78)
> > at
> > org.locationtech.geomesa.accumulo.data.AccumuloDataStoreFactory.createDataStore(AccumuloDataStoreFactory.scala:25)
> > at
> > org.geotools.data.DataAccessFinder.getDataStore(DataAccessFinder.java:130)
> > at org.geotools.data.DataStoreFinder.getDataStore(DataStoreFinder.java:89)
> > at geomesa.gdelt.GDELTIngest.main(GDELTIngest.java:116)
> >
> >
> > Thanks,
> > Jeff
> >
> > On Tue, Nov 17, 2015 at 5:01 PM, Jake Kenneally <jdk2pq@xxxxxxxx> wrote:
> > > Hi Jeff,
> > > I've just updated the geomesa-gdelt project so that you should be able to
> > > depend on the latest version of GeoMesa for that tutorial. Try doing a
> > > `git
> > > pull origin master` on the geomesa-gdelt directory and building again to
> > > see
> > > if things are working.
> > >
> > > If not, try building the latest release of GeoMesa
> > > (https://github.com/locationtech/geomesa/releases/tag/geomesa-1.1.0-rc.6)
> > > with `build/mvn clean install -DskipTests` from the geomesa directory, and
> > > then try building the geomesa-gdelt project.
> > >
> > > If things still aren't working, let me know, and I'll keep working on it.
> > >
> > > Thanks,
> > > Jake
> > >
> > >
> > > On 11/17/2015 03:52 PM, Jeff Gold wrote:
> > > > Greetings. I’ve got GeoMesa working great with some basic loading and
> > > > querying and I’m now trying to run this ingest example:
> > > > http://www.geomesa.org/geomesa-gdelt-analysis/
> > > >
> > > > But my mvn clean install is failing for geomesa-gdelt.git.
> > > >
> > > > It is failing on:
> > > >
> > > >
> > > > http://repo.boundlessgeo.com/main/org/locationtech/geomesa/geomesa-core-accumulo1.5/1.0.0-rc.4-SNAPSHOT/maven-metadata.xml.
> > > > Return code is: 504 , ReasonPhrase:Gateway Timeout.
> > > >
> > > >
> > > > When I browse the URL in a web browser, the URL doesn’t exist beyond
> > > > locationtech. The only folder below locationtech is geogig (no
> > > > geomesa).
> > > >
> > > >
> > > > Any ideas?
> > > >
> > > >
> > > > Thanks,
> > > > Jeff
> > > > _______________________________________________
> > > > geomesa-users mailing list
> > > > geomesa-users@xxxxxxxxxxxxxxxx
> > > > To change your delivery options, retrieve your password, or unsubscribe
> > > > from this list, visit
> > > > http://www.locationtech.org/mailman/listinfo/geomesa-users
> > >
> > > _______________________________________________
> > > geomesa-users mailing list
> > > geomesa-users@xxxxxxxxxxxxxxxx
> > > To change your delivery options, retrieve your password, or unsubscribe
> > > from
> > > this list, visit
> > > http://www.locationtech.org/mailman/listinfo/geomesa-users
> > _______________________________________________
> > geomesa-users mailing list
> > geomesa-users@xxxxxxxxxxxxxxxx
> > To change your delivery options, retrieve your password, or unsubscribe from
> > this list, visit
> > http://www.locationtech.org/mailman/listinfo/geomesa-users
> >
> >
> > _______________________________________________
> > geomesa-users mailing list
> > geomesa-users@xxxxxxxxxxxxxxxx
> > To change your delivery options, retrieve your password, or unsubscribe from
> > this list, visit
> > http://www.locationtech.org/mailman/listinfo/geomesa-users
> _______________________________________________
> geomesa-users mailing list
> geomesa-users@xxxxxxxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> http://www.locationtech.org/mailman/listinfo/geomesa-users