| Re: [geomesa-users] Hortonworks/Geomesa distributed ingestion, error. |
I did try this, and I thought it would work. In my shell scrip that launches the load I included these lines: myHortonVersion=`hadoop version | grep "^This command" | sed " s/^.*hdp.// " | sed "s/.hadoop.hadoop-common.*//" | sed "s/^/-Dhdp.version=/" ` echo “determined this is the local hortonworks version:" echo ${myHortonVersion} export JAVA_OPTS=${myHortonVersion} echo "Using this Java Option: “ echo ${JAVA_OPTS} geomesa ingest \ -u ${accumulo_user} -p ${accumulo_pw} \ -i ${accumulo_instance_id_param} -z ${zookeeper_param} \ -c myGeomesa.gcam -s gcam-schema \ -C gcam-reader \ When I run it I still get the same error, pasted below: [andrew@gzet loadscripts]$ . load_gcam.sh using these java opts: -Dhdp.version=2.4.0.0-169 Using this Java Option: -Dhdp.version=2.4.0.0-169 Using GEOMESA_HOME = /home/andrew/data/geo/geomesa-1.2.4/dist/tools/geomesa-tools-1.2.4 Creating schema gcam-schema Running ingestion in distributed mode Submitting job - please wait... Unable to parse '/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a URI, check the setting for mapreduce.application.framework.path java.lang.IllegalArgumentException: Unable to parse '/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a URI, check the setting for mapreduce.application.framework.path when I inspect the error further I find the mapreduce.tar.gz that the code is looking for here: /usr/hdp/2.4.0.0-169/hadoop/mapreduce.tar.gz We see that we are looking at the wrong path, albeit with the right version embedded in it. The way the version option is assembled from the JAVA_OPTS into the URI for the file it’s searching for needs adjusting. Is there a way you can pass that in an an option too? many thanks with this Andrew Andrew J Morgan CEO, Bytesumo Limited Tel: +44 (0)7970130767 E-mail: andrew@xxxxxxxxxxxx Bytesumo Limited - Registered Company in England and Wales 33 Brodrick Grove, London, SE2 0SR, UK. Company Number: 8505203
|