Thursday, November 27, 2014

Failed to load hive table in pig using hcatalog

Symptom:

In pig, loading hive table using HCatLoader fails with below error message:
grunt> a = load 'passwords' using org.apache.hcatalog.pig.HCatLoader(); 
2014-11-25 11:19:35,474 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve org.apache.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Details at logfile: /root/pig_1416943163875.log

Root Cause:

Option "-useHCatalog" is not specified when starting pig.

Solution:


1. Import needed jars in HADOOP_CLASSPATH.

Take hive-0.13 for example:
export HADOOP_CLASSPATH=/opt/mapr/hive/hive-0.13/hcatalog/share/hcatalog/hive-hcatalog-core-0.13.0-mapr-1409.jar:/opt/mapr/hive/hive-0.13/hcatalog/share/hcatalog/hive-hcatalog-streaming-0.13.0-mapr-1409.jar:/opt/mapr/hive/hive-0.13/hcatalog/share/hcatalog/hive-hcatalog-pig-adapter-0.13.0-mapr-1409.jar:/opt/mapr/hive/hive-0.13/hcatalog/share/hcatalog/hive-hcatalog-server-extensions-0.13.0-mapr-1409.jar

2. Start pig using "-useHCatalog" option.

pig -useHCatalog
Then it should work:
grunt> a = load 'passwords' using org.apache.hive.hcatalog.pig.HCatLoader(); 
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/mapr/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/mapr/hbase/hbase-0.94.21/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2014-11-25 11:25:43,929 [main] INFO  hive.metastore - Trying to connect to metastore with URI thrift://xx.xx.xx.xx:9083
2014-11-25 11:25:43,965 [main] INFO  hive.metastore - Connected to metastore.



No comments:

Post a Comment

Popular Posts