Friday, December 12, 2014

Hbase/M7 sample JAVA code to scan a table

This is just a memo for a sample java code to scan a hbase/M7 table.
Source Code -- MaprHbaseTest.java
 import java.io.IOException;
     import org.apache.hadoop.conf.Configuration;
     import org.apache.hadoop.hbase.HBaseConfiguration;
     import org.apache.hadoop.hbase.KeyValue;
     import org.apache.hadoop.hbase.client.HTable;
     import org.apache.hadoop.hbase.client.Result;
     import org.apache.hadoop.hbase.client.ResultScanner;
     import org.apache.hadoop.hbase.client.Scan;
 public class MaprHbaseTest {
         private static Configuration conf = null;
         static {
                 conf = HBaseConfiguration.create();
                 //conf.set("hbase.table.namespace.mappings","t1:/,t15:/tables,t3:./,t20:/goose/tables");
                 conf.set("mapr.htable.impl","com.mapr.fs.MapRHTable");
                 //conf.set("fs.default.name", "maprfs://xxx:7222");
                 conf.set("hbase.zookeeper.quorum","n1a,n2a,n3a");
                 conf.set("hbase.zookeeper.property.clientPort","5181");
                 conf.set("fs.maprfs.impl", "com.mapr.fs.MapRFileSystem");
                 conf.set("hadoop.spoofed.user.uid","2000");
                 conf.set("hadoop.spoofed.user.gid","2000");

                 conf.set("hadoop.spoofed.user.username","mapr");
         }

         //Scan (or list) a table
         public static void getAllRecord (String tableName) {
         try{
              HTable table = new HTable(conf, tableName.getBytes());
              Scan s = new Scan();
              ResultScanner ss = table.getScanner(s);
              for(Result r:ss){
                  for(KeyValue kv : r.raw()){
                     System.out.print(new String(kv.getRow()) + " ");
                     System.out.print(new String(kv.getFamily()) + ":");
                     System.out.print(new String(kv.getQualifier()) + " ");
                     System.out.print(kv.getTimestamp() + " ");
                     System.out.println(new String(kv.getValue()));
                  }
              }
         } catch (IOException e){
             e.printStackTrace();
         }
     }

         public static void main(String[] args) {

                 try {
                         String tablename = "/basetest";
                         if (args != null && args.length>0) {
                          tablename = args[0];
                         }
     //To access M7 tables always prefix /
                         MaprHbaseTest.getAllRecord(tablename);
                 } catch (Exception e) {
                         e.printStackTrace();
                 }

         }
 }
Note: Make sure uid/gid/username are correct.
[mapr@n1a ~]$ id
uid=2000(mapr) gid=2000(mapr) groups=2000(mapr),2001(shadow)
Env:
[root@n1a hbasejava]# cat  env.sh
export CLASSPATH=/root/hao/case/hbasejava:/opt/mapr/hbase/hbase-0.94.21/*:/opt/mapr/hbase/hbase-0.94.21/lib/*:/opt/mapr/hadoop/hadoop-0.20.2/*:/opt/mapr/hadoop/hadoop-0.20.2/lib/*
Note: The MaprHbaseTest.java exists in /root/hao/case/hbasejava
Test:
source env.sh
javac MaprHbaseTest.java
java MaprHbaseTest <table name>
If no table found, below error will occur:
14/12/12 14:24:05 WARN client.HConnectionManager$HConnectionImplementation: Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for table: abc, row=abc,,99999999999999
 at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:151)
 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:1059)
 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1121)
 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1001)
 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:958)
 at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:279)
 at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:169)
 at MaprHbaseTest.getAllRecord(MaprHbaseTest.java:28)
 at MaprHbaseTest.main(MaprHbaseTest.java:53)
org.apache.hadoop.hbase.TableNotFoundException: abc
 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1139)
 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1001)
 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:958)
 at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:279)
 at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:169)
 at MaprHbaseTest.getAllRecord(MaprHbaseTest.java:28)
 at MaprHbaseTest.main(MaprHbaseTest.java:53)

2 comments:

  1. Can you share your pom.xml for dependency's?

    ReplyDelete
  2. We can do it! Adopting the mindset of winners, we can cope with any task despite how difficult, urgent, or large it is. Tell us when you need your paper done and be sure our writers will do https://topswriting.com/review/payforessay it on time. A team of pro writers is always at your hand so you can supercharge your GPA with some great academic content.

    ReplyDelete

Popular Posts