Friday, December 12, 2014

Hbase/M7 sample JAVA code to scan a table

This is just a memo for a sample java code to scan a hbase/M7 table.
Source Code -- MaprHbaseTest.java
 import java.io.IOException;
     import org.apache.hadoop.conf.Configuration;
     import org.apache.hadoop.hbase.HBaseConfiguration;
     import org.apache.hadoop.hbase.KeyValue;
     import org.apache.hadoop.hbase.client.HTable;
     import org.apache.hadoop.hbase.client.Result;
     import org.apache.hadoop.hbase.client.ResultScanner;
     import org.apache.hadoop.hbase.client.Scan;
 public class MaprHbaseTest {
         private static Configuration conf = null;
         static {
                 conf = HBaseConfiguration.create();
                 //conf.set("hbase.table.namespace.mappings","t1:/,t15:/tables,t3:./,t20:/goose/tables");
                 conf.set("mapr.htable.impl","com.mapr.fs.MapRHTable");
                 //conf.set("fs.default.name", "maprfs://xxx:7222");
                 conf.set("hbase.zookeeper.quorum","n1a,n2a,n3a");
                 conf.set("hbase.zookeeper.property.clientPort","5181");
                 conf.set("fs.maprfs.impl", "com.mapr.fs.MapRFileSystem");
                 conf.set("hadoop.spoofed.user.uid","2000");
                 conf.set("hadoop.spoofed.user.gid","2000");

                 conf.set("hadoop.spoofed.user.username","mapr");
         }

         //Scan (or list) a table
         public static void getAllRecord (String tableName) {
         try{
              HTable table = new HTable(conf, tableName.getBytes());
              Scan s = new Scan();
              ResultScanner ss = table.getScanner(s);
              for(Result r:ss){
                  for(KeyValue kv : r.raw()){
                     System.out.print(new String(kv.getRow()) + " ");
                     System.out.print(new String(kv.getFamily()) + ":");
                     System.out.print(new String(kv.getQualifier()) + " ");
                     System.out.print(kv.getTimestamp() + " ");
                     System.out.println(new String(kv.getValue()));
                  }
              }
         } catch (IOException e){
             e.printStackTrace();
         }
     }

         public static void main(String[] args) {

                 try {
                         String tablename = "/basetest";
                         if (args != null && args.length>0) {
                          tablename = args[0];
                         }
     //To access M7 tables always prefix /
                         MaprHbaseTest.getAllRecord(tablename);
                 } catch (Exception e) {
                         e.printStackTrace();
                 }

         }
 }
Note: Make sure uid/gid/username are correct.
[mapr@n1a ~]$ id
uid=2000(mapr) gid=2000(mapr) groups=2000(mapr),2001(shadow)
Env:
[root@n1a hbasejava]# cat  env.sh
export CLASSPATH=/root/hao/case/hbasejava:/opt/mapr/hbase/hbase-0.94.21/*:/opt/mapr/hbase/hbase-0.94.21/lib/*:/opt/mapr/hadoop/hadoop-0.20.2/*:/opt/mapr/hadoop/hadoop-0.20.2/lib/*
Note: The MaprHbaseTest.java exists in /root/hao/case/hbasejava
Test:
source env.sh
javac MaprHbaseTest.java
java MaprHbaseTest <table name>
If no table found, below error will occur:
14/12/12 14:24:05 WARN client.HConnectionManager$HConnectionImplementation: Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for table: abc, row=abc,,99999999999999
 at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:151)
 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:1059)
 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1121)
 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1001)
 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:958)
 at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:279)
 at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:169)
 at MaprHbaseTest.getAllRecord(MaprHbaseTest.java:28)
 at MaprHbaseTest.main(MaprHbaseTest.java:53)
org.apache.hadoop.hbase.TableNotFoundException: abc
 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1139)
 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1001)
 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:958)
 at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:279)
 at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:169)
 at MaprHbaseTest.getAllRecord(MaprHbaseTest.java:28)
 at MaprHbaseTest.main(MaprHbaseTest.java:53)

2 comments:

  1. Can you share your pom.xml for dependency's?

    ReplyDelete
  2. A fairly detailed script, I ran it in the browser, but it seems to me that doesn't work very well. How about optimizing the code? My friend is more knowledgeable in the java script, so I turned to him for advice. After all discussion about bugs, I think with guidance on java more details I can understand what's wrong.

    ReplyDelete

Popular Posts