Monday, December 8, 2014

How to use openTSDB with MapR

This article shows the steps to install and configure openTSDB rpm on MapR cluster.
Note, these steps are not official or QAed steps, for test only.
Env: MapR 4.0.1 Demo sandbox.

1.  Install the necessary packages

yum groupinstall 'Development Tools' -y
yum install -y git autoconf libtool rpm-build telnet gnuplot liberation-sans-fonts

2. Download and install the latest openTSDB rpm from http://opentsdb.net/

wget https://github.com/OpenTSDB/opentsdb/releases/download/v2.0.1/opentsdb-2.0.1.noarch.rpm
rpm -ivh opentsdb-2.0.1.noarch.rpm

3. Replace the build asynchbase jar with mapr-asynchbase jar.

yum install mapr-asynchbase
cp <path_to_installed_asynchbase_jar> <opentsdb_install_root>/build/third_party/hbase/<asynchbase_jar_name>
Eg:
cp /opt/mapr/asynchbase/asynchbase-1.4.1/asynchbase-1.4.1-mapr-1407.jar /usr/share/opentsdb/lib/asynchbase-1.6.0.jar

4.  Update core-site.xml for namespace mapping for tables tsdb creates. (tsdb, tsdb-uid, tsdb-tree, tsdb-meta)

<property>
  <name>hbase.table.namespace.mappings</name>
  <value>tsdb:/tsdb,tsdb-uid:/tsdb-uid,tsdb-tree:/tsdb-tree,tsdb-meta:/tsdb-meta</value>
</property>
<property>
  <name>fs.mapr.asynchbase.flushonput</name>
  <value>true</value>
</property>

5. Restart warden

service mapr-warden stop
service mapr-warden start

6. Configure OpenTSDB

vi /etc/opentsdb/opentsdb.conf
# If using MapR 4.0.1 client with asynchbase 4.0.1 or newer you are ready to go otherwise change this setting - there was a bug and it is fixed in the 4.0.1 client and accompanying release of asynchbase-mapr
tsd.storage.enable_compaction = false
# FOR TESTING ONLY: Create metrics automatically generally should be off, but for all testing cases, turn it on
tsd.core.auto_create_metrics = true
# Specify a valid path to your tables - the client will know these are MapR-DB Tables and NOT HBase tables because of this path! - You will create these in Step 4.
tsd.storage.hbase.data_table = /tsdb
tsd.storage.hbase.uid_table = /tsdb-uid
tsd.storage.hbase.tree_table = /tsdb-tree
# asynchbase-mapr doesn’t actually use zk_quorum - but it needs to be set to something
tsd.storage.hbase.zk_quorum = maprdemo:5181

7. Set environment variables

vi /opt/mapr/conf/env.sh
export COMPRESSION=NONE
export HBASE_HOME=/opt/mapr/hbase/hbase-0.94.21/
export GDFONTPATH=/usr/share/fonts/liberation
export GNUPLOT_DEFAULT_GDFONT=LiberationSans-Regular

8. Create Tables on MapRDB

/usr/share/opentsdb/tools/create_table.sh

9. Copy MapR jars into opentsdb.

Take MapR 4.0.1 for example:
\cp -f /opt/mapr/hadoop/hadoop-0.20.2/lib/*.jar /usr/share/opentsdb/lib 
rm /usr/share/opentsdb/lib/netty-3.6.2.Final.jar

10. Start daemon

nohup tsdb tsd --port=4242 --staticroot="/usr/share/opentsdb/static/" --cachedir="/tmp/opentsdb"  --table=/tsdb --uidtable=/tsdb-uid --auto-metric &

11. Create Metrics

su - mapr
tsdb mkmetric mymetric.stock 

12. Import Sample Data into a file called tmp_input

vi tmp_input
mymetric.stock 1407165399 196.30 symbol=VOD.L
mymetric.stock 1407165399 484.20 symbol=BP.L
mymetric.stock 1407165401 224.15 symbol=BARC.L
mymetric.stock 1407165402 196.30 symbol=VOD.L
mymetric.stock 1407165403 484.15 symbol=BP.L
mymetric.stock 1407165404 224.15 symbol=BARC.L
mymetric.stock 1407165405 196.30 symbol=VOD.L
mymetric.stock 1407165405 484.15 symbol=BP.L
mymetric.stock 1407165406 224.15 symbol=BARC.L
mymetric.stock 1407165407 196.30 symbol=VOD.L
mymetric.stock 1407165408 484.15 symbol=BP.L
mymetric.stock 1407165409 224.15 symbol=BARC.L
mymetric.stock 1407165410 196.30 symbol=VOD.L
mymetric.stock 1407165411 484.15 symbol=BP.L
mymetric.stock 1407165412 224.15 symbol=BARC.L
mymetric.stock 1407165413 196.30 symbol=VOD.L
mymetric.stock 1407165414 484.15 symbol=BP.L
mymetric.stock 1407165415 224.15 symbol=BARC.L
mymetric.stock 1407165416 196.30 symbol=VOD.L
mymetric.stock 1407165417 484.15 symbol=BP.L
mymetric.stock 1407165417 224.15 symbol=BARC.L
mymetric.stock 1407165418 196.30 symbol=VOD.L
mymetric.stock 1407165419 484.15 symbol=BP.L
mymetric.stock 1407165422 224.15 symbol=BARC.L
mymetric.stock 1407165422 196.30 symbol=VOD.L
mymetric.stock 1407165423 484.255 symbol=BP.L

13. Import file and the data is loaded into table '/tsdb'

tsdb import tmp_input

14. Check web page

http://<server-name>:4242/ 
Choose dates August 3 to August 5 timeframe
Choose metrics as mymetric.stock and press enter.


1 comment:

  1. These essay writing services can be found online by reading essay pro review reviews, which will assist you in selecting the top essay writing services and avoiding being a victim of scam. This may be avoided by keeping track of how properly the website is designed and maintained, as well as the credentials and experience of the team members.

    ReplyDelete

Popular Posts