Tuesday, July 1, 2014

Pressure testing tool for Impala -- HAOTPS

HAOTPS is a handy pressure testing tool in JAVA written by me.
It was created for Oracle, but this version is only for Impala compatible.
Please refer to Pressure testing tool for Greenplum -- HAOTPS for details.

What are the differences?

1. JDBC driver
This Impala version is referring to <Configuring Impala to Work with JDBC>.
The JDBC driver we are using is "org.apache.hive.jdbc.HiveDriver".
Class.forName("org.apache.hive.jdbc.HiveDriver").newInstance();
2. Database-less
This version does not support "database" in the configuration file, so you can specify any database name. For example, 
#sid is the database name we want to connect in the GP system
$sid       = test_database
3.  Download link
haotps_impala.zip
4.  Usage
unzip haotps_impala.zip
cd haotps_impala
source haotps.profile
vim test.conf
java haotps test.conf

Example

[root@admin haotps_impala]# java haotps test.conf
======1.checking the config file test.conf ......
======2.checking constraint of the config ......
======3.collecting all the bind variable data ......
log4j:WARN No appenders could be found for logger (org.apache.hive.jdbc.HiveQueryResultSet).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
======4.running 2 threads, each thread with 10 queries......
----------------Result--------------
Average response time(ms) =   242.15
Average TPS               =   7.84
When it's running, we can see the active queries in below page:
http://<target host>:25000/queries

Note:

Due to the hive-jdbc bug in cdh4.2 version mentioned below:
[BUG] JDBC using PreparedStatement can not close the statements properly.
I suggest you use hive-jdbc cdh5 or above version. The hive-jdbc in my attached zip file is cdh4.2 version.
If you are using CM5 parcels installation, you can specify below CLASSPATH in haotps.profile:
export CLASSPATH=/opt/cloudera/parcels/CDH/lib/hive/lib/*:/opt/cloudera/parcels/CDH/lib/hadoop/*:.

No comments:

Post a Comment

Popular Posts