Hive on Tez : How to control the number of Mappers and Reducers

Goal:

How to control the number of Mappers and Reducers in Hive on Tez.

Hive on Tez : How to identify which DAG ID for which Hive query in the same DAG Application Master

Goal:

Hive on Tez : How to identify which DAG ID for which Hive query in the same DAG Application Master

Hive on Tez : How to identify the reused YARN containers

Goal:

Hive on Tez : How to identify the reused YARN containers

Drill errors with "JDK Java compiler not available - probably you're running Drill with a JRE and not a JDK"

Symptom:

Drill query fails with error message:"JDK Java compiler not available - probably you're running Drill with a JRE and not a JDK".

Pig job fails with "org.apache.hadoop.mapreduce.counters.LimitExceededException: Too many counters: 121 max=120"

Symptom:

Pig job fails with "org.apache.hadoop.mapreduce.counters.LimitExceededException: Too many counters: 121 max=120".

How to limit the size for each zookeeper transaction log

Goal:

How to limit the size for each zookeeper transaction log.

How to purge Zookeeper Transaction Log and Snapshots

Goal:

How to purge Zookeeper Transaction Log and Snapshots.

How to read or dump the Zookeeper Transaction Log and Snapshots

Goal:

How to read or dump the Zookeeper Transaction Log and Snapshots.

How to troubleshoot Hive local task in a separate JVM process

Goal:

How to troubleshoot Hive local task in a separate JVM process.
Especially when troubleshooting issues of Map Join's local task, we may want to put this local task into a separate JVM process instead of in the Hive CLI process.

How to use Drill to parse ResourceManager Rest API results

Goal:

ResourceManager REST API provides all detailed information regarding YARN applications, metrics of YARN cluster, etc.
This article provides a simple demo on how to use Drill to query the result of the REST APIs.
One example use case is to show the largest YARN applications which are currently running.