Error java.lang.NoSuchMethodException when running spark-sql-perf with Hive Metastore 3.x

Symptom:

When using spark-sql-perf with a Hive Metastore 3.x, it fails with error:

java.lang.NoSuchMethodException: 
org.apache.hadoop.hive.ql.metadata.Hive.alterTable(java.lang.String, org.apache.hadoop.hive.ql.metadata.Table, org.apache.hadoop.hive.metastore.api.EnvironmentContext)

Env:

Hive 3.x

Root Cause:

This is due to the latest branch of spark-sql-perf is still using Hive 2.x artifacts to compile, but here our Hive Metastore is 3.x version.

And between Hive 2.x and 3.x, the signature for method "alterTable" in java class "org.apache.hadoop.hive.ql.metadata.Hive" changed a lot.

In Hive 3.x:

$ grep alterTable ./ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java |grep public
  public void alterTable(Table newTbl, boolean cascade, EnvironmentContext environmentContext,
  public void alterTable(String fullyQlfdTblName, Table newTbl, EnvironmentContext environmentContext,
  public void alterTable(String fullyQlfdTblName, Table newTbl, boolean cascade,
  public void alterTable(String fullyQlfdTblName, Table newTbl, boolean cascade,
  public void alterTable(String catName, String dbName, String tblName, Table newTbl, boolean cascade,
  public void alterTable(String catName, String dbName, String tblName, Table newTbl, boolean cascade,

In Hive 2.3.8:

$ git checkout rel/release-2.3.8
$ grep alterTable ./ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java |grep public
  public void alterTable(String tblName, Table newTbl, EnvironmentContext environmentContext)
  public void alterTable(String tblName, Table newTbl, boolean cascade, EnvironmentContext environmentContext)

As we can see from above, in Hive 3.x, there is no such signature with below 3 input parameters' data type anymore: 

(java.lang.String, org.apache.hadoop.hive.ql.metadata.Table, org.apache.hadoop.hive.metastore.api.EnvironmentContext)

Solution:

This is a workaround by just using Hive 2.x jars in the pre-built 2.x tarball:

https://hive.apache.org/downloads.html

And then in spark-defaults.conf, change it to above jars:

spark.sql.hive.metastore.jars /xxx/apache-hive-2.3.8-bin/lib/*