Env:
impalad version 1.3.1-cdh5 RELEASE (build )hive-jdbc-0.10.0-cdh4.2.0.jar
Symptom:
When using "PreparedStatement" to execute SQLs in Impala in a loop, it can not close the statements properly.From "http://<impalad IP>:25000/queries" page, you may see many queries are running in parallel although you are using one single connection.
Many of them are in "FINISHED" state, but not closed.
If the SQL count in the loop is above the resource pool capacity(default pool is 20), below error message shows:
java.sql.SQLException: Admission for query id=78415bdbab8f54e0:3cbe172156d17c99 exceeded timeout 60000ms at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:159) at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:147) at org.apache.hive.jdbc.HivePreparedStatement.executeImmediate(HivePreparedStatement.java:187) at org.apache.hive.jdbc.HivePreparedStatement.executeQuery(HivePreparedStatement.java:149) at ClouderaImpalaJdbcExample2.main(ClouderaImpalaJdbcExample2.java:40)
Reproduce:
ClouderaImpalaJdbcExample.java is using "java.sql.Statement" and it works fine. Class.forName(JDBC_DRIVER_NAME);
con = DriverManager.getConnection(CONNECTION_URL);
for (int i=1;i<=21;i++) {
Statement stmt = con.createStatement();
ResultSet rs = stmt.executeQuery(SQL_STATEMENT);
System.out.println("\n== Begin Query Results ======================");
while (rs.next()) {
System.out.println(rs.getString(1));
}
System.out.println("== End Query Results =======================\n\n");
stmt.close();
rs.close();
}
ClouderaImpalaJdbcExample2.java is using "java.sql.PreparedStatement" and it can reproduce the issue. Class.forName(JDBC_DRIVER_NAME);
con = DriverManager.getConnection(CONNECTION_URL);
for (int i=1;i<=21;i++) {
PreparedStatement stmt=con.prepareStatement(SQL_STATEMENT);
stmt.setString(1,"abc");
ResultSet rs = stmt.executeQuery();
System.out.println("\n== Begin Query Results ======================");
while (rs.next()) {
System.out.println(rs.getString(1));
}
System.out.println("== End Query Results =======================\n\n");
stmt.close();
rs.close();
}
Fix:
This issue is fixed in latest hive-jdbc, for example, hive-jdbc-0.12.0-cdh5.0.2.jar.If you are using CM5 parcels installation, you can specify below CLASSPATH:
export CLASSPATH=/opt/cloudera/parcels/CDH/lib/hive/lib/*:/opt/cloudera/parcels/CDH/lib/hadoop/*:.

B5E722E5
ReplyDeleteTunceli
Uşak
Osmaniye
Elazığ
Yozgat
Isparta
Gümüşhane
Muğla
Kütahya
Excellent technical write-up! I appreciate how you identified the JDBC PreparedStatement issue, explained the root cause, and walked through the solution in a clear, practical way. Posts like this are incredibly valuable for developers troubleshooting similar problems.
ReplyDeleterestaurant equipment auctions
D05B096C
ReplyDeleteKırşehir
Siirt
Aydın
Kırıkkale
Rize
Bitlis
Aksaray
Kastamonu
Mardin