Spark writing to S3 failed: java.lang.NoSuchMethodError: org.apache.hadoop.util.SemaphoredDelegatingExecutor.

Symptom:

When using Spark writing to S3, the insert query failed:

java.lang.NoSuchMethodError: org.apache.hadoop.util.SemaphoredDelegatingExecutor.<init>(Lcom/google/common/util/concurrent/ListeningExecutorService;IZ)V
	at org.apache.hadoop.fs.s3a.impl.StoreContext.createThrottledExecutor(StoreContext.java:292)
	at org.apache.hadoop.fs.s3a.impl.DeleteOperation.<init>(DeleteOperation.java:206)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.delete(S3AFileSystem.java:2468)
	at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.cleanupJob(FileOutputCommitter.java:532)
	at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.abortJob(FileOutputCommitter.java:551)
	at org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.abortJob(HadoopMapReduceCommitProtocol.scala:242)
	at org.apache.spark.sql.rapids.GpuFileFormatWriter$.write(GpuFileFormatWriter.scala:262)

Env:

spark-3.2.1-bin-hadoop3.2

hadoop-aws-3.2.1.jar

aws-java-sdk-bundle-1.11.375.jar

Solution:

After upgrading  hadoop-aws-3.2.1.jar to hadoop-aws-3.2.3.jar, it works fine.

$  ls -altr $SPARK_HOME/jars|grep -i aws
-rw-rw-r--  1 xxx xxx 98732349 Jul 26  2018 aws-java-sdk-bundle-1.11.375.jar
-rw-rw-r--  1 xxx xxx   506819 Jul 19 10:03 hadoop-aws-3.2.3.jar