Showing posts with label RAPIDS. Show all posts
Showing posts with label RAPIDS. Show all posts
Monday, May 3, 2021

Understand Decimal precision and scale calculation in Spark using GPU or CPU mode

›
Goal: This article research on how Spark calculates the Decimal precision and scale using GPU or CPU mode. Basically we will test Addition/S...
12 comments:
Thursday, April 29, 2021

How to use Spark Operator to run Spark job with Rapids Accelerator

›
Goal: This article shares the steps on how to run Spark job with Rapids Accelerator using Spark Operator in a Kubernetes Cluster.
Tuesday, April 27, 2021

Rapids Accelerator compatibility related to spark.sql.legacy.parquet.datetimeRebaseModeInWrite

›
Goal: This article talked about the compatibility of Rapids Accelerator for Spark regarding parquet writing related to parameters spark.sql....
Tuesday, April 20, 2021

How to use latest version of Rapids Accelerator for Spark on EMR

›
Goal: This article shows how to use latest version of Rapids Accelerator for Spark on EMR.  Currently the latest EMR 6.2 only ships with Rap...
Thursday, April 8, 2021

How to use NVIDIA Nsight Systems to profile a Spark job on Rapids Accelerator

›
Goal: This article explains how to use NVIDIA Nsight Systems to profile a Spark job on Rapids Accelerator.
Sunday, April 4, 2021

How to enable GpuKryoRegistrator on RAPIDS Accelerator for Spark

›
Goal: This article shares the steps to enable GpuKryoRegistrator on RAPIDS Accelerator for Spark.
Thursday, March 25, 2021

concat_ws example on Spark with RAPIDS Accelerator

›
Goal: This is a quick example of operator contact_ws on Spark with RAPIDS Accelerator.
Wednesday, March 24, 2021

Hands-on native cuDF Pandas UDF

›
Goal: This article will help show some hands-on steps to play with native cuDF Pandas UDF on Spark with RAPIDS Accelerator for Apache Spark .
Tuesday, March 23, 2021

How to run the pandas cudf_udf test for RAPIDS Accelerator for Apache Spark

›
Goal: How to run the pandas cudf_udf test for RAPIDS Accelerator for Apache Spark .
Friday, March 19, 2021

Understanding RAPIDS Accelerator For Apache Spark's supported timezone

›
Goal: This article explains the current supported timezone for " RAPIDS Accelerator For Apache Spark ".
Thursday, March 18, 2021

Spark Tuning -- Adaptive Query Execution(3): Dynamically optimizing skew joins

›
Goal: This article explains Adaptive Query Execution (AQE)'s "Dynamically optimizing skew joins" feature introduced in Spark 3...

What Dataset API is not supported for RAPIDS Accelerator for Apache Spark

›
Goal: This article explains what Dataset API is not supported for RAPIDS Accelerator for Apache Spark.
Wednesday, March 17, 2021

Spark Tuning -- Adaptive Query Execution(2): Dynamically switching join strategies

›
Goal: This article explains Adaptive Query Execution (AQE)'s "Dynamically switching join strategies" feature introduced in Spa...
Tuesday, March 16, 2021

Spark Tuning -- Adaptive Query Execution(1): Dynamically coalescing shuffle partitions

›
Goal: This article explains Adaptive Query Execution (AQE)'s "Dynamically coalescing shuffle partitions" feature introduced in...
Monday, March 15, 2021

Spark Tuning -- Dynamic Partition Pruning

›
Goal: This article explains Dynamic Partition Pruning (DPP) feature introduced in Spark 3.0.
Wednesday, March 10, 2021

Understanding RAPIDS Accelerator For Apache Spark parameter -- spark.rapids.memory.gpu.allocFraction and GPU pool related ones.

›
Goal: This article explains the RAPIDS Accelerator For Apache Spark parameter -- spark.rapids.memory.gpu.allocFraction and other GPU memory...
Tuesday, March 9, 2021

Understanding RAPIDS Accelerator For Apache Spark parameter -- spark.rapids.memory.pinnedPool.size

›
Goal:  This article explains the RAPIDS Accelerator For Apache Spark parameter -- spark.rapids.memory.pinnedPool.size .
Wednesday, February 24, 2021

Spark on GPU -- Hands on GCP Dataproc to test Spark on GPU using RAPIDS

›
Goal: This article shares a quick hands-on experience to test Spark on GPU using RAPIDS in GCP Dataproc. We will provide step-by-step instr...