This is a cookbook for scala programming. 1. Define a object with main function -- Helloworld. object HelloWorld { def main(args: Array...

This article shows a sample code to load data into Hbase or MapRDB(M7) using Scala on Spark. I will introduce 2 ways, one is normal load us...

Hive is trying to embrace CBO(cost based optimizer) in latest versions, and Join is one major part of it. Understanding join best practices ...

Hive table contains files in HDFS, if one table or one partition has too many small files, the HiveQL performance may be impacted. Sometime...

Goal: This article explains what is the difference between Spark HiveContext and SQLContext.

Goal: How to build and use parquet-tools to read parquet files. Solution: 1. Download and Install maven. Follow below link: http://...

Goal: This article explains the configuration parameters for Oozie Launcher job.

Goal: How to understand PageRank algorithm in scala on Spark. This article explains each step using sample data.

The fundamental idea of MRv2(YARN) is to split up the two major functionalities -- resource management and job scheduling/monitoring, into...

Env: Hive 0.13 Impala 1.4.1 Symptom: Hive fails to read the parquet table created by Impala with below error: FAILED: RuntimeExceptio...