This is a cookbook for scala programming. 1. Define a object with main function -- Helloworld. object HelloWorld { def main(args: Array...

Hive table contains files in HDFS, if one table or one partition has too many small files, the HiveQL performance may be impacted. Sometime...

Hive is trying to embrace CBO(cost based optimizer) in latest versions, and Join is one major part of it. Understanding join best practices ...

Many commands can check the memory utilization of JAVA processes, for example, pmap, ps, jmap, jstat. What are the differences? Before we ...

This article shows a sample code to load data into Hbase or MapRDB(M7) using Scala on Spark. I will introduce 2 ways, one is normal load us...

Goal: How to build and use parquet-tools to read parquet files. Solution: 1. Download and Install maven. Follow below link: http://...

Goal: This article explains what is the difference between Spark HiveContext and SQLContext.

Goal: This article explains the configuration parameters for Oozie Launcher job.

Goal: This article provides the SQL to list table or partition locations from Hive Metastore. Env: Hive metastore 0.13 on MySQL Root ...

Goal: How to understand PageRank algorithm in scala on Spark. This article explains each step using sample data.