Saturday, December 12, 2020

What does "enable_table_replication" do internally in Hbase replication?

Goal:

This article explains what does the command "enable_table_replication" do internally in Hbase replication by looking into the source code.

It also explains the difference between below 2 commands which are shown on different articles.

hbase shell> enable_table_replication "t1"

vs.

hbase shell> disable 't1'
hbase shell> alter 't1', {NAME => 'column_family_name', REPLICATION_SCOPE => '1'}
hbase shell> enable 't1'

Env:

Hbase 1.1.8

Analysis:

1. Hbase Source code analysis for "enable_table_replication"

a. "enable_table_replication" is a ruby command in hbase shell

Inside hbase-shell/src/main/ruby/shell/commands/enable_table_replication.rb,

it is calling replication_admin.enable_tablerep(table_name).

b. "enable_tablerep"

Inside hbase-shell/src/main/ruby/hbase/replication_admin.rb,

it is calling @replication_admin.enableTableRep(tableName).

c. "enableTableRep"

Inside hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java,

it is calling:

    checkAndSyncTableDescToPeers(tableName, splits);
setTableRep(tableName, true);

d. "checkAndSyncTableDescToPeers"

Inside hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java,

it is to Create the same table on peer when not exist, and Throw exception if the table exists on peer cluster but descriptors are not same.

e. "setTableRep"

Inside hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java,

      if (isTableRepEnabled(htd) ^ isRepEnabled) {
boolean isOnlineSchemaUpdateEnabled =
this.connection.getConfiguration()
.getBoolean("hbase.online.schema.update.enable", true);
if (!isOnlineSchemaUpdateEnabled) {
admin.disableTable(tableName);
}
for (HColumnDescriptor hcd : htd.getFamilies()) {
hcd.setScope(isRepEnabled ? HConstants.REPLICATION_SCOPE_GLOBAL
: HConstants.REPLICATION_SCOPE_LOCAL);
}
admin.modifyTable(tableName, htd);
if (!isOnlineSchemaUpdateEnabled) {
admin.enableTable(tableName);
}

Basically it checks the value of hbase.online.schema.update.enable(default=true).

If hbase.online.schema.update.enable=true, it modify the REPLICATION_SCOPE for ALL column families to true.

Else, it will firstly disable table, modify REPLICATION_SCOPE for ALL column families to true, and then enable table.

2. Differences

Based on above analysis, "enable_table_replication" can help create the table in target peer if not exist and detect differences on table if exist.

It can modify the REPLICATION_SCOPE for ALL column families.

It checks if hbase.online.schema.update.enable=true, and then decides if disable/enable table is needed.


No comments:

Post a Comment

Popular Posts