Friday, January 9, 2015

Hands on Promotable Mirror in MapR 4.0.2

Promotable Mirror is a new feature in MapR 4.0.2.
My understanding is, as long as the volumes share the same original source volume, you can change the mirror chain to failover the source - mirror relationships.

Here is a hands on lab demo:

1. Create below cascading mirror chain.

srcvol -> mirvol1 -> mirvol2
   |
   -----> mirvol3 
Using below commands:
maprcli volume create  -name srcvol -path /srcvol
maprcli volume create  -name mirvol1 -source srcvol@demo.mapr.com  -path /mirvol1 -type mirror
maprcli volume create  -name mirvol2 -source mirvol1@demo.mapr.com -path /mirvol2 -type mirror
maprcli volume create  -name mirvol3 -source srcvol@demo.mapr.com  -path /mirvol3 -type mirror
Check current volume infomation.
All "mirrorDataSrcVolume" of mirrors are pointing to "srcvol", which means they all share the same data source.
"mirrortype" is newly introduced column, and "volumetype" is legacy column.

For "mirrortype":
3 - new RW volume
    Can be converted to type 2 to make mirror
2 - new RO mirror volume (aka promotable mirror)
    Can be converted to type 3 to make read/write
1 - old RO mirror volume
0 - old RW volume
maprcli volume list -filter "[volumename==*vol*]" -columns volumename,type,mirrortype,mirrorSrcVolume,mirrorDataSrcVolume,volumetype
mirrortype  mirrorDataSrcVolume  mirrorSrcVolume  volumename                   volumetype
3                                                 mapr.resourcemanager.volume  0
2           srcvol               srcvol           mirvol1                      1
2           srcvol               mirvol1          mirvol2                      1
2           srcvol               srcvol           mirvol3                      1
3                                                 srcvol                       0

2. If "srcvol" is lost, let's break the mirroring from srcvol to "mirvol1" and "mirvol3" by enabling read/write of "mirvol1" and "mirvol3".

maprcli volume modify -name mirvol1 -type rw
maprcli volume modify -name mirvol3 -type rw
After that, the topology is:
srcvol   mirvol1 -> mirvol2
   
         mirvol3 
Note: it doesn't change the "mirrorDataSrcVolume" and "mirrorSrcVolume", but the types.
maprcli volume list -filter "[volumename==*vol*]" -columns volumename,type,mirrortype,mirrorSrcVolume,mirrorDataSrcVolume,volumetype
mirrortype  mirrorDataSrcVolume  mirrorSrcVolume  volumename                   volumetype
3                                                 mapr.resourcemanager.volume  0
3           srcvol               srcvol           mirvol1                      0
2           srcvol               mirvol1          mirvol2                      1
3           srcvol               srcvol           mirvol3                      0
3                                                 srcvol                       0

3. Let's make some changes to "srcvol", "mirvol1" and "mirvol3", to make them diverge.

touch /mapr/demo.mapr.com/srcvol/srcvol_change1
touch /mapr/demo.mapr.com/mirvol1/mirvol1_change1
touch /mapr/demo.mapr.com/mirvol3/mirvol3_change1
And also sync the change from "mirvol1" to "mirvol2":
Sync:
maprcli volume mirror push -name mirvol1
Async:
maprcli volume mirror start -name mirvol2

4. When "srcvol" comes back, let's decide to use "mirvol3" as the source of "srcvol" firstly to sync the changes happened in "mirvol3".

maprcli volume modify -name srcvol -readonly true
maprcli volume modify -name srcvol -type mirror -source mirvol3@demo.mapr.com
The topology becomes:
mirvol3 -> srcvol   mirvol1 -> mirvol2
It still doesn't change the "mirrorDataSrcVolume" or "mirrorSrcVolume" of "mirvol1/2/3".
But changes the "mirrorSrcVolume" of "srcvol" to "mirvol3", and the types.
Note:
  • "mirrorDataSrcVolume" of "srcvol" is still itself;
  • "mirrorSrcVolume" of "mirvol3" is still showing as "srcvol", however its "volumetype"=0 which means it is now a RW volume.
maprcli volume list -filter "[volumename==*vol*]" -columns volumename,type,mirrortype,mirrorSrcVolume,mirrorDataSrcVolume,volumetype
mirrortype  mirrorDataSrcVolume  mirrorSrcVolume  volumename                   volumetype
3                                                 mapr.resourcemanager.volume  0
3           srcvol               srcvol           mirvol1                      0
2           srcvol               mirvol1          mirvol2                      1
3           srcvol               srcvol           mirvol3                      0
2           srcvol               mirvol3          srcvol                       1
Now let's sync the changes from "mirvol3" back to "srcvol":
Sync:
maprcli volume mirror push -name mirvol3
or
Async:
maprcli volume mirror start -name srcvol
After that:
# ls srcvol
mirvol3_change1

5. Fallback to "srcvol" as the source for both "mirvol1" and "mirvol3".

5.a Make "mirvol1" and "mirvol3" readonly firstly.

maprcli volume modify -name mirvol1 -readonly true
maprcli volume modify -name mirvol3 -readonly true

5.b Make last mirroring of "mirvol3" to "srcvol".

Sync:
maprcli volume mirror push -name mirvol3

5.c Make "srcvol" as the source of "mirvol1" , "mirvol3" again.

maprcli volume modify -name srcvol -type rw
maprcli volume modify -name mirvol1 -type mirror
maprcli volume modify -name mirvol3 -type mirror
The topology becomes original:
srcvol -> mirvol1 -> mirvol2
   |
   -----> mirvol3
Note: we can see "mirrorDataSrcVolume" and "mirrorSrcVolume" of "srcvol" still exist after fallback.
maprcli volume list -filter "[volumename==*vol*]" -columns volumename,type,mirrortype,mirrorSrcVolume,mirrorDataSrcVolume,volumetype
mirrortype  mirrorDataSrcVolume  mirrorSrcVolume  volumename                   volumetype
3                                                 mapr.resourcemanager.volume  0
2           srcvol               srcvol           mirvol1                      1
2           srcvol               mirvol1          mirvol2                      1
2           srcvol               srcvol           mirvol3                      1
3           srcvol               mirvol3          srcvol                       0

5.d Make changes to "srcvol":

touch /mapr/demo.mapr.com/srcvol/srcvol_newchange

5.e Start mirroring of "srcvol":

Sync:
maprcli volume mirror push -name srcvol
or
Async:
maprcli volume mirror start -name mirvol1
maprcli volume mirror start -name mirvol3
After that:
[root@maprdemo demo.mapr.com]# ls mirvol*
mirvol1:
mirvol3_change1  srcvol_newchange

mirvol2:
mirvol1_change1

mirvol3:
mirvol3_change1  srcvol_newchange

6. Thought: If management decide to use a newly created "supersrcvol" as the source of "srcvol", what should we do?

If we do like this, it will fail because they do not share the same "mirrorDataSrcVolume".
maprcli volume create  -name supersrcvol -path /supersrcvol
maprcli volume modify -name srcvol -type mirror -source supersrcvol@demo.mapr.com
ERROR (22) -  Can not update the 'source' property of Volume 'srcvol' because new source volume creator volumeuuid didn't match'
Currently:
maprcli volume list -filter "[volumename==*vol*]" -columns volumename,type,mirrortype,mirrorSrcVolume,mirrorDataSrcVolume,volumetype
mirrortype  mirrorDataSrcVolume  mirrorSrcVolume  volumename                   volumetype
3                                                 mapr.resourcemanager.volume  0
2           srcvol               srcvol           mirvol1                      1
2           srcvol               mirvol1          mirvol2                      1
2           srcvol               srcvol           mirvol3                      1
3           srcvol               mirvol3          srcvol                       0
3                                                 supersrcvol                  0
To make it work, we need to make "supersrcvol" as the mirror of "srcvol" family in the begining. And then switch the role.
maprcli volume remove -name supersrcvol
maprcli volume create -name supersrcvol -source mirvol2@demo.mapr.com  -path /supersrcvol -type mirror
maprcli volume modify -name srcvol -readonly true
maprcli volume modify -name supersrcvol -type rw
maprcli volume modify -name srcvol -type mirror -source supersrcvol@demo.mapr.com
The topology becomes:
supersrcvol -> srcvol -> mirvol1 -> mirvol2
                 |
                 -----> mirvol3
Now:
maprcli volume list -filter "[volumename==*vol*]" -columns volumename,type,mirrortype,mirrorSrcVolume,mirrorDataSrcVolume,volumetype
mirrortype  mirrorDataSrcVolume  mirrorSrcVolume  volumename                   volumetype
3                                                 mapr.resourcemanager.volume  0
2           srcvol               srcvol           mirvol1                      1
2           srcvol               mirvol1          mirvol2                      1
2           srcvol               srcvol           mirvol3                      1
2           srcvol               supersrcvol      srcvol                       1
3           srcvol               mirvol2          supersrcvol                  0



No comments:

Post a Comment

Popular Posts