Tuesday, May 19, 2015

Drill Workshop -- Impersonation

Env:

Drill 1.0

Theory: 

When you enable impersonation, Drill executes client requests as the user logged in to the client.
Drill impersonation only supports File System storage plugin as of version 1.0.

Goal:

Know how to enable impersonation.
Know the behaviors of impersonation.

Workshop:

1. How to enable impersonation.

Edit drill-override.conf on all Drillbits nodes to set drill.exec.impersonation.enabled to true, and restart all drillbits.
For example:
$ cat drill-override.conf
drill.exec: {
  cluster-id: "MyCluster-drillbits",
  zk.connect: "h2.poc.com:5181,h3.poc.com:5181,h4.poc.com:5181",
  sys.store.provider.zk.blobroot: "maprfs:///mydrill/",
  impersonation.enabled: true
}
If it is a MapR Cluster, make sure below settings exist.(By default they exist out of box)
$ cat drill-env.sh |grep MAPR
export MAPR_IMPERSONATION_ENABLED=true
export MAPR_TICKETFILE_LOCATION=/opt/mapr/conf/mapruserticket
After restarting all drillbits, run below query to verify the settings for impersonation:
>  select * from sys.boot where name like '%impersonation%';
+-------------------------------------------------+----------+-------+---------+----------+-------------+-----------+------------+
|                      name                       |   kind   | type  | status  | num_val  | string_val  | bool_val  | float_val  |
+-------------------------------------------------+----------+-------+---------+----------+-------------+-----------+------------+
| drill.exec.impersonation.enabled                | BOOLEAN  | BOOT  | BOOT    | null     | null        | true      | null       |
| drill.exec.impersonation.max_chained_user_hops  | LONG     | BOOT  | BOOT    | 3        | null        | null      | null       |
+-------------------------------------------------+----------+-------+---------+----------+-------------+-----------+------------+
2 rows selected (2.665 seconds)

2. Behavior when using "admin" user.

In the past, "admin" user is normally used to logon sqlline, and it is the default user if you ignore "-n" option of sqlline.
After impersonation is enabled, if "admin" user does not exist, the query may fail.
Starting from 1.0 version, function "user" and "current_schema" are added to help check current user information.
[testuser@h1 ~]$ alias sqlline
alias sqlline='/opt/mapr/drill/drill-1.0.0/bin/sqlline -u jdbc:drill:zk=h2.poc.com:5181,h3.poc.com:5181,h4.poc.com:5181 -n admin -p admin'
[testuser@h1 ~]$ sqlline
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
apache drill 1.0.0
"the only truly happy people are children, the creative minority and drill users"
0: jdbc:drill:zk=h2.poc.com:5181,h3.poc.com:5> select user,current_schema from sys.version;
+--------+-----------------+
|  user  | current_schema  |
+--------+-----------------+
| admin  |                 |
+--------+-----------------+
1 row selected (2.328 seconds)
0: jdbc:drill:zk=h2.poc.com:5181,h3.poc.com:5> use dfs.tmp;
+-------+--------------------------------------+
|  ok   |               summary                |
+-------+--------------------------------------+
| true  | Default schema changed to [dfs.tmp]  |
+-------+--------------------------------------+
1 row selected (0.099 seconds)
0: jdbc:drill:zk=h2.poc.com:5181,h3.poc.com:5> show files;
Error: SYSTEM ERROR: java.io.IOException: Error getting user info for current user, admin


[Error Id: ef420f05-8884-4aa5-bbad-e98aa7732959 on h2.poc.com:31010]

  (org.apache.drill.exec.planner.sql.QueryInputException) Failure handling SQL.
    org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():187
    org.apache.drill.exec.work.foreman.Foreman.runSQL():902
    org.apache.drill.exec.work.foreman.Foreman.run():240
    java.util.concurrent.ThreadPoolExecutor.runWorker():1142
    java.util.concurrent.ThreadPoolExecutor$Worker.run():617
    java.lang.Thread.run():745
  Caused By (java.io.IOException) Error getting user info for current user, admin
    com.mapr.fs.MapRFileSystem.lookupClient():603
    com.mapr.fs.MapRFileSystem.lookupClient():640
    com.mapr.fs.MapRFileSystem.listMapRStatus():1367
    com.mapr.fs.MapRFileSystem.listStatus():1427
    com.mapr.fs.MapRFileSystem.listStatus():78
    org.apache.hadoop.fs.FileSystem.listStatus():1519
    org.apache.hadoop.fs.FileSystem.listStatus():1596
    org.apache.hadoop.fs.FileSystem.listStatus():1576
    org.apache.drill.exec.store.dfs.DrillFileSystem.list():697
    org.apache.drill.exec.planner.sql.handlers.ShowFileHandler.getPlan():97
    org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():177
    org.apache.drill.exec.work.foreman.Foreman.runSQL():902
    org.apache.drill.exec.work.foreman.Foreman.run():240
    java.util.concurrent.ThreadPoolExecutor.runWorker():1142
    java.util.concurrent.ThreadPoolExecutor$Worker.run():617
    java.lang.Thread.run():745 (state=,code=0)

[All file/view permissions are based on file system permission completely.]

3. Querying a file requires read permission of the file and execute permission of the directory.

a. No "read" permission of the file.

The file is owned by root with permission "640", and "testuser" does not belong to root group.
The user fails to query the file.
$ sqlline -u jdbc:drill:zk=h2.poc.com:5181,h3.poc.com:5181,h4.poc.com:5181 -n testuser

> select user,current_schema from sys.version;
+-----------+-----------------+
|   user    | current_schema  |
+-----------+-----------------+
| testuser  | dfs.drill       |
+-----------+-----------------+
1 row selected (0.099 seconds)
> show files;
+-----------------+--------------+---------+---------+-----------+-----------+--------------+------------------------+--------------------------+
|      name       | isDirectory  | isFile  | length  |   owner   |   group   | permissions  |       accessTime       |     modificationTime     |
+-----------------+--------------+---------+---------+-----------+-----------+--------------+------------------------+--------------------------+
| rootonly.csv    | false        | true    | 6       | root      | root      | rw-r-----    | 2015-05-19 20:40:25.0  | 2015-05-19 20:40:25.022  |
...
+-----------------+--------------+---------+---------+-----------+-----------+--------------+------------------------+--------------------------+
11 rows selected (0.084 seconds)

> select * from `rootonly.csv`;
Error: SYSTEM ERROR: java.io.IOException: 2049.24401.580276 /drill/rootonly.csv (Input/output error)

b. No "execute" permission of the directory.

The directory is owned by root with permission "754", and "testuser" does not belong to root group.
The user can show files in that directory, but can not select the files inside.
> show files;
+-----------------+--------------+---------+---------+-----------+-----------+--------------+------------------------+--------------------------+
|      name       | isDirectory  | isFile  | length  |   owner   |   group   | permissions  |       accessTime       |     modificationTime     |
+-----------------+--------------+---------+---------+-----------+-----------+--------------+------------------------+--------------------------+
| rootdir         | true         | false   | 1       | root      | root      | rwxr-xr--    | 2015-05-19 20:50:53.0  | 2015-05-19 20:51:17.258  |
12 rows selected (0.12 seconds)

> show files in rootdir;
+-----------+--------------+---------+---------+--------+--------+--------------+------------------------+--------------------------+
|   name    | isDirectory  | isFile  | length  | owner  | group  | permissions  |       accessTime       |     modificationTime     |
+-----------+--------------+---------+---------+--------+--------+--------------+------------------------+--------------------------+
| root.csv  | false        | true    | 6       | root   | root   | rwxrwxrwx    | 2015-05-19 20:51:17.0  | 2015-05-19 20:51:17.255  |
+-----------+--------------+---------+---------+--------+--------+--------------+------------------------+--------------------------+
1 row selected (0.09 seconds)


> select * from `rootdir/root.csv`;
Error: PERMISSION ERROR: Not authorized to read table [rootdir/root.csv] in schema [dfs.drill]


[Error Id: 5ee1c67b-7eeb-49b2-954f-58b19e6fa0bd on h3.poc.com:31010]

  (org.apache.hadoop.security.AccessControlException) User testuser(user id 6001)  does not have access to maprfs:/drill/rootdir/root.csv 

4. "Show files" requires read permission of the directory.

a. No "read" permission of the directory

# hadoop fs -chmod 750 /drill
# hadoop fs -ls -d /drill
drwxr-x---   - root root         24 2015-05-21 03:50 /drill

> show files;
Error: PERMISSION ERROR: User testuser(user id 6001)  does not have access to /drill
After adding "read" permission of the directory:
# hadoop fs -chmod 754 /drill
# hadoop fs -ls -d /drill
drwxr-xr--   - root root         24 2015-05-21 03:50 /drill

> show files;
+--------------------------+--------------+---------+---------+-----------+-----------+--------------+------------------------+--------------------------+
|           name           | isDirectory  | isFile  | length  |   owner   |   group   | permissions  |       accessTime       |     modificationTime     |
+--------------------------+--------------+---------+---------+-----------+-----------+--------------+------------------------+--------------------------+
| view_b.view.drill        | false        | true    | 198     | userb     | userb     | rwxr--r--    | 2015-05-19 23:47:04.0  | 2015-05-19 23:47:04.161  |
...
24 rows selected (0.152 seconds)

5. "Show tables" to list the views requires read permission of the view file and read+execute permission of the directory.

a. No "read" permission of the view file.

# hadoop fs -chmod 777 /drill
# hadoop fs -chmod 770 /drill/testuserview.view.drill
# hadoop fs -ls /drill/testuserview.view.drill
-rwxrwx---   3 root root        208 2015-05-19 21:56 /drill/testuserview.view.drill

> show tables like '%testuserview%';
+--+
|  |
+--+
+--+
No rows selected (0.122 seconds)
After adding "read" permission of the view file:
# hadoop fs -chmod 774 /drill/testuserview.view.drill
# hadoop fs -ls /drill/testuserview.view.drill
-rwxrwxr--   3 root root        208 2015-05-19 21:56 /drill/testuserview.view.drill

> show tables like '%testuserview%';
+---------------+---------------+
| TABLE_SCHEMA  |  TABLE_NAME   |
+---------------+---------------+
| dfs.drill     | testuserview  |
+---------------+---------------+
1 row selected (0.167 seconds)

b. No "read" permission of the directory.

# hadoop fs -chmod 774 /drill/testuserview.view.drill
# hadoop fs -chmod 773 /drill
# hadoop fs -ls /drill/testuserview.view.drill
-rwxrwxr--   3 root root        208 2015-05-19 21:56 /drill/testuserview.view.drill
# hadoop fs -ls -d /drill
drwxrwx-wx   - root root         24 2015-05-21 03:50 /drill

> show tables like '%testuserview%';
+--+
|  |
+--+
+--+
No rows selected (0.117 seconds)

c. No "execute" permission of the directory.

# hadoop fs -chmod 776 /drill
# hadoop fs -ls -d /drill
drwxrwxrw-   - root root         24 2015-05-21 03:50 /drill

> show tables like '%testuserview%';
+--+
|  |
+--+
+--+
No rows selected (0.114 seconds)
Minimum requirement is read permission of the view file and read+execute permission of the directory:
# hadoop fs -chmod 775 /drill
# hadoop fs -ls -d /drill
drwxrwxr-x   - root root         24 2015-05-21 03:50 /drill
# hadoop fs -ls /drill/testuserview.view.drill
-rwxrwxr--   3 root root        208 2015-05-19 21:56 /drill/testuserview.view.drill

> show tables like '%testuserview%';
+---------------+---------------+
| TABLE_SCHEMA  |  TABLE_NAME   |
+---------------+---------------+
| dfs.drill     | testuserview  |
+---------------+---------------+
1 row selected (0.148 seconds)

6. "Create or Drop view/table" requires read+write+execute permission of the directory.

a. No "execute" permission of directory

# hadoop fs -chmod 756 /drill
# hadoop fs -ls -d /drill
drwxr-xrw-   - root root         23 2015-05-21 03:44 /drill

> drop view view_user_a;
Error: PERMISSION ERROR: Not authorized to read view [view_user_a] in schema [dfs.drill]

b. No "read" permission of directory

# hadoop fs -chmod 753 /drill
# hadoop fs -ls -d /drill
drwxr-x-wx   - root root         24 2015-05-21 03:50 /drill

> drop view view_user_a;
Error: PERMISSION ERROR: Not authorized to list or query tables in schema [dfs.drill]

c. No "write" permission of directory

# hadoop fs -chmod 755 /drill
# hadoop fs -ls -d /drill
drwxr-xr-x   - root root         24 2015-05-21 03:50 /drill

> drop view view_user_a;
Error: PERMISSION ERROR: User testuser(user id 6001)  does not have access to /drill/view_user_a.view.drill

Reference:

Configuring User Impersonation

No comments:

Post a Comment

Popular Posts