tag:blogger.com,1999:blog-929270410515568702.post4926511653371293140..comments2024-03-09T22:16:11.028-08:00Comments on Open Knowledge Base: How to use Drill to parse ResourceManager Rest API resultsOpenKBhttp://www.blogger.com/profile/02892129494774761942noreply@blogger.comBlogger1125tag:blogger.com,1999:blog-929270410515568702.post-47486027731382766672017-03-02T13:42:18.326-08:002017-03-02T13:42:18.326-08:00Some reader is asking how to parse the job configu...Some reader is asking how to parse the job configuration. Actually that is related to Job History Server Rest API instead of RM rest API. Please refer to below Doc:<br />https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/HistoryServerRest.html#Job_Conf_API<br /><br />One example is:<br />1. Download the json file which contains the configuration for specific job ID:<br />curl -v -X GET -H "Content-Type: application/json" http://v7.poc.com:19888/ws/v1/history/mapreduce/jobs/job_1473379941569_0096/conf > /mapr/my2.cluster.com/tmp/restapi/hs_data.json<br /><br /><br />2.Using Drill to query it to get what is the value for parameter "hive.query.string":<br />with tmp as<br />(<br />select flatten(t.conf.property) as col from dfs.tmp.`restapi/hs_data.json` t<br />)<br />select tmp.col.`value` from tmp where tmp.col.`name`='hive.query.string';<br /><br />+---------------------------------+<br />| EXPR$0 |<br />+---------------------------------+<br />| select count(*) from passwords |<br />+---------------------------------+<br />1 row selected (0.232 seconds)<br /><br />OpenKBhttps://www.blogger.com/profile/02892129494774761942noreply@blogger.com