Spark SQL Thrift Server
Spark SQL is a Spark module for structured data processing. Unlike the basic Spark RDD API, the interfaces provided by Spark SQL provide Spark with more information about the structure of both the data and the computation being performed. It starts a custom instance of hive-sever2 and enabled user to register spark based table in order to make the data accessible to hive clients.
Show Source Code
module.exports =
deps:
ssl: module: 'masson/core/ssl', local: true
iptables: module: 'masson/core/iptables', local: true
krb5_client: module: 'masson/core/krb5_client', local: true, required: true
java: module: 'masson/commons/java', local: true, auto: true
hadoop_core: module: 'ryba/hadoop/core', local: true
hdfs: module: 'ryba/hadoop/hdfs_client'
hdfs_nn: module: 'ryba/hadoop/hdfs_nn', required: true
hive_server2: module: 'ryba/hive/server2'
spark_client: module: 'ryba/spark/client', local: true, auto: true
spark_thrift_server: module: 'ryba/spark/thrift_server'
yarn_nm: 'ryba/hadoop/yarn_nm'
tez: module: 'ryba/tez', local: true
configure :
'ryba/spark/thrift_server/configure'
commands:
'install': [
'ryba/spark/thrift_server/install'
'ryba/spark/thrift_server/start'
'ryba/spark/thrift_server/check'
]
'check':
'ryba/spark/thrift_server/check'
'stop':
'ryba/spark/thrift_server/stop'
'start':
'ryba/spark/thrift_server/start'