gotitinc / mongo-hive Goto Github PK

Having this error running (from user hive)

Closing file descriptor root
Adding fragment value root to mongodb.
Executing command: hadoop fs -rm -r -f /incomingMongoData/minibars/data_transform/output
15/11/27 22:43:32 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 360 minutes, Emptier interval = 0 minutes.
Moved: 'hdfs://sandbox.hortonworks.com:8020/incomingMongoData/minibars/data_transform/output' to trash at: hdfs://sandbox.hortonworks.com:8020/user/hive/.Trash/Current
Executing command: hadoop fs -mkdir -p /incomingMongoData/minibars/data_transform/output/root
Executing command: hadoop fs -copyFromLocal /tmp/onefold_mongo/minibars/data_transform/output/root/part-00000 /incomingMongoData/minibars/data_transform/output/root/
Traceback (most recent call last):
File "./onefold.py", line 513, in
main()
File "./onefold.py", line 509, in main
loader.run()
File "./onefold.py", line 422, in run
self.load_dw()
File "./onefold.py", line 377, in load_dw
if self.dw.table_exists(self.dw_database_name, self.dw_table_name):
File "./dw_util.py", line 289, in table_exists
r = self.execute_sql(database_name, "show tables", True)
File "./dw_util.py", line 88, in execute_sql
conn = pyhs2.connect(host=self.host, port=self.port, authMechanism="NOSASL", database='default')
File "/usr/lib/python2.6/site-packages/pyhs2/init.py", line 7, in connect
return Connection(_args, *_kwargs)
File "/usr/lib/python2.6/site-packages/pyhs2/connections.py", line 47, in init
res = self.client.OpenSession(TOpenSessionReq(username=user, password=password, configuration=configuration))
File "/usr/lib/python2.6/site-packages/pyhs2/TCLIService/TCLIService.py", line 154, in OpenSession
return self.recv_OpenSession()
File "/usr/lib/python2.6/site-packages/pyhs2/TCLIService/TCLIService.py", line 165, in recv_OpenSession
(fname, mtype, rseqid) = self._iprot.readMessageBegin()
File "/usr/lib64/python2.6/site-packages/thrift/protocol/TBinaryProtocol.py", line 140, in readMessageBegin
name = self.trans.readAll(sz)
File "/usr/lib64/python2.6/site-packages/thrift/transport/TTransport.py", line 58, in readAll
chunk = self.read(sz - have)
File "/usr/lib64/python2.6/site-packages/thrift/transport/TTransport.py", line 159, in read
self.__rbuf = StringIO(self.__trans.read(max(sz, self.__rbuf_size)))
File "/usr/lib64/python2.6/site-packages/thrift/transport/TSocket.py", line 120, in read
message='TSocket read 0 bytes')
thrift.transport.TTransport.TTransportException: TSocket read 0 bytes

Hi ,
I am using your code with Hive
'https://github.com/big-data-europe/docker-hive'

I get this error :
Traceback (most recent call last):
File "./onefold.py", line 513, in
main()
File "./onefold.py", line 509, in main
loader.run()
File "./onefold.py", line 422, in run
self.load_dw()
File "./onefold.py", line 377, in load_dw
if self.dw.table_exists(self.dw_database_name, self.dw_table_name):
File "/home/mongo-hive/dw_util.py", line 292, in table_exists
r = self.execute_sql(database_name, "show tables", True)
File "/home/mongo-hive/dw_util.py", line 91, in execute_sql
conn = pyhs2.connect(host=self.host, port=self.port, authMechanism="NOSASL", database='default',user=self.user,password=self.password)
File "/usr/local/lib/python2.7/dist-packages/pyhs2/init.py", line 7, in connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pyhs2/connections.py", line 47, in init
res = self.client.OpenSession(TOpenSessionReq(username=user, password=password, configuration=configuration))
File "/usr/local/lib/python2.7/dist-packages/pyhs2/TCLIService/TCLIService.py", line 154, in OpenSession
return self.recv_OpenSession()
File "/usr/local/lib/python2.7/dist-packages/pyhs2/TCLIService/TCLIService.py", line 165, in recv_OpenSession
(fname, mtype, rseqid) = self._iprot.readMessageBegin()
File "/usr/local/lib/python2.7/dist-packages/thrift/protocol/TBinaryProtocol.py", line 148, in readMessageBegin
name = self.trans.readAll(sz)
File "/usr/local/lib/python2.7/dist-packages/thrift/transport/TTransport.py", line 60, in readAll
chunk = self.read(sz - have)
File "/usr/local/lib/python2.7/dist-packages/thrift/transport/TTransport.py", line 161, in read
self.__rbuf = BufferIO(self.__trans.read(max(sz, self.__rbuf_size)))
File "/usr/local/lib/python2.7/dist-packages/thrift/transport/TSocket.py", line 132, in read
message='TSocket read 0 bytes')
thrift.transport.TTransport.TTransportException: TSocket read 0 bytes

This is the way I run the script :
./onefold.py --mongo mongodb://mongodb_1:27017
--source_db test
--source_collection Data
--hiveserver_host localhost
--hiveserver_port 10000

How can i fix it ?

Best Regards,