CODEWITHSUNDEEP
javahadoophivebigdatathrift
tooptoop4
tooptoop4

Reputation: 330

hiveserver2 org.apache.thrift.transport.TTransportException error when running 2nd query after minute of inactivity

I make a JDBC connection from my SQL tool (ie Squirrel SQL, Oracle SQL Developer) to HiveServer2 (running on remote server) with port 10000.

I am able to run some queries successfully. I then do something else (not in the SQL tool) for 1-2minutes and then return to my SQL tool and attempt to run a query but I get this error: org.apache.thrift.transport.TTransportException: java.net.SocketException: Software caused connection abort: socket write error

If I now disconnect and reconnect in my SQL tool I can run queries again. But does anyone know what HiveServer2 settings I should change to prevent the error? I assume something in hive-site.xml

From the hiveserver2 logs below, can see an exact 1 minute gap from 30th min to 31stmin where the disconnect happens.

2018-04-05T03:30:41,706  INFO [HiveServer2-Handler-Pool: Thread-36] session.SessionState: Resetting thread name to  HiveServer2-Handler-Pool: Thread-36
2018-04-05T03:30:41,712  INFO [HiveServer2-Handler-Pool: Thread-36] session.SessionState: Updating thread name to c81ec0f9-7a9d-46b6-9708-e7d78520a48a HiveServer2-Handler-Pool: Thread-36
2018-04-05T03:30:41,712  INFO [HiveServer2-Handler-Pool: Thread-36] session.SessionState: Resetting thread name to  HiveServer2-Handler-Pool: Thread-36
2018-04-05T03:30:41,718  INFO [HiveServer2-Handler-Pool: Thread-36] session.SessionState: Updating thread name to c81ec0f9-7a9d-46b6-9708-e7d78520a48a HiveServer2-Handler-Pool: Thread-36
2018-04-05T03:30:41,719  INFO [HiveServer2-Handler-Pool: Thread-36] session.SessionState: Resetting thread name to  HiveServer2-Handler-Pool: Thread-36
2018-04-05T03:31:41,232  INFO [HiveServer2-Handler-Pool: Thread-36] thrift.ThriftCLIService: Session disconnected without closing properly.
2018-04-05T03:31:41,233  INFO [HiveServer2-Handler-Pool: Thread-36] thrift.ThriftCLIService: Closing the session: SessionHandle [c81ec0f9-7a9d-46b6-9708-e7d78520a48a]
2018-04-05T03:31:41,233  INFO [HiveServer2-Handler-Pool: Thread-36] service.CompositeService: Session closed, SessionHandle [c81ec0f9-7a9d-46b6-9708-e7d78520a48a], current sessions:0
2018-04-05T03:31:41,233  INFO [HiveServer2-Handler-Pool: Thread-36] session.SessionState: Updating thread name to c81ec0f9-7a9d-46b6-9708-e7d78520a48a HiveServer2-Handler-Pool: Thread-36
2018-04-05T03:31:41,233  INFO [HiveServer2-Handler-Pool: Thread-36] session.SessionState: Resetting thread name to  HiveServer2-Handler-Pool: Thread-36
2018-04-05T03:31:41,233  INFO [HiveServer2-Handler-Pool: Thread-36] session.SessionState: Updating thread name to c81ec0f9-7a9d-46b6-9708-e7d78520a48a HiveServer2-Handler-Pool: Thread-36
2018-04-05T03:31:41,233  INFO [HiveServer2-Handler-Pool: Thread-36] session.HiveSessionImpl: Operation log session directory is deleted: /var/hive/hs2log/tmp/c81ec0f9-7a9d-46b6-9708-e7d78520a48a
2018-04-05T03:31:41,233  INFO [HiveServer2-Handler-Pool: Thread-36] session.SessionState: Resetting thread name to  HiveServer2-Handler-Pool: Thread-36
2018-04-05T03:31:41,236  INFO [HiveServer2-Handler-Pool: Thread-36] session.SessionState: Deleted directory: /var/hive/scratch/tmp/anonymous/c81ec0f9-7a9d-46b6-9708-e7d78520a48a on fs with scheme file
2018-04-05T03:31:41,236  INFO [HiveServer2-Handler-Pool: Thread-36] session.SessionState: Deleted directory: /var/hive/ec2-user/c81ec0f9-7a9d-46b6-9708-e7d78520a48a on fs with scheme file
2018-04-05T03:31:41,236  INFO [HiveServer2-Handler-Pool: Thread-36] hive.metastore: Closed a connection to metastore, current connections: 1

Environment:

Upvotes: 1

Views: 2999

Answers (1)

tooptoop4
tooptoop4

Reputation: 330

  1. change default timeout of AWS ELB from 1min to 20min
  2. Use SimbaHive driver with Squirrel (no timeout) or use standard apache hive2 driver with IBM data studio (no timeout)

Upvotes: 0

Related Questions