#14828: Test Hortonworks with the new simba type drivers

tomconrad reported 2016-11-16T23:02:57Z · last modified 2017-01-29T19:09:26Z

Test Hortonworks with the new simba type drivers

doc

Dev	tomconrad
QA	tariqrahiman

Priority	Low
Complexity	Unknown

Component	Hive
Version	18.0

Hi Tariq,

Would you or one of the team do some research and testing with the new Hortonworks simba type drivers? We should update our documentation to reflect the new drivers. Does Horton support both simba and apache drivers now?

Thanks, Tom

This is from a customer support email... Since our docs didn't include these drivers, he didn't know which to install.

Hi Niels,

For the new Hortonworks JDBC drivers , hive-exec is not included in the package.. See attached

12 attachments

customer.png

2016-11-16T23:02:57Z

6 KB
Apachehive21.png

2017-01-07T14:00:39Z

223 KB
Apachehive20.png

2017-01-07T14:00:39Z

205 KB
Apachehive12.png

2017-01-07T14:00:39Z

199 KB
simbaDriver1.png

2017-01-07T14:11:01Z

99 KB
simbaDriver.png

2017-01-07T14:12:28Z

231 KB
cloudera_hive_simba.png

2017-01-07T14:17:44Z

213 KB
datastudio.log

2017-01-09T05:35:03Z

6 KB
hive21DriversBundledWithHDP25.png

2017-01-09T05:35:03Z

418 KB
hivearchive.png

2017-01-09T05:36:58Z

43 KB
hortonworksdriversfromSite.png

2017-01-09T12:52:28Z

317 KB
HortonworksDriver.log

2017-01-09T12:52:28Z

3 KB

All Comments (12) Change History

dutondeumesh 2017-01-09T11:42:12Z · (edited)

Does Horton support both simba and apache drivers now?

Apache Drivers :

Connection Test	ADS	Result	screenshot
HDP 2.5 - apache-hive-1.2.0	18.0.0-preview-1	Pass	hive1.2
HDP 2.5 - apache-hive-2.0.1	18.0.0-preview-1	Pass	hive2.0
HDP 2.5 - apache-hive-2.1.1[bundled with Hortonworks]	18.0.0-preview-1	Pass	hive2.1

dutondeumesh 2017-01-07T14:23:24Z · (edited)

Simba Drivers 1.0 - HDP 2.5 :

I have downloaded Simba hive drivers from here . please refer

Then copied below drivers to ADS -

hive_metastore

hive_service

HiveJDBC4

TCLIServiceClient

While connecting to ADS, it throws below error. screenshot

java.lang.ClassNotFoundException: org.apache.hive.jdbc.HiveDriver

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:264)

at com.aquafold.aquacore.open.rdbms.drivers.hive.uꍹꁇꉅꍶif10.n(Unknown Source)

at com.aquafold.aquacore.open.rdbms.drivers.hive.uꍹꁇꉅꍶif10.b(Unknown Source)

at \\.\\.\\हिñçêČάй語简�?한\\.I⢙⢭ꑭꇇ2.hY(Unknown Source)

at \\.\\.\\हिñçêČάй語简�?한\\.I⢙⢭ꑭꇇ2.s(Unknown Source)

at \\.\\.\\हिñçêČάй語简�?한\\.I⢙⢭ꑭꇇ2.getConnection(Unknown Source)

at \\.\\.\\हिñçêČάй語简�?한\\.L⣴ͪꃌꈌthrows.dO(Unknown Source)

at \\.\\.\\हिñçêČάй語简�?한\\.bꏟ⡞ꇕꈷ$4.bHs(Unknown Source)

at \\.\\.\\हिñçêČάй語简�?한\\.bꏟ⡞ꇕꈷ$4.asS(Unknown Source)

at \\.\\.\\हिñçêČάй語简�?한\\.jꏮᡍꈡꀁ.doInBackground(Unknown Source)

at javax.swing.SwingWorker$1.call(SwingWorker.java:295)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at javax.swing.SwingWorker.run(SwingWorker.java:334)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

I have also tried above scenario with previous ADS versions(15,16,17) and HDP(2.3,2.4) versions -> above mention error is observed.

Simba Drivers 1.0 - HDP 2.5 :

I have downloaded Simba hive drivers from here . please refer

Then copied below drivers to ADS -

hive_metastore

hive_service

HiveJDBC4

TCLIServiceClient

While connecting to ADS, it throws below error. screenshot

java.lang.ClassNotFoundException: org.apache.hive.jdbc.HiveDriver

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:264)

at com.aquafold.aquacore.open.rdbms.drivers.hive.uꍹꁇꉅꍶif10.n(Unknown Source)

at com.aquafold.aquacore.open.rdbms.drivers.hive.uꍹꁇꉅꍶif10.b(Unknown Source)

at \\.\\.\\हिñçêČάй語简�?한\\.I⢙⢭ꑭꇇ2.hY(Unknown Source)

at \\.\\.\\हिñçêČάй語简�?한\\.I⢙⢭ꑭꇇ2.s(Unknown Source)

at \\.\\.\\हिñçêČάй語简�?한\\.I⢙⢭ꑭꇇ2.getConnection(Unknown Source)

at \\.\\.\\हिñçêČάй語简�?한\\.L⣴ͪꃌꈌthrows.dO(Unknown Source)

at \\.\\.\\हिñçêČάй語简�?한\\.bꏟ⡞ꇕꈷ$4.bHs(Unknown Source)

at \\.\\.\\हिñçêČάй語简�?한\\.bꏟ⡞ꇕꈷ$4.asS(Unknown Source)

at \\.\\.\\हिñçêČάй語简�?한\\.jꏮᡍꈡꀁ.doInBackground(Unknown Source)

at javax.swing.SwingWorker$1.call(SwingWorker.java:295)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at javax.swing.SwingWorker.run(SwingWorker.java:334)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

I have also tried above scenario with previous ADS versions(15,16,17) and HDP(2.3,2.4) versions -> above mention error is observed.

dutondeumesh 2017-01-07T14:18:04Z

I have also performed test on cloudera 5.5 with simba drivers which are here - I am able to connect to ADS
Please refer this screenshot

dutondeumesh 2017-01-09T05:44:13Z · Edited by

SachinPrakash

Clarification regarding Fail scenario i.e HDP 2.5 - apache-hive-2.1.118.0.0-preview-1Failhive2.1

The drivers which I have used are copied from here . Please refer.

I have attached log file for fail scenario. Please refer attached datastudio.log

But if I copy Apcahe hive drivers[Apache hive - 2.1.0.2.5.0.0-1245] which are bundled with Hortonworks -> I am able to connect to ADS successfully. screenshot

[SP] Since we only support the drivers that are bundled w/ the distribution that the user is using, the fail scenario is invalid. Update the failed scenario to pass

dutondeumesh 2017-01-09T12:56:25Z

Updated result for Apache hive drivers 2.1 here

I have downloaded Hortonworks drivers from hortonworks site. The downloaded file name appears as Simba_HiveJDBC41_1.0.36.1046.zip. Then I removed default hive drivers from ads -> lib -> drivers and added below new downloaded drives.

TCLIServiceClient.jar
HiveJDBC41.jar
hive_service.jar
hive_metastore.jar

Still I am not able to connect to ADS - Please refer screenshot and attached HortonworksDriver.log

SachinPrakash 2017-01-09T15:42:08Z

@ajinkya: for this comment, can you work w/ Mujahid & see if you can determine what the issue is.

ajinkya(*) 2017-01-10T14:19:25Z · (edited)

@sachin sir,

I've gone through codes and docs with the help of @mujahid, below are our findings:

--- The driver path required for connection for hive2 using Simba drivers is: com.simba.hive.jdbc41.HS2Driver

It is observed that the path is not used anywhere in the whole project.

Probably we will need to enhance CoreHiveDriver to point at the new driver path.

-- We tried the same in a small connection program using the above path with given jars and now it was not showing the ClassNotFoundException but after this we have observed one more issue here:

-- In order to establish the connection, having the drivers only is not sufficient but also need a license file as SimbaApacheHiveJDBCDriver.lic. This file needs to be placed in the same folder where the drivers are kept. Please go through the reference_1(1st para on Page 9) & reference_2 which mention these details about license file. Please refer the exception screenshot for details.

Please let us know if our understandings are correct or not.

tomconrad 2017-01-16T16:03:33Z · (edited)

Hi Sachin,

I downloaded the Simba v1.0.36 drivers from the Hortonworks site https://hortonworks.com/downloads/ and added them to \General\Tom\Hortonworks Connectors\Simba1.0.36.1046_Hive. The latest Simba drivers from the Simba site are 1.0.39 but we should go with what Hortonworks provides.

Thanks, Tom

tomconrad 2017-01-16T16:17:51Z

svn #52702

Hi QA Team,

The Simba drivers should now work in ADS for a Hortonworks connection. Please test with the drivers provided by the Hortonworks site https://hortonworks.com/downloads/. Although the Simba drivers are generalized drivers and probably will work with other distributions, we currently will only support them for Hortonworks.

Thanks in advance, Tom

tomconrad 2017-01-16T23:09:20Z · (edited)

V18 documentation is updated.

https://www.aquaclusters.com/app/home/project/public/aquadatastudio/wikibook/Documentation18/page/Apache-Hive-JDBC-Drivers/Apache-Hive-JDBC-Drivers

[SP] Pls specify links to updated docs

tariqrahiman 2017-01-16T23:36:14Z

Verified in ADS v18.0.0-preview-12

shilpasingh(*) 2017-01-29T14:39:54Z

@Tariq - can you close this issue , if no further actions are needed?

Search Tips

Aqua Data Studio / nhilam

Test Hortonworks with the new simba type drivers

12 attachments

Issue #14828

Completion

Search Tips

Aqua Data Studio / nhilam

Title

Test Hortonworks with the new simba type drivers

12 attachments

Issue #14828

Completion