#11911: NPE on schema compare of two tables in Apache Hive

anilkalshetti reported 2014-04-22T06:09:29Z · last modified 2014-09-23T05:53:14Z

NPE on schema compare of two tables in Apache Hive

Dev	tomconrad
QA	tariqrahiman

Priority	Major
Complexity	Unknown

Component	Hive
Version	15.0

Aqua Data Studio 14.0.11
Build #: 37136
Built on: 2014-Apr-15 07:02:13 AM

Aqua Data Studio 15.0.0-dev-246

Operating Environment: Ubuntu 13.04 LTS

Database: Apache Hive 0.12
Distribution: Apache

Java Version: Java 1.7.0_51

Issue: NPE on schema compare of two tables in Apache Hive

Steps to reproduce:
1. Create Database and Table using below script,

CREATE DATABASE `anil`

CREATE DATABASE `tariq`

CREATE TABLE `anil`.`order` (

o_id int,

o_name string,

o_address string,

o_price int)

ROW FORMAT DELIMITED

STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat'

CREATE TABLE `tariq`.`order` (

o_id int,

o_name string,

o_address string,

o_price int)

ROW FORMAT DELIMITED

STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat'

2. Database and tables has been created.
3. ADS menubar, go to Tools-> Compare Tools-> Schema Compare.
4. 'Schema Compare Tool' window opens, [Refer Screenshot1]
select Server 1 - ApacheHive- database- anil
select Server 2 - ApacheHive- database- tariq
5. In 'Schema Objects' block, click on Tables.
In both Objects block 'order' table has been selected.
Now click on 'Compare' button
6.'Schema Compare Results' tab opens. [Refer Screenshot2]
Click on 'order' in name column, so both table 'Create Table' script is shown.
7. Now, Click on ' Compare Data' option icon.

'Results Compare' tab opens, which shows 'null' text
and in view log it shows NPE.

java.lang.NullPointerException

at \\...\\ .\\हिñçêČάй語简�?한\\.Eᢤꃣꁔꀸ.a(Unknown Source)

at \\...\\ .\\हिñçêČάй語简�?한\\.Eᢤꃣꁔꀸ.atY(Unknown Source)

at \\...\\ .\\हिñçêČάй語简�?한\\.o⠸ꋴꌤ̈́ 9 const.start(Unknown Source)

at \\...\\ .\\हिñçêČάй語简�?한\\.Eᢤꃣꁔꀸ.E(Unknown Source)

at \\...\\ .\\हिñçêČάй語简�?한\\.Eᢤꃣꁔꀸ.l(Unknown Source)

at \\...\\ .\\हिñçêČάй語简�?한\\.Eᢤꃣꁔꀸ.<init>(Unknown Source)

at \\...\\ .\\हिñçêČάй語简�?한\\.N⡑ꂰꆰ⢹.xv(Unknown Source)

at \\...\\ .\\हिñçêČάй語简�?한\\.Fꂁꈋꆧꎠ interface instanceof.action(Unknown Source)

at \\...\\ .\\हिñçêČάй語简�?한\\.hᠼ⡢ꁂꐊ import.actionPerformed(Unknown Source)

at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)

at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)

at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)

at javax.swing.DefaultButtonModel.setPressed(Unknown Source)

at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)

at com.jidesoft.plaf.basic.BasicJideButtonListener.mouseReleased(Unknown Source)

at java.awt.AWTEventMulticaster.mouseReleased(Unknown Source)

at java.awt.Component.processMouseEvent(Unknown Source)

at javax.swing.JComponent.processMouseEvent(Unknown Source)

at java.awt.Component.processEvent(Unknown Source)

at java.awt.Container.processEvent(Unknown Source)

at java.awt.Component.dispatchEventImpl(Unknown Source)

at java.awt.Container.dispatchEventImpl(Unknown Source)

at java.awt.Component.dispatchEvent(Unknown Source)

at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)

at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)

at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)

at java.awt.Container.dispatchEventImpl(Unknown Source)

at java.awt.Window.dispatchEventImpl(Unknown Source)

at java.awt.Component.dispatchEvent(Unknown Source)

at java.awt.EventQueue.dispatchEventImpl(Unknown Source)

at java.awt.EventQueue.access$400(Unknown Source)

at java.awt.EventQueue$2.run(Unknown Source)

at java.security.AccessController.doPrivileged(Native Method)

at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source)

at java.awt.EventQueue$3.run(Unknown Source)

at java.security.AccessController.doPrivileged(Native Method)

at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source)

at java.awt.EventQueue.dispatchEvent(Unknown Source)

at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)

at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)

at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)

at java.awt.EventDispatchThread.pumpEvents(Unknown Source)

at java.awt.EventDispatchThread.run(Unknown Source)

8 attachments

Screenshot1.png

2014-04-22T06:09:29Z

226 KB
Screenshot2.png

2014-04-22T06:09:31Z

308 KB
CDH5ADS15dev290.png

2014-06-09T10:34:29Z

285 KB
CDH5ADS15dev314.png

2014-06-09T10:34:30Z

277 KB
CDH5ADS14015.png

2014-06-09T10:34:30Z

271 KB
MapR3ADS14015.png

2014-06-09T10:34:31Z

266 KB
MapR3ADS15dev314.png

2014-06-09T10:34:31Z

271 KB
CDH5Hive12ADS150311.png

2014-09-19T12:13:53Z

327 KB

All Comments (5) Change History

tomconrad 2014-04-24T17:04:43Z

Also ADS 15.0.0-dev-251

Hi Anil, Please test on Hortonworks, Mapr and Apache if you have them available. I could only test on Cloudera.

Thanks in advance,

Tom

tariqrahiman 2014-04-24T21:30:48Z

Verified in 14.0.11-3

anilkalshetti 2014-06-09T10:33:19Z

Aqua Data Studio 15.0.0-dev-314
Build #: 38339
Built on: 2014-Jun-06 05:37:08 PM

Aqua Data Studio 14.0.15

Checked this issue on all the distributions listed below.
a] [Apache Hadoop2.4, Hive 0.13, Distribution: Apache, Connection Type: Hiveserver1]
[Apache Hadoop2.4, Hive 0.12, Distribution: Apache, Connection Type: Hiveserver1]
b] Cloudera Quickstart VM CDH5 [Hive 0.12] , Distribution: Cloudera, Connection Type: Hiveserver2]
c] Hortonworks HDP2.1 Quickstart VM [Hive 0.13] , Distribution: Hortonworks, Connection Type: Hiveserver2]
Hortonworks HDP2.0 Quickstart VM [Hive 0.12] , Distribution: Hortonworks, Connection Type: Hiveserver2]
d] MapR-VM-3.0 [Hive 0.11] , Distribution: MapR, Connection Type: Hiveserver2]

Still the issue is observed for below distributions for both ADS 14.0.15 and ADS 15.0.0-dev314 builds
b] Cloudera Quickstart VM CDH5 [Hive 0.12] , Distribution: Cloudera, Connection Type: Hiveserver2]
d] MapR-VM-3.0 [Hive 0.11] , Distribution: MapR, Connection Type: Hiveserver2]

Please refer screenshots:
CDH5ADS15dev314.png
CDH5ADS15dev290.png
CDH5ADS14015.png

MapR3ADS15dev314.png
MapR3ADS14015.png

tomconrad 2014-09-17T17:38:23Z

Hi Anil, I think this is a result of uncaught database exceptions. Because of these exceptions, result sets were not being created. This caused the NPE. In my case, I was getting return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask when doing comparisons against Hive. The select was failing!!

Anyway, please give this a try. Now, you should at least get an error message and statement if there are any database exceptions. Also, would you please try using the compare tool against some other databases just to make sure I didn't break something else. I did do some testing and it seemed OK.. Thanks in advance, Tom

anilkalshetti 2014-09-19T11:50:53Z

@tomconrad

Hi Thomas, You are right, I too got the same exception as you commented, and its thrown from database.

//Please refer screenshot CDH5Hive12ADS150311.png

It's Ok then, we can Verify this issue.

As per your request, I tested compare tool for below databases, and it works fine.

List of databases:

1] Apache Hive 13

2] Hortonworks Hive 13

3] MySQL 5.7.4

4] DB2-LUW7

5] Oracle 12c

6] Teradata 14 and Teradata 15

7] MongoDB 2.6.4

Search Tips

Aqua Data Studio / nhilam

NPE on schema compare of two tables in Apache Hive

8 attachments

Issue #11911

Completion

Search Tips

Aqua Data Studio / nhilam

Title

NPE on schema compare of two tables in Apache Hive

8 attachments

Issue #11911

Completion