#15025: Hive - Nested Struct's are not displaying correctly in the schema tree.

tomconrad reported 2017-02-15T23:37:44Z · last modified 2017-02-21T05:01:34Z

Hive - Nested Struct's are not displaying correctly in the schema tree.

Dev	tomconrad
QA	tariqrahiman

Priority	Minor
Complexity	Unknown

Component	Hive
Version	17.0

The following create table example does not display correctly in the schema tree and in auto-completion:

CREATE TABLE `tom1`.`test_struct_layout0` (

`outer_struct` struct<outer_struct_field1:struct<inner_struct1_field1:string,inner_struct1_field2:string>,outer_struct_field2:struct<inner_struct2_field1:string,inner_struct2_field2:string>,outer_struct_field3:struct<inner_struct3_field1:string,inner_struct3_field2:string>>)

2 attachments

Querybuilder_Tooltip.png

2017-02-16T13:57:39Z

121 KB
Issue_15025_verified.png

2017-02-16T13:57:40Z

401 KB

All Comments (8) Change History

tomconrad 2017-02-15T23:56:08Z · (edited)

v17 svn #53196

v18 svn #53198

SachinPrakash 2017-02-15T23:58:35Z

Impacted areas to test include: auto-completion, schema tree & query builder.

tomconrad 2017-02-16T01:43:12Z · (edited)

Hi QA team,

Please test structs and nested struct table columns and confirm that they are displaying correctly in the schema tree as well as in auto-completion. See example above. Also please test Query Builder. Query Builder will display the tree at it's highest level. In the case of the above example - `outer_struct`. I was only able to test using Cloudera Hive. Please test across other hive distributions.

Thanks in advance, Tom

shilpasingh(*) 2017-02-16T02:43:32Z

@Umesh - Verify this throughly at all areas for v17 and update the issue. Take this as your first task of the day.

dutondeumesh 2017-02-16T13:53:42Z

I have tested issue on all distributions with different drivers versions. Please go through the below table.

Distribution	Details	ConnectionType	Driver version	ADS 17.0.11-12	ADS 18.0.0-preview-73
Apache	Apache Hadoop2.7	HiveServer2	Default Hive drivers w/ ADS, 1.0 for v17 and 1.1 for v18	Works fine	Works fine
Apache	Apache Hadoop2.7	HiveServer2	2.1	Works fine	Works fine
Cloudera	CDH 5.8	HiveServer2	Default Hive drivers w/ ADS, 1.0 for v17 and 1.1 for v18	Works fine	Works fine
Cloudera	CDH 5.8	HiveServer2	Drivers bundled w/ the distribution : 1.2	Works fine	Works fine
Cloudera	CDH 5.8	Impala	Impala 2.5	Works fine	Works fine
Hortonworks	HDP2.5	HiveServer2	Default Hive drivers w/ ADS, 1.0 for v17 and 1.1 for v18	Works fine	Works fine
Hortonworks	HDP2.5	HiveServer2	Drivers bundled w/ the distribution :1.2	Works fine	Works fine
Hortonworks	HDP2.5	SparkSQL1.6	Drivers bundled w/ the distribution :1.2	Works fine	Works fine
MapR	MapR5.2	HiveServer2	Default Hive drivers w/ ADS, 1.0 for v17 and 1.1 for v18	Works fine	Works fine
MapR	MapR5.2	HiveServer2	Drivers bundled w/ the distribution :1.2	Works fine	Works fine
Amazon EMR	EMR5.2	HiveServer2	Default Hive drivers w/ ADS, 1.0 for v17 and 1.1 for v18	Works fine	Works fine
Amazon EMR	EMR5.2	HiveServer2	Amazon Drivers : 1.0	Works fine	Works fine

I have tested issue on all distributions with different drivers versions. Please go through the below table.

Distribution	Details	ConnectionType	Driver version	ADS 17.0.11-12	ADS 18.0.0-preview-73
Apache	Apache Hadoop2.7	HiveServer2	Default Hive drivers w/ ADS, 1.0 for v17 and 1.1 for v18	Works fine	Works fine
Apache	Apache Hadoop2.7	HiveServer2	2.1	Works fine	Works fine
Cloudera	CDH 5.8	HiveServer2	Default Hive drivers w/ ADS, 1.0 for v17 and 1.1 for v18	Works fine	Works fine
Cloudera	CDH 5.8	HiveServer2	Drivers bundled w/ the distribution : 1.2	Works fine	Works fine
Cloudera	CDH 5.8	Impala	Impala 2.5	Works fine	Works fine
Hortonworks	HDP2.5	HiveServer2	Default Hive drivers w/ ADS, 1.0 for v17 and 1.1 for v18	Works fine	Works fine
Hortonworks	HDP2.5	HiveServer2	Drivers bundled w/ the distribution :1.2	Works fine	Works fine
Hortonworks	HDP2.5	SparkSQL1.6	Drivers bundled w/ the distribution :1.2	Works fine	Works fine
MapR	MapR5.2	HiveServer2	Default Hive drivers w/ ADS, 1.0 for v17 and 1.1 for v18	Works fine	Works fine
MapR	MapR5.2	HiveServer2	Drivers bundled w/ the distribution :1.2	Works fine	Works fine
Amazon EMR	EMR5.2	HiveServer2	Default Hive drivers w/ ADS, 1.0 for v17 and 1.1 for v18	Works fine	Works fine
Amazon EMR	EMR5.2	HiveServer2	Amazon Drivers : 1.0	Works fine	Works fine

dutondeumesh 2017-02-16T14:05:57Z · (edited)

While testing, I came across tool-tip issue in Query Builder. Tool-tip not displayed properly after hover a mouse on columns in the query builder.

Please refer attached - Querybuilder_Tooltip.png

Observations

1 - Works fine in ADS dev-73

2 - Tested in dev-80 - Issue observed.

3 - No build is available from dev-74 to dev-79.

4- Preview-73 - Issue Observed

5 - Issue is observed for complex datatypes only.

Please refer below sql

CREATE TABLE `table4` (

`full_name` array<string>,

`deductions` map<string, bigint>,

`address` struct<flat_no:int,landmark:array<string>>)

COMMENT 'table contains complex data type'

ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' COLLECTION ITEMS TERMINATED BY '#' MAP KEYS TERMINATED BY '$'

STORED AS TEXTFILE

TBLPROPERTIES ('COLUMN_STATS_ACCURATE'='{\"BASIC_STATS\":\"true\"}', 'numFiles'='0', 'numRows'='0', 'rawDataSize'='0', 'totalSize'='0')

tomconrad 2017-02-16T16:20:17Z

Hi Umesh,

I opened another issue for the tooltip problem since it is related to QB and linked it to this issue.

Thanks, Tom

tariqrahiman 2017-02-16T18:58:39Z

Verified in ADS 17.0.11-12 - Schema Tree node for Tables, Indexes, Query Builder, ER Modeler and Autocompletion.

Search Tips

Aqua Data Studio / nhilam

Hive - Nested Struct's are not displaying correctly in the schema tree.

2 attachments

Issue #15025

Completion

2 issue links

Issue #15028

Issue #10816

Search Tips

Aqua Data Studio / nhilam

Title

Hive - Nested Struct's are not displaying correctly in the schema tree.

2 attachments

Issue #15025

Completion

2 issue links

Issue #15028

Issue #10816