Easily find issues by searching: #<Issue ID>
Example: #1832
Easily find members by searching in: <username>, <first name> and <last name>.
Example: Search smith, will return results smith and adamsmith
Aqua Data Studio 17.0.0-rc-22
Build #: 46570
Built on: 2015-Oct-22 03:35:55 PM
Database: Hive 1.2.1
Hortonworks HDP 2.3.0 - Hive 1.2.1
Drivers Link- Hive 1.2.1
[ACID Transactions enabled]
Connection Type: HiveServer 2
Distribution: Hortonworks
Issue: "Stored As" field mapping improvement
This issue is related to Issue #13791
1] For Storage Format- TEXTFILE
a] Currently ADS uses, below mapping:
b] Mapping should be
InputFormat: org.apache.hadoop.mapred.TextInputFormat,
OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Reason: OutputFormat: " org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat " is deprecated.
This is not updated in Hive confluence Language manual .
Found correct information from Hive Java Docs [Deprecated Classes]- Ref.Link1 [Hive 0.10] Link2 [Hive 1.2.1]
Link Summarizes to below text
iv] Output of command [desc extended testseq] also matches with above Input and Output format, still Stored As field is not reflected properly.
As this Input and Output format is mapped to SEQUENCEFILE, it should be displayed in Stored As field.
Please refer screenshot2.png
|
|
145 KB
|
|
212 KB
|
|
187 KB
Hi Anil, Maybe I am misunderstanding what you are saying. Please clarify. ADS uses the following storage handlers to map table properties and alter table dialogs. I test the output format handler only to determine the short name. For example: "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"; maps to TEXTFILE.
@Tom
In order to detect, 'Stored As' field for both Create Table and Alter Table in ADS
Can we map
A] TEXTFILE to both storage handler classes -
a] "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"
b] "org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat"
and
B] SEQUENCEFILE to both storage handler classes -
a] "org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat"
b] "org.apache.hadoop.hive.ql.io.SequenceFileOutputFormat"
@Tom
In order to detect, 'Stored As' field for both Create Table and Alter Table in ADS
Can we map
A] TEXTFILE to both storage handler classes -
a] "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"
b] "org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat"
and
B] SEQUENCEFILE to both storage handler classes -
a] "org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat"
b] "org.apache.hadoop.hive.ql.io.SequenceFileOutputFormat"
Verified in ADS 17.0.0-rc-24
and ADS 18.0.0-dev-10
Please refer screenshot: Fixed13794.png
Verified in ADS 17.0.0-rc-24
and ADS 18.0.0-dev-10
Please refer screenshot: Fixed13794.png
Verified in ADS v17.0.0-rc-49
Anil will check other distributions and create a new issue if we have missed any where else.
Verified in ADS v17.0.0-rc-49
Anil will check other distributions and create a new issue if we have missed any where else.
Issue #13794 |
| Closed |
| Fixed |
| Resolved |
Completion |
| No due date |
| Fixed Build ADS 18.0.0-dev-9 & ADS 17.0.0-rc-24 |
| No time estimate |
1 issue link |
relates to #13791
Issue #13791Hive - Add correct format value to the "Stored As" field rather then using a default value of "INPUT/OUTPUT FORMAT" |
Hi Anil, Maybe I am misunderstanding what you are saying. Please clarify. ADS uses the following storage handlers to map table properties and alter table dialogs. I test the output format handler only to determine the short name. For example: "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"; maps to TEXTFILE.