#15387: Cassandra - JDBC should be enhanced to handle the new types

tomconrad reported 2017-07-18T15:01:08Z · last modified 2018-02-08T16:10:05Z

Cassandra - JDBC should be enhanced to handle the new types

Dev	kunalkalbande
QA	tomconrad

Priority	Low
Complexity	Unknown

Component	DB - Cassandra
Version	18.5

Looks like we need to enhance the JDBC to handle new types for V3.0

1. Here is the ddl to create the table. Open a Cassandra QA window and execute the following:

CREATE TABLE "new_datatypes" (

date_s date,

smallint_s smallint,

time_s time,

tinyint_s tinyint,

PRIMARY KEY(smallint_s)

) WITH

"bloom_filter_fp_chance"='0.01' AND "dclocal_read_repair_chance"='0.0' AND "gc_grace_seconds"='864000' AND "read_repair_chance"='0.1'

AND compaction = {'class':'SizeTieredCompactionStrategy'}

AND compression = {'sstable_compression':'org.apache.cassandra.io.compress.LZ4Compressor'}

INSERT INTO new_datatypes(smallint_s, date_s, time_s, tinyint_s)

VALUES(45,'2017-07-03','19:10:12',10)

INSERT INTO new_datatypes(smallint_s, date_s, time_s, tinyint_s)

VALUES(44,'2017-07-04','19:10:12',10)

INSERT INTO new_datatypes(smallint_s, date_s, time_s, tinyint_s)

VALUES(42,'2017-07-03','19:10:12',10)

INSERT INTO new_datatypes(smallint_s, date_s, time_s, tinyint_s)

VALUES(40,'2017-07-03','19:10:12',10)

2. When you do a select on this table in the QA window, select * from new_datatypes, you get a java.lang.StringIndexOutOfBoundsException: String index out of range: -1 exception. See here.

1 attachment

CassandraError.png

2017-07-18T15:01:08Z

135 KB

All Comments (11) Change History

tomconrad 2017-07-18T15:15:41Z · (edited)

These new types need to be added to TypesMap.class.

"org.apache.cassandra.db.marshal.ByteType"

"org.apache.cassandra.db.marshal.TimeType"

"org.apache.cassandra.db.marshal.SimpleDateType"

"org.apache.cassandra.db.marshal.ShortType"

The error is caused because the comparator used in line 1165 of CassandraResultSet.createColumn is not finding the new types and allowing the code path to enter the if statement -

if (validator == null) {

int index = valueType.indexOf("(");

assert index > 0;

String collectionClass = valueType.substring(0, index);

index is set to -1 and so the substring fails. Also note that the assert is not turned on and so doesn't catch it.

This is also a problem for data extraction for Aquascript Database Schema and Data Exporter and was discovered while doing db regression testing. VA #10125. When testing the fix please also regression test Aquascript Database Schema and Data Exporter using Cassandra.

sagarzond 2017-09-20T14:08:18Z

For Short and Bytes data types, We have successfully implemented handler by overrding AbstractJdbcType<Short> and AbstractJdbcType<Byte>.

We are not getting, how to implement same thing for Date & Time type. On cassandra forum, they had latest implementation by overriding AbstractType<Integer> and AbstractType<Long>. I did same implementation but on ADS UI its showing value in Number. I also tried by overring AbstractType<Date> but its showing incorrect date after UI.

Please guide which will be correct implementation for handling Date and Time data type.

supriyasarade 2017-10-04T13:29:40Z

This issue is verified on Build no.: ads-windows-x64-18.5.0-dev-8.

tomconrad 2017-10-04T22:12:14Z

Hi Sagar,

Would you also add the tinyint keyword into the datatypes section of SQLCassandraSyntax.java file? This keyword is not showing up with a blue color in the Query Analyzer window.

Also, if possible, since the changes were made in several svn check in's, would you please add the svn numbers for those check in's to this issue for reference.

Thanks in advance, Tom

tomconrad 2017-10-06T19:39:48Z

Code Reviewed

supriyasarade 2017-10-09T13:33:33Z

This issue is verified on Build no.: ads-windows-x64-18.5.0-dev-12.

tomconrad 2017-11-15T17:54:18Z

While doing global functional testing relative to Cassandra, I discovered that some of the tools are not working

with the new types.

1. SchemaScriptGenerator when including insert statements.

2. Export tool when generating the output data.

3. Import tool for csv and Excel when creating the insert statement string for the new data types

4. Query Analyzer results when doing a select against the new types.

#1, #2 and #4 are related to there not being JDBC_TYPE listed for the new types. See CassConstants.java.

Put a breakpoint on line 231 of ETLUtil.java. (columnDataTypeName = CassColumnType.getSQLTypeByJDBCType(columnDataTypeName);)

and run the SchemaScriptGenerator tool.

These are the errors that I am seeing in the datastudio.log

11/15 09:45:47.900 CassColumnType.getSQLTypeByJDBCType SQL data type is not found for JDBC type=[ShortType]

11/15 09:45:48.470 CassColumnType.getSQLTypeByJDBCType SQL data type is not found for JDBC type=[SimpleDateType]

11/15 09:45:49.119 CassColumnType.getSQLTypeByJDBCType SQL data type is not found for JDBC type=[TimeType]

11/15 09:45:49.671 CassColumnType.getSQLTypeByJDBCType SQL data type is not found for JDBC type=[ByteType]

#3, the created insert statement for the import has quotes for the smallint and tinyint fields. This might be

because it does not know how to translate those columns.

Please have a look at these.

Pankaj.Isarka 2017-11-16T15:37:47Z

Tom,

We have fixed review comments. Revision ID : #55612.

This changes will get available in next build.

tomconrad 2017-11-16T17:44:37Z · (edited)

Hi Pankaj,

I am still getting these errors in the log when executing Schema Script Generator or simply doing a query in the Query Analyzer window with the new types. Please have a look. I think your jdbc mapping is incorrect in CassColumnType.java, The data type mapping should be different from the jdbc mapping.

_jdbcTypeMap.put(DATA_TYPE_SMALLINT, SQL_TYPE_SMALLINT);

_jdbcTypeMap.put(DATA_TYPE_TINYINT, SQL_TYPE_TINYINT);

Thanks, Tom

11/16 09:21:02.985 CassColumnType.getSQLTypeByJDBCType SQL data type is not found for JDBC type=[ShortType]

11/16 09:21:02.986 CassColumnType.getSQLTypeByJDBCType SQL data type is not found for JDBC type=[ByteType]

11/16 09:21:02.986 CassColumnType.getSQLTypeByJDBCType SQL data type is not found for JDBC type=[ShortType]

11/16 09:21:02.986 CassColumnType.getSQLTypeByJDBCType SQL data type is not found for JDBC type=[ByteType]

11/16 09:21:02.986 CassColumnType.getSQLTypeByJDBCType SQL data type is not found for JDBC type=[ShortType]

11/16 09:21:02.986 CassColumnType.getSQLTypeByJDBCType SQL data type is not found for JDBC type=[ByteType]

11/16 09:21:02.987 CassColumnType.getSQLTypeByJDBCType SQL data type is not found for JDBC type=[ShortType]

11/16 09:21:02.987 CassColumnType.getSQLTypeByJDBCType SQL data type is not found for JDBC type=[ByteType]

11/16 09:22:21.375 CassColumnType.getSQLTypeByJDBCType SQL data type is not found for JDBC type=[ShortType]

11/16 09:22:21.376 CassColumnType.getSQLTypeByJDBCType SQL data type is not found for JDBC type=[ByteType]

Pankaj.Isarka 2017-11-17T15:49:30Z

Hi Tom,

Fixed reviewed comment on revision :55615

tomconrad 2017-11-20T17:30:58Z

svn #55620 - Cleaned up typing. New types should only be allowed in v3.0 and greater in the Create Table visual editor.

Search Tips

Aqua Data Studio / nhilam

Cassandra - JDBC should be enhanced to handle the new types

1 attachment

Issue #15387

Completion

1 issue link

Issue #15487

Search Tips

Aqua Data Studio / nhilam

Title

Cassandra - JDBC should be enhanced to handle the new types

1 attachment

Issue #15387

Completion

1 issue link

Issue #15487