#14411: If result set contains a lot of columns, Grid Results processing freezes ADS

JennyNishimura reported 2016-04-29T20:55:27Z · last modified 2016-05-17T02:53:29Z

If result set contains a lot of columns, Grid Results processing freezes ADS

Dev	JennyNishimura
QA	tariqrahiman

Priority	Major
Complexity	Unknown

Component	Q Analyzer - Grid Results
Version	17.0

Using MongoDB, documents can contain a lot of fields (over 100,000 fields and sub-fields). When user runs a query that returns all fields, it takes a long time to process the Grid view and ADS becomes unresponsive.

In File -> Options -> Results -> Grid Results, add a new option to specify the maximum number of columns allowed. If the number of columns in the result set exceeds this limit, there will be an error message in the Grid view.

7 attachments

ErrorMessageInGrid.png

2016-04-29T23:45:55Z

28 KB
ads_17_scenario_a.png

2016-05-03T12:02:33Z

220 KB
ads_18_scenario_b.png

2016-05-03T12:02:33Z

212 KB
ads_18_scenario_a.png

2016-05-03T12:02:33Z

211 KB
ads_17_scenario_b.png

2016-05-03T12:02:33Z

238 KB
scenario_d.png

2016-05-04T13:36:41Z

57 KB
scenario_c.png

2016-05-04T13:36:41Z

87 KB

All Comments (10) Change History

JennyNishimura 2016-04-29T23:36:51Z · (edited)

- Added File -> Options -> Results -> Grid Results -> Maximum number of columns allowed. Default is 50,000. A value of 0 means that there is no limit.

- If any query produces number of columns that exceeds the limit, the Grids view will display an error message. All other views will still display the results.

SachinPrakash 2016-04-29T23:40:27Z

@jenny: pls provide a screenshot showing the error message

JennyNishimura 2016-04-29T23:47:14Z

Please see the screenshot in ErrorMessageInGrid.png

dutondeumesh 2016-05-03T12:04:20Z · (edited)

Scenario A : With simple insert query and Default ADS Memory

1. Set maximum number of columns allowed as 3

2. Open Query Analyzer(Mongo JS) and run below insert script

db.mycol1.insert({_id :1,name : 'U',sal : 123444,city:'Pune'})

3. Now run below query

db.mycol1.find()

Result : Error message displayed in Grid tab

4. Set maximum number of columns allowed as 5

5. Open a new Query Analyzer(Mongo JS) and run below query

db.mycol1.find()

Result : Result displayed in Grid view tab

Scenario B : With large fields(100,001),ADS Memory = 4048M and only "Show grid" icon is enable

1. Set maximum number of columns allowed as 50,000

2. Open Query Analyzer(MongoSQL) and run insert script given in the test2.ddl file.(could not attached this file because of size)

Insert script will take 5-6 minute to insert data.

3. Now run below select query

SELECT FLATTEN * FROM "AQ6011147711932106"

Result : Error message displayed in Grid tab

4. Set maximum number of columns allowed as 0. 0 means that there is no limit.

5. Open a new Query Analyzer(MongoSQL) and run below select query

SELECT FLATTEN * FROM "AQ6011147711932106"

Result : Result displayed in Grid tab

dutondeumesh 2016-05-03T12:05:05Z

Verified in ADS 17.0.4-4, 18.0.0-devi-128

dutondeumesh 2016-05-04T13:36:20Z

Reopened for below two scenarios

Version: 18.0.0-devi-129

Scenario C : For doubt Clarification

1. Set maximum number of columns allowed as 3

2. Open Query Analyzer(MongoSQL) and run below insert script

insert into test1(_id,name,sal) values(1,'Umesh',30000)

insert into test2(_id,name,sal,city) values(1,'Umesh',30000,'Pune')

3. Select and run both query at same time

select * from test1

select * from test2

Result : Error message displayed in Grid tab. Is correct behavior ?

Here can we Show two grid views one is showing error message for second select query and one is showing Result for first select query

Please refer attached scenario_c.png

Scenario D : ADS becomes unresponsive (ADS Memory = 4048M and only "Show grid" icon is enable)

1. Set maximum number of columns allowed as 0

2. Run select query which has large fields(100,001)

3. Run simple insert query in same Query Analyzer window

For example : insert into test3(_id,name,sal,city) values(1,'Umesh',30000,'Pune')

Here ADS becomes unresponsive

This behavior observed in version 17 also

Please refer attached scenario_d.png

SachinPrakash 2016-05-04T15:45:59Z

Scenario C : For doubt Clarification

Result : Error message displayed in Grid tab. Is correct behavior ?

OR

This is the correct behavior

SachinPrakash 2016-05-04T15:47:02Z

Scenario D : ADS becomes unresponsive (ADS Memory = 4048M and only "Show grid" icon is enable)

This is the reason why a max columns setting has been introduced. The grid takes a very large amount of memory & rendering time when the # of columns gets extremely large.

dutondeumesh 2016-05-05T03:43:54Z

Thanks for doubt clarification. Marking this issues as verified

tariqrahiman 2016-05-17T02:53:29Z

Confirmed that the patch is working from customer

Search Tips

Aqua Data Studio / nhilam

If result set contains a lot of columns, Grid Results processing freezes ADS

7 attachments

Issue #14411

Completion

1 issue link

Issue #14798

Search Tips

Aqua Data Studio / nhilam

Title

If result set contains a lot of columns, Grid Results processing freezes ADS

7 attachments

Issue #14411

Completion

1 issue link

Issue #14798