Here is a sample document in my collection
{ "asset": "dsafjkkk", "epochTimeInMillis":35647, "anomalyResult": { "name": "dsfljdfaslkj", "modelResults": [ { "modelName": "abc", "anomalyScore": 12.49153623540901 }, { "modelName": "pqr", "anomalyScore": 12.38601205387994 } ] } }
Following 2 queries give different results.
1) Simply by dividing one of the columns by 1000 the flattening behavior changes.
2) Result_2 is repeat the row. It should have returned only 1 record.
Query_1
select FLATTEN_ARRAY asset, epochTimeInMillis, anomalyResult.modelResults.modelName as model, anomalyResult.modelResults.anomalyScore as score
from test
Result_1
asset epochTimeInMillis model score
dsafjkkk 35647 abc 12.49153623540901
dsafjkkk 35647 pqr 12.38601205387994
Query_2
select FLATTEN_ARRAY asset, epochTimeInMillis/1000 asepochTimeInSeconds, anomalyResult.modelResults.modelName as model, anomalyResult.modelResults.anomalyScore as score
from test
Result_2
asset epochTimeInMillis/1000 model.0 model.1 score.0 score.1
dsafjkkk 35.647 abc pqr 12.49153623540901 12.38601205387994
dsafjkkk 35.647 abc pqr 12.49153623540901 12.38601205387994
dsafjkkk 35.647 abc pqr 12.49153623540901 12.38601205387994
dsafjkkk 35.647 abc pqr 12.49153623540901 12.38601205387994
|
120 KB
|
240 KB
![]() |
3 KB
|
92 KB
When we applied an aggregation in Query It gives wrong result set because we pass the wrong parameters in pipeline.
I have changed selectCore function in mongoStatement file.
Fix seems correct. Please test that scenarios reported on issue #9629 are still working after this source-code change.
Fix seems correct. Please test that scenarios reported on issue #9629 are still working after this source-code change.
I have tested all the scenario on issue #9629.It does not effect the results.The results comes in
For Issue #13625
The results are coming correct but are not display properly.So shall I Write own display logic for it ? [SP] Can you explain what you mean by results are not displayed properly? How about providing a screenshot which might help in understanding
I have tested all the scenario on issue #9629.It does not effect the results.The results comes in
For Issue #13625
The results are coming correct but are not display properly.So shall I Write own display logic for it ? [SP] Can you explain what you mean by results are not displayed properly? How about providing a screenshot which might help in understanding
For Query1 : select FLATTEN_ARRAY asset, epochTimeInMillis, anomalyResult.modelResults.modelName as model, anomalyResult.modelResults.anomalyScore as score from test
Result :
obj{ "asset" : 'dsafjkkk', "epochTimeInMillis" : 35647.0, "anomalyResult" : { "modelResults" : [ { "modelName" : 'abc', "anomalyScore" : 12.49153623540901 }, { "modelName" : 'pqr', "anomalyScore" : 12.38601205387994 } ] } }
For Query2:
select FLATTEN_ARRAY asset, epochTimeInMillis/1000 as epochTimeInSeconds, anomalyResult.modelResults.modelName as model, anomalyResult.modelResults.anomalyScore as score from test
Before Changes In Code :
Result :
obj{ "asset" : 'dsafjkkk', "epochTimeInSeconds" : 35.647, "model" : [ [ 'abc', 'pqr' ], [ 'abc', 'pqr' ] ], "score" : [ [ 12.49153623540901, 12.38601205387994 ], [ 12.49153623540901, 12.38601205387994 ] ] }
After Changes In code :
Result:
{ "asset" : 'dsafjkkk', "epochTimeInSeconds" : 35.647, "model" : [ 'abc', 'pqr' ], "score" : [ 12.49153623540901, 12.38601205387994 ] }
Please Refer this Images for result:mail.png
For Query1 : select FLATTEN_ARRAY asset, epochTimeInMillis, anomalyResult.modelResults.modelName as model, anomalyResult.modelResults.anomalyScore as score from test
Result :
obj{ "asset" : 'dsafjkkk', "epochTimeInMillis" : 35647.0, "anomalyResult" : { "modelResults" : [ { "modelName" : 'abc', "anomalyScore" : 12.49153623540901 }, { "modelName" : 'pqr', "anomalyScore" : 12.38601205387994 } ] } }
For Query2:
select FLATTEN_ARRAY asset, epochTimeInMillis/1000 as epochTimeInSeconds, anomalyResult.modelResults.modelName as model, anomalyResult.modelResults.anomalyScore as score from test
Before Changes In Code :
Result :
obj{ "asset" : 'dsafjkkk', "epochTimeInSeconds" : 35.647, "model" : [ [ 'abc', 'pqr' ], [ 'abc', 'pqr' ] ], "score" : [ [ 12.49153623540901, 12.38601205387994 ], [ 12.49153623540901, 12.38601205387994 ] ] }
After Changes In code :
Result:
{ "asset" : 'dsafjkkk', "epochTimeInSeconds" : 35.647, "model" : [ 'abc', 'pqr' ], "score" : [ 12.49153623540901, 12.38601205387994 ] }
Please Refer this Images for result:mail.png
For Issue #13625
The results are coming correct but are not display properly.So shall I Write own display logic for it ? [SP] Can you explain what you mean by results are not displayed properly? How about providing a screenshot which might help in understanding
@ambikesh.shukla: I still don't understand what is exactly the display problem that you state that still occurs after your fix is applied. Please give us more specific details. The screenshot provided only compares the before & after behaviours, but does not indicate what the display problem is.
For Issue #13625
The results are coming correct but are not display properly.So shall I Write own display logic for it ? [SP] Can you explain what you mean by results are not displayed properly? How about providing a screenshot which might help in understanding
@ambikesh.shukla: I still don't understand what is exactly the display problem that you state that still occurs after your fix is applied. Please give us more specific details. The screenshot provided only compares the before & after behaviours, but does not indicate what the display problem is.
When we display the result after changing the code : In "MongoUtils" class there a function named
currentPos : 1
result : empty
In previous case the column name was "anomalyResult.modelResults" (Aliases are not applied), then it goes to
public static Object getFlattenedObject(String flattenName, DBObject obj, boolean metadata) function and return key and value pair list
But my case column name was different [model,score] (Aliases are applied by mongo command ) then separate list is return one modelname and another for anomaly score.
Then 2 BasicDBList is created one for model name and one for anomaly score
list 1:
index - 0 -> 12.49153623540901
index - 1 -> 12.38601205387994
list 2 :
Index - 0 -> abc
index - 1 -> pqr
According to display logic It takes the first element of list1 and then iterate 2 times for list 2
Again it takes 2 element of list 1 and iterate 2 times for list 2.
When we display the result after changing the code : In "MongoUtils" class there a function named
currentPos : 1
result : empty
In previous case the column name was "anomalyResult.modelResults" (Aliases are not applied), then it goes to
public static Object getFlattenedObject(String flattenName, DBObject obj, boolean metadata) function and return key and value pair list
But my case column name was different [model,score] (Aliases are applied by mongo command ) then separate list is return one modelname and another for anomaly score.
Then 2 BasicDBList is created one for model name and one for anomaly score
list 1:
index - 0 -> 12.49153623540901
index - 1 -> 12.38601205387994
list 2 :
Index - 0 -> abc
index - 1 -> pqr
According to display logic It takes the first element of list1 and then iterate 2 times for list 2
Again it takes 2 element of list 1 and iterate 2 times for list 2.
@Sachin:
Please refer attachments: 13625Explained.txt and 13625Image.png
Refer [ File : "MongoUtils.java"
Method: private static void expandArrays(DBObject object, String[] columnNames,
int currentPos, List<DBObject> result) throws SQLException ]
@Sachin:
Please refer attachments: 13625Explained.txt and 13625Image.png
Refer [ File : "MongoUtils.java"
Method: private static void expandArrays(DBObject object, String[] columnNames,
int currentPos, List<DBObject> result) throws SQLException ]
For Issue #13625
The results are coming correct but are not display properly.So shall I Write own display logic for it ? [SP] Can you explain what you mean by results are not displayed properly? How about providing a screenshot which might help in understanding
Indeed, there was a problem in flattening the arrays when math expressions were present.
The latest version of Ambikesh's patch is correct. I have integrated it in our source-code repository.
For Issue #13625
The results are coming correct but are not display properly.So shall I Write own display logic for it ? [SP] Can you explain what you mean by results are not displayed properly? How about providing a screenshot which might help in understanding
Indeed, there was a problem in flattening the arrays when math expressions were present.
The latest version of Ambikesh's patch is correct. I have integrated it in our source-code repository.
All working well.I have check in to branch with svn 2880.
The MongoStatement.java file sent by e-mail also contained changes to convertJSScriptableToJDBCResultSet() method (not related to this issue). That's why I have applied the proposed fix on my side.
@ambikesh: If svn #2880 contains only the proposed changes made to MongoStatement.selectCore() method, then you can keep it on your repository, otherwise please rollback that changes.
All working well.I have check in to branch with svn 2880.
The MongoStatement.java file sent by e-mail also contained changes to convertJSScriptableToJDBCResultSet() method (not related to this issue). That's why I have applied the proposed fix on my side.
@ambikesh: If svn #2880 contains only the proposed changes made to MongoStatement.selectCore() method, then you can keep it on your repository, otherwise please rollback that changes.
Verified in ADS17.0.0-beta-20
Please refer screenshot: Fixed-ADS17-beta-20.png
Verified in ADS17.0.0-beta-20
Please refer screenshot: Fixed-ADS17-beta-20.png
Issue #13625 |
Closed |
Fixed |
Resolved |
Completion |
No due date |
Fixed Build beta-18,bi-studio 1.4.7.2 |
No time estimate |
1 issue link |
is broken by #9629
Issue #9629Wrong program beheviour when complex WHAT and WHERE clauses in the statement |
When we applied an aggregation in Query It gives wrong result set because we pass the wrong parameters in pipeline.
I have changed selectCore function in mongoStatement file.