Easily find issues by searching: #<Issue ID>
Example: #1832
Easily find members by searching in: <username>, <first name> and <last name>.
Example: Search smith, will return results smith and adamsmith
Extremely large collection in MongoDB 3.6.2, I set max rows to 5 and run db.collection_name.find() and it seems to run forever and fill up all available memory. I suspect it is not respecting that I only asked for 5 rows.
I have the following Mongo setup:
[cid:image002.jpg@01D3AA5A.699494C0]
Notice the # of documents in each collection.
I opened a fresh ADS with a fresh datastudio.log. I then proceeded to issue a db.ica_activitylog_warus.find() in a new query window with a 5 record limit. Worked fine.
Then I opened a new query window and did the same on ica_activitylog_amit, also worked fine.
I then I opened a new query window and performed db.ica_activitylog.find() with a 5 record limit. The entire ADS eventually locked up, not responding, consuming all available memory. After many minutes of waiting at a blank screen with the app not responding, I finally got in the text window:
>[JS] Script lines: 1-3 -----------------------------
db.ica_activitylog.find();
________________________________
>[Error] Script lines: 1-3 --------------------------
java.lang.OutOfMemoryError: GC overhead limit exceeded
________________________________
0 record(s) affected
[Executed: 2/20/2018 2:42:56 PM] [Execution: 7m 3s]
The grid window was blank.
I then closed ADS and attached datastudio.log and startup.log.
John, each record is less than 1K. Please I've been trying to tell you that your code seems to be pulling in the entire dataset, then displaying the first X rows. I have a very large memory allocation:
vmarg.2=-Xmx4096M
vmarg.3=-XX:MaxPermSize=192m
That query singlehandedly filled up all available memory.
|
|
32 KB
|
22 KB
|
|
4 KB
|
|
721 KB
|
|
5 KB
The problem is that the max results is ignored during array flattening; the max results is respected only after the array flattening. During the array flattening, millions of rows could be created in memory due to multiple columns with array values, resulting in out of memory condition. Due to the way garbage collection works, the application will come to a crawl during extremely low memory situation, causing ADS to appear hanging and unresponsive.
The fix is to also respect the max results during array flattening.
Fixed:
Fixed:
Also fixed in Fluidshell:
Also fixed in Fluidshell:
Handle condition when max result is set to zero, which is treated as no limit:
Handle condition when max result is set to zero, which is treated as no limit:
Based on the screenshots in ADSMONGO.DOCX, the customer executes the following query in Query Analyzer (MongoJS) with Max Result = 5:
db.ica_activitylog.find();
Based on my current debugging, it seems that unlike the "SELECT ..." in the regular Query Analyzer, the db.ica_activitylog.find() executed in Query Analyzer (MongoJS) may be returning the entire dataset first, then only the Max Results is displayed in the results panel.
While I continue to debug this issue, please have the customer try the following query instead to see if this will temporarily solve the customer's issue:
db.ica_activitylog.find().limit(5);
Based on the screenshots in ADSMONGO.DOCX, the customer executes the following query in Query Analyzer (MongoJS) with Max Result = 5:
db.ica_activitylog.find();
Based on my current debugging, it seems that unlike the "SELECT ..." in the regular Query Analyzer, the db.ica_activitylog.find() executed in Query Analyzer (MongoJS) may be returning the entire dataset first, then only the Max Results is displayed in the results panel.
While I continue to debug this issue, please have the customer try the following query instead to see if this will temporarily solve the customer's issue:
db.ica_activitylog.find().limit(5);
"db.ica_activitylog.find().limit(5);" solves the issue for the customer.
This issue is now resolved. The memory aspect of the issue will be handled in Issue #14242.
"db.ica_activitylog.find().limit(5);" solves the issue for the customer.
This issue is now resolved. The memory aspect of the issue will be handled in Issue #14242.
Issue #15532 |
| Verified |
| Fixed |
| Resolved |
Completion |
| No due date |
| Fixed Build v18.0.19-6, v18.5.0-11, v19.0.0-beta-51 |
| No time estimate |
2 issue links |
relates to #15730
Issue #15730"Edit Collection Data (Top 1000)" in MongoDB fails with OutOfMemory |
relates to #14242
Issue #14242MongoJS QA Window - Memory Check for Flatten Grid Results |
The problem is that the max results is ignored during array flattening; the max results is respected only after the array flattening. During the array flattening, millions of rows could be created in memory due to multiple columns with array values, resulting in out of memory condition. Due to the way garbage collection works, the application will come to a crawl during extremely low memory situation, causing ADS to appear hanging and unresponsive.
The fix is to also respect the max results during array flattening.