Over MySQL connection, I've tested with queries that return a 3000+ rows resultset.

It is unlikely this is a bug in ADS as this has been reporting, reviewed and discussed and excessive amount of times.  MySQL JDBC driver is the only driver that caches the full resultset before allowing ADS to iterate the results, but in 12.0 we added this to prevent this ...

                if (_settings.isMySQL() ) {
                    //By default, ResultSets are completely retrieved and stored in memory. In most cases this is the most efficient way to operate, and
                    // due to the design of the MySQL network protocol is easier to implement. If you are working with ResultSets that have a large number
                    // of rows or large values, and cannot allocate heap space in your JVM for the memory required, you can tell the driver to stream the
                    // results back one row at a time.
                    _currentStatement = _conn.createStatement( ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY );
                } else {
                    _currentStatement = _conn.createStatement();
                }

The problem you may be experiences may just be related to how the JVM garbage collects.

The MySQL JDBC driver tuning explained above seems to help when there is no enough free memory so that the JDBC Driver can fully cache the ResultSet. But current issue is not about this, as former query runs fit memory without any problems. The problem is that the used memory continues to increase from one query launch to another. I can see on the memory graph when the garbage collector frees orphaned objects from memory, but the problem is that after this garbage collection the amount of used memory doesn't go back (aprox) to the amount it was used before issuing these queries (~70-90 MB). Forcing a GC by clicking on the "trash can" icon doesn't help.

I'm pretty sure this is not a JDBC Driver / JVM GC issue but a GUI one because for example I can run the same query under FluidShell, no matter how many times, and it never goes out of memory and forcing a GC by clicking on the trash can icon releases the memory to its initial state (~70-90 MB used memory).

Investigating it further, I noticed that this memory leak issue doesn't occur when only the "Grid" output type is enabled, but occurs when "Text" output type is enabled. Therefore, it seems the memory is leaked inside the vincaed editor. (the read-only one that displays the resultset output) :-)

Please list here how the "Text" read-only editor is currently instantiated / reused, i.e. what are the calls on your  code for this GUI component when the "Execute (CTRL + E)" action is triggered. Is the output text from previous run cleared (in order to reuse the component) or another component is created? Are there any other clean-ups called on this step, for this component?

The root cause of this problem seems to be the fact that you reuse the JEditorPane component that displays the output on the Text Tab by recreating its underlined document: pane.setDocument(pane.getEditorKit().createDefaultDocument());

However, references to the old document remains on the GUI decorators of the JEditorPane component. Even if you currently call document.remove(o, length); or JEditorPane.setText(""); on the old document before replacing it with the new one, the old document still references the big chunk of data (the output of the former resultset) on the BaseDocument.lastModifyUndoEdit field.

Therefore, JEditorPanes should never be reused this way. I know that the pane.setDocument(pane.getEditorKit().createDefaultDocument());  call is currently used in your code as a way of releasing memory, but this works properly only if the pane (JEditorPane) GUI component, together with its decorators (side bars etc) are no more on the Swing Hierarchy (i.e. are not reused) and are going to be released by the GC.

Instead, you should recreate the entire JEditorPane that displays the Text result set every time the Run action is triggered.

Changed the priority of this issue as it could leads to OOME if several queries (with large result sets) are run on the same Query Analyzer window and the Text output is enabled.

You are correct in that we rebuild the JEditorPane in this fashion.  If you are correct on the behavior of JEditorPane then there would be a memory leak here.  The code in QueryPanel.java/QueryThread.java is involved, so I'll move this to 14.0 for Fung and if the changes are not to big he can backport to 13.0.

SVN trunk/r31100 - Made change to recreate the JEditorPane upon each query execution.

Did the following test:
(a) create a test table in MySQL with 4K+ rows, the size of each each is ~32K bytes
(b) launch ADS with -Xmx512M
(c) open a query analyzer and make sure only Text tab will be displayed, nothing else (no Grid, Pivot Grid, etc.)
(d) select * from test_table limit 1000
(e) CTRL+E
(f) click the 'trash can' icon to GC
(g) repeat (e) and (f)

Before r31100, my ADS usually run out of memory after repeating (e) and (f) 3 times; after r31100, I can repeat (e) and (f) quite a few times.

One thing I noticed is: if I move the scrollbar attached to the editor pane, then memory usage increased significantly, and the used memory does not seem to be freed up after GC.

changes look low impact, lets backport to 13.0.

One thing I noticed is: if I move the scrollbar attached to the editor pane, then memory usage increased significantly, and the used memory does not seem to be freed up after GC.

I think this behaviour occurs because Netbeans lazily loads the paragraph views, i.e. initially just the paragraphs that are on the visible area (viewport) are rendered. As you scroll down, more paragraph views are instantiated and thus the memory used increases.

When you notice that the memory usage is high (after scroll actions), if you run a short resultset query (to get rid of the old JEditorPane output) then launch the GC, is the memory freed up or not?

Please send us an updated build and I'll investigate if there are still some memory leaks on these scenarios.

> When you notice that the memory usage is high (after scroll actions), if you run a short resultset query (to get rid of the old JEditorPane output) then launch the GC, is the memory freed up or not?

No, it won't be garbage-collected in 1-2 minutes most of time after a new query is executed (even a query with nothing returned). If I wait long enough, say 2-4 minutes, it will be freed up.

Backport from trunk to 13.0 branch is done:
   SVN r31100 - trunk
   SVN r31105 - 13.0 branch

If moving scrollbar indeed causes memory leak, we can reopen this issue or create a new issue.

The initial memory leak seems to be fixed in the ADS 14-dev6 build, i.e. I can release the memory by running a short/empty resultset, then launch the GC.

The memory spikes on scroll actions don't seem to be a memory leak, because the amount of used memory also decreases when GC is launched, i.e. the orphaned objects (paragraph views etc) are released from the heap memory.

However, there is still a memory leak related to the Query Analyzer Window and that's why I'm reopening this issue:

• I open a new Query Analyzer window, enable just the Text output;
• I run a long result-set query and while the results are displayed, the amount of used memory increases as expected;
• I close this Query Analyzer Window, then try several times to release the orphaned objects by launching the GC, but it doesn't work, i.e. the amount of used memory is high;
• there are two workarounds that helps on this stage:
  • I open a new Query Analyzer Window, then I can successfully release the memory by launching the GC. Therefore, there could be some static references to GUI components of the Query Analyzer Window that blocks the GC from releasing the orphaned objects;
  • I wait a few minutes then launch the GC again and this time (but not always) the memory is released.

I attach two stack traces extracted from heap dumps, that could indicate the root of this memory leak. Unfortunately there are obfuscated elements on the stack so I don't know what classes (from the top of the trees) are keeping these references. Please take a look.

There seems to be a common reference held by these classes (see the elements in bold below):

b \\...\\ .\\हिñçêČάй語简�?한\\.Xꏕꅴꊃ᠗
    avv com.aquafold.datastudio.queryanalyzer.gᠲᛇꅺꐕ assert interface
        imp \\...\\ .\\हिñçêČάй語简�?한\\.Tꃋ͔͡ꃍ Boolean super
            visComp javax.swing.JPanel @ 0xd4f1dc70
                layoutMgr java.awt.BorderLayout
                    center javax.swing.JScrollPane
                        mouseWheelListener org.netbeans.modules.editor.lib2.view.DocumentViewOp
 

On the first scenario, the reference is held by the selected text from the "Current database" combobox, as I changed the current database before running the long-resultset query. On the second scenario, I ensure that the text on this comobobox does not remain selected after changing the current database.

This could indicate that the Query Analyzer Window in not properly removed from the Swing hierarchy and that are still references to it (or some GUI components from it) that prevent the GC from releasing the big NbEditorDocument instance. Could you please debug using the steps described above?

As you can see from the last two screenshots, the orphaned NbEditorDocument instance holds big data in two fields: text and lastModifyUndoEdit which actually doubles the amount of used memory. This happens because before replacing the JEditorPane output with another one (from one query run to another), there is an obsolete JEditorPane.setText("") call:

javax.swing.JEditorPane.setText(JEditorPane.java:1470)
com.aquafold.datastudio.queryanalyzer.gᠲᛇꅺꐕ assert interface.BI
com.aquafold.datastudio.queryanalyzer.gꌎ᡼⣻ꏧ.run
java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:216)

which actually copies the big data chunk on the lastModifyUndoEdit field. This means that the JEditorPane.setText(""); call does not have the intended scope (release memory) but actually keeps a copy of the big chunk on the lastModifyUndoEdit field.

As now the Text component is recreated entirely (the fix made on SVN change r31100), it would be better if you remove the setText("") call indicated on the stack trace above. This way less memory would be allocated from one query run to another.

com.common.ui.frame.CEditorPanelView$2 -> \\...\\ .\\हिñçêČάй語简�?한\\.Xꏕꅴꊃ᠗:
    com.common.ui.frame.CEditorPanelView this$0 -> avv
    javax.swing.JPopupMenu constructPopupMenu(javax.swing.JComponent,int,int) -> a
    void showMenu(javax.swing.JPopupMenu,javax.swing.JComponent,int,int) -> a

Merged change of SVN trunk/r31100 back to 12.0 branch, i.e. the followings are the same:
   SVN r31100 - trunk
   SVN r31105 - 13.0 branch
   SVN r31111 - 12.0 branch
This fix is to recreate editor pane upon each query execution.

Changes below should take care of problem caused by setText(""):
   SVN r31116 - 12.0 branch
   SVN r31117 - 13.0 branch
   SVN r31118 - trunk

In my test case, after this fix is applied, additional ~50MB heap space is freed up.

I did some more test using 12.0/SVN r31116:
(1) Launch ADS (-Xmx512M) with no tabs opened. The Memory Usage displayed in the lower-right corner shows:  (50 : 193 : 494 MB)
(2) Open a Query Analyzer tab. The Memory Usage becomes (56 : 193 : 494 MB).
(3) Change the "Database:" combo box to db_1 which is the database where the long result-set query can be executed.
(4) Execute the long result-set query, then run the GC.  The Memory Usage becomes (138 : 327 : 494 MB).
(5) Repeats (4).  The Memory Usage becomes (216 : 494 : 494 MB).
(6) Repeats (4) a few more times. The Memory Usage remains as (216 : 494 : 494 MB).
(7) Change the "Database:" combo box to db_2.
(8) Execute the long result-set query (the query will fail this time), then run the GC.  The Memory Usage becomes (138 : 327 : 494 MB) which is similar to (4).
(9) Repeats (8) a few times. The Memory Usage remains as (138 : 327 : 494 MB).
(10) Change the "Database:" combo box back to db_1.
(11) Execute the long result-set query (the query will finish this time), then run the GC.  The Memory Usage becomes (216 : 494 : 494 MB) which is similar to (5).
(12) Repeats (11) a few times. The Memory Usage remains as (216 : 494 : 494 MB).
(13) Close the Query Analyzer tab opened in (2) and then run the GC.  The Memory Usage remains unchanged (216 : 494 : 494 MB).
(14) Wait for 5-10 minutes and run the GC again.  The Memory Usage goes down to (50-56 : 193 : 494 MB) which is similar to (1)-(2).

(a) I ran Eclipse Memory Analyzer Tool (MAT) after (13), MAT detects a org.netbeans.modules.editor.lib2.document.CharContent object (screenshot attached) and says it could be a problem suspect.  I ran Eclipse Memory Analyzer Tool again after (14), the org.netbeans.modules.editor.lib2.document.CharContent is no longer reported (screenshot attached).  Any idea about this netbeans object?
(b) Don't know why the Used Memory increased again at step (5).
(c) After step (6) if you wait for 5-10 minutes and then run the GC, the Memory Usage will go down to (138, 327, 494 MB) which is similar to (4).
(d) After step (9) if you wait for 5-10 minutes and then run the GC, the Memory Usage will go down to (56, 193, 494 MB) which is similar to (2).
(e) Also modified PopupMenu related code in both CEditorPane and CEditorPanelView, and did some test, does not seem to affect results described above.

The org.netbeans.modules.editor.lib2.document.CharContent objects are the low-level classes that encapsulates the chars of the document (i.e. the model data of the JEditorPane). Inside Eclipse MAT, if you scroll down below the last "Problem Suspect X", there is a "Hint" paragraph that gives you a hierarchical view of the references to these low-level containers that prevents them from being released from the heap memory.

Obviously, the NbEditorDocument instance and its CharContent chunks (i.e. the objects that are actually stored on the heap memory) come from the Text output (the JEditorPane), but we have to find out where does these references come from. On the Hints paragraph, if you look on the top part of the tree, you'll notice that there are references to the JPanel that holds the Text output. This means that there are some references left to the GUI components of the Query Analyzer Window, remained after this component has been removed from the Swing hierarchy. This should not happen and I think this is the root cause why the NbEditorDocument cannot be released from the heap memory.

Fortunately, I've found another workaround that might help identifying the cause of this memory leak: after step 13) , if you trigger the File -> Options or File -> Set Window Title actions, some references will be deleted and then the memory can be released by launching the GC. Other workarounds such as opening a new text editor, FluidShell window etc after step 13) won't work. Therefore, I think there is something related to the JTextField components that releases the unwanted references.

Below are two reference trees extracted with Eclipse Mat after step 13). Referenced objects are on the top, the referrers are below them (stack traces obtained with the 14-dev7 build).

    text org.netbeans.modules.editor.NbEditorDocument @ 0xd440cf10
      model \\...\\ .\\हिñçêČάй語简�?한\\.eꂚꂉꆿᠩ 7 @ 0xd44090a8
      [1] java.lang.Object[2] @ 0xd44179c0
        table javax.swing.ArrayTable @ 0xd44179b0
           clientProperties javax.swing.JPanel @ 0xd440e888
              visComp \\...\\ .\\हिñçêČάй語简�?한\\.Mꋒꄷ⠺ᢇ @ 0xd1e38f50
                 imq com.aquafold.datastudio.queryanalyzer.qꌯꅋꂈꁐ @ 0xd1dc2f90
                     avv \\...\\ .\\हिñçêČάй語简�?한\\.k⡃⢪ꍐꊽ for @ 0xd1e7dd58
                       b java.awt.AWTEventMulticaster @ 0xd1e7dc30
                         a java.awt.AWTEventMulticaster @ 0xd1e7dc18
                           a java.awt.AWTEventMulticaster @ 0xd1f6a018
                             keyListener \\...\\ .\\हिñçêČάй語简�?한\\.eꂚꂉꆿᠩ 7 @ 0xd1dbae28
                               clientComponent sun.awt.im.CompositionAreaHandler @ 0xd1d16f40
                                 compositionAreaHandler sun.awt.im.InputMethodContext @ 0xd1a49ed8
                                   inputContext com.aquafold.datastudio.mainframe.Eᛏꈳꇾꊮ switch @ 0xd187aed0
                                     currentFocusCycleRoot class java.awt.KeyboardFocusManager @ 0xc58dda70 System Class
                                       gCS \\...\\ .\\हिñçêČάй語简�?한\\.V͠ꏪꃣꌪ 6 if @ 0xd1abaa58 XYL Thread
                                       parent javax.swing.JDialog @ 0xdcfa3cd8 »
                                       [0] java.lang.Object[10] @ 0xd0e63900 »
                                       parentFrame com.aquafold.datastudio.mainframe.ShortcutToolBar @ 0xd18ac630 »
                                       [24] java.lang.Object[64] @ 0xd1008620 »
                                       fi class com.aquafold.aquaeditor.bridge.DiffTmpBridge @ 0xc903dfc8 »
                     qr com.aquafold.datastudio.queryanalyzer.rᡉᡛꃠ᠘ return @ 0xd1e271f0 »

On the second run, I've added a KeyboardFocusManager.getCurrentKeyboardFocusManager().setGlobalCurrentFocusCycleRoot(null); call inside PaneFactory.closedPane() method. This way the currentFocusCycleRoot object disappears from the reference tree, but the memory leak is still there. Could this happen due to the qr reference (the last node from the tree, i.e. one of the top referrers )?

    text org.netbeans.modules.editor.NbEditorDocument @ 0xd4a15908
      model \\...\\ .\\हिñçêČάй語简�?한\\.eꂚꂉꆿᠩ 7 @ 0xd4a16aa8
        [1] java.lang.Object[2] @ 0xd4a22d78
          table javax.swing.ArrayTable @ 0xd4a22d68
            clientProperties javax.swing.JPanel @ 0xd4a16f80
              visComp \\...\\ .\\हिñçêČάй語简�?한\\.Mꋒꄷ⠺ᢇ @ 0xd4865cd8
                imq com.aquafold.datastudio.queryanalyzer.qꌯꅋꂈꁐ @ 0xd4864c68
                  avv \\...\\ .\\हिñçêČάй語简�?한\\.k⡃⢪ꍐꊽ for @ 0xd487ee88
                    b java.awt.AWTEventMulticaster @ 0xd487ed60
                     a java.awt.AWTEventMulticaster @ 0xd487ed48
                       a java.awt.AWTEventMulticaster @ 0xd487ed30
                         keyListener \\...\\ .\\हिñçêČάй語简�?한\\.eꂚꂉꆿᠩ 7 @ 0xd485c8e0
                           clientComponent sun.awt.im.CompositionAreaHandler @ 0xd1b22d18
                             compositionAreaHandler sun.awt.im.InputMethodContext @ 0xd1b20e08
                               inputMethodWindowContext class sun.awt.im.InputContext @ 0xc91ee280 System Class
                               inputContext com.aquafold.datastudio.mainframe.Eᛏꈳꇾꊮ switch @ 0xd18a3308 »
                  qr com.aquafold.datastudio.queryanalyzer.rᡉᡛꃠ᠘ return @ 0xd48647d0 »

If these hints doesn't help you to identify the cause of the memory leak, could you please fully deobfuscate these reference trees so I can understand what the referenced objects and the referrers are?

These reference trees can be obtained by clicking on the "Problem Suspect X" -> "Details" link inside Eclipse Mat.

The inputContext (class type com.aquafold.datastudio.mainframe.Eᛏꈳꇾꊮ switch ) reference from the trees above, has a timer field inside which probably expires and that's why after a few minutes of waiting after step 13), the memory can be released.

"Path to GC Roots" supposes to explain why an object cannot be garbage collected.  If I run a 'Path To GC Roots -> exclude weak/soft references' on the very first 'Problem Suspect', then nothing shows up; however, if I chose 'Path To GC Roots -> with all references', then netbeans' EditorRegistry and EditorRegistry.Item are displayed between ClassLoader and ADS' CEditorPane (derived from JEditorPane) (see attached screenshot).  Could this be the root cause?  Below is the definition of EditorRegistry.Item:
    static final class Item extends java.lang.ref.WeakReference<javax.swing.text.JTextComponent>

When a JEditorPane (actually, a CEditorPane) is recreated upon a query execution, ADS did not do anything about the old instance of JEditorPane, is ADS supposed to do something to remove the old instance from EditorRegistry.items?

I don't think the EditorRegistry.Item weak reference blocks the GC from releasing the document and its underlying data chunks, because by definition a weak reference means that the GC can release the object if there are no more strong references, but just weak references /  no references at all, isn't it?

I think the last workaround I've listed on my previous comment is a good key while hunting this specific memory leak:

Fortunately, I've found another workaround that might help identifying the cause of this memory leak: after step 13) , if you trigger the File -> Options or File -> Set Window Title actions, some references will be deleted and then the memory can be released by launching the GC. Other workarounds such as opening a new text editor, FluidShell window etc after step 13) won't work. Therefore, I think there is something related to the JTextField components that releases the unwanted references.

It seems that some JTextField instantiations deletes some strong references (see the last two reference trees above) and after that, the GC is able to release these objects (the NbEditorDocument and the CharContent chunks) from memory.

I don't have the source-code, therefore I'm unable to help more, but reference trees above give hints about some InputContext instances that might be updated when new JTextField is created (e.g. the "Set Window Title" action), thus releasing the strong reference. Do you have any tweaks / static fields related to the InputContext?

Former reference trees contained a JTextField node that I noticed is the text field associated to the "Current Database" combo box (because I changed the current database and the text-field remains with its text selected when I close the Query Analyzer Window). However, after changing the current database, if I deselect the text of this text field, run the long-resultset query, then this object doesn't appear any more on the reference trees.

I don't know if this information helps you.

I just spoke to Niels, since after step 14, used memory eventually will be freed up, Niels is OK with this and I have marked this issue Resolved.

Thank you all for your input and help.