Implementation note: For pie chart only, not sunburst. Manipulate the in-memory data iterator to physically group the small values into one bucket, named "Others". An example: if the data iterator contains 10 data points, the result could contain only 6 data points if 5 of them are small values.

Note that if all values are below the small value threshold, then they will all get grouped into a single "Others" category.

Add a new section in the Options popup, between the "Show Empty Categories" and "Center Hole Size" sections:

• New section name: Small Values
  • "Group Small Values" checkbox -- default is unchecked
  • "Group Values Less Than" percentage slider between 1% and 100% inclusive -- default is 3%.

Note that the small values settings is per worksheet and applies to all measures in a worksheet.

From sagarzond:

Nhi,

     We have implemented code as below : 

private void groupSmallValue() {

//TODO remove hardcoding
   BIValueNode valueObj = _data.getValueObject();
   if(valueObj != null) {
      Double total = valueObj.getValues().stream().mapToDouble(a -> Double.valueOf(Objects.toString(a, null))).reduce(0, (a, b) -> a + b);
      List values = valueObj.getValues();
      List keys = valueObj.getKeys();

      Iterator valueIterator = values.iterator();
      Iterator keyIterator = keys.iterator();
      Double otherValue = new Double(0);

      while(valueIterator.hasNext()) {
         Double value = (Double) valueIterator.next();
         Object key = keyIterator.next();

  
         if(((value * 100) / total ) < 30){
      otherValue += value;
            valueIterator.remove();
            keyIterator.remove();
         }
      }

      if(otherValue != 0){
         keys.add("Other");
         values.add(otherValue);
      }

      if(_data instanceof  BIChartDataImpl) {
         BIChartDataImpl dataImpl = ((BIChartDataImpl) _data);
         dataImpl.setComponentCount(values.size());
      }
   }
}

    With this implementation having following questions :

1. We are removing value from BIModel. For single dimention this will work. But what if user drag second dimention then will not find removed values.
2. Check attachment : pie-chart-small-group. We also need to remove Red squared value(shown in attachment). This value getting fetched from worksheet. So if we remove value from worksheet then other type of reports also get impacted.

The above code changes are incorrect. The BIValueNode was not designed with modifications in mind. Thus modifying the object can have unknown consequences. Instead, you should construct a new object with a new keys and values list containing the modified values.

Also as pointed out by the screenshot attachment, the color legend will also need to be updated to add the "Others" category. We don't want to remove the small categories from the color map because in a multiple chart cell scenario, a category with small value in one chart cell may have a large value in another chart cell. The contents of the color legend is constructed in BIModelAccordionsHelper.buildDimensionColors(). There's a special scenario where a "Grand Total" entry is added to the color legend. Thus you want to add the "Others" category in a similar way.

This grouping of small values and adding "Others" to color legend should be done only if the right condition is met. Basically do this only if the "group small values" settings == TRUE, chart type == Pie chart, and there is exactly 1 dimension field in the Chart Properties deck. For the latter, you can use the BIModelAccordionsHelper.getAccordionDimensionFields() to check the number of dimension fields.

Note that I've updated the issue description regarding the color legend selection behavior and tooltip behavior.

Nhi, 

    Having following questions :

• Color legend getting initialized before pie chart gets rendered. Because of this newly added category not getting shown. Do we have any way to refresh or reload color legend as pie chart calculation completed.  [Nhi] You should not have to do anything special. As long as "Others" category is added to color legend first, the Pie chart rendering will automatically use the color assignments in the color legend.
• How can we have same color for pie chart category and color legend.  [Nhi] Chart rendering automatically use the color assignments in the color legend. The color for each slice is obtained via BIDetailDataPoint.getColor(). There's a lot of charting "framework" code that provides a lot of the charting functionalities that are common to all chart types.
• What should happen when user click on small category. Right now its showing nothing as small values grouped into Other category.  [Nhi] Just default behavior, which is chart appears grayed out. No code changes needed for this. The issue description is updated to reflect this.

We cannot disable the View Data for "Others" because doing so will greatly reduce the value add of the feature to group small values.

When the user clicks View Data, the Summary tab should expand the category into the individual small value categories. The Underlying tab should contain all rows that corresponds to the same. Basically it should behave as if all of the individual small pie slices were selected.

Since the individual cell coordinates are needed for this functionality, then during the grouping logic you have to save the necessary info so that this is available later for calculating the individual cell coordinates. I don't remember on top of my head what info is required, so you'll have to analyze the existing VA code. Please don't study the ADS code as it is irrelevant in VA. The only reason the ADS functionality is mentioned is to show an example of the pie slice that contains the grouped values.

Nhi,

    I tried to implement View data option by storing small values keys and used it while showing data inside view data -> Underlying. It worked but didnt think this will be better approach because existing code uses cell coordinate. So better approach will be like using cell coordinates of small values. I didn't find any way to find cell coordinate by its key.

    Can you please direct me for better approach and how to implement it. 

Code Review (SVN #55536):

1. BIData
   • groupSmallValues()
     • Why is "Double total" recalculating the total? Can it use "_data.getTotal()" instead?
     • The concept of "accordingModel.setOldKeys(...)" and "accordingModel.setOldValues(...)" are incorrect. This assumes that there's only 1 pie chart in the chart view area. This will lead to incorrect behavior for the following scenario:
       • Open pie-chart.vizx in VA
       • Goto Worksheet 2
       • Hover over the green slice in each of the 4 pie charts and select View Data.
       • Result: the content of the View Data is identical for each pie chart, which is incorrect.
       • Expected: the content of the View Data should be the actual content for each particular pie chart.
     • Invoking "valueIterator.remove()" inside the "while(valueIterator.hasNext())" loop is generally not good programming practice. You are simultaneously using the iterator state and modifying the state. I am surprised that this does not result in ConcurrentModificationException.
     • The implementation of this method is incomplete. Trace the implementation of the BIData.doSort() method. It invokes valueObj.doSort(), which also does the same sorting on the "syncList". The grouping of small values has to be done for the "syncList" as well. Because of this, the following scenario fails:
       • Open pie-chart.vizx in VA
       • Goto Worksheet 3
       • Note the data label value for the biggest orange slice is "California : 1,124,349", which is correct.
       • Click on Options, check the "Group Small Values" checkbox, and move the slider a bit to the right.
       • Result: The data label values for the same slice is different from before, which is incorrect. The data label values are actually incorrect for all slices.
       • Expected: Grouping of small values should not change the value of ungrouped values.

Functional Review:

1. Scenario 1:
   • Open pie-chart.vizx in VA
   • Goto Worksheet 1
   • DnD State to Color deck
   • Scroll to the end of the color legend
   • Result: The Other category appears in the color legend
   • Expected: Since the "Group Small Values" settings is NOT turned on, the Other category should NOT be added to the color legend.
2. Scenario 2:
   • Open pie-chart.vizx in VA
   • Goto Worksheet 3
   • Click on Options and check the "Group Small Values" checkbox
   • Move the Small Values slider a little to the right
   • Result: The Undo action is not available to undo each of the previous 2 actions.
   • Expected:
     • Changing the checkbox should produce an undo object.
     • When I move the slider, the undo object should be generated after the mouse is released, thus creating only a single undo even though the slider has be moved over a range of values.
3. Scenario 3:
   • Open pie-chart.vizx in VA
   • Goto Worksheet 3
   • Click on Options and check the "Group Small Values" checkbox
   • Move the Small Values slider a little to the right
   • Save the workbook.
   • Close the workbook and open it in VA again.
   • Result: The "Group Small Values" and the slider are not persisted.
   • Expected: The "Group Small Values" and the slider should be persisted.

Behavior Change Required:

The slider under "Group Small Values" is too sensitive and not user friendly. Please make the following changes:

1. The min value should be 0% and the max value should be 10%.
2. The slider should have a tooltip showing the current value of the slider.

Nhi,

• groupSmallValues()
  • Why is "Double total" recalculating the total? Can it use "_data.getTotal()" instead? SZ : Fixed
• The min value should be 0% and the max value should be 10%. --> SZ : Fixed
• The slider should have a tooltip showing the current value of the slider. --> SZ : Fixed
• The implementation of this method is incomplete. Trace the implementation of the BIData.doSort() method. It invokes valueObj.doSort(), which also does the same sorting on the "syncList". The grouping of small values has to be done for the "syncList" as well. Because of this, the following scenario fails:
  • Open pie-chart.vizx in VA
  • Goto Worksheet 3
  • Note the data label value for the biggest orange slice is "California : 1,124,349", which is correct.
  • Click on Options, check the "Group Small Values" checkbox, and move the slider a bit to the right.
  • Result: The data label values for the same slice is different from before, which is incorrect. The data label values are actually incorrect for all slices.
  • Expected: Grouping of small values should not change the value of ungrouped values.
  • SZ : Fixed

[Nhi]

• The background color of the small values slider should be white, not gray.
• The grouping of smaller values need to be applied to all values in "getSyncObjects()", not _hiddenMeasureValues. Because of this, the following scenario failed:
  • ​Open pie-chart.vizx in VA
  • Goto to Worksheet 4
  • Click on Options and check the "Group Small Values" checkbox
  • Move the small values slider a bit to the right
  • Hover over the California slice
  • Result: The value for Tax Amount in the tooltip is incorrect.
  • Expected: The value for Tax Amount in the tooltip be correct. In this case it should match the last value of the data label for California. See the attached tooltip.png.

Nhi,

    We have fixed with following issues :

• The background color of the small values slider should be white, not gray.
• The grouping of smaller values need to be applied to all values in "getSyncObjects()", not _hiddenMeasureValues. Because of this, the following scenario failed:
  • ​Open pie-chart.vizx in VA
  • Goto to Worksheet 4
  • Click on Options and check the "Group Small Values" checkbox
  • Move the small values slider a bit to the right
  • Hover over the California slice
  • Result: The value for Tax Amount in the tooltip is incorrect.
  • Expected: The value for Tax Amount in the tooltip be correct. In this case it should match the last value of the data label for California. See the attached tooltip.png.

Fixed with following functional comments :

1. Scenario 1:
   • Open pie-chart.vizx in VA
   • Goto Worksheet 1
   • DnD State to Color deck
   • Scroll to the end of the color legend
   • Result: The Other category appears in the color legend
   • Expected: Since the "Group Small Values" settings is NOT turned on, the Other category should NOT be added to the color legend.
2. Scenario 2:
   • Open pie-chart.vizx in VA
   • Goto Worksheet 3
   • Click on Options and check the "Group Small Values" checkbox
   • Move the Small Values slider a little to the right
   • Result: The Undo action is not available to undo each of the previous 2 actions.
   • Expected:
     • Changing the checkbox should produce an undo object.
     • When I move the slider, the undo object should be generated after the mouse is released, thus creating only a single undo even though the slider has be moved over a range of values.
3. Scenario 3:
   • Open pie-chart.vizx in VA
   • Goto Worksheet 3
   • Click on Options and check the "Group Small Values" checkbox
   • Move the Small Values slider a little to the right
   • Save the workbook.
   • Close the workbook and open it in VA again.
   • Result: The "Group Small Values" and the slider are not persisted.
   • Expected: The "Group Small Values" and the slider should be persisted.

Could you please help us to solve following issue  :

1. The concept of "accordingModel.setOldKeys(...)" and "accordingModel.setOldValues(...)" are incorrect. This assumes that there's only 1 pie chart in the chart view area. This will lead to incorrect behavior for the following scenario:

• Open pie-chart.vizx in VA
• Goto Worksheet 2
• Hover over the green slice in each of the 4 pie charts and select View Data.
• Result: the content of the View Data is identical for each pie chart, which is incorrect.
• Expected: the content of the View Data should be the actual content for each particular pie chart.

In this case problem is like when we group small values data into other caordinate then all cell cordinate gets changed. But co-ordinate mapping does not get changed into PivotDataModel. How cell coordinate getting calculated in Pie Chart ? Can you please assist us to solve this problem.

The required implementation is far more complex than we originally thought. The rendering of grouped small values is relatively simple. However the view data, keep only and exclude functionalities are much more complicated to correctly implement. Further, the current code changes also break Merged Measures (Single and Multiple Rings) when Group Small Values are enabled.

Please revert all changes made in this issue. We'll defer this feature to a later release.

Deferred to future release.