Decimal separator and decimals settings ignored when aggregating decimal fields

Thanks for sharing a pragmatic Twig solution in #16 🐛 Decimal separator and decimals settings ignored when aggregating decimal fields Needs work @Ben Greenberg. Drupal 10 is now on Twig 3 → , so the URL is https://twig.symfony.com/doc/3.x/filters/number_format.html, but the syntax is the same.

@akram khan Thanks for the patch, could you explain the changes you made so we are all clear as your reroll added more removals than @ramil g did.

We have various screenshots on this issue, so removing the PHP related issue tag and needs screenshots.

Thanks @ramil g. The test looks great and it's red/green on test-only/with fix. 🚀 it!

Moving priority due to the PHP error that is triggered
https://www.drupal.org/docs/develop/issues/fields-and-other-parts-of-an-... →

Trigger a PHP error through the user interface, but only under rare circumstances or affecting only a small percentage of all users, even if there is a workaround.

I tested on Drupal 11.x today, standard install and reproduce the problem. Applied the diff and the problem was fixed. Updated credit.

There are no screenshots in the issue summary. Ah, I see they are in #44. @asad_ahmed, when adding screenshots it helps everyone working on the issue if they are available from the issue summary. This save reviewers and committers from hunting through the comments to find the correct images.

Can someone explain why the test is not testing the scenario in the Issue Summary? The test seems to be testing a different problem. It is a problem I encountered when testing this. The problem was that I could not change the aggregate settings from 'Sum' to anything else. Settings to NW to understand the testings.

@quietone I don't believe the screenshots in #44 accurately display the root problem for why the decimal settings change between aggregation being on and off.

I added the screenshot from #38 which identifies where this stems from to the issue summary, which displays the accidental copy/paste error in https://git.drupalcode.org/project/drupal/-/commit/684b4a036e736b16d21a1..., which manifests itself into the originally reported issue as well as other Views UI issues including the error I mentioned in #56 when I bumped the priority and which @ramil g wrote the tests against.

Also I added a bunch more screenshots to the IS illustrating both problems.

One thing I don't know how to change the title to indicate the original issue is resolved + issues with any numeric aggregated field? Any suggestions?

LMK if there’s anything I can do to get this patch committed. It should be a slam dunk, but I may be missing something that’s holding it up.

To reiterate, this issue was introduced accidentally due to a copy/paste error, and it has been there for quite some time.
https://git.drupalcode.org/project/drupal/-/commit/684b4a036e736b16d21a1...

@joelpittet I don't think there was anything in particular holding it up, but I think it's a victim of the following:

1. The RTBC queue has been constantly fluctuating between 75-150 for several months, it feels impossible to get it under 50, so it's easy for individual issues to slip through when the list is very long. This is despite ~150 commits to 11.x in the past month and however many other issues moved to needs work in the same period.

2. It was RTBC and un-RTBCed a couple of times, that meant it wasn't the oldest issue in the RTBC queue until this week, even though it was more-or-less RTBC for a lot longer.

This is the search I use when trying to approach the RTBC queue FIFO:
https://www.drupal.org/project/issues/search/drupal?text=&assigned=&subm... →

The 'status changed' column is usually a good indicator, but it's not perfect - because it counts from the most recent status change, not the first change to that status. On the other hand, there are issues that were first RTBCed five years ago and have 100 comments since because they were never really RTBC, so 'first RTBC' would also not be accurate for different issues.

3. The title made the issue look a lot more complex than it was, so at least for me I expected to be looking at it for at least an hour before I'd feel comfortable committing it. It can take longer to get to those issues. Obviously a quick scan of the MR would have indicated it's pretty straightforward but you have to look at them first, and that goes back to point #1 and #2.

None of these are good reasons for an issue to get held up, but I think they probably are the reasons.

Committed/pushed to 11.x and cherry-picked to 11.1.x, 10.5.x, and 10.4.x, thanks!

catch → closed merge request !10837

Thanks for the context, @catch—much appreciated! I was pretty sure it would get in eventually, just didn’t want it to be one of those “not really RTBC” cases.

I’m really glad this bug is finally squashed—I kept running into it with aggregation enabled on all my migrations!

Automatically closed - issue fixed for 2 weeks with no activity.

I know this is probably not the right place but in case someone else has the same problem as me, I just wanted to note that this fix breaks the SUM and AVG functions on the commerce_order.total_price__number field using 10.4.6 and Commerce 3.x. Putting back 'field' => 'numeric' in getAggregationInfo() for SUM and AVG makes them working again for the commerce_order.total_price__number field like before. I wanted to share this even if it is only a temporary "bandaid" solution for a couple projects, I will investigate further for a better long term alternative when I have time. Thank you.

This has also caused a regression for us with a pretty simply view after upgrading to 11.1.6

The view uses a COUNT aggregation on the changed field to output the number of nodes changed per month for a given date range. In 11.1.5 this output a numeric count, now it outputs the formatted changed date.

Reverting this commit fixes it again.

I've attached an example view. Should this be reverted?

Steps to reproduce:
- Install standard on 11.1.6
- Install rest module
- Import attached view
- Go to /test/count
- Notice HTML in changed_1 field
- Revert commit
- Notice counts in changed_1 field

Reading through this issue a bit more, it seems like from the last screenshot in the IS the field handler should not have been removed from COUNT and COUNT DISTINCT?

Tracking the regression over in 🐛 [regression] COUNT aggregation no longer outputs count value for some fields Active I've got a test case going so far.

I'm afk at the moment but this needs a revert. See related issue.

The further I dive into this the more I'm seeing that this should be reverted (x-posted with @catch).

We do want to use the Numeric plugin for displaying aggregation data, configuring entity field based formatter settings that get applied to the aggregated value is never going to work properly for all cases.

The real bug here is why the Numeric plugin isn't being used when the field is first being added - this is what causes the original WSOD error because it's using EntityField::buildGroupByForm, then the next time you edit it it's using the parent (HandlerBase) and submitGroupByForm doesn't have the group_columns fields which then passes NULL to array_filter.

We already have some special handling in the Numeric field plugin around float precision when $this->definition['float'] is set, so maybe we can figure out how to use that to allow Decimal settings in aggregation output.

We also obviously need a lot more test coverage for aggregation output, some of which can be seen over in 🐛 [regression] COUNT aggregation no longer outputs count value for some fields Active

Revert MR up

Maybe these errors are connected to this long standing issue: https://www.drupal.org/project/drupal/issues/2230909 🐛 Simple decimals fail to pass validation Needs work

Hello,

I've just updated from 10.4.5 to 10.4.6 and it breaks my view that uses aggregation set to "count DISTINCT". The format plural configuration has gone since the 'field' => 'numeric' key/value has been removed from the "count_distinct" under getAggregationInfo() method.

I was using the count DISTINCT and the format plural to display the nodes referencing the listed nodes.

Is there an alternative way to do the same now that the 'field' => 'numeric' is removed, or the removal was a mistake?

More info on this issue I've just opened: Upgrading from 10.4.5 to 10.4.6 removes views format plural on Content ID field → .

Thank you.

RTBCing the revert MR.

This reportedly also caused 🐛 hook_views_post_execute(ViewExecutable() not working after update to 10.4.6 Active .

Committed the revert MR to the four relevant branches - this will go out in the next patch release. Back to needs work for the original problem.

@catch, It looks like there may be a regression—thanks for catching that.

Ideally, we'd add more tests here to show the new regression, no? The test in here shows the regression we'd been living with for a while.

See the screenshot of the various problems this solves in #58 #2735997-58: Decimal separator and decimals settings ignored when aggregating decimal fields → on top of it looking to be a copy/paste mistake (I keep saying that but looking at the diff it might be hard to see what I am saying), I wish dawehner could confirm this... sigh

joelpittet → reopened merge request !10837

There's test coverage for at least one of the bugs this caused in https://git.drupalcode.org/project/drupal/-/merge_requests/11761/diffs I agree we should try to get that in before this issue gets worked on again, but ideally we have even more tests to cover the issues other users have reported.

I don't think this is a copy-paste mistake at all, this is intended and required for aggregation output to be functional, see my comment in #74

@acbramley see the test we have here, there was a regression fixed here as well. The screenshot shows the field handler was copied to all the handlers where in D7 it was only on 2 (that's why it appears to be a copy/paste mistake, it wasn't copied over like that on any of the other related commits at the time, in my deep dive/hunt for the root of the problem). Removing them let the default shine through (EntityField in our case).

https://git.drupalcode.org/project/drupal/-/commit/d15e2ea581dac9e112f15...

The here is the field we are testing aggregation on (id).
https://git.drupalcode.org/project/drupal/-/blob/11.x/core/modules/views...

Sorry if I sound defensive, I am in a rush to get out the door... I will look a bit later to see if I can understand a bit deeper what is going on between the two regressions. It feels like it needs to be some sort of fallback in my guess, but it is likely way more complicated than that.

Removing them let the default shine through (EntityField in our case)

That's my point, you can't use those views handlers on aggregated fields (see my comment above for reasoning)

It works for id because it happens to work on certain fields, see 🐛 [regression] COUNT aggregation no longer outputs count value for some fields Active for more info on that.

Also the test you're pointing to that uses https://git.drupalcode.org/project/drupal/-/blob/11.x/core/modules/views... doesn't actually test the output of the view itself, just what's on the row result, which isn't actually what gets output. E.g the row result for the timestamp aggregation is correct, but the output isn't.