Using facets on numerical fields makes them disappear

Created on 30 April 2024, about 2 months ago
Updated 10 June 2024, 18 days ago

Problem/Motivation

As shown in 🐛 Facets stop working after upgrade from v1.1 to v2.0.1 Active , https://github.com/meilisearch/meilisearch/issues/4060 leads to empty responses when using numerical fields.
As outlined in https://www.meilisearch.com/docs/reference/api/facet_search, Meilisearch (currently) cannot index numerical values in a useful way.

One option is to index those fields as strings, which can work flawlessly, but can also pose other side effects.
Think of typo tolerance (matching completely different IDs for example could be unexpected) or sorting (which would be alphanumerical, then).

I am unsure what to make of this.

If it stays a non-feature then it should at least be added to the documentation/module's front page.

Maybe there are solutions to this or a discussion with Meilisearch devs makes sense.
I just wanted to note this down so others that may be affected can find this issue.

🐛 Bug report
Status

Needs work

Version

2.0

Component

Facets

Created by

🇦🇹Austria tgoeg

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @tgoeg
  • 🇩🇪Germany hexabinaer Berlin, Germany

    Thanks for noting this! If nothing less it helped me understanding what my current problem is.

  • First commit to issue fork.
  • Status changed to Needs work 20 days ago
  • 🇸🇮Slovenia bcizej

    @tgoeg Thanks for reporting this.

    I have added a draft MR that could be tested. The idea behind the solution/workaround is before indexing items, go through all the fields and for any numeric types add a duplicated field to the index with __facet__ prefix and a value converted to string. Then when generating facets we go through each one and check if any of them are numeric type - if they are run the facet query on meilisearch with prefixed __facet__ instead of the original one. At the end move those __facet__ results to the proper/original facet.

    Patch can be tested but it is nowhere ready for merging or production, it is in mvp state and there are issues with filterableAttributes re-setting sometimes, losing the __facet__ prefixed attributes etc.

    Can be tested by applying the patch, having a numeric field in the index and reindexing the entire index.

  • 🇦🇹Austria tgoeg

    Hi and thanks for your quick patch!

    It will take some time to check this, as I am currently buried in work with no end in sight, can @hexabinaer probably jump in for the time being?

    I think the algorithm is legit and most likely a better approach as it keeps the numeric fields in their correct form and allows sorting, etc. and only converts to strings for the purpose of facetting, which I think should be OK for values with different digits.
    This should definitely be tested, as I also had to workaround this problem recently as I had a numeric field indexed as string for faceting to work properly, but also wanted to use it to sort results. That didn't work out properly, as it was 0<value<100 and so the single digit values were sorted in the wrong way, i.e. alphanumerically, 9 > 80, which is not what visitors expect.
    As you only add a numeric field for faceting, this should be cleaner.

    When looking at the patch, this line jumped out:

    -      $facetResults = $this->getFacetData($query->getIndex()->id(), array_keys($facets), [$filters], $keys);
    +      $facetResults = $this->getFacetData($index, array_keys($facets), [$filters], $keys);
    

    In all the other locations

    $index=$query->getIndex()
    

    but not

    $index=$query->getIndex()->id()
    

    You know I am no real dev so this might be totally OK, just wanted to mention it.

Production build 0.69.0 2024