Support '*' as value in manually created Search API queries

Created on 13 March 2024, 10 months ago
Updated 25 April 2024, 8 months ago

To create range queries like ds_changed:[2024-03-04T00:00:00Z TO *], Solarium expects a NULL as the second value:

$query->addCondition('changed', ['2024-03-04', NULL], 'BETWEEN');

For a netter DX we should also support '*' as value:

$query->addCondition('changed', ['2024-03-04', '*'], 'BETWEEN');

I am facing an issue with the addition of this commit from https://www.drupal.org/project/search_api_solr/issues/3400167 🐛 Filter value formatting might fail on custom Search API field types Fixed when using custom Filter on date_range fields. The field indexed in Solr is of type Date range. I use Drupal 10.2.3 with Search api solr 4.3.2
Everything worked as expected in Search api solr 4.3.0 version.

Steps to reproduce:
1. Set the "date range from" value to a specific date, e.g., "2024-03-04T00:00:00Z".
2. Set the "date range to" value as "*"(indicating a undefined range). - the formatFilterValue function returns 0 instead of the value itself.
I believe this case needs to be treated because Solr throws an exception in those cases:
"error":@ "msg":"Wrong order: 2024-03-04 TO 0000", "trace":"java.lang.IllegalArgumentException: Wrong order: 2024-03-04 TO 0000\n\tat org.apache.lucene.spatial.prefix.tree.NumberRangePrefixTree.toRangeShape(NumberRangePrefixTree.java:108)\n\tat

I would really appreciate if this can be solved since I am getting no results in my search.
Thank you!
Ioana

Feature request
Status

Fixed

Version

4.0

Component

Code

Created by

🇷🇴Romania ioana apetri

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @ioana apetri
  • 🇷🇴Romania ioana apetri

    So far, I've found that if I get rid of the next plus, the normal flow works perfectly.

    --- a/src/Plugin/search_api/data_type/DateRangeDataType.php
    +++ b/src/Plugin/search_api/data_type/DateRangeDataType.php
    @@ -11,7 +11,7 @@
    * id = "solr_date_range",
    * label = @Translation("Date range"),
    * description = @Translation("Date field that contains date ranges."),
    - * fallback = "date",
    + * fallback_type = "date",
    * prefix = "dr"
    * )
    */

  • 🇷🇴Romania ioana apetri

    The fix for the issue I described above could be this patch. Can be reviewed?
    Thank you!

  • 🇵🇹Portugal joao.ramos.costa

    Hi @ioana petri,

    I was having the same issue in a project and hadn't updated from 4.3.0, waiting for time to look into the matter, but I can confirm your patch resolved the problem and solr_date_range with a custom filter using undefined range works smootly.
    Solr version: 8.7.0
    Search_api_solr: 4.3.2

    Let's let more people test this and change the ticket status.

    Thanks a lot!

  • Status changed to Needs work 9 months ago
  • 🇩🇪Germany mkalkbrenner 🇩🇪

    I think there's some misunderstanding here. (Nevertheless there might be an issue with the code.)

    If you have a date field in the index, you can do range queries like [X TO Y].
    If you have a date range field in the index, you usually query it with a specific date.

    Querying a data range field with a range query might be supported by Solr, but is not very common. At least we don't support it via the module at the moment.

  • 🇧🇪Belgium herved

    Patch #4 works well for me.
    We could also add a test, possibly in \Drupal\Tests\search_api_solr\Kernel\Processor\DateRangeTest

    #6: I can also confirm 🐛 Filter value formatting might fail on custom Search API field types Fixed introduced a regression, as we are not able to pass * for date range fields anymore.
    I suppose not a lot of sites use date range fields but passing * when querying such fields is very common https://solr.apache.org/guide/8_7/working-with-dates.html#date-range-for...

  • 🇩🇪Germany mkalkbrenner 🇩🇪

    I agree that querying a date range field with * is common and must be supported. And yes, the patch in #4 might fix that.

    But it is something else than reported in the issue description!
    The description is about querying a date range field with something [2024-03-04T00:00:00Z TO *], which I think should not be supported.

    A query like [2024-03-04T00:00:00Z TO *] should only be supported for a date field, which stores a single date, not a range.

  • 🇩🇪Germany mkalkbrenner 🇩🇪

    The patch in #4 is wrong as it would allow queries like > *

  • 🇩🇪Germany mkalkbrenner 🇩🇪

    BTW the already existing way in Solarium and Search API is to pass NULL, not '*'. Solarium converts NULL values into '*' in range queries!
    Nevertheless we could add a small conversion code in search_api_solr in case someone creates a Search API Query manually instead via views.

  • Status changed to Needs review 9 months ago
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.2 & MySQL 8
    last update 9 months ago
    31 pass
  • 🇩🇪Germany mkalkbrenner 🇩🇪

    After reviewing the code, I'm convinced we don't have an issue here, but a support request. '*' was never supported in our queries, but NULL.

    So I converted this issue into a feature request, to additionally support the wildcard character '*' for range queries as input.
    And I added some more tests.

  • 🇧🇪Belgium herved

    Thanks for your input,

    The description is about querying a date range field with something like [2024-03-04T00:00:00Z TO *], which I think should not be supported.
    A query like [2024-03-04T00:00:00Z TO *] should only be supported for a date field, which stores a single date, not a range.

    I'm confused. From Solr's doc:

    Solr’s DateRangeField supports the same point in time date syntax described above (with date math described below) and more to express date ranges.

    So I understand that [2024-03-04T00:00:00Z TO *] is perfectly valid, which would filter on >= 2024-03-04T00:00:00Z
    Or am I missing something?

    solr.DateRangeField was implemented in https://issues.apache.org/jira/browse/SOLR-6103 which does mention such ranges.

  • 🇩🇪Germany mkalkbrenner 🇩🇪

    date range queries vs. date range fields vs. Search API DateRange Processor

    These are all different things even if they all contain data range in their name.

    So I understand that [2024-03-04T00:00:00Z TO *] is perfectly valid, which would filter on >= 2024-03-04T00:00:00Z

    Correct, but >= * isn't.

    I'm confused. From Solr's doc:

    Solr’s DateRangeField supports the same point in time date syntax described above (with date math described below) and more to express date ranges.

    Yes it supports the same point in time date syntax when writing value to the these fields at index time!
    If you index the date range [2003 TO 2005] you could perfectly query it like my_date_range_field:2004.
    But if you query it like my_date_range_field:[2000 TO 2004], the result is kind of undefined as 2003 is in that range while 2005 is not.

  • 🇧🇪Belgium herved

    If I understand correctly, solr.DateRangeField uses the spatial functionality.
    So in your example it would return it, since they intersect. unless I'm mistaken?

  • Status changed to Fixed 9 months ago
  • 🇩🇪Germany mkalkbrenner 🇩🇪

    If I understand correctly, solr.DateRangeField uses the spatial functionality.
    So in your example it would return it, since they intersect. unless I'm mistaken?

    You could create the query using a Search API query and it will be executed (like before with NULL instead of *) if you use Solr as backend. And you simply get the result Solr returns.

    But it is undefined in Search API itself as the database backend doesn't support it.

  • 🇧🇪Belgium herved

    We are using solr of course, and a custom facet, querying a date_range solr field ().

    Patch #12 is not working for me.

    +++ b/src/Plugin/search_api/backend/SearchApiSolrBackend.php
    @@ -3472,7 +3474,16 @@ class SearchApiSolrBackend extends BackendPluginBase implements SolrBackendInter
    +      if ('*' === $value) {
    

    Unless.. was is meant to be if ('*' === $v) { ?

    Thanks for your support!

  • 🇩🇪Germany mkalkbrenner 🇩🇪

    This has been corrected in git already before the release. We have tests, at least some ;-)

  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024