Erratic results when searching with Live API

Created on 31 July 2024, 5 months ago

Problem/Motivation

When searching with the Live API, sometimes the results are incomplete or missing, particularly over slower internet connections. As the results are cached, if incomplete results are returned they will also be cached, so the same incomplete result will be shown again.

As an example, I set the browser to simulate 3G network speed, and tested in DrupalPod.

I recorded a short video showing a search for the term "Disclosure".

https://www.youtube.com/watch?v=olpryJW8lNU

Steps to reproduce
  1. Open a Project Browser instance either locally of in DrupalPod.
  2. Set the browser network to a slower speed.
  3. Type in the searchbox.

Proposed resolution

One issue is that the display updating is to "jumpy" with the search initiated on every keypress, but the display updating after a delay, so it's not clear which keypress has been processed. This probably will put excessive load on servers when a larger number of users are searching. I would suggest requiring a minimum of 3 characters before searching. I also think it should wait a defined period after pressing a key before searching, If any additional keys are pressed during the wait, the timer would reset. I'm not sure what the delay should be. I'd guess somewhere between 100 and 500ms.

The second issue is the timeout on calling the API may be too short, so the reply is getting missed or truncated. Hopefully the timeout can be increased to mitigate this.

The third is that invalid responses get cached. I think the solution for this is to check that it received a valid response before storing to the cache. Perhaps if the response is invalid we could display it, but then try again after a defined period if there have been no further searches.

🐛 Bug report
Status

Active

Version

2.0

Component

Code

Created by

🇮🇪Ireland lostcarpark

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @lostcarpark
  • 🇮🇳India prashant.c Dharamshala

    I also faced the same issue:

    1. Sometimes the API takes too long to fetch the results, even if the internet speed is good.
    2. In case incomplete results are cached, even with the cache clear it is not resetting the results, I had to reinstall the module
    3. In my case it could only fetch around 24 results out of 8k and pagination till 2 pages worked, The pager appeared but the other pages had not data, trying to replicate to get a screenshot but no success yet.
  • 🇮🇪Ireland lostcarpark

    Interesting point on pagination. I'd suggest it makes no sense to try to send back all the results since there could be a lot, and most users won't go past the first page or two. So the API should return the first X results, and the total number of matches, and the front end should be able to request additional pages as required.

  • 🇮🇳India prashant.c Dharamshala

    I agree, on the initial page load instead of loading all the results we should load only the first page results and load the next results on the fly based on the page number.

  • 🇮🇪Ireland lostcarpark

    Also, there will need to be some form of cache invalidation. If maintainers update their modules and don't see the updates in PB fairly quickly, they will lose confidence in the system. Perhaps the API could send a list of updated module IDs, and PB would discard any results containing those modules from its cache.

  • 🇮🇳India prashant.c Dharamshala

    A few more issues, filters behaving weirdly. Overall very unstable, I would say.

    https://www.awesomescreenshot.com/video/30109604?key=65c4ed47642ae06794a...

  • 🇺🇸United States chrisfromredfin Portland, Maine

    I can't reproduce the spottiness, even simulating 3G right now. AND, there's an open separate issue for creating a UI and a Drush-based way of resetting the Key-Value store.

    So, can this be closed (outdated) or can other folks still replicate the spottiness here?

  • 🇮🇳India narendraR Jaipur, India

    Now, we need to click the search icon after entering text, so it seems all points raised in this issue are outdated. Therefore, this issue can be marked as closed (outdated).

  • 🇮🇳India narendraR Jaipur, India

    Closing this issue as Closed(cannot reproduce),

Production build 0.71.5 2024