Why is there a queue?

Created on 19 April 2024, 7 months ago
Updated 21 April 2024, 7 months ago

Problem/Motivation

3. I don't think I fully understand the purpose of the queue functionality or how you envisioned users using it.

a) I want to automatically send bulletins from the website, so requiring a user to go to the queue and click a button to actually send the queued items is an extra step I'd like to avoid. How can I disable the queue functionality so bulletins are immediately sent?

b) Furthermore, the way items are presented on the queue page is not very user friendly. The raw XML is displayed which isn't that helpful to most users and is very verbose so a queue with just a handful of items becomes a rather lengthy page. I expected to be able to edit and/or remove items from the queue, but you can't. You can only send them, which seems rather limiting.

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

πŸ’¬ Support request
Status

Fixed

Version

1.0

Component

Documentation

Created by

πŸ‡ΊπŸ‡ΈUnited States swirt Florida

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @swirt
  • πŸ‡ΊπŸ‡ΈUnited States RichardDavies Portland, Oregon
  • πŸ‡ΊπŸ‡ΈUnited States swirt Florida
  • πŸ‡ΊπŸ‡ΈUnited States RichardDavies Portland, Oregon
  • πŸ‡ΊπŸ‡ΈUnited States RichardDavies Portland, Oregon

    This is probably my biggest sticking point with the module right now (figuring out how to bypass or automate sending the queue) so any advice here would be greatly appreciated. Thank you!

  • πŸ‡ΊπŸ‡ΈUnited States swirt Florida

    The short version is the queue is mandatory, for reliability. If the send failed for whatever reason (api is down etc) and it was not queued. The send would just be lost and other than maybe an error or exception captured some place, it would not be re-sendable. With the queue it will keep trying them, so nothing gets lost.
    The other reason for the queue is performance. Some uses of this module may trigger 100 items that supposed to be sent to govDelivery with a single node save. If it just sent them all on demand, it would be a blocking process, so you would save your node and wait for minutes while it sends 100 or 1000 items. It would lead to both a bad user experience on the site and potentially timeouts that would result in a loss of data being sent.

    I'll have to reacquaint myself with the cron setup, but it should just process whatever is in the queue when cron runs. You should never have to push the process queue button except for in your testing during initial setup.

    Regarding the lack of pretty-ness to the queue, it is not meant for the average user. It is meant for a developer to see exactly what is being queued and sent. The payload is sent as is. If it was converted to human readable, it would likely obscure a data problem if one existed. The queue list is just so you can see what is there for debugging and initial setup. A way to monitor like perhaps cron is not firing it.

    If things are running properly, you will likely have only a few items living in the queue at any one time.

  • πŸ‡ΊπŸ‡ΈUnited States swirt Florida

    Oh, I remember now... we never built the cron portion. #3099053: Add option for cron processing of queue β†’ It was not a priority for our needs and there are two workarounds.

    The queue can be processed in these two rough ways using Jenkins, cron, circle CI , or acquia cloud (or anything else that can call commands).

    1. A call to `drush queue:govdelivery_bulletins run
    2. A GET request made to the trigger endpoint `/api/govdelivery_bulletins/queue` uses basic auth will trigger processing of the queue.

    And yes this documentation is lacking completely. Sorry about that.

  • πŸ‡ΊπŸ‡ΈUnited States RichardDavies Portland, Oregon

    Thanks for the explanation. The performance and reliability aspects make sense, as well as the UI only being a developer/debugging tool.

    I agree, it would be ideal if there was a setting to have the cron send any queued items. But we can workaround that limitation using one of the methods you suggested. The second method of using the API trigger endpoint does require a Unix timestamp (it would be nice if it defaulted to the current time) but I suppose we could just use a very far future timestamp in every request if we don't want to have to programmatically calculate a timestamp.

  • πŸ‡ΊπŸ‡ΈUnited States swirt Florida

    If it is bash or cron, something like this ought to give you the timestamp to append.

      timestamp=$(date +%s)
    
  • Status changed to Fixed 7 months ago
  • πŸ‡ΊπŸ‡ΈUnited States swirt Florida

    I just committed three FAQs to the README.md that essentially cover these answers. Thank you RichardDavies for taking the time to ask great questions.

  • Status changed to Fixed 7 months ago
  • πŸ‡ΊπŸ‡ΈUnited States swirt Florida

    This was released as part of 8.x-1.10 β†’

Production build 0.71.5 2024