This will go out with beta 11
Patch #67 seems to no longer apply cleanly to 10.3
I haven't decided yet whether to make this part of the UI, or make it a test BatchOperation that could be run to clear them out.
Probably a UI would be more visible / discover-able.
This has been created with the exception filters, pagination, and results counters on the BatchOpLog table on the BatchOperation page, and the BatchOperation list on the BatchOperation list page. These have been added as separate issues.
I am calling this done, since the comps are all attached. Thank you to all who worked on it.
This has been released with 1.0.1-beta8 →
This has been released with 1.0.1-beta8 →
This has been released with 1.0.1-beta8 →
The ability to add the following methods to your custom Batch Operation class has been added.
/**
* Optional method that is called once BEFORE the run of the batch.
*
* This can be used to turn on maintenance mode, disable indexing, or
* anything else that needs to be done before running the BatchOperation.
*
* @param array $sandbox
* The sandbox by reference. Used for keeping state in the batch.
*
* @return string
* A message that will be added to the BatchOpLog
*/
public function preBatchMethod(&$sandbox): string;
/**
* Optional method that is called once AFTER the run of the batch.
*
* This can be used to undo anything that was done in preBatchMethod(). Like
* turn off maintenance mode, re-enable indexing, or
* anything else that needs to be done after running the BatchOperation.
*
* @param array $sandbox
* The sandbox by reference. Used for keeping state in the batch.
*
* @return string
* A message that will be added to the BatchOpLog
*/
public function postBatchMethod(&$sandbox): string;
This is resolved and will go out with Beta8
Example of the hook_update
https://www.drupal.org/docs/drupal-apis/update-api/updating-entities-and... →
Skyriter and I discussed this on a call.
He found that sleep() paused the process but when the processing resumed after sleep(), the processing was still slow. It did not speed up the way he saw it happen when the process is actually killed and restarted. This leads me to think that that the sleep() does not allow post processing to run (like search indexing...), because the sleep is still technically using the resources preventing other processes from running, so they stack up.
What we discussed as an option is that we could connect the batch size to how much it does before calling it quits and picks up the next batch when cron runs. This would likely solve the problem but it would add the uncertainty of when the next batch would run and when the entire BatchOperation would be considered complete.
If this is the chosen path there are two issues that would be blocking it
1.
✨
Add an option to run a BatchOperation via cron
Active
2.
✨
Add base fields to BatchOpLog to save items to process, and last item processed
Active
swirt → created an issue.
This article was a nice help https://www.digitalnadeem.com/drupal/how-to-create-a-custom-filter-form-...
and this one will be https://zanzarra.com/blog/drupal-custom-pager-and-table-header-sortable-...
https://www.drupal.org/project/codit_batch_operations/issues/3432548#com... ✨ UX design for script run sub-module Active
I don't have it fully pretty or filterable / sortable but the data and link are present.
BlakeOrgan, kesmith, skyriter and I met and worked through 4 rounds of revisions. Here are the comps for the 4th round that BlakeOrgan created.
- Batch operations list
- Batch operation info page - includes table or revision logs
- Batch operation run confirmation
- Batch processing progress screen (provided by Drupal core batch)
- Batch operations log list
- Batch operation log
I did some work on this last night. It is incomplete, but making progress.
These changes look great. Thank you @skyriter
I am changing the status of this to 'Needs more info' because I think the appearance of this warning is the result of something else and I need more information from the person that saw this originally.
Released as part of 1.0.1-beta7 →
Released as part of 1.0.1-beta7 →
This has been resolved
This was released as part of 8.x-1.10 →
This was released as part of 8.x-1.10 →
This was released as part of 8.x-1.10 →
The notes went out as part of
8.x-1.10 →
However this issue needs more work so making it active.
This was released as part of 8.x-1.10 →
This was released as part of 8.x-1.10 →
This was released as part of 8.x-1.10 →
I just update the code example with your notes, and added a specific FAQ about deduping.. Thank you @RichardDavies.
Merged the initial commit. Setting this back to Active so it will continue to receive patches.
I added a Link to the custom module on The Department of Veterans Affairs.
The GovDelivery Bulletins module was built for the VA, but is generic enough so other government agencies can use it.
Thank you @RichardDavies for all the supporting information. I reworked the admin settings to be more readable and more informative. I also added details to the README (which also become the project page). Thanks to you this module's settings are a lot easier to understand.
I added an FAQ to cover this.
I am working to clear up those descriptions and add something to the README as well.
These settings default to off initially as a safety. Enabling them is intended to be an intentional choice. You would not want test sends accidentally spamming your GovDelivery Bulletin list.
I just fixed the identified points of confusion.
I just committed three FAQs to the README.md that essentially cover these answers. Thank you RichardDavies for taking the time to ask great questions.
If it is bash or cron, something like this ought to give you the timestamp to append.
timestamp=$(date +%s)
Closing this as no longer the right version.
Oh, I remember now... we never built the cron portion. #3099053: Add option for cron processing of queue → It was not a priority for our needs and there are two workarounds.
The queue can be processed in these two rough ways using Jenkins, cron, circle CI , or acquia cloud (or anything else that can call commands).
- A call to `drush queue:govdelivery_bulletins run
- A GET request made to the trigger endpoint `/api/govdelivery_bulletins/queue` uses basic auth will trigger processing of the queue.
And yes this documentation is lacking completely. Sorry about that.
The short version is the queue is mandatory, for reliability. If the send failed for whatever reason (api is down etc) and it was not queued. The send would just be lost and other than maybe an error or exception captured some place, it would not be re-sendable. With the queue it will keep trying them, so nothing gets lost.
The other reason for the queue is performance. Some uses of this module may trigger 100 items that supposed to be sent to govDelivery with a single node save. If it just sent them all on demand, it would be a blocking process, so you would save your node and wait for minutes while it sends 100 or 1000 items. It would lead to both a bad user experience on the site and potentially timeouts that would result in a loss of data being sent.
I'll have to reacquaint myself with the cron setup, but it should just process whatever is in the queue when cron runs. You should never have to push the process queue button except for in your testing during initial setup.
Regarding the lack of pretty-ness to the queue, it is not meant for the average user. It is meant for a developer to see exactly what is being queued and sent. The payload is sent as is. If it was converted to human readable, it would likely obscure a data problem if one existed. The queue list is just so you can see what is there for debugging and initial setup. A way to monitor like perhaps cron is not firing it.
If things are running properly, you will likely have only a few items living in the queue at any one time.
I just moved them all and will close this issue in favor of the children.
I'm just getting started on our GovDelivery integration, so I might have some more suggestions in the future as I continue using the module.
I am happy to help, but please try to create separate issues for each suggestion/question. And if you have discovered an answer and learned by bumping your head into it a few times, please feel empowered to create a Merge Request on the issue.
I am no longer on a project that is using this module so my familiarity with it is diminishing over time.
This one I have looked into, and as you suspected RichardDavies it is a limitation of Guzzle. And I have been frustrated by exactly this issue several times. In some cases the full error can be found in the php log, and sometimes not. I do not recall the specific situation under which they show in logs.
I will leave this issue open because I would like to address it in the documentation.
2a Definitely needs to be addressed.
2b is partially the result of drupal.org Readme conversion, which in this case is a good thing. The project page in this case is pulled from the README.md which also feeds the help page for the module when it is installed. If you go to the help page the links will point to the places in your local site where the links will work properly. On drupal.org when those links are brought into the project page, it makes them non-links, which I think is intentional, but may be a defect.