Update to version 2.2 results in error warning updating database.

You jumped versions and didn't update your database to account for the change made between 2.1.3 and 2.1.4.
Update to 2.1.4 first and make sure you do a database update.

This still feels like an issue with the module itself as composer require 'drupal/honeypot:^2.1' will automatically update to 2.2 which seems 2.2 should include whatever update hooks going for 2.1 to 2.2 instead of creating this error situation.

As I did not set composer to honeypot^2.2 but because of how the versioning system works composer it will automatically jump subversions. Hence my intentional comment that 2.2 should be a 3.x version as that would greatly reduce the likelihood of others have this same issue.

There is also the issue that the current composer require text for 2.1 needs to be updated to "composer require 'drupal/honeypot:2.1.4' " as the current text will not grab the 2.1 version but the 2.2 which has a high chance of creating this same issue for others.

I agree with @staris.

Database breaking change should go in a new branch. Here you should have created branch 3.0.x.

As noted by @staris, composer will automatically jump to version 2.2.0 and the update will fail.

I have the same problem.
I made an update in the old way, admin-extend- available updates and it broke with the same error. The update was recommended from 2.1.3 to 2.2.0. (see screenshot)

Hello,

Waiting for a corrective version, just force update to version 2.1.4 with the DB update, then update normaly ... and all is fine. You can do this even you previously update to 2.2.0

composer require 'drupal/honeypot:2.1.4'
drush updatedb
drush cr
composer require 'drupal/honeypot'
drush updatedb
drush cr

I have this error too. While there is a work-around, it makes the upgrade unnecessarily time consuming. Maintainer should not have removed old version 2.x db update functions until branch 3.x.

Confirming this is not a good solution, and I think it would still make sense to have a 3.x branch for this breaking change.

This causes a lot of manual work if using ^2 or ^2.1 which is absolutely typical. Any maintainer plans?

I added the issue template and a workaround / fix to the issue summary:

composer require drupal/honeypot:2.1.4
drush updb
composer require drupal/honeypot:^2.2

Just want to flag that folks noting solutions like this:

composer require drupal/honeypot:2.1.4
drush updb
composer require drupal/honeypot:^2.2
drush updb

Is really probably missing quite a few steps for most people. I think most people update on a local environment and then deploy changes to development, test, and live environments. Note that you'll need to deploy 2.1.4 and run drush updb all the way to your live environment before updating to 2.2. - I updated the issue summary with this note.

Also, it would be helpful if somebody updated the 2.2.0 release notes to note these required steps.

I'm going to mark as Needs Review because that feels most appropriate to me.

@sonfd yes of course, was just the most simple example (while most simple means the last drush updb could be left out currently). Better than a release note notice would be to improve the situation for anyone who hasn't updated yet... PLUS release note information.

@Anybody Agreed 100%. Just wanted to make sure that it was flagged for folks less experienced with updates - It's easy for folks to get tripped up when making updates and deploying to another environment. (For example, it's not uncommon to see an uninstall and composer remove in the same commit / deploy.)

Actual status is “Needs work” as you outlined work to be done.

Hi @sonfd

Is really probably missing quite a few steps for most people. I think most people update on a local environment and then deploy changes to development, test, and live environments. Note that you'll need to deploy 2.1.4 and run drush updb all the way to your live environment before updating to 2.2. - I updated the issue summary with this note.

I think it is what I already specified on my post #6, right ?

@TR As others have pointed out this is a breaking change in a minor release which is not expected according to semantic versioning. For people using automated deployment system, they will have to specifically require and deploy 2.1.4 or ~2.1.4 to move to another feature release. If there are incompatible changes to previous versions it should be done in a new 3.x major release.

As the change from #3464350 📌 Remove obsolete update hooks and update tests Fixed is due to the update hooks "not [being] relevant anymore in 2.2.x", is there a chance that we could get the commit 4be4c190 reverted in the major version 2.x, with a new patch release (e.g. 2.2.1), so there are no breaking changes for people updating on 2.x? In my opinion some "unnecessary" code is better than breaking changes on such a wide-spread module.

I've also just run into this issue having been upgraded directly from 2.1.3 to 2.2.0, with my composer constraint being "^2.1" for honeypot.

Expecting people to do multiple deploys for a minor release of a module isn't reasonable. Please revert the changes in #3464350 to put the update hooks back.

Personally, I'd be inclined to leave old update hooks in for an entire major release of a module, so people are free to update from any 2.x branch to 3.x.

Having a smooth update method is important, when having 100+ , 1000+, or more.

Drupal 11 was released early.
Perhaps a new 3.0.x branch for Drupal 11 to avoid complicating the update process.
To have any clean up of old code, and drop support for Drupal 9
- If no changed in features or logic, the 2.2.x feels logical too

Personally, I would like the 3.0.x if we do have a new logic.

In all cases, thank you for taking the time to maintain such an important module.

Should the hook_update_last_removed() implementation be removed from 2.2.x? My understanding is that the hook is to ensure that updates are not skipped if they are removed and required. Since these database updates are not required for 2.2.x, the last removed update probably does not need to be reported. I assume the updates can remain in 2.1.x, and if they are removed, the last removed would need to be set there (unless they are not required for the latest 2.1.x version either).

We ran into this same issue. I think for the moment I'm only going to update to 2.1.4 in this deployment and we can do the 2.2 update at a later time (rather than needing to do 2 separate deployments right at the same moment).

Hah, i ran into this on a site we update only intermittently and uninstalled and reinstalled rather than mess with the update hooks. But then i ran into this on a site we update like every couple weeks and came to look for this issue!

Agree that these update hooks should be re-added and kept working throughout the 2.x release cycle.

I agree with @tr for this issue we need two deployment’s.

Answer #6 worked well for me. Thank you so much @larryla

Thanks @slown

It's just unfortunate that there is no reaction from maintainers on this thread, and by putting a new version correcting the problem for those who will update later.

What reaction are you looking for? You need to update to 2.1.4 before you update to 2.2.0. That's what I said in response to the initial post, and I said it within a half hour of when it was posted.

If you're waiting for a different answer, it's not coming, because I gave you the answer.

The suggestions above are short-sighted. Create a new major version? Well, aside from the fact that there are no API changes or "database breaking" changes or anything that would justify a new major version, you would STILL have to update to 2.1.4 before going to 3.0.0. So that doesn't solve anything does it? Don't remove the update functions? Well then they need to be ported to D11, just like they were ported to D10 and D9 and D8. When does it stop? Changing the composer command suggested on the project page? Not in my control - that's automatically generated and filled in on drupal.org. Add warning text to the release notes? Do you think anyone reads those *before* they have a problem?

Drupal has a well-known built-in mechanism for removing update functions so they don't have to be kept around forever, and that well-known built-in mechanism was used here. Just like it is frequently used in core to remove update functions between minor core releases.

The "error" message you see is the safety mechanism Drupal core uses to keep you from skipping a needed update. You're *supposed* to see it if an update is skipped.Not everyone is updating from 2.1.3 you know, some are updating from 2.1.2, 2.1.1, 2.1.0,

And 2.1.4 is marked as a "recommended" release on the project page, while 2.2.0 is *not*.

So what do you suggest as a course of action? Maybe composer shouldn't be automatically installing a release that isn't marked as recommended. Maybe the UI shouldn't be either. Neither of those things can be changed by the code in this module. Regardless, there is a simple step that you can take if you see this "error" message, and that's make sure you upgrade to 2.1.4 first, and run the database updates, before you upgrade to a higher version. No other action is necessary. If you have some clever solution that can ensure that, then I am all ears, but I don't know of one.

#6 is a good work around for this issue. It fixed it for me.

In my opinion, the reason for a new major version is exactly what was described by TR: 2.2.0 is not recommended version. Without it, composer would update to 2.2.0 and Drupal will be listing the module on the /admin/reports/updates/update list.
What's more - platforms like Pantheon, that has built-in automated process for update will fail on this and it will require manual updates to one version and next one after that.
And website owners would have to define version in composer.json to prevent the upgrade from hapening, if the newest version is not the one that is recommended.
Also, solution from #6 is a workaround but it causes that PROD upgrade would have to be split into two releases. Woth the major version it also would be two releases, but at least there wouldn't be automatic jump between versions.
And, BTW, some people read all release notes - for example I did. And I didn't see a warning that this update should be done version by version :)

So, as you said TR - it is not in scope of this module to change the way how composer or Drupal updates work, but I think the way how module is released should consider how composer works (even if we think it is not perfect).

@TR

- Using composer update on CI/CR (Continuous Integration/Continuous Release) system breaks the site with a 500 error, forcing site admins to 'composer require drupal/honeypot 2.1.4' instead of “composer update”. Fortunately, most of those installation will be tested in a pre-prod environment. But nevertheless, it requires two deployments, even on prod sites.
- The Automatic upgrade project (soon to be part of core) would not know not to install 2.2.0 before going to 2.1.4. Automatic upgrades does not look for “recommended” because “recommended” is not marked anywhere in the code. So there is no way beforehand to know of this issue. So those using Automatic upgrade will fail.
- Drupal 10.1, 10.2, 10.3, 10.4 never removes any updates, they are removed only on the next major branch like 11.0. This is why it is required to update to 10.3 before upgrading to 11.0.

The fix is quite trivial, port back updates to the 2.2.x branch and create a new 3.0.x branch without the updates. That way you match Drupal core behavior and CI/CR systems are not broken. Furthermore, following that rule will make Automatic Update (soon is core), do it's magic.

Sorry @tr for my late reply to your post #25
I was just expecting a minor version 2.2.1 which includes a test of the version of the module data into the current DB to know if it needs to be updated or not ... not for me (updates are done), but for those who don't have updated yet and to avoid them to lose time with an extra manual process. That's all :-)

I agree with @larryla. I think that's the best path forward. The sooner that's done, the fewer people will have this issue.

Ok, after reading this and after reviewing further in the maintainer slack I think there are a few additional points to make.

This issue probably belongs in closed works as designed, but personally I'd leave it open for discoverability.

I've updated the workaround to include the note about deploying between the composer commands.

In general I think hook updates removed should generally be reserved for major version updates. This is not documented anywhere that I can find though so this is left up to the maintainer (in this case @TR).
After further discussion I think @TR made the right decision.

For updating honeypot for Drupal 11 there were three possible paths.
1. Update 2.x to support Drupal 11 dropping old update hooks and using update hooks removed.
2. Create a 3.x that supports Drupal 11 with update hooks removed (2.x would not support D11 in this scenario)
3. Upgrade 2.x to support Drupal 11 and keep update hooks (including updating tests for these update hooks). Then create a 3.x release that has update hooks removed.

While this thread may not realize it, it is asking for option 3 which is more work than is being suggested in this thread. The tests all would need to be updated for Drupal 11, not just added back in, there is the additional concern about what happens if you remove update hooks last removed, do the newly readded updates re-run? Will that break more things?

In this scenario as a user I'd prefer option 2 over option 1, but as a maintainer I fully understand and would have probably chosen option 1 myself.

This is an inconvenience as a consumer of the module, but many times I feel like option 2 which is a new major version of the module to support the new major version of Drupal to be more disruptive when updating major versions of Drupal.

This issue has a one time workaround so I'd vote for documenting it and leaving it be.

@nicxvan You must also take into account composer updates and upcoming automated updates. Those two only look at the main version number.

If, as a maintainer you provide option 1, for sure you will break many pipelines and automations.

We are not discussing tests here, but the fact that composer automatically jumps to a version that will break because missing updates.

Also a module supports only Drupal 11, Composer will not try to update the module on a Drupal 10 site. If a new version of a module supports only Drupal 11, we do not need a new branch. We can only change the info and composer.json file to require D11, even if the maintainer release a very minor version, like 2.1.5. In that example, 2.1.4 supports D10 and 2.1.5 supports D11.

Fair point.

However, there are no automatic updates yet and for this specific case the change is already released. I don't think you can cleanly roll that back.

Maybe once automatic updates are here the major update will be the way to go.

This would need to be documented for module maintainers though.

I think option 2 would have preserved automatic updates and prevented this composer issue. But like I said, can you cleanly rollback removing update hooks?

I don't think you can cleanly roll that back.

Fair point! (sorry for using the same acknowledgment 😀)

We should remember that this is not only Drupal automated updates but all processes that might be used for upgrading the website, that are impacted here. Considering changing in the versioning requires a lot of effort and also, probably shouldn't be done to already existing versions, maybe at least update of the release notes for version 2.2.0 https://www.drupal.org/project/honeypot/releases/2.2.0 → can be done to help people who may run into this issue?

#6 worked great for mine

Considering that composer with a version range of "drupal/honeypot": "^2.1", jumps from 2.1 => 2.2, we should strongly consider that removing install hooks between those 2 minor releases will lead to exactly the problems seen here. For our site, we have to pin this directly on composer require drupal/honeypot:2.1.4, roll out a release to live. Then switch back to ^2.1. I think this issue here isn't that there aren't BC breaking changes in functionality of the module but that composer itself sorta doesn't support what was done. It essentially makes that hook_update_last_removed a BC breaking change. Composer also doesn't care what is set as the default version on the project page on d.o. It only looks at versions in composer.json.

If, however, this module is going to introduce BC breaking changes like this in minors, I don't think as a site owner I have much choice but to do what the maintainer has asked and roll back to a pinned version.

thanks @larryla in #6 for Drupal 10.3.5 :)

#6 works for me too. Thanks @larryla.

I agree with Lucas #13
We did exactly what you said in all our projects

Updated the Honeypot module from ~2 to ~2.1.0 as a TEMP step for a smoother update process.

Proposed resolution

• Change drupal/honeypot to ~2.1.0 in the composer.json file, to ensure a smoother update process.
• This is a TEMP change to have the first round of updates and deployment in a smoother way
• Also after a couple of months ( with all our projects updated ) we can switch to "drupal/honeypot": "~2.2.0", to pave the way to upgrade to Drupal 11 by then.

Nice work @ajab natshah!

Adding your comments to the IS. Feel free to update it at your convenience.

Please confirm that we have enough information captured (IS and comments) in this ticket to mark it as fixed.

Cleaning up the Issue Summary might help discover this issue, though.

Yeah, there is a workaround and that's nice, but we really shouldn't normalize this kind of thing as a reasonable way to do releases. Minor versions of a module should include all database updates necessary to upgrade from one minor version to another.

100% agreed that dropping update scripts in minor releases (or patch releases for that matter) is a bad decision and will cause more problems for site admins in the long run, especially folks who are less technically minded.

+1 with avoiding this in the future, this is a BC break and needs a major release.
Having to pin/unpin in composer is tedious.
If every module were to do that, it would be quite a hassle.

I completely agree with @damienmckenna, @herved, @loopy1492, and others - removing update hooks is a bad idea.

-mike

+1000 :-) this is not a great way of having to do multiple deploys to upgrade the module

Oy vey! I was struck by this one as well :)

Just wanted to add another confirmation that (a) @larryla's solution in #6 fixes this issue, but is a bit of a pain if you need to deploy the fix through multiple environments (especially for multiple sites), as you need to hop on two separate release trains. And (b) I agree with some of the others above that update hooks should not be removed in a minor version.

Of course it's the maintainer's prerogative, but it does create a bit of friction for all module users who don't update every point release :)

I'm still curious about my question from #19 of whether update 8104 is actually required before updating to 2.2. If not, couldn't the last removed be set to 8103, and most sites would avoid this issue. 8104 is just a cache clear if the Tour module was installed, and I'm not sure why you would need to ensure that had happened before updating to 2.2.

The workaround described in the issue worked perfectly for me. Downgrading first to version 2.1.4, running the database updates, and then upgrading to 2.2.0 resolved the problem without issues. Thanks for documenting this!

Wow. #25 sounds so rude and so dense (and actually wrong) compared to how things were said by others.
What reaction was he looking for?
Maybe he just expected you to admit you did not do things the best way and to consider doing them differently next time?
I'm not sure what's worse here, the way you handled this technically or your reaction.
I will certainly be less inclined to recommend the use of this module after reading this.

@pbouchereau : You react to a perceived "rude" post by insulting my intelligence and competence? So much for raising the tone ...

My post in #25 was in response to #24 where I was accused of "no reaction" and "not fixing" the issue.

For the record, I DID post a complete and concise response to the original problem in #2, which I posted exactly 24 minutes after the issue was opened. I call that EXTREMELY RESPONSIVE. I deliberately waited to release 2.2.0 until I knew I had time to deal with unexpected problems, and my quick response shows that I was on the job. (A job I do completely as a volunteer by the way - I don't get paid to do Drupal in any way shape or form.)

My "What reaction are you looking for?" was a rhetorical question, because obviously I HAD already given a response and IMO no further response was needed. Evidently the poster didn't want to accept the response, or had seen the previous 22 posts and assumed they were all viable "fixes". Which is why I followed with a detailed and accurate post about why all the suggestions for "fixing" the issue were non-starters.

And I DID worry about potential problems with the 2.2.0 release, (and I expressed that worry in the issue queue), given that virtually NO ONE in the community was helping out with any of the issues involved with porting this module to Drupal 11 (one or two people, out of >150k users). LOTS of people were badgering me to make a release, but the only testing had be done by me, using MY workflow, and following ALL of Drupal's recommendations and best practices learned from my 17 years doing Drupal development. I kept asking people to test, but no one did. I put it off for a good long while waiting for people to test and point out problems, and when NO ONE said anything, I went ahead and released it.

This update caused an inconvenience to some people who use a different workflow and development practices than I do. And none of them said anything until after the fact.

The "error" message is simply Drupal core's way of telling you that there was an update skipped and you have to go back and ensure you update to 2.1.4 before you go to 2.2.0. There is absolutely no chance of screwing up your site here - the 2.2.0 code will run just fine with the 2.1.4 database structure, and the 2.1.4 code will run just fine with the 2.2.0 database structure. There are no backwards-incompatible changes.

If you encounter this situation, correcting it is as simply as installing the 2.1.4 code, running the database update, then installing the 2.2.0 code. Period. Easy. Requires no exceptional amount of work other than updating the module twice, which again is EXACTLY what you would have to do if I named the release 3.0.0 instead of 2.2.0.

And again, how you do propose to deal with this? I see no suggestions from you, just insults. I also see absolutely no contribution to this module, ever, in any way shape or form.

Also, I was the one who developed those update functions, who tested those update functions with my own deployment workflow, who wrote test cases for those update functions, and who begged the community to test that code. That process took about 18 months, since there was almost NO contribution from the community of 150,000+ users. The removed updates were about a year old at that point.

I was the only one working on porting this module to D11, and I didn't want to spend the effort porting all the old update tests to D11. And I didn't want to keep carrying around update code, some of which were almost 9 years old. That is MY CHOICE since I'm the one doing all the work.

If anyone from the community had stepped in and said "no, we still need that - here I'll help to do the work" then those update functions might still be in there. Or if someone had said "hey I use a different workflow than you and this will be inconvenient", then those update functions might still be in there. But Drupal is famously a "do-ocracy" - meaning those who do the work get to make the decisions. THAT WAS NOT YOU.

This issue has been open for more than 4 months now. There is nothing left to say IMO. I left it open as a courtesy so that people who encountered an unfamiliar "error" message and didn't know how to search the issue queue could find the solution and share experiences. And perhaps so that someone could figure out a way to prevent people from being inconvenienced by this update. But that hasn't happen, and now that it's degenerated into personal attacks I don't feel the need to keep this open any longer.