Tugboat previews

dmundra → closed merge request !121

dmundra → reopened merge request !121

Followed https://www.drupal.org/docs/develop/git/using-git-to-contribute-to-drupa... → to add the config. Had to close and re-open the merge request to have it show up on the branch but it did once I did that.

Look for 'View live preview' via Tugboat under the branch name. The preview stays up for 5 days after that the MR will need to be closed and reopened.

Critically there is no encryption plugin installable through the UI which renders TFA impossible to setup and test inside of Tugboat. NW for resolution

As a maintainer it would be great to have MR environments so I can save tons of time evaluating MRs without needing to pull it locally.

As an actual (co)maintainer, I wonder how useful this is:

With previews only existing 5 days and most issues taking longer will we see an increase in issue noise? (although that may have just increased to 30 days in 📌 Increase Tugboat Preview expiration to 30 days Active , better but still possible noise for longer lived issues if users are closing/opening mr's to generate previews).

Is code updated when new code is pushed? If so are these new environments or continuations of the previous environment?

In this basic form there is a decent amount of setup that still needs to be done from an admin before they can test much/any of TFA's code. Add an encryption key, setup an encryption profile (which requires an encryption module), before TFA can be enabled, add a user, add a token, etc.

As a maintainer In the amount of time that takes I can (if at a terminal) easily checkout the MR and use an existing deployed lab to test (for context in my local lab I have multiple users, multiple tokens, multiple token types, user roles, and requirements setups configured to test most usage scenarios). There may be an advantage to this when I'm on a mobile device without being near a development terminal.

Presumably some of that setup time could be reduced by configuring the environment inside the template.

Beyond that I'm trying to think how often would I use this. I often review code first and then load into the IDE to validate API usage, sometimes with a requirement for a breakpoint in XDEBUG. To perform those operations I would already have the code in my local lab where I can view the changes.

How often would users use this? How many that do use this it will perform an actual test rather than a "it loaded" test which provide us little to no validation. We are not a theme, our issues are not usually visual they are technical. How much of our code is testable without edge cases that require server changes?

I don't have a good answer for those questions. Leaving this open for opinions from other maintainers and community on if tugboat adds value for our workflow.

Thank you @cmlara. Your questions are appropriate. Another related module is looking at setting up tugboat previews to configure them with enough setup to be useful for testing https://www.drupal.org/project/field_encrypt/issues/3512408 📌 Tugboat preview should configure field_encrypt Active

Is code updated when new code is pushed? If so are these new environments or continuations of the previous environment?

Yes tugboat is updated whenever code is pushed to the open merge request.

We think having the previews would lower the barriers for folks to help test changes especially if there is enough setup already done for them or instructions provided to do that setup.

I think you are right, usefulness will differ module to module so it really up to you and the folks who are invested in this module.

Those are my thoughts.

As a maintainer of a bunch of modules, I find the tugboat previews very helpful and a large time savings. Your mileage may vary.

Is code updated when new code is pushed? If so are these new environments or continuations of the previous environment?

When the MR is updated, the Tugboat environment rebuilds automatically within a few minutes in the same environment (the url does not change).

In this basic form there is a decent amount of setup that still needs to be done from an admin before they can test much/any of TFA's code. Add an encryption key, setup an encryption profile (which requires an encryption module), before TFA can be enabled, add a user, add a token, etc. ... in my local lab I have multiple users, multiple tokens, multiple token types, user roles, and requirements setups configured to test most usage scenarios)

I don't know the specifics of what it takes to setup these items but assuming you are using drush commands or some other terminal commands, they could be run by adding them to the config.yml. Doing so means all co-maintainers and any new co-maintainers would have access to the power and benefit of your local lab type setup. Sounds like and easily achievable win to me.

Presumably some of that setup time could be reduced by configuring the environment inside the template.

100% accurate.

sometimes with a requirement for a breakpoint in XDEBUG.

Tugboat would not work for needing to test at xdebug breakpoints. I believe adding access to the environment terminal is on the roadmap, but xdebug is not. If you need that level of investigation then you'd still need your local. It comes up for me on occasion that I need to inspect patch solutions at that level, but it is fairly rare.

How often would users use this? How many that do use this it will perform an actual test rather than a "it loaded" test which provide us little to no validation.

This often depends on the issue being addressed. In this initial addition, there is not much to test other than "it loaded, and the module is enabled" After this is merged, on UX issues it can help people see the change visually or on multiple devices which screenshots can't really do. When you are working in your "lab" setup, how easy is it for you to preview the changes on your mobile device? Tugboat previews makes it very easy. It makes it easy for others, without extensive lab set-up to see it too.

Critically there is no encryption plugin installable through the UI which renders TFA impossible to setup and test inside of Tugboat.

This sounds like you've identified a new Feature request for the module. Maybe a test encryption plugin that only gets enabled via a drush command to make testing in general easier?

...but still possible noise for longer lived issues if users are closing/opening mr's to generate previews)

I have not found this to be the case. If someone has submitted a MR and the TugBoat environment expires, it is not usually the submitter that rebuilds it. It is usually the maintainer that rebuilds it right before reviewing it. Generally Drupal people are nice, they usually say "please" and "thank you" and they try not to create noise for maintainers. I also know that it is on the roadmap for the Tugboat implementation here on Drupal.org to be adding a link/button to rebuild them without needing to close and re-open or rebase the MR.

In closing, you are right adding tugboat without following up with a bit of additional setup would not gain much. But with a bit of additional setup added onto the tugboat config could make this very helpful.

CivicActions (via nerdstein's contributions) supported this module all the way back to 2014 right after its inception. It is a module we use on several large sites and wanted to give back to help make it easier to contribute to / support in the future.

This sounds like you've identified a new Feature request for the module. Maybe a test encryption plugin that only gets enabled via a drush command to make testing in general easier?

One already exists as a testing module and we use it as part of unit Functional tests, however Drupal must be configured to use such modules on 'production' sites (not UI accessible to my knowledge).

After this is merged, on UX issues it can help people see the change visually or on multiple devices which screenshots can't really do.

We have very little in the way of UX issues directly, and those that do touch UX do not have any real customization to them, renderings would generally be controlled by site themes and the Drupal Core API meaning we have little need for multi-platform testing on a daily basis, if something is broken it is (in the current code) most likely going to be transferred to core as a FormAPI flaw.

When you are working in your "lab" setup, how easy is it for you to preview the changes on your mobile device?

Fairly easily, un-comment a port setting on the lab config and the site is directly exposed, even easier when its on one of internally routed dev machines where it has an internal hostname assigned to it (although that is rare).

Tugboat previews makes it very easy. It makes it easy for others, without extensive lab set-up to see it too.

That is the part I am wondering how useful it is, I'm involvded in many of the deeper issue so perhaps my vision is a bit narrow. I haven't kept any statistical logging of how often I reached an issue that would be acceptable to approve on just a visual without a lab setup.

Doing so means all co-maintainers and any new co-maintainers would have access to the power and benefit of your local lab type setup.

Fair, though these would likely involve a little more complex of a setup as some of the data is to my knowledge not exposed via direct commands. Nothing impossible to do, just another maintenance point to consider if the burden is worth the return, which is the root question, how often tugboat would actually help move issues forward, if it is is frequent enough it may be worth the maintenance burden

I think you are right, usefulness will differ module to module so it really up to you and the folks who are invested in this module.

The simple community question would be "What issues have you worked on in the TFA queue in the past year or two that Tugboat would have allowed you to avoid a local lab and expend less time/effort testing , and in what way would it have done so?"