Contrib.social

Adopt (semver) versioning for DrupalCi images

Open on Drupal.org →
Created on 24 November 2023, over 1 year ago

Problem/Motivation

Previously under DrupalCi the majority of image usage was controlled by the DA. To this design images were either Production or Dev images.

With the migration to GitLab the Images have transitioned to a status where they are now a base for CI operations. The DA no longer directly controls all operations, maintainers are encouraged to extend/modify the CI process as needed to fit their project needs.

The DrupalCI images are currently the foundational base of the gitlab_templates repository which is encouraged as the base for configuring GItLab based CI on G.D.O.

This transition us to a state where the DrupalCi Environment images are now being used in a manner that depends upon reliably signaling changes and maintainers needing the tools to control adoption of those changes.

As an example the work on 📌 Split PHP image into php(cli/apache) and yarn(node/nightwatch) Needs review was promoted from the dev images to the tagged production images yesterday (Context: during a US holiday which for many continues until Monday). This change can/does break CI testing for some projects.

Thankful my projects just end up not running Code Coverage and this created a soft fail, however if I had adopted Mutation testing (which depends upon code coverage) it would have been a hard fail.

While I was aware this work was in-progress, and suspected I would need to eventually update my Gitab template files there was no advance warning that this change would occur yesterday and that I needed to update all my projects before the holiday.

Addtionaly for point-in-time testing (needing to back and test older dev cycles) having reliably retrievable images would also help.

The gitlab_templates project could handle most of the issues regarding when/how to upgrade to a new major and how to announce it, the images just need to be tagged in a way that allows the templates to do so.

Naming like drupalci/apache-php-8.2:X.Y.Z (with a side label of drupalci/apache-php-8.2:latest pointing to the latest in the highest major and drupalci/apache-php-8.2:X pointing to the highest version in the specified major) would likely be sufficient for templates to refer to these images.

How we want to sync the majors/minors would be the biggest question.

I would suggest that at least the majors be synced in such a manner that one could use the major specific tag across all currently maintained CI images without expecting errors to occur due to tooling (in other words: the only errors we should expect would be related to the version of PHP and not the rest of the image.)

Steps to reproduce

N/A

Proposed resolution

Adopt SEMVER type numbering for images being pushed to Docker Hub,
Any changes that could break existing installs (such as removing programs, removing libraries, disabeling modules, etc) be done as a major release.
Any changes adding new features (adding new modules, tools, upgrading versions of tools without changing their interface methods, etc) be done as a Minor release
Any changes that only fix faults in the existing images be done as a patch release.

Remaining tasks

TBD

User interface changes

None

API changes

Adds a definition of the images being considered API.

Data model changes

None

✨ Feature request
Status

Active

Component

Container Build Process

Created by

🇺🇸United States cmlara

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @cmlara
  • Comment over 1 year ago →
    🇺🇸United States moshe weitzman Boston, MA

    As an aside, all these same issues apply to gitlab_templates too. There we want our cake and eat it too. We want sites to automatically pickup changes to the templates but occasionally we break stuff, either by mistake or on purpose.

  • Comment over 1 year ago →
    🇺🇸United States cmlara

    Opened #3404175: Adopt (semver) versioning for gitlab_templates → to discuss the gitlab_templates equivalent of this.

  • Comment over 1 year ago →
    🇫🇷France andypost

    building script needs improvement here

  • Comment 6 months ago →
    🇺🇸United States cmlara

    I will also note that if at the same time we move everything into a central
    drupalci/php project we can avoid the issues of needing to know what base image type being used as drupalci/php:8.4-Ubuntu could be aliased to drupalci/php:8.4 which would make CI management significantly easier for version selection.

  • Comment about 1 month ago →
    🇫🇷France andypost
  • Comment 20 days ago →
    🇫🇷France andypost

    IIRC the reason to keep only latest image is security because distros getting updates and PHP has monthly release cycle, so no reason to bloat storage...

    So semver should be local and applicable for nightly somehow

  • Comment 20 days ago →
    🇺🇸United States cmlara
    IIRC the reason to keep only latest image is security because distros getting updates

    Although the image is avaliable doesn’t mean a site has to use it. We see this all the time on DockerHub older releases are kept so they can be referred to in the future.

    This is analogous to arguing all the Drupal 9 core releases should be deleted due to the fact they are known vulnerable.

    and PHP has monthly release cycle, so no reason to bloat storage...

    First off, retention and semver are different.

    Semver would be about ensuring images are generally the same and compatible between build, PHP majors and minors. This could be done even with the current system of never retaining anything except the latest build.

    Additional one could set a retention window for images (6 months of same “major” and at least the last major of a semver for a few years?) to reduce storage bloat and yet still allow retaining images.

    I will also add that shared storage of being in the same repository could allow image layer reuse further reducing size (this would require streamlining the builds a bit).

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024