Add jq and yq packages to images

Clarifying: this originally was intended to ask for kislyuk/yq

There is mikefarah/yq however it notes it supports a smaller syntax than kislyuk/yq

ksislyuk/yq works by converting XML to JSON allowing jq to use its natural language support.

idea looks good, personally I think jq is the most common and it should be enough

who can review dependencies and security consequences?

I think jq is the most common and it should be enough

The yq package adds support for XML which is likley just as important given we now have been modifying the phpunit config to support differences in core versions. As noted above yq uses jq for the parsing engine and is primarily a converter.

who can review dependencies and security consequences

A good question, to my knowledge this has not been requested before in this queue and there is no documented review policy. Given the lack of policy it would likley need to be done by a queue admin (yourself?) as none of us can speak to an undefined standard.

I mean adding yq requires to add python interpreter with few dependencies, IMO it's useless for CI image

I mean adding yq requires to add python interpreter

The python interpreter is already on the images and is used by gitlab_templates to provision the mkdocs stage (in other words: we are not likely to accept the removal of the python interpreter anytime soon) .

with few dependencies,

Fair it may add more dependencies into python.

If there is a binary XML->JSON->XML and YAML->JSON->YAML converter that could be a reasonable alternative to the yq package.

IMO it's useless for CI image

https://git.drupalcode.org/project/gitlab_templates/-/blob/main/scripts/... is an example of where we are creating a 51 line script for what could likely be done more efficiently and reliably a 1 line yq command. I've had other discussions in Slack where we are making the jobs harder to maintain, more fragile (dependent upon another HTTP GET, and can be broken by mismatch in versions downloaded vs version of the gitlab template) (these are incidents that have occurred in the past) by using external PHP Scripts.

Downloading extra scripts is somewhat of an anti-pattern in GitLab, and the tricks that gitlab_templates team currently use do not currently exist for GitLab Components meaning the ability to inline commands can become even more important long term.

The script alone proves that we need to make changes to XML files as part of the CI process which renders the argument 'useless' as moot and only leaves 'is it worth the cost vs savings'.

Looks it's only few packages python3-argcomplete python3-toml python3-xmltodict python3-yaml

root@821a35c6e52f:/var/www/html# apt install jq yq
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libjq1 python3-argcomplete python3-toml python3-xmltodict python3-yaml
The following NEW packages will be installed:
  jq libjq1 python3-argcomplete python3-toml python3-xmltodict python3-yaml yq
0 upgraded, 7 newly installed, 0 to remove and 30 not upgraded.
Need to get 409 kB of archives.
After this operation, 1357 kB of additional disk space will be used.
Do you want to continue? [Y/n] ^C
root@821a35c6e52f:/var/www/html# apt install jq   
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libjq1
The following NEW packages will be installed:
  jq libjq1
0 upgraded, 2 newly installed, 0 to remove and 30 not upgraded.
Need to get 206 kB of archives.
After this operation, 481 kB of additional disk space will be used.
Do you want to continue? [Y/n] 

As jq is instalkled as dependency here's MR

merged to dev so it can be tested https://git.drupalcode.org/project/drupalci_environments/-/jobs/4611960

As jq is instalkled as dependency here's MR

I generally recommend to be explicit.

For all we know next year yq could change and not use the jq package (highly unlikely however if they do we loose jq and don’t realize it)

@cmlara thanks updated commit to both yq jq https://git.drupalcode.org/project/drupalci_environments/-/commit/5d1853...

PS: this week new PHP releases will out, meantime I did update prod images with fresher PECL