Missing packaging script information by installing not from Drupal.org needs documentation.

the Drupal.org packaging script is placing its own information automatically, which should not be used as reference to add such info manually but if the datatables.info.yml is missing that part I think best is to contact drupalorg infrastructure channel. I will try to ping somebody on Slack and will ask what's going on.

Thanks for the finding! +1

Removed the related issue because it can possibly confuse maintainers of different projects. But yeah, I know what you try to say: it happens somewhere else too. Good finding! Thanks. I will track this.

I just closed #3398727-3: Packaging script information missing → as "Works as designed", too. Please see my answer there. You probably need to use git_deploy → , since this information is "added by Drupal.org packaging script", and not to be found in Git.

@Chris64: I now you are very careful but can you please check if you maybe run in the same confusion like this reporte here? https://www.drupal.org/project/responsive_menu/issues/3398727 💬 Packaging script information missing Closed: works as designed

Indeed, the info.yml file is OK,

# Information added by Drupal.org packaging script on 2024-01-18
version: '2.0.1'
project: 'datatables'
datestamp: 1705542838

sudo drush  pml | grep datatable
  Views                             DataTables (datatables)                                                 Enabled    2.0.1

In fact, no problem with most of the sites. However, for one of them the datatables directory was not updated by the composer. And the file was wrong. Problem solves (re)moving the directory. Kind of composer error got,

In GitDownloader.php line 342:
  Source directory */web/modules/contrib/datatables has unpushed changes on the current branch:
  Branch 2.0.1 could not be found on any remote and appears to be unpushed 

And composer stop.

Thank you @dww, indeed looking directly to the release source is also an sure means to check such a problem.

looking directly to the release source is also an sure means to check such a problem.

As I sad in #3 💬 Packaging script information missing Closed: works as designed

the Drupal.org packaging script is placing its own information automatically, which should not be used as reference to add such info manually

... which can be read as: or to expect it to be in other release sources ;-) I just wanted to be kind and not too "know-it-all" since there could also be other issues overlapping here. This is why I didn't wanted to close this issue too soon. Apart from that, it wasn't clear from the initial report if the report is referring to the Drupal.org release version or not. I actually expected the first, because I knew that this is added by Drupal.org so why reporting if not.

Apart from that: for me closing this is too hasty, because there is another issue @Chris64 has perfectly detected here under the hood:

As a consequence, the following drush command does not display any version number,
drush pml | grep datatables

Which resolves for me into the issue: "If Drupal.org is not the release source with its automated package info, Drush pml then only works with limited functionality and should use other sources for its version information." ;-)

If that's the agreement we should move it to Drush. But please! Wait until we collect opinions.

@diqidoq, the version information provide by drush in pml comes from the info.yml files I think. Hence if no version information are found in this files '' is written by the drush command. Coherent. That was the case.

Some information are missing from my part to fully understand the situation. I was working with a development version, without version information, as required. The problem has been faced when trying to install the release version via composer on a development environment. What is not clear for me is how to make this installation without clearing the current module directory. Clearing is a walkaround. Probably simply creating a new branch and switching on it before the composer update. If a problem exists, it is a composer-git problem. Not a drush problem one. The situation is particular since is like a feedback loop between development and release.

@Chris64: Again, during development, if you've got a git checkout (source) of a given project, not a packaged release (dist), you'll need either git_deploy → or composer_deploy → for Update Manager in core (and therefore drush package manager commands) to work.

@diqidoc: Yes, this works as designed. We don't have to re-open it. There was never a bug here. People are (rightfully) confused that if they're working with a source checkout that the .info.yml changes aren't done for them and they need to use another tool to have it done automatically.

Thanks,
-Derek

p.s. Drush itself can't decide how to get this info from other places. That's not for drush to decide, IMHO. It's for the site builder to determine based on their workflow and needs.

Meanwhile, you might want to open a separate issue in drush as a UI improvement to print out installed projects for which there's no package metadata and therefore it cannot handle certain commands for. That'd be reasonable, but drush doesn't use d.o for issue management. So there's no where to move this issue to. If you're inspired to try to improve drush in that way, please report it at GitHub.

Thanks again,
-Derek

Meanwhile, you might want to open a separate issue in drush as a UI improvement to print out installed projects for which there's no package metadata and therefore it cannot handle certain commands for.

Thats exactly what was planned to do and why I kept this issue open as maintainer here for my later reference and to furthermore try to help @Chris64 to understand the possible connections and possible new issues which need to be opened. An issue needs not to be closed immediately before the descision hasn't been made yet how to go on and I explicitely explained that before that a possible new issue is required for Drush. And I would like to ask you kindly please to accept this decision from the maintainers view point and to not change back the status to prevent edit wars.

All good. No war intended. Do whatever you need. I'm just saying you can stop fishing for if there's actually a packaging script bug here, which there is not. 😅

Enjoy,
-Derek

@dww: No offense or fishing intended here. I see why you are after it, appreciate your contributions, and it's all good with the Drupal.org packages ;-). And I was kind, polite and thankful for your try to help here, in chat, and in the other issues, as you will remember. But I had to restrain myself a little towards you to ask for calming down a little while seeing you rushing over the issues without asking and giving time to communicate between users.

I'm just saying you can stop fishing for if there's actually a packaging script bug here

@dww: I think this is the initial misunderstanding here. Read my comments here and on Slack. Even my first question in the round. I never really believed there is one (issue). I was just communicating "case-open" while asking "when this started to occure in issues". And I tried to explain users what the automated packing script is doing without cutting somebody short and while keeping things "more open" to detect maybe other possible issues crossing this.

Apart from that, please consider not everybody here is a "native tongue". So there are possible misunderstandings coming into mix. And I am not only refering to my own native language but also to all others contributing here. Try to imagine you would need to communicate here, in other issues, in chat, in maintainer mails, in message bords, in release notes, etc not in your language but in German ;-) or French. Over days and nights.

I hope I was able to clear up a few misunderstandings and how I appreciate you but also others here reporting and the efforts of all of us in the community.

I have to respond. In fact finally my real problem is not that no version exists. My real problem is that composer doesn't update the directory. Here the problem version matter only as symptom of non updating. And my goal is to update the directory with composer, overcoming the obstruction.

@dww, do you want to tell me that the modules git_deploy or composer_deploy allow me to update the directory in such a case? If they are only to get the version I don't need them now. My sentiment is those modules have no utility for my problem.

Your purpose are surely true. The problem is that probably we don't have the same problems in mind. Whence some kind of dialogue of the deaf.

The problem is not to know if the original problem exists. The problem is now that some thing wrong appends and we would like to know more about it. It is why @dqd don't want to close the issue.

composer remove and composer require make disappear the problem. Unfortunately it is not a good initiative, since remove erase the modules directory in contrib/ with its local .git... And with it obviously the problem disappear.

@Chris64: Thanks for coming back and elaborating your investigations. Let's shift the issue target a little bit away from its initial title which is somewhat misleading and obviously triggering contributors to the install systems of Drupal thinking there is a possible issue they need to take a look at.

In fact it is NOT an automated Packaging script issue. And that's what I actually tried to say from the beginning as you maybe remember. While keeping it open to discuss underlying issues and solutions, because there was no information from which install method this has been caused from. And while doing this for others, @dww's and my ways have crossed and now there is the assumption that I do not understand the reasons for that the packaging script is missing, when using git clone and similar. Hah :-)

In fact - from my side - it is the awaited behaviour, that git clones and similar install methods do not have a version information coming with it since it isn't a release yet but a git branch. And so I assume from other related ways to install.

This title is a first idea ,where this issue maybe goes. But I won't move it to core yet before we both do not solved the way how users can circumvent this experience and how to make a maybe (documentation?) out of it. Then we rather should start a new minor core documentation issue providing the solution.

@dqp your insistence to get more light about this subject seems justify. More information got. Not trivial. Some things have to be known about this problem.

(the modules appearing here are coincidences, linked to home changes, and not release state.)

The elements automatically add by Drupal.org packaging script in the info.yml when a new release is created are the 3 keys version, project and datestamp.

If for some reasons those information are not in a module of an installation some problems could happen.

Some errors happen in some pages of the applications (see 📌 Handle malformed module info in the translation status form Active ).

Some others happen when trying to update the installation via composer. Indeed, it seems that if the information does not exists the update system is blinded about the module, and the module is not updated,

Loading composer repositories with package information
Updating dependencies
Nothing to modify in lock file
Installing dependencies from lock file (including require-dev)
Nothing to install, update or remove

To update a module with such a problem the update have to be forced. There is several means to do that.
Remove and reinstall the module, with composer remove and composer require. However it is not always possible by this means. For example if the module has dependencies. Then a message like the following happens from composer,
Removal failed, drupal/entity_reference_revisions is still present, it may be required by another package. See `composer why drupal/entity_reference_revisions`.
Others obstructions could exist for removing a module. For example, now removing the dependent module, the message could be,

Running composer update drupal/paragraphs
Loading composer repositories with package information
Updating dependencies
Lock file operations: 0 installs, 0 updates, 2 removals
  - Removing drupal/entity_reference_revisions (1.11.0)
  - Removing drupal/paragraphs (1.17.0)
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 0 installs, 0 updates, 2 removals
  - Removing drupal/paragraphs (1.17.0)
Deleting /var/www/ligamen.eu/web/modules/contrib/paragraphs - deleted

In GitDownloader.php line 342:

  Source directory /var/www/ligamen.eu/web/modules/contrib/entity_reference_revisions has unpushed changes on the current branch:
  Branch 1.11 could not be found on any remote and appears to be unpushed

And the second module is not deleted. Thus such a message happens in a remove operation. In some case the release could be downloaded but not installed,

Running composer update drupal/entity_reference_revisions
Loading composer repositories with package information
Updating dependencies
Lock file operations: 1 install, 0 updates, 0 removals
  - Locking drupal/entity_reference_revisions (1.11.0)
Writing lock file
Installing dependencies from lock file (including require-dev)
Nothing to install, update or remove

Here the module is not updated.
Furthermore the idea to write in the info.yml file some arbitrary information for the missing ones seems not working.
If composer remove does not work a sure method is to remove or better move (mv and save) the module folder from contrib/.
An other solution is suggested by @dww in #4 💬 Packaging script information missing Closed: works as designed : use a module that update the info.yml file.

Therefore, if an installation has modules with abnormal info.yml files the update with composer is not trivial.

Moreover, firstly the application maintainer have to know the modules with such a problem. How to do that? For Drupal 10.2 fortunately a simple means exists to get such an information but not regular: see 📌 Handle malformed module info in the translation status form Active

Secondly, does a module with the missing information could be regular and accepted? A modified module version should not have the same version than a release one. And development version to be pushed should have not such information. So it seems possible. And composer foresee such a case since in its documentation for status,

If you often need to modify the code of your dependencies and they are installed from source, the status command allows you to check if you have local changes in any of them.

In the principle such a command allow therefore to found code difference. However how the composer know the release version to compare if this information is missing? Hence this command is not sure for this purpose.

@Chris64 thanks for taking the time to investingate! +1 We will try to collect some more to maybe create a doc page for it. Things will change next time anyway when Drupal issues (and releases?) move to Gitlab.

I would add to the documentation clearly in front the suggestion to rather NOT use any not from Drupal.org installed modules on production sites and to only use such git clones and similar for development and contribution only. Also Drupal.org security checks will not be guaranteed in this case. For most contributors this isn't a new information ;-) and this is what @dww wanted to say. The update functionality only works with official releases or dev snapshots provided on the project page for now.

Let's focus now on makeing DataTables 100% compatible with the features the library provides and let's concenttrate on that.

Million thanks for you refforts here. @chris64: as always very much appreciated!

A good documentation is wellcome.
Yes @dqd, still some work for DataTables.
And thank you for your support!