Decide on the future of the user guide

Created on 28 August 2024, 4 months ago

The current user guide parsing is not something we can sustain as Drupal.org moves to Drupal 11. Possible solutions are:

Convert to regular documentation pages & guides

This is how this was solved for the β€œofficial docs.” The user guide becomes regular documentation pages, editable on Drupal.org.

Stay as-is, moving most of the build to GitLab CI

Right now builds happen on our batch server, which is non-ideal. There is no transparency for what happened in a build, I have to copy console output to Slack. The tools need to be installed at a system-level, and we are rebuilding those systems to be container-based. We are running the scripts from this repository in a context that makes me uncomfortable from a security perspective; access to the sites is more direct than other code we run. Builds should be done in this project in GitLab CI, with πŸ“Œ Convert build scripts to asciidoctor Active resolved, and the HTML output artifact is what Drupal.org imports.

The feeds import process and surrounding customization would need to be rebuild for Drupal 11. This is not ideal for ongoing maintenance.

Use GitLab pages

Everything is self-contained in this repository. The user guide is not part of www.drupal.org and does not appear in Drupal.org site search.

πŸ“Œ Task
Status

Active

Version

11.0

Component

Scripts

Created by

πŸ‡ΊπŸ‡ΈUnited States drumm NY, US

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @drumm
  • πŸ‡ΊπŸ‡ΈUnited States volkswagenchick San Francisco Bay Area

    +1 to Use GitLab pages

  • πŸ‡©πŸ‡ͺGermany joachim namyslo Kulmbach πŸ‡©πŸ‡ͺ πŸ‡ͺπŸ‡Ί

    If I may make a suggestion: let's try out what would happen if we convert the English original to MKdocs. I can only speak from my own experience. Ascidock is not necessarily the most technically simple solution for motivating the average Joe to contribute to the continued existence of the Drupal user manual. Markdown should be simple enough for many people to use. MK-Docks automates the build process with one script and one general configuration file per language. Even converting existing HTML into Markdown is not that difficult. MKdocs material could even be hosted using github pages.

  • πŸ‡ΊπŸ‡ΈUnited States eojthebrave Minneapolis, MN

    Whatever we do here I would like to see us reduce the burden on the DA for maintaining one-off infrastructure and workflows that the current system creates.

    Personally I would love for this to remain in Git, and not become standard nodes. For all the same reasons outlined when we originally decided to go this path. I think it lends to better editorial control, consistency, and supports things like translations in a way that would be really hard without VCS. And selfishly, as the person who currently does the majority of the content and screenshot maintenance for the guide doing so via editing individual nodes would be a nightmare.

    If we keep doing the importing to Drupal.org as nodes I think moving the build process to GitLab and creating the HTML artifacts there would be fairly straight forward, and a great idea.

    The feeds import process and surrounding customization would need to be rebuild for Drupal 11. This is not ideal for ongoing maintenance.

    We also import the user guide content on Drupalize.Me, and use a custom migrate source plugin and a couple of migration YAML files that's far less code, and easier to maintain, than the asciidoc_feeds code used by Drupal.org. I'm happy to share that work if it's ever useful.

    The user guide is not part of www.drupal.org and does not appear in Drupal.org site search.

    This for me is a big reason for not using GitLab pages (or Mkdocs) or anything else that is self contained. I'm worried that it potentially makes it less discoverable. I'm not opposed, something like learn.drupal.org could be cool. But I think burying it in GitHub pages could have a negative effect on people's already pretty negative view of Drupal's documentation. There are some benefits to going this route too, if for example we built a static site out of the user guide it could potentially be more shiny and have a different kind of impact on people's perception of the documentation. I think ideally if we do go this route that the site is themed to be consistent with the Drupal.org brand so that it continues to be as official as possible.

    Converting to Markdown is an interesting idea (aside, there are static site generators that support asciidoc natively). Doing so would require figuring out a few technical things, here's a few I can think of off hand:

    • Ebooks. Asciidoc has far superior support for ebook generation. Though I'm sure it's possible with Markdown. Probably requires something like converting to a single .md file, and then using pandoc to create the books.
    • Markdown doesn't support reference style links. In asciidoc for example we can do [[extend-module]] and it'll be converted to a link to the document with that ID and the text of the link will be the title of the document. We use this a lot in the content of the guide. I'm sure we could come up with a custom solution in Markdown, but that's a thing we would need to figure out.
    • Indexing. Asciidoc supports the creation of an index by using ((Term,term)) style metadata in documents. This is another thing we use a lot. And again, could probably be figured out with some kind of frontmatter in Markdown but isn't something Markdown supports natively.

    I've tried using tools like pandoc in the past to automate conversion to Markdown as an experiment. But because we use features of asciidoc that have no Markdown equivalent it fails pretty miserably. So the conversion would either be manual. Or someone would need a script a custom conversion tool.

    I think it would be a significant amount of work to convert things. And I'm not convinced that doing so would suddenly get more people contributing. Personally, it doesn't make much difference to me if I'm editing the files in asciidoc or markdown so I'm happy to do either.

    And ... I think that if it is converted to markdown there are a lot more options for using something like Mkdocs and just general support in that many more people use Markdown on a regular basis.

  • πŸ‡©πŸ‡ͺGermany joachim namyslo Kulmbach πŸ‡©πŸ‡ͺ πŸ‡ͺπŸ‡Ί

    I tell you what I did.

    Used mkall.sh and built the guide just as usual
    Then I used this little guy with this particular addon and all I have to do to get the 11.x version of the guide now, is to fix the tables.

    Besides that, importing the HTML files from the guide's German version worked effortlessly.

    So perhaps there is a chance we can host markdown files directly on GitHub pages in the future so we can iterate much faster then?

    Converting markdown into other formats like epub and mobi as well as PDF shouldn't be that hard I like to test that as soon as I have fixed the tables in the markdown of the guide. If it helps I can take the English source and upload a fixed markdown version here.

    mkdocks is, the right approach, at least iin my opinion. This way, we can continue to work directly in the GitHub editor and, if necessary, simply build a new output every x days or hours. Even I can manage that, and I think I know a lot less about DevOps than some other members of the Drupal community. Perhaps a version written in Markdown would just help to speed up the whole process, and thus relieve the DA relatively fast.

  • πŸ‡ΊπŸ‡ΈUnited States drumm NY, US

    We also import the user guide content on Drupalize.Me, and use a custom migrate source plugin and a couple of migration YAML files that's far less code, and easier to maintain, than the asciidoc_feeds code used by Drupal.org. I'm happy to share that work if it's ever useful.

    Yes, it would be good to see this.

  • πŸ‡©πŸ‡°Denmark ressa Copenhagen

    I agree @eojthebrave:

     

    The user guide is not part of www.drupal.org and does not appear in Drupal.org site search.

    This for me is a big reason for not using GitLab pages (or Mkdocs) or anything else that is self contained. I'm worried that it potentially makes it less discoverable. I'm not opposed, something like learn.drupal.org could be cool.

    Whatever the solution ends up being, keeping it on the drupal.org domain and not on a third-party domain, is crucial in my opinion.

    Personally I would love for this to remain in Git, and not become standard nodes. [...] I think it lends to better editorial control, consistency, and supports things like translations in a way that would be really hard without VCS.

    I agree -- this is the way to do it, to keep it structured and coherent, and importantly translatable. And you're doing a stellar job with that already, so thanks for that!

    @eojthebrave:

    Also, regarding the conversation about converting the files to Markdown. Drupalize.Me has code for importing markdown documents like this into Drupal via the Migrate API as well. And would be open to sharing that too.

    @joachim namyslo:

    Used mkall.sh and built the guide just as usual. Then I used this little guy with this particular addon and all I have to do to get the 11.x version of the guide now, is to fix the tables.

    Markdown is a joy to format text with, so I would definitely second that. It would be interesting if you each did a trial run of the conversion process, and shared the results, so that the process could be compared for efficiency, quality, ease of use, and so on?

Production build 0.71.5 2024