Decide on the future of the user guide

Created on 28 August 2024, 5 months ago

The current user guide parsing is not something we can sustain as Drupal.org moves to Drupal 11. Possible solutions are:

Convert to regular documentation pages & guides

This is how this was solved for the “official docs.” The user guide becomes regular documentation pages, editable on Drupal.org.

Stay as-is, moving most of the build to GitLab CI

Right now builds happen on our batch server, which is non-ideal. There is no transparency for what happened in a build, I have to copy console output to Slack. The tools need to be installed at a system-level, and we are rebuilding those systems to be container-based. We are running the scripts from this repository in a context that makes me uncomfortable from a security perspective; access to the sites is more direct than other code we run. Builds should be done in this project in GitLab CI, with 📌 Convert build scripts to asciidoctor Active resolved, and the HTML output artifact is what Drupal.org imports.

The feeds import process and surrounding customization would need to be rebuild for Drupal 11. This is not ideal for ongoing maintenance.

Use GitLab pages

Everything is self-contained in this repository. The user guide is not part of www.drupal.org and does not appear in Drupal.org site search.

📌 Task
Status

Active

Version

11.0

Component

Scripts

Created by

🇺🇸United States drumm NY, US

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @drumm
  • 🇺🇸United States volkswagenchick San Francisco Bay Area

    +1 to Use GitLab pages

  • 🇩🇪Germany joachim namyslo Kulmbach 🇩🇪 🇪🇺

    If I may make a suggestion: let's try out what would happen if we convert the English original to MKdocs. I can only speak from my own experience. Ascidock is not necessarily the most technically simple solution for motivating the average Joe to contribute to the continued existence of the Drupal user manual. Markdown should be simple enough for many people to use. MK-Docks automates the build process with one script and one general configuration file per language. Even converting existing HTML into Markdown is not that difficult. MKdocs material could even be hosted using github pages.

  • 🇺🇸United States eojthebrave Minneapolis, MN

    Whatever we do here I would like to see us reduce the burden on the DA for maintaining one-off infrastructure and workflows that the current system creates.

    Personally I would love for this to remain in Git, and not become standard nodes. For all the same reasons outlined when we originally decided to go this path. I think it lends to better editorial control, consistency, and supports things like translations in a way that would be really hard without VCS. And selfishly, as the person who currently does the majority of the content and screenshot maintenance for the guide doing so via editing individual nodes would be a nightmare.

    If we keep doing the importing to Drupal.org as nodes I think moving the build process to GitLab and creating the HTML artifacts there would be fairly straight forward, and a great idea.

    The feeds import process and surrounding customization would need to be rebuild for Drupal 11. This is not ideal for ongoing maintenance.

    We also import the user guide content on Drupalize.Me, and use a custom migrate source plugin and a couple of migration YAML files that's far less code, and easier to maintain, than the asciidoc_feeds code used by Drupal.org. I'm happy to share that work if it's ever useful.

    The user guide is not part of www.drupal.org and does not appear in Drupal.org site search.

    This for me is a big reason for not using GitLab pages (or Mkdocs) or anything else that is self contained. I'm worried that it potentially makes it less discoverable. I'm not opposed, something like learn.drupal.org could be cool. But I think burying it in GitHub pages could have a negative effect on people's already pretty negative view of Drupal's documentation. There are some benefits to going this route too, if for example we built a static site out of the user guide it could potentially be more shiny and have a different kind of impact on people's perception of the documentation. I think ideally if we do go this route that the site is themed to be consistent with the Drupal.org brand so that it continues to be as official as possible.

    Converting to Markdown is an interesting idea (aside, there are static site generators that support asciidoc natively). Doing so would require figuring out a few technical things, here's a few I can think of off hand:

    • Ebooks. Asciidoc has far superior support for ebook generation. Though I'm sure it's possible with Markdown. Probably requires something like converting to a single .md file, and then using pandoc to create the books.
    • Markdown doesn't support reference style links. In asciidoc for example we can do [[extend-module]] and it'll be converted to a link to the document with that ID and the text of the link will be the title of the document. We use this a lot in the content of the guide. I'm sure we could come up with a custom solution in Markdown, but that's a thing we would need to figure out.
    • Indexing. Asciidoc supports the creation of an index by using ((Term,term)) style metadata in documents. This is another thing we use a lot. And again, could probably be figured out with some kind of frontmatter in Markdown but isn't something Markdown supports natively.

    I've tried using tools like pandoc in the past to automate conversion to Markdown as an experiment. But because we use features of asciidoc that have no Markdown equivalent it fails pretty miserably. So the conversion would either be manual. Or someone would need a script a custom conversion tool.

    I think it would be a significant amount of work to convert things. And I'm not convinced that doing so would suddenly get more people contributing. Personally, it doesn't make much difference to me if I'm editing the files in asciidoc or markdown so I'm happy to do either.

    And ... I think that if it is converted to markdown there are a lot more options for using something like Mkdocs and just general support in that many more people use Markdown on a regular basis.

  • 🇩🇪Germany joachim namyslo Kulmbach 🇩🇪 🇪🇺

    I tell you what I did.

    Used mkall.sh and built the guide just as usual
    Then I used this little guy with this particular addon and all I have to do to get the 11.x version of the guide now, is to fix the tables.

    Besides that, importing the HTML files from the guide's German version worked effortlessly.

    So perhaps there is a chance we can host markdown files directly on GitHub pages in the future so we can iterate much faster then?

    Converting markdown into other formats like epub and mobi as well as PDF shouldn't be that hard I like to test that as soon as I have fixed the tables in the markdown of the guide. If it helps I can take the English source and upload a fixed markdown version here.

    mkdocks is, the right approach, at least iin my opinion. This way, we can continue to work directly in the GitHub editor and, if necessary, simply build a new output every x days or hours. Even I can manage that, and I think I know a lot less about DevOps than some other members of the Drupal community. Perhaps a version written in Markdown would just help to speed up the whole process, and thus relieve the DA relatively fast.

  • 🇺🇸United States drumm NY, US

    We also import the user guide content on Drupalize.Me, and use a custom migrate source plugin and a couple of migration YAML files that's far less code, and easier to maintain, than the asciidoc_feeds code used by Drupal.org. I'm happy to share that work if it's ever useful.

    Yes, it would be good to see this.

  • 🇩🇰Denmark ressa Copenhagen

    I agree @eojthebrave:

     

    The user guide is not part of www.drupal.org and does not appear in Drupal.org site search.

    This for me is a big reason for not using GitLab pages (or Mkdocs) or anything else that is self contained. I'm worried that it potentially makes it less discoverable. I'm not opposed, something like learn.drupal.org could be cool.

    Whatever the solution ends up being, keeping it on the drupal.org domain and not on a third-party domain, is crucial in my opinion.

    Personally I would love for this to remain in Git, and not become standard nodes. [...] I think it lends to better editorial control, consistency, and supports things like translations in a way that would be really hard without VCS.

    I agree -- this is the way to do it, to keep it structured and coherent, and importantly translatable. And you're doing a stellar job with that already, so thanks for that!

    @eojthebrave:

    Also, regarding the conversation about converting the files to Markdown. Drupalize.Me has code for importing markdown documents like this into Drupal via the Migrate API as well. And would be open to sharing that too.

    @joachim namyslo:

    Used mkall.sh and built the guide just as usual. Then I used this little guy with this particular addon and all I have to do to get the 11.x version of the guide now, is to fix the tables.

    Markdown is a joy to format text with, so I would definitely second that. It would be interesting if you each did a trial run of the conversion process, and shared the results, so that the process could be compared for efficiency, quality, ease of use, and so on?

  • 🇭🇺Hungary Balu Ertl Budapest 🇪🇺

    All valuable points absolutely worth finding some freetime to read through all your arguments. However, until then just briefly dropping here a question: is this thread the good place to think about whether we want to consider updating “The Next Big Edition” of the user guide for Drupal CMS instead? Or maybe I have missed the point when it has been already decided that the User Guide must be about Core?

  • 🇩🇰Denmark ressa Copenhagen

    Having a Drupal CMS User Guide is a great idea @baluertl!

    Since Drupal CMS builds on top of the content of the User Guide, it could be added as a new Chapter 14, before the last chapter, like this?

    • Chapter 13. Security and Maintenance
    • Chapter 14. Drupal CMS
    • Chapter 15. Final Thoughts

    ... or it could get its own Drupal CMS User Guide?

  • 🇭🇺Hungary Balu Ertl Budapest 🇪🇺

    Glad to hear others also like the idea. From what I gather about the Starshot project through its communications, its aim seems to focus on serving an audience that Drupal Core alone has struggled to engage: non-coder site builders who prefer working exclusively through a graphical user interface. While I can’t speak on behalf of the original authors of the User Guide (Jennifer, Amber, Joe, and many others whose contributions deserve to be honored, though I regretfully can’t recall all their names), my experience working on the Hungarian translation left me with the impression that it was written with a very similar audience in mind.

    As Drupal 8 was era-shifting in its terms, I believe, Drupal CMS is also expected to be so too. Publishing the 1. edition of the User Guide at the time of such a groundbreaking change was an extremely valuable aid to assist many over the gap. I think Drupal CMS's initial adoption could be further supported if we dedicate the 2nd edition of the User Guide solely to them.

    Oppositely, I don't really see the point in mixing Drupal Core × Drupal CMS in the same book. It just makes no sense at all to me, because those are two very distinctive products sold to different audiences.

  • 🇸🇰Slovakia poker10

    Adding Drupal CMS documentation related issue: 📌 [META] Track 3: Drupal Starshot documentation Active . There were also some Slack discussions about this in the #starshot channel.

    I think the most important point from there which was also mentioned in #4 and #7 here, is to have the docs on drupal.org domain and indexable by search engines, not on drupalcode.org, which is hidden a not indexed (currently).

  • 🇩🇰Denmark ressa Copenhagen

    It looks like a Drupal CMS guide is underway, see the comment #3454527-13: [META] Track 3: Drupal Starshot documentation posted just now.

  • 🇭🇺Hungary Balu Ertl Budapest 🇪🇺

    In light of Lenny's recent updates 📌 [META] Track 3: Drupal Starshot documentation Active , it seems there’s no need to shift the target audience of this User Guide project to focus on the Drupal CMS product. A dedicated documentation effort is already being handled by the excellent team at Drupalize.me. As such, I’m withdrawing the point I raised in #8 📌 Decide on the future of the user guide Active , as it’s now clear this isn’t a concern.

    That said, I’m left wondering whether Drupal Core (which is primarily aimed at developers, unlike Drupal CMS) truly needs a hands-on guide at all.

  • 🇩🇪Germany joachim namyslo Kulmbach 🇩🇪 🇪🇺

    That's a very good point.

    From the last 13 years, I know that all Drupal developers I've encountered during that time prefer reading English texts rather than translating. Users who are just starting to explore software like Drupal are very grateful when you provide them with information about the software in their native language. These are two completely different points of view. I would like to know whether we host the documentation directly on drupal.org or use a static page generator like Vitepress.

    I don't think we can easily activate the multilingual features of Drupal for drupal.org. Nevertheless, I would be happy if we could localize the documentation for Drupal CMS.
    The question that concerns me here is whether this would really work if we did it through the documentation on Drupal.org. If the answer to that question is yes, then nobody needs any new system to continue offering the user guide in multiple languages.

Production build 0.71.5 2024