Document high-level API concepts in an easier format

I have been thinking about this a fair bit, sparked by recently spending way too many hours to get some documentation to display correctly on api.drupal.org. Once that task was done, the next thing I did was look at what Symfony does. The reStructuredText (RST) format certainly looks much better for writing example code than what we have now. As for prose, I don't think there is as much of a difference, but still RST is easier. I did note that in the Symfony docs one can't view a single php file.

For the proposals

1. Doing this is more likely to lead to improvements than the longer path of creating an issue, MR, and so.

2. What I think about here are, in no order, a) how the docs will be structured, b) need to follow a page of instructions to get setup to contribute, c) who has commit access, d) may not improve the documentation, e) adds a new place to find documentation without removing one. But I do think this should be pursued. And proposal 1 can happen independently of the decision on this.

And with both 1 and 2 we don't have to wait 1 week for api.drupal.org to be re-parsed for the doc to be updated.

Agree with most of the proposed resolution. My thoughts:

1. Move long form texts to a new /docs folder in core (and contrib). The folder contains markdown files. Publish the docs via mkdocs or any other static site generator (SSG) that core prefers. mkdocs is the the SSG behind DDEVand Drush docs. We already support this in our Gitlab templates. Benefits:
   1. Once we are fully on Gitlab, it becomes super friendly to non devs and lazy devs to submit an MR via the Gitlab UI to change the docs.
   2. Docs can change along with code
   3. Docs automatically vary by branch.
2. Retire the api module and use Doctum or similar external project to generate our API docs. We can start with Drupal 12, and keep the API module docs as more or less static for older versions.

> I really dislike using phpdoc for prose.

Long paragraphs of text which aren't tied to a particular piece of code could definitely be moved into non-PHP text files.

Personally, I find it fiddly to have to keep thinking about word-wrapping all the time, and text files wouldn't fix that, but IDEs do handle it wrapping text these days.

It's worth bearing in mind that API module parses things like function and class names into links, as well as @see tags (and probably other things too), so that we'd need more than just an OOTB parser.

> The Drupal handbook can be updated by anyone with permissions, but it's never clear to me how or when to make a big structural update to pages

I've been saying for years we have a problem with our handbooks, and it's partly technical and partly cultural. I remember saying this around 2018 when I applied to be Documentation maintainer.

The difficulty with making big changes is both technical, because changing structure requires advanced permissions, and also requires changes to multiple fields in multiple nodes. Contrast with something like Wikipedia, where structure is just content -- you change ToCs and categories which are inside the content.

It's also cultural because there isn't the invitation to make those kinds of changes. People can't tell that it's ok. It feels like you might be treading on people's toes if you make big edits. Again, on Wikipedia, there is the invitation to 'be bold', which is backed by the ease of reverting edits. With Drupal nodes, comparing or restoring revisions is difficult, and again, because what amounts to one big structural edit might be materialised in a number of different nodes, a reversion might not be properly made.

My worst example of this is the documentation about installing Drupal. Unless it's been fixed recently, you start off on one page, then you immediately get sent somewhere else if you're installing with Composer, and then THAT page turns out to be deep in a HUGE docs section all about low-level Composer stuff, when all you wanted was the commands to get started. I've been wanting to fix this for YEARS, and I don't know where to start. On Wikipedia I'd've just deleted a bunch of pages and merged things.

We're also REALLY bad at keeping our docs up to date.

I needed to look up hook_schema() stuff the other day, and the best thing I could find was in the Drupal 7 section: https://www.drupal.org/docs/7/api/schema-api/data-types → .

The huge advantage of putting docs in the same repository as Drupal core is that the docs are kept versioned.

So I'd be very strongly against moving high-level docs from the core repository into the handbook.

> Find another way to provide version controlled high-level API documentation, whether .rst or a similar format, potentially rendering this on api.drupal.org using

This gets a +1 from me, though as the risk of bikeshedding, why rST and not Markdown, which we already use for README files? That would have the advantage that we can easily move content between README and docs files.

> And with both 1 and 2 we don't have to wait 1 week for api.drupal.org to be re-parsed for the doc to be updated.

That's something that we could change, though couldn't we?

For example, if docs files all live in a MODULE/api folder, the API module could be made to store a hash of that folder and compare that more regularly to know whether to re-parse it.

I've also been saying for years that when Drupal 8 started, we left our API docs stuck in the D7 days. We document hooks in API docs, but we still haven't added documentation for all our other structures and extensible surfaces -- plugins, plugin types, services, events, etc etc etc. Instead of a MODULE.api.php file, modules should have a MODULE/api folder, which contains a hooks.api.php file and other things too. So new documentation text files could go in there.

This gets a +1 from me, though as the risk of bikeshedding, why rST and not Markdown, which we already use for README files? That would have the advantage that we can easily move content between README and docs files.

I thought everyone used markdown until I found out Symfony uses rST, I would also prefer Markdown unless there's some specific limitation to using it that rST solves, I assume for Symfony it's because they use Sphinx to generate the HTML pages with navigation etc. - we would need something like that.

I have updated the issue summary with the options from #5.

And I thought I should mention that Symfony uses an internal tool, the Symfony Docs Builder to build their docs.

I do have a question about documentation that is not in *api.php file. We have a lot of documentation in class and method docs, including code examples. What happens to that? After some browsing through Laravel and Symfony API I didn't find an example where any class or method had an extra paragraph for explanation or example code. Anyone have an example?

I think replacing the API module with something else is out of scope for this issue, and a pretty complicated topic. API module does a lot of things, many of which are specific to Drupal.

This issue should just be about moving long topics in PHP files which aren't attached to specific PHP code into text files, and adding functionality to API module to parse them.

It's a shame that we didn't invent a Drupalism for standalone topics, as we've used @defgroup:

 * @defgroup hooks Hooks

but @defgroup is also used to group code-specific documentation together.

If we'd invented, say, a @topic tag, it would have made deprecating it easier!

I do have a question about documentation that is not in *api.php file. We have a lot of documentation in class and method docs, including code examples. What happens to that?

I think that in most cases that would stay where it is at least at the beginning. But once we have the longer form docs, we might want to move some of those longer explanations or code examples there if they're sufficiently high level.