Migrate documentation into the new system

I’ve opened migration for the remaining books and added links to the issue summary:

• Site Building Guide →
• TO BE MIGRATED →
• Reference →
• Reference →
• Tutorials and site recipes →

Migrating or deleting book pages directly helps the Drupal.org migration past Drupal 7. The book module and content type will not be on the site in the future.

Now-useless pages can and should be deleted. I think https://www.drupal.org/migration-archive → might have been the place to collect pages to be deleted.

(fixing link)

@drumm, thanks for making a new list.

I have skimmed through all those except "Migration archive" and did not find anything that I thought need to be moved to the new documentation system. I think this is also supported by no one else has come along and moved pages either.

Because you said that "Migration archive" was probably for items to delete I tried to move "Reference" there. Instead I get get this error, "You tried to migrate over 30 child nodes. Usually a guide consists of a smaller number of pages. To prevent possible mistakes, please select a smaller sub-set of book pages for migration at a time." I tried 3 more times with smaller sections but kept getting the same message. So, it there some way to make it easier to move them?

I suppose that question applies to deleting the pages as well. Although, I must admit I don't feel that I have the knowledge to delete community history.

I have skimmed through all those except "Migration archive" and did not find anything that I thought need to be moved to the new documentation system. I think this is also supported by no one else has come along and moved pages either.

I think that is a too broad statement. Firstly, there is a lot of content under these guides. I skimmed too and found a lot that is useful, but needs updating. Secondly, people are finding the pages using search and are happy with that. The don't care which system the pages are in.

Anyway, we aren't interested in moving this mostly D7 stuff to the new system. We don't have the resource to do it properly. So either we delete or we make a static archive of the pages. (A minor clean-up before archiving is necessary.)

Thanks for looking at this. Some pages on the https://www.drupal.org/to-be-migrated → list have been migrated, for example:

OLD
Disable Drupal (>=8.0) caching during development
https://www.drupal.org/node/2598914 →

NEW
Disabling and debugging caching
https://www.drupal.org/docs/develop/development-tools/disabling-and-debu... →

Since the old page can now be deleted and removed from https://www.drupal.org/to-be-migrated → , should we add a new "Page status" called "Migrated", or should I create an issue and request deletion of the old doc page? Or is there another method?

The reference book does look like it has a lot that can be deleted, and may have some hidden gems.

I'd like to go ahead and bulk delete:

• PHP block snippets https://www.drupal.org/node/21867 →
• PHP block visibility settings https://www.drupal.org/node/60317 →
• PHP page snippets https://www.drupal.org/node/23220 →

and their child pages. The PHP input format and block visibility are anti-patterns now.

I deleted https://www.drupal.org/node/2598914 → and replaced with a redirect for the new page, thanks ressa.

We do need child issues like #2832228: Review and migrate 'To be migrated' book → for batches of pages that are achievable. Each page can be:

• Migrated
• Deleted with no replacement
• Deleted and replaced with a redirect

@drumm, Deleting those sections makes sense to me.

Yes skimming is a broad statement. What I did was go through an overwhelming majority of the pages. I was relying on the title and the heading to be indicators. And I did not keep track of the tens of pages that I read more closely. I was focusing on finding pages that I thought were worth saving.

I skimmed too and found a lot that is useful, but needs updating.

If you list those pages then we can work on were to move them to.

Secondly, people are finding the pages using search and are happy with that

I have no evidence to support that. What I do know is that there are issues in the core queue for correcting links to any of the 'older' documentation.

I have made some issues as suggested.

It may make sense to take a different approach. Instead of a small group hunting for pages to keep it could be better to ask the community. Say, make an issue that section X will be deleted by some date. And invite people to provide links to the pages they find useful.

Checking which pages in the TO BE MIGRATED section are worth to migrate seems a time-consuming activity. There could be already documentation pages that cover the same topic, the covered topic is obsolete / not relevant anymore, or the information provided is not (anymore) correct.

Even with more people doing the task, it could take time to handle all those pages, or even to agree on what to do which every one of those pages.

I agree with apaderno - we should minimize the effort used to clean-up.

If quietone and ressa create issues with suggested deletions, I'm happy to review (and delete). Then we are only 3 people "wasting" our time.

It helps with "task" issues like https://www.drupal.org/project/documentation/issues/3441968 →

@drumm: Yes, go ahead and delete PHP block snippets, PHP block visibility settings, PHP page snippets

I skimmed too and found a lot that is useful, but needs updating.

@hansfn, What is your plan for these pages? Can you share the links?

My plan / wish is to remove as much as possible because we 1) don't have the resources to update them and / or 2) IMO we add too much non-Drupal specific docs. That doesn't mean that these pages aren't useful ...

In general, to repeat myself, the problem is what to do with D7 specific stuff. We have migrated a lot already that we maybe would deleted now.

These are now deleted:

• PHP block snippets https://www.drupal.org/node/21867 →
• PHP block visibility settings https://www.drupal.org/node/60317 →
• PHP page snippets https://www.drupal.org/node/23220 →

Code used was:

$page = node_load(23220);
$queue = book_menu_subtree_data($page->book);
$nids = [];
while ($item = array_shift($queue)) {                                                                                                                                                                         print $item['link']['nid'] . ' ' . $item['link']['title'] . PHP_EOL;
  $queue += $item['below'];
  $nids[] = $item['link']['nid'];
}
node_delete_multiple($nids);

So deleting more trees of pages is technically straightforward.

And agreed on this not being an ideal way to spend time, but this is what we need to do whenever there is a migration to something new. There is twice as much technical debt as long as there are 2 ways to do documentation. Unless we bulk delete everything or auto-migrate outdated/etc book pages to the current system, there's unfortunately no way around the sheer volume of docs that were a good idea at the time.

I get overwhelmed when I start looking at this. So much nice stuff contributed over the years, but auto-migrating it will probably just create a maintenance nightmare.

Anyway, @drumm could you delete all sections named something with D5 / D6 Migration archive → ? Maybe all of it is D5 / D6, but quicker to review when the obvious stuff is gone.

As I completed #3442307: Decide what to do with 'Tutorials and site recipes' → I added 3 new capabilities:

• Removed the 30-book-page cap on bulk migrations. However, migrations via the UI still only migrate into child pages of a single guide. So this is actually only limited usefulness. https://git.drupalcode.org/project/drupalorg/-/commit/7a6290a27763e966fa...
• Added a drush command to flatten a tree of book pages into a single page. https://www.drupal.org/docs/7/howtos/book-drupal-7-the-essentials → had many chapters with ~15-25 tiny child pages. This command recursively merges the child pages into the parent page, adding h2/h3/etc headings for the titles, moving attached files to the parent, and adding redirects to replace the deleted pages. https://git.drupalcode.org/project/drupalorg/-/commits/7.x-3.x?ref_type=...
• Drafted a process for migrating a book page to be a guide. After a couple og_menu-related issues, this looks like it is working well. However, on next edit, the character limit for the guide’s description will be enforced; that’s probably why we didn’t build this before. This can be adapted to move a tree of book pages with hierarchy if needed.

Anyway, @drumm could you delete all sections named something with D5 / D6 Migration archive? Maybe all of it is D5 / D6, but quicker to review when the obvious stuff is gone.

Done, and I’ve reviewed & completed the deletes requests in the child issues.

I looked more closely at the Migration archive → tonight. It mostly for Drupal 5 and 6 with some Drupal 7 or 8 mentions, but all outdated. The exception could be the archive of past events → . There might be community history there that is not anywhere else. For example this page → lists the developers who were at the sprint in Antwerp 2005.

2008-03-03: DrupalCon Boston 2008 > Contributing to Drupal: A guide for everyone → should allow to download slides, but all the given links return a 404 error.

Since there are child issue, adding meta to the title.

The links mentioned in #87 are for lullabot.com, so d.o never stored that data.

I meant that a page giving only three links which return a 404 error is not much helpful. It could be probably removed too.

Agreed the past events should be migrated.

I updated https://www.drupal.org/node/233045 → to removed the dead links.

The migration tooling so far has been really geared toward discouraging huge hierarchies of pages, but this section does greatly benefit from hierarchy, and should be migrated with some of it. I may be able to do it with some manual work to get additional automation going. In the meantime:

• Should the year-level of pages, like https://www.drupal.org/node/299564 → for 2006, be removed? So there is a simple list of all the events here.
• Should individual events, like https://www.drupal.org/forum/general/news-and-announcements/2006-08-07/2... → , or Summer of Code years, like https://www.drupal.org/google-summer-of-code/2006 → be merged into a single page, with headings for the sub-pages?
• Where would be a good place for these pages to land in a documentation guide?

Is there some way we could migrate the events to d.o - community/events/past-events → The content itself is probably not that useful, but the fact the events took place is nice to preserve.

And maybe all the Google Summer of Code events could be removed - we have https://groups.drupal.org/google-summer-code

Is there some way we could migrate the events to d.o - community/events/past-events The content itself is probably not that useful, but the fact the events took place is nice to preserve.

Some of the old events looked like they would be a bit large to flatten into a single page, which would be required for moving into community events.

And maybe all the Google Summer of Code events could be removed - we have https://groups.drupal.org/google-summer-code

We also need to figure out the best way to handle groups.drupal.org, which could end up being a static archive. I wouldn't move anything there. That did lead me to find https://www.drupal.org/community/contributor-guide/reference-information... → , where those pages could land.

Some of the old events looked like they would be a bit large to flatten into a single page, which would be required for moving into community events.

I was thinking just keeping the title - really. Looking at https://www.drupal.org/node/46559 → (as an example) the sub pages doesn't have much value.

Yes, if History of Google Summer of Code → was turned it a doc guide, you could just move the GSOC stuff there.

Found this issue about pages in the 'Migration Archive'. #1243274: Plan to archive DrupalCon/event and GSOC/GCI content from drupal.org → . I haven't read it in depth and am adding this comment as a reminder to do so.

Just updating the IS to strikeout the recent fixes.

I think the only remaining work here and in child issues is for me to automate bulk moving the remaining 5,806 book pages.

Of course, if there is any remaining that is outdated or duplicitous, and can be deleted, please call that out and that can be done.

I got together a quick script to migrate book hierarchies and have migrated:

• https://www.drupal.org/drupalorg/docs/archive-of-past-events →
• https://www.drupal.org/community/contributor-guide/reference-information... →

$page = node_load(33887);
$destination_guide = node_load(28580);
$test_mode = TRUE;
//$test_mode = FALSE;

if ($destination_guide->type !== 'guide') {
  print 'Not a guide!';
  return;
}

$queue = book_menu_subtree_data($page->book);
$nids = [];
$mlid_nid = [];
while ($item = array_shift($queue)) {
  if (empty($mlid_nid)) {
    $mlid_nid[$item['link']['plid']] = $destination_guide->nid;
  }
  $mlid_nid[$item['link']['mlid']] = $item['link']['nid'];
  $queue += $item['below'];
  $nids[$item['link']['nid']] = [
    'parent' => $mlid_nid[$item['link']['plid']],
    'type' => empty($item['below']) ? 'documentation' : 'guide',
  ];
}

foreach (node_load_multiple(array_keys($nids)) as $node) {
  print $node->title . ' (' . $node->nid . ') → ' . $nids[$node->nid]['parent'] . PHP_EOL;
  if ($node->type !== 'book') {
    print 'SKIPPING ' . $node->type . PHP_EOL;
    // TODO if this was to be a guide, bad things will happen!
    continue;
  }

  $node->type = $nids[$node->nid]['type'];
  $node->language = 'en';

  // Remove the page from book schema.
  if (!$test_mode) {
    book_node_delete($node);
  }
  $node->book = [];

  $node->menu = [
    'link_title' => $node->title,
    'menu_name' => 'menu-og-' . $nids[$node->nid]['parent'],
    'parent' => 'menu-og-' . $nids[$node->nid]['parent'] . ':0',
    'plid' => 0,
    'enabled' => 1,
  ];

  // Do not create alias during intermediate save.
  $node->path = ['pathauto' => FALSE];

  // Create a revision for the migration.
  $node->revision = TRUE;
  $node->log = t('Migrating to new documentation system.');

  $node_wrapper = entity_metadata_wrapper('node', $node);

  // Summary is a new field for documentation. Lets auto-fill it
  $node_wrapper->field_summary->set(truncate_utf8(strip_tags($node->body[LANGUAGE_NONE][0]['value']), 140, TRUE, FALSE));
  if (!$test_mode) {
    $node_wrapper->save();
  }

  // Use normal path aliasing.
  $node->path = ['pathauto' => TRUE];

  // Do not create a revision for the second save.
  $node->revision = FALSE;

  // Empty out summary attached to body field.
  $node->body[LANGUAGE_NONE][0]['summary'] = '';
  // Open comments.
  $node->comment = COMMENT_NODE_OPEN;

  // Set the group reference for the node. After the initial save so field
  // changes have kicked in.
  $node_wrapper->og_group_ref_documentation->set(intval($nids[$node->nid]['parent']));
  if ($node->type === 'guide') {
    $node_wrapper->{OG_GROUP_FIELD}->set(TRUE);
    $node->og_menu = TRUE;
    $node_wrapper->field_new_page_and_guide_review->set(1);
  }
  if (!$test_mode) {
    $node_wrapper->save();
  }
}

Now that all books have been triaged in child issues, we are left with 123 orphaned book pages. There’s some gems like https://www.drupal.org/teapot → that should have a place to land, and others that can be deleted.

@drumm, can you supply the list of links?

The orphaned pages are now at https://www.drupal.org/node/3522465 →

I took a first pass through deleting placeholders, obvious D6 & earlier, and a couple terrible security ideas. 101 pages remain

Deleted the docs for unsupported project

• Ember style guide
• Web Engagement Management
• Usage of the Default user picture styles module
• How to use the Faceted Search
• Moneybird Commerce
• Profile Location Configuration and Customization

Created an issue in these projects, that have a D8 stable release, to migrate the docs.

Forward module - 📌 Migrate module documentation from old book style Active
GA Push 📌 Migrate documentation from old book-style Active
Module - Visual Website Optimizer 📌 Migrate module documentation from old book style Active
Single Page Apps / 📌 Migrate module documentation from old book style Active
Term Merge 📌 Migrate module documentation from old book style Active

@drumm, The can be deleted but I don't have edit accees, the bitcahce → project is unsupported.

https://www.drupal.org/docs/extending-drupal/contributed-modules/contrib... →
https://www.drupal.org/docs/extending-drupal/contributed-modules/contrib... →

@drumm, I deleted the pages for "unsupported modules and modules only compatible with Drupal 6 or earlier" for the list of orphaned list here and the Contributed module documentation archive → .

What about the pages for modules that do not have a stable D7 release?

I deleted a couple pages that had docs on how to use jQuery which are readily available elsewhere on the internet, and some pages related to the obsolete Dreditor browser extension.

• Get Radio button value through jQuery
• Get Select box value through jQuery
• Filing a followup
• My program to count $R(x)$ and $\breve R(x)$
• Prefix Suffix Title -- this was 3 sentences of text related to a non-existent module and a couple comments saying this module doesn't exist
• Setting up a local test site to work on Git related patches for d.o -- content related to setting up Git to create patches for Drupal 6, no longer relevant and the updated version of this exists in the contributor guide.
• facebook rss... -- the page content was a note that said "In case you are wondering this is impossible ..." 🤷
• hirji time conversion in drupal -- some example PHP code for doing date conversions that is covered elsewhere on the internet and not specific to Drupal.
• style don't create the resource when clean URL is enabled with Nginx like inverse proxy --- this page contained a question about nginx that should have probably been a forum post, and had no answers or additional comments.

I did a quick pass through the orphaned book pages ( https://www.drupal.org/node/3522465 → ) and deleted a couple of pages for themes that didn’t have D7 or later versions.

The can be deleted but I don't have edit accees, the bitcahce project is unsupported.

And those have been deleted too, thanks!

What about the pages for modules that do not have a stable D7 release?

I think we have to keep those for now, there will be plenty of people with D7 sites that they need to figure out, and documentation has potential to help.

It's really great with all the clean up, though many Drupal 7 doc pages indexed in search engines now give a 404, since they have been deleted, without having a redirect to a replacement page, probably because there wasn't one.

This is unfortunate, since these pages may both be linked to from other pages on the internet, or on drupal.org, and the topics are still themselves relevant in Drupal 11, 12 etc. so people will still be looking for the info.

It's Drupal 7 pages like these:

https://www.drupal.org/node/1995428 →
Step 4 - Rename default.settings.php to settings.php
Install Drupal 7 in one-minute (or Drupal 6) Using Webhost Control Panel ... Step 4 - Rename default.settings.php to settings.php · Step 5 - Putting WAMP ...

https://www.drupal.org/node/2608620 →
Increase Maximum File Size
11 Jul 2017 — Although many other PHP settings are configurable by using ini_set() in the settings. ... Administer user profiles in Drupal 7 · Administering ...

As I see it, there are two options:

1. Handcraft precise redirects

   Either add a redirect to the corresponding page, if it exists, or a fitting section or page on drupal.org. This is labour intensive, and may not be realistic. Though, it could be considered for the top 50 of 404'ing Drupal 7 documentation pages?

2. Do an automatic search

   If a node is not present on drupal.org -- if possible, do a look up to somehow get the title of the deleted node, and do a search like https://www.drupal.org/search/site/Increase%20Maximum%20File%20Size?f%5B... → , perhaps with a text like "Sorry, this page is gone, but we did a search, maybe it can help?"

We can continue putting redirects in by hand. Where should the two pages you linked go to?

We do keep records of deleted nodes, but now is not a good time to engineer new functionality as we’re migrating the site. That is something we could build once documentation is fully migrated, if the search for the previous page title is tested and shown to be useful.

If I recall Correctly, The page "Install Drupal 7 in one-minute (or Drupal 6) Using Webhost Control Panel", was Written by me , and I had wanted it deleted anyway. because it was completely outdated. My Opinion, get rid of it. thank you. I wouldn't have deleted it myself, years ago, but I did not have that level of authority.

The page "Increase Maximum File Size", If I remember correctly, Was a huge mass of mostly irrelevant comments, But which terminated with a detailed explanation of mine, on how to ' Definitively' achieve an increase of file size, Keystroke by keystroke, mouse click by mouse click.. Addressing new bees common mistake of saving INI files. as Ini.txt file in windows. and other similar newbie errors, related simply to day-one-drupal-users lack of experience.

Aside from my detailed guide, which was actually in 2-parts, but I forget The differentiation between my two parts, If I review, I would let that page fade away into history. If you were gonna look at it, you might recognize what I am talking about, and you might pull my new. b information out into a brand new how-to guide, but aside from send it to oblivion please

"All the best; intended." ~ Granpa Chris

Thanks for a fast reply @drumm. I have experienced a fair number of drupal.org 404's recently, after doing a search for Drupal stuff in the search engines lately, so my best guess was that a bulk deletion had been done, without redirect. So when this issue appeared in my dashboard, I was reminded of it.

So those two pages are just examples I quickly looked up, to use as examples as dead links, so I don't have an opinion on a redirect, sorry.

Until the new Drupal is launched (and possibly automated search on title?) we could have a "404 Not found documentation pages" issue, where we continually add 404's, and then set up redirects?

Thanks for sharing @christopher james francis rodgers :)

Though in this case, the question is not so much about resurrecting pages, but making sure that users are redirected to a page or section about the topic. So a user may not be looking for a one-minute set up, but info about "default.settings.php" and "settings.php" and find that link in the search engine. But it's a dead end on drupal.org. So we could instead redirect them to something that is as close to that subject.

So the second example I used could be redirected to https://www.drupal.org/docs/8/core/modules/file/overview → .

Another thing: It might be worth considering adding a "This is archived/old content" banner or something, like the warnings that api.drupal.org just got, making it clearer that some of these are old or archived documentation pages. It could be added automatically at the top of all pages under certain paths? See for example, a page like this:

https://www.drupal.org/docs/extending-drupal/contributed-modules/contributed-module-archive/contrib-modules-for-building-the-site-functionality/dates-and-events/datecalendar/drupal-6-datecalendar/date-locale-time-zone

https://www.drupal.org/docs/extending-drupal/contributed-modules/contrib... →

A user may easily miss that this doc page is for an older version. A rule could be, that since it's under https://www.drupal.org/docs/extending-drupal/contributed-modules/contributed-module-archive a message like this could be shown at the top of the page?