Refactor bootstrap to better utilize the kernel

Problem/Motivation

Drupal's bootstrap is independent of the Kernel, meaning our path towards Kernel based testing is rocky.

Proposed resolution

Make the kernel bootstrap Drupal and itself by adding a static kernel factory method to Drupal\Core\DrupalKernel thus DrupalKernel::createFromRequest(Request $request);Refactor index.php/installer/etc to use this instead of bootstrap phases. We will need to provide backwards compatibility for drush so drush si and drush en aren't broken (used by testbot).

Remaining tasks

None

User interface changes

None

API changes

conf_path() gets deprecated in favour of DrupalKernel::getSitePath() or DrupalKernel::findSitePath()

Super globals are removed from the bootstrap process and drupal_bootstrap() becomes deprecated. Instead DrupalKernel::createFromRequest($request) can be called to retrieve a kernel then the boot method called in intialize the container and other systems. Alternatively, a kernel can be directly instantiated and the boot() method called. If manually creating a kernel, Settings must be manually initialized.

Original report by @username

We currently do quite a bit of work before we can instantiate DrupalKernel. Because of this, early configuration and servies that would normally go on the service container are configured by global variables. We also have code in this early bootstrap that requires access to the request (conf_path, drupal_environment_initialize, request_uri, drupal_valid_test_ua, etc), but we don't have the request until DrupalKernel::handle() is called, which is too late, so we're still accessing global $_SERVER. I hope we can all agree that needing access to global request variables defeats the purpose of using Symfony's HttpKernel. These are fixable problems for a running Drupal site. We can move things like conf_path to kernel wrappers (middleware) that get the request and set environment variables that DrupalKernel can pick up (a bit cleaner than globals). One problem with this approach is our installer. It requires variables like $databases and $config_directories to be mutable globals. Since we have a different front controller for the installer (install.php), this is fixable, just a PITA. The real problem comes with simpletest. It fires off a batch api request, which runs through the main front controller. Drupal is loaded like it was a normal request, then WebTestBase::setUp changes a bunch of global variables around and calls the installer functions directly. This carefully orchestrated global mutation makes it impossible to encapsulate our bootstrap configuration. I worked on this for several days, using a number of different approaches, and each failed in new and unexpected ways because of simpletest and the installer. We also don't have unit tests for what would break here, making the process of refactoring prohibitively difficult. I very much wanted to start this issue with a patch, but I've got nothing. I tried to trim down what I was working on to something simple that was at least a step forward, but the best I can do is just move our existing code around. I don't see a way forward while WebTestBase exists, but I'm hoping someone else sees something I don't. If this cannot be fixed in D8, we might want to consider rethinking how we use DrupalKernel so that it isn't an awkward mix of our old bootstrap and what a Symfony app normally looks like.