add a way to manually use the test site from a Functional test

Could the MR be updated for 10.1 please.

Added control of this feature with a BROWSERTEST_PRESERVE_SITE env var, and added documentation to phpunit.xml.dist.

Sounds like a useful idea.

I usually have to look at screenshots in the pipeline to see what went wrong.

Reading the comments not 100% sure how to get it working but maybe the CR will help clear that up.

Let's have this documented in the wiki and the change record can just refer to that. I think this is the page, PHPUnit Browser test tutorial → .

Proposed documentation text:

Interacting with the test site

The test site can be preserved after the test has run, to allow the developer to interact with it as with a normal site and to inspect the database tables. This is useful for debugging and developing tests, when the HTML output files are insufficient to understand a problem.

To preserve a test site instead of having it be deleted in the test’s tear down phase, do one of the following:

• Set the class property $preserveSiteInTearDown to TRUE in your test class
• Set the environment variable BROWSERTEST_PRESERVE_SITE to ‘true’ on the command line when running phpunit
• Set the environment variable BROWSERTEST_PRESERVE_SITE to ‘true’ in your phpunit.xml file

Then, after the test has run, do the following:

1. Find the test site prefix. You can get this from the test site directory which was created during the test in your sites/ directory. It will be of the form ‘simpletest/test_123456'. The ‘test_123456’ is the test site prefix.
2. Set up the sites.php file if it is not already present.
3. Add code to return the test site directory as the site. The simplest way is to completely replace the main site with the test site:
   

   $sites = [
   ‘example.com’ => ‘simpletest/TEST_PREFIX',
   ];</ol>
   

You can then use your site as you would a normal Drupal site. The admin user login is ‘admin’/‘admin’.

You will need to delete the site directory and the test site database tables manually.

—————-

TODO: it occurs to me writing these instructions that it would be a nice DX touch to output the test site prefix in the phpunit output when the site is being preserved.

PS. It’s not a wiki!!!

Wrote CR & draft documentation.

Ready for review again.

Read the Cr and feel comfortable removing that tag.

Leaving the documentation tag as don't know the process for getting that page updated.

But thinks seem clearer based on what's there. Going to mark.

I have been wanting to test this and today I finally did. I am pleased to say that it works in my environment which is DDEV and with a separate database for testing.

TODO: it occurs to me writing these instructions that it would be a nice DX touch to output the test site prefix in the phpunit output when the site is being preserved.

Yes, that would be a nice to have.

The text in #14 for the doc page is fine with me. It does help to see it here though instead of waiting for this to be committed.

I left comments and suggestions in the MR, about the comments, that do need to be looked at.

> Yes, that would be a nice to have.

Done, but I suspect this is going to bog this issue down, as the way I've done it isn't ideal and I don't see a better way.

It's not ideal because printing from within the test process causes PHPUnit to say the test was risky because it printed something. I think that's acceptable, because the developer knows they're doing something for debugging.

I experimented with using a PHPUnit extension, same way that HtmlOutputLogger is a PHPUnit extension, but that doesn't work because outside of the test's process, we can't get the site ID.

The only way that I can think of would be to either write the ID to a file, or have the PHPUnit extension look at all the folders in sites/simpletest and pick the newest one.

Re-ran unit test multiple times but unfortunately fails each :(

Fixed text, and tweaked the wording of the later paragraph as it says 'this' and 'it' for things which are really far away now.

Trying to keep up with this one quicker.

So now it appears to be nightwatch failing consistently? Re-ran 4 times

I don't see how Nightwatch can be failing because of this??

I'm running with this patch on a contrib project to see why its test are failing. Without this patch, the JS test randomly passes. With this patch, it gives me the following output, mainly: Test Printed Unexpected Output

Assertion Succeeded (Constraint: is equal to 'pt-pt', Value: 'pt-pt')
Post Condition Method Called (Drupal\Tests\tmgmt\FunctionalJavascript\TranslatorJavascriptTest::failOnJavaScriptErrors)
Post Condition Method Finished:
- Drupal\Tests\tmgmt\FunctionalJavascript\TranslatorJavascriptTest::failOnJavaScriptErrors
Test Passed (Drupal\Tests\tmgmt\FunctionalJavascript\TranslatorJavascriptTest::testMultipleLocalToSingleRemoteMapping)
After Test Method Called (Drupal\Tests\tmgmt\FunctionalJavascript\TranslatorJavascriptTest::tearDown)
After Test Method Called (Drupal\Tests\tmgmt\FunctionalJavascript\TranslatorJavascriptTest::tearDownErrorHandler)
After Test Method Finished:
- Drupal\Tests\tmgmt\FunctionalJavascript\TranslatorJavascriptTest::tearDown
- Drupal\Tests\tmgmt\FunctionalJavascript\TranslatorJavascriptTest::tearDownErrorHandler
Test Printed Unexpected Output
Test site is preserved with site ID 75952671.

Test Considered Risky (Drupal\Tests\tmgmt\FunctionalJavascript\TranslatorJavascriptTest::testMultipleLocalToSingleRemoteMapping)
This test printed output: Test site is preserved with site ID 75952671.

That's normal.

The test prints this:

> This test printed output: Test site is preserved with site ID 75952671.

so that you know how to access the test site. Without it, you'd have to do an ls of the sites/simpletest (IIRC) folder and find which numeric-named folder is the most recently created.

Is there any way around that? A Risky test makes me think something in the test failed. If no way around, we should definitely mention that implication in the CR.

Not easily.

The way to produce output in PHPUnit is with a PHPUnit extension, similar to how we handle HTML output with HtmlOutputLogger.

But the test site ID is inside the test, and a PHPUnit extension doesn't have access to that data.

The way that HtmlOutputLogger manages to pass information between the two systems is by setting an env variable when the test starts, which gives the location of a file to write to. The test can then use tat env var to write a list of written HTML files:

HtmlOutputLogger extension:

    if ($this->browserOutputFile) {
      touch($this->browserOutputFile);
      putenv('BROWSERTEST_OUTPUT_FILE=' . $this->browserOutputFile);
    }

In the test:

  protected function initBrowserOutputFile() {
    global $base_url;
    $browser_output_file = getenv('BROWSERTEST_OUTPUT_FILE');

We'd have to do something similar: create a PHPUnit extension which sets and env var which is a filepath, and the test system writes the test ID into that file, and after the test has run the extension can output a message about the site ID.

I don't think the test can write the site ID to an env var, as then IIRC PHPUnit will complain that the env has changed during the test.

I wasn't sure it was worth that much effort to write all that, seeing as if you're enabling this for a test, it's probably that your test is failing anyway.