Don't be misled by the "Testing result:" metric on the project page: this module has almost no tests. (It hasn't been very difficult to guarantee a workable state by manually testing things here and there after working on bugs/new functionality... but it's getting there.)
Three types of tests are needed:
1. Functional tests covering the login procedure.
A lot of functionality (e.g. combinations of configuration settings) cannot be tested on a lower level than simulating the full login flow, because we need to exercise the SAML PHP Toolkit library. This means we need to generate SAML messages - and AFAICT there isn't another feasible way than to install an IdP locally and simulate the login flow through that, using browser tests.
For a local / custom setup, SimpleSAMLphp can be installed, with some simple/custom authentication provider. (https://github.com/Jamedjo/docker-test-saml-idp is a good basis.)
It would probably be nicer if we could use LightSAML/LiteSAML instead, which I have no experience with yet but which looks promising at a first glance. We could hopefully create a session-less IDP setup (with just a hardcoded list of test users that are automatically logged in) that can work on the same domain.
I'm not sure that installing SimpleSAMLphp in any way as a dev dependency would work on drupal.org's test infrastructure. Installing LightSAML/LiteSAML as part of a test module should work. Unfortunately it seems the old LightSAML-IDP library is abandoned and has not been implemented on top of the newer LiteSAML yet, so that would take work. (I'm updating the project links on a documentation page → .)
2. Tests that exercise user creation / synchronization.
This is an important part of the module functionality, and can hopefully be done using kernel tests rather than needing browser tests / category 1.
But not yet, because our dependency on externalauth v1 has required us to write spaghetti code.
In order to keep our current guarantee of not saving user accounts with corrupted data and be able to write more cleanly separated code, we'll need to either start using our own copy of the ExternalAuth service or work to get it incorporated into a new version of the externalauth module.
Update: I was documenting plans for that new service at #3179148-6: Redo authmap_alter / register events. → . But after giving it more thought, I believe that externalauth is better off getting a lot more extension work than that. The result is 🌱 Datahandler plugins, (optionally) mappable from login providers Active and that's going to be a lot of work. The end result, if that ever happens, is that the bulk of user creation/synchronization work (plus tests) is all moved into externalauth.
3. Miscellaneous tests
I have not been diligent in adding tests for other parts of the module functionality that are not dependent on category 1 and 2. (For instance: checking that the configuration form works well, or the "Use Drupal login, bypassing SAML IdP" permission.) This is because the first two categories of tests are more important, and it was already a challenge to keep up with the issue queue, given the time I have to work on this module. (Whether that's a good excuse, I don't know.)
Test reuse
Category 1 tests can be incorporated by other SAML modules (simplesamlphp_auth, saml_sp, etc.) if they want. Category 2 may be used as a basis by those same modules, though it will need rework.
(simplesamlphp_auth is in the same boat as us: there are tests, but the actual most important functionality isn't tested. saml_sp has no tests.)
Active
3.0
Code
Not all content is available!
It's likely this issue predates Contrib.social: some issue and comment data are missing.
Thanks. In 2023 I've finally dived into Docker enough that I can hack my way around projects, and I'm using this for a local test setup now. (With modified saml20-sp-remote.php containing various test SP data.)