It seems Drupal doesn't like triggering the kill switch after headers have already been sent. This includes writing to the cache during the kernel.terminate event.
This can be reproduced with the minimal install profile with just replica_cache enabled and configured as per the README, as well as $config['system.logging']['error_level'] = 'verbose';
for verbose error reporting.
Simply login to the site and go to /admin and the error appears.
RuntimeException: Failed to start the session because headers have already been sent by "/app/vendor/symfony/http-foundation/Response.php" at line 368. in Symfony\Component\HttpFoundation\Session\Storage\NativeSessionStorage->start() (line 152 of /app/vendor/symfony/http-foundation/Session/Storage/NativeSessionStorage.php).
Full backtrace:
Symfony\Component\HttpFoundation\Session\Storage\NativeSessionStorage->start() (Line: 164)
Drupal\Core\Session\SessionManager->startNow() (Line: 118)
Drupal\Core\Session\SessionManager->start() (Line: 323)
Symfony\Component\HttpFoundation\Session\Storage\NativeSessionStorage->getBag('attributes') (Line: 249)
Symfony\Component\HttpFoundation\Session\Session->getBag('attributes') (Line: 271)
Symfony\Component\HttpFoundation\Session\Session->getAttributeBag() (Line: 81)
Symfony\Component\HttpFoundation\Session\Session->set('ignore_replica_server', 1612152100) (Line: 69)
Drupal\Core\Database\ReplicaKillSwitch->trigger() (Line: 139)
Drupal\replica_cache\Cache\ReplicaDatabaseBackend->triggerKillSwitch() (Line: 98)
Drupal\replica_cache\Cache\ReplicaDatabaseBackend->setMultiple(Array) (Line: 181)
Drupal\Core\Cache\DatabaseBackend->set('module_implements', Array, -1, Array) (Line: 180)
Drupal\Core\Cache\ChainedFastBackend->set('module_implements', Array) (Line: 337)
Drupal\Core\Extension\ModuleHandler->writeCache() (Line: 42)
Drupal\Core\EventSubscriber\RequestCloseSubscriber->onTerminate(Object, 'kernel.terminate', Object)
call_user_func(Array, Object, 'kernel.terminate', Object) (Line: 142)
Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch(Object, 'kernel.terminate') (Line: 100)
Symfony\Component\HttpKernel\HttpKernel->terminate(Object, Object) (Line: 32)
Stack\StackedHttpKernel->terminate(Object, Object) (Line: 684)
Drupal\Core\DrupalKernel->terminate(Object, Object) (Line: 22)