Exposed form not displayed when using "Input required" option

It works if I select the "Always build output" option..

The problem appears to be this:

class ViewfieldFormatterDefault extends FormatterBase {
..
  public function viewElements(FieldItemListInterface $items, $langcode) {
..
        if (!empty($view->result) || $always_build_output) {

This doesn't have any logic to check if the exposed form is supposed to be displayed.