500 Internal Server Error when using Form in popup mode

When debugging the EditableFieldsFieldFormatter.php on line 230, it looks like it's going into a loop and if I comment out this whole part, the node is shown with the editable field.

// Popup version.
if ($this->getSetting('behaviour') === 'popup') {}

I think that the problem is in the return, because it is returning the same field on the same display and therefore it goes into a loop. If I change this to "teaser" it works. Now just the "Edit" link is displayed twice when I save the form in the popup.

Example of the working widget. As I mentioned above, I had to change the "Select fallback display mode:" to "teaser".

I'm not sure what would be the best fix for the code. As a temporary solution I am creating a new display view for that mode with the wanted fields and selecting it as a fallback.