Does not work with Layout Builder

The switcher doesn't actually work when I place the views block in layout builder and click the switch it doesn't actually switch. The other view display in in the same view. Maybe i'm not understanding how this should work?