At the moment, these fields are added to the entity response automatically: type, entity_type, bundle, id, label, transform_mode.
There can multiple reasons for not exposing all this information and letting the user make the decision.
Thus, I suggest we show all returnable fields in the configurations page along the entity's other fields.
Further, exposing the fields in the configurations, it eases the (re)use of plugins and altering the fields before showing - e.g. type casting the ID as integer.
I am currently unable to assess how much effort it requires to make this change.
Active
1.0
Code
Do be aware that only ID and label are actually fields. However they are also fields that are not exposed for display purposes, otherwise they would already show up.
There might be other ways to add these in a configurable manner on transform modes, like display plugins or directly on the transform mode.