There are 2 issues when using HTML5 widget with unlimited color values:
1. there is no way to remove a value. This is because the input will always return #000000, as it doesn't support null value (tested in chrome)
2. a new #000000 value will be added on each edit and save even if no color has selected (this is caused by the same problem, the html5 color input doesn't support null value)
For HTML5 widget I suggest to remove the empty element on edit and to add a "remove" action for each element.
2.0
Code
The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.
Not all content is available!
It's likely this issue predates Contrib.social: some issue and comment data are missing.
Core now provides a button to remove any value in a multivalue field, since 📌 Allow for deletion of a single value of a multiple value field Fixed was merged. I will leave this issue open so that Drupal sites older than 10.2 can use this patch, but I don't intend to merge this in.
That said, there could be value instead to suppress the default behaviour of adding a new value whenever the entity is edited. I will open a separate issue for that.