Missing *.min.js.map files when aggregation is on

I am getting this in the Chrome dev console and it might just be a development issue vs. regular use issue.

DevTools failed to load source map: Could not load content for https://mysite/files/css/optimized/admin-toolbar.css.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE

Hello,
Same Problem on drupal 9.5.10

DevTools failed to load source map: Could not load content for http://127.0.0.1/sites/default/files/js/optimized/dialog-min.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://127.0.0.1/sites/default/files/js/optimized/index.umd.min.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://127.0.0.1/sites/default/files/js/optimized/jquery.form.min.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://127.0.0.1/sites/default/files/js/optimized/shepherd.min.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://127.0.0.1/sites/default/files/css/optimized/style.css.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE

@savage2003 thanks for your work. I patched the AssetOptimizer.php and the messages do not apper anymore.
#14 works for me.
Thanks,
Marcel

#14 worked for me on Drupal 10.2.2 & advagg 6.0.0-alpha1, thanks @savage2003.