File sanitation not removing unused replacement character and dedup issues

Created on 24 February 2025, 13 days ago

Problem/Motivation

When setting the file sanitation up, the unused replacement character is remaining as a valid character.

eg. if you have a file called `file_name.txt` and have a replacement character '-' the filename will not be changed. I would expect this change the file name to `file-name.txt`

Also if you have something like `_-` or ` _` and deduplication and replacement character of `-` then the previous 2 examples will change that part of the filename to `-` which is kind of inconsistent to in one place treating the the unused replacement character as valid to the replacing in another situation.

Proposed resolution

I see there 2 methods to treat this.
1. Treat the unused characters as invalid chars and convert them to the replacement character which would make it consistent with the deduplication process.
2. Add a new parameter to allow the user to decide if they want to remove the unused replacement characters and then resolve the problem with the deduplication.

I also think that there is a problem with dedup process in that it treats the underscore, hyphens and dot as a single character and will merge them together. eg. `_-.` will be converted into the replacement character.

🐛 Bug report

Status

Active

Version

10.2 ✨

Component

file system

Created by

🇦🇺Australia gordon Melbourne

Live updates comments and jobs are added and updated live.

Comments & Activities

Issue created by @gordon
Comment 12 days ago →
🇳🇿New Zealand quietone
Changes are made on on 11.x (our main development branch) first, and are then back ported as needed according to the Core change policies → .
Comment 11 days ago →
🇦🇺Australia kim.pepper 🏄‍♂️🇦🇺Sydney, Australia
Would be great to have a test that covered this.
Comment 6 days ago →
🇦🇺Australia kim.pepper 🏄‍♂️🇦🇺Sydney, Australia
I think this is the expected behaviour.

'_' is a valid separation character and santization only replaces invalid characters.

'_-' gets converted to '-' because we de-duplicate separation characters

contrib.social Blog FAQ Discussions

Production build 0.71.5 2024