Proposal: create two new stream-wrappers in core:
- public-cache://
- private-cache://
These stream-wrappers should shadow the behaviour of their equivalents (public:// and private://), but be used for storing files that can be regenerated using other sources: for example, image-styles and aggregated CSS/JS should use these new stream-wrappers.
Background
I've recently inherited a project which has roughly 9GB stored beneath sites/default/files. About half of that are files that can be regenerated, such as image thumbnails.
I synchronize that data with my local dev environment so that my dev work can be test, and we regularly sync that data to our staging environment, but those sync-tasks take a significant amount of time. I've improved the speed of that sync by adding --exclude parameters to the rsync task, but this feels like a fragile solution, and one that I've had to repeat on multiple projects.
Separating regenerable content from persistent files
If we implement this in core, I would typically use sites/default/files as the standard public:// file-path, and sites/default/generated-files for cache-public://.
This change would make it trivial to only sync/backup necessary files.
Target users
Sites with a significant amount of auto-generated files. As the bulk of this is often images, image-heavy sites (such as photography sites) or sites with a large number of image-styles would benefit.
Some use-cases
- Faster synchronization between environments
E.g. Synchronization from live to a stage environment (for testing), a training environment, or to a local development environment.
- Faster backups, requiring less disk-space
- Excluding generated files from HA cluster replication
Auto-generated files could be stored on local disk (or even an in-memory disk) for each individual web-head, rather than a shared/replicated disk such as NFS or GlusterFS. This can also help avoid 'split-brain' reconciliation.
Backward compatibility
I'd suggest that by default, if the variable file_cache_public_path is not specified, it should inherit the path provided in file_public_path. This would mean that existing sites are not affected, and new sites operate in the same way, unless the developer deliberately reconfigures the site. The feature is an optional enhancement, and doesn't detract from the current experience.
Slides
I've created a short slidedeck to introduce the topic in a BOF: slides are available at http://www.slideshare.net/manarth/drupal-feature-proposal-two-new-stream...
