Research: How to implement auto-save with multiple users viewing edits

Leaving assigned to myself because I still working on the summary to describe the issue.

Unfortunately we don't have the auto-save feature I am describing for issues on drupal.org😜

effulgentsia → credited hchonov → .

Adding @hchonov to this issue per #3462360-20: Partial preview updates: update preview of modified component only, not entire component tree → .

We are currently under the assumption that we can require Content Moderation for this functionality. It should work "out of the box" but Experience Builder may want to create a specific UI for saving and viewing revisions.

Content moderation has severe limitations - for example if media, taxonomy terms (via tags autocomplete) or other similar entities are created within experience builder while editing a node, it is not able to track their publishing state along with the node, so the media entity or taxonomy term will 'leak' into the live site.

These problems are fully solved by workspaces, so I'd strongly recommend looking at that first especially since it's stable in core now. Workspaces also supports full-site preview, so once a draft revision of an entity is saved, it can be properly viewed in context in the rest of the site - including in views listings and any other contexts it might appear.

The disadvantage of workspaces at the moment is that the 'workspace' concept is always exposed, but #3095414: Consider adding the concept of temporary workspaces → is already open to address this, and would eventually allow for a lite/transparent interface for workspaces that could fully replace content moderation.

The initial idea for auto-save is to use the shared temp-store. It was determined that doing a full entity save for auto-save would not be a good solution because

Another reason that auto-saving can't do a full entity save is validation. For example when first creating a node in a form, the entity is not a valid node (no title for example), so entity validation would fail. Saving without validating would create invalid revisions which if they were ever published could break custom logic, lead to information disclosure, or might not even be allowed with the database schema. Even once an entity is valid, it could be temporarily invalid again in the form state - e.g. if I delete the current content of the title field to replace it, the title field will be empty but required and autosave would be blocked (or risky) until there's something in there again.

@catch thanks for the feedback.

1. I guess I don't know how much of the limitations of Content Moderation would be a dealbreaker for all sites that want to use Experience Builder.

   It seems like the drawbacks Content Moderation that Experience Builder would face such as entities "leaking" into the live site would be similar to just creating a node saving it as unpublished. Any media entities or terms would still "leak" into the live site.

   I am not up-to-date as far as how much of hurdle it is use workspaces on a site. Is there friction as far as site building, contrib compatibility, or custom code? I won't want it to be a hurdle to people using XB if it was a dependency

   My understanding from the reason Workspaces was originally developed was or organizations that had pretty strict requirements for this type of entity "leaking" into a live. I am not sure if everybody who wants XB would have the same requirements

   Maybe Workspaces could be the suggested method for sites that have this concern. We could ship a recipe or a XB/Workspaces integration module

2. Another reason that auto-saving can't do a full entity save is validation.

   yep once we expose entity field editing within XB this will start to be a problem

   I am not sure if the XB UI will actually have to be an entity form. We could do the auto-save and even a real save via a JS call like are we doing now with the experience_builder.api.preview route which calls \Drupal\experience_builder\Controller\SdcController::preview and sends it the layout, which maps to the ComponentTreeStructure on the back-end, and `model`, which maps to `ComponentPropsValues`. The client could send other data related to the entity fields that are updated.

   Right now the front-end react UI doesn't really have a concept of XB Canvas is a field on an entity. It mostly just knows about component, props and slots.

Is there friction as far as site building, contrib compatibility, or custom code?

Disclaimer that I haven't actually used the Drupal 10 version of workspaces in production, but it's completely transparent - the only case where it wouldn't be is hard-coded entity-like or entity-related storage that does not use the entity revisions/views/entity query system properly. The only friction is that you have the workspaces concept exposed, vs. 'only' workflow in content moderation. Recipes could probably smooth out any initial set up now.

My understanding from the reason Workspaces was originally developed was or organizations that had pretty strict requirements for this type of entity "leaking" into a live.

It's only partly about the 'leaking', but as much about being able to publish sets of changes at the same time (which IMO is potentially useful for any site bigger than a single-person-blog), and having whole-site-preview so that changes can be reviewed together and in context (relevant for read-only collaborative preview if not quite collaborative editing as such).

The initial idea for auto-save is to use the shared temp-store. It was determined that doing a full entity save for auto-save would not be a good solution

It seems like you are only listing the drawbacks of the alternative solution. Have you considered what are the limitations and drawbacks of the current solution? For example, are we able to show these auto-save states in the content listing for when the content was last updated?

If Content Moderation cannot be a requirement for this functionality then another implementation would need to be part of this research spike.

At minimum, we could have a soft dependencies like this. If we can provide better experience by having a hard dependency, we could consider that too.

These problems are fully solved by workspaces, so I'd strongly recommend looking at that first especially since it's stable in core now.

I agree with @catch that Workspaces from UX perspective would likely to be closer to the ideal experience (in the long run). With Workspaces, it is possible to edit multiple things and preview them together. For example, when I make changes to a design token as well as components, it's likely that those changes would be better reviewed together. Same as when we make code changes; if we were limited to making changes just in a single file per MR, it would be really hard to actually review changes. I think we should at least consider this as an option since we believe it could provide the best experience.

TBD: What information would we need to save in the shared tempstore?

FYI, there's an existing implementation of this in the Blokkli module which may be worth reviewing: https://git.drupalcode.org/project/paragraphs_blokkli/-/blob/1.0.x/src/E....

@lauriii re #15

For example, are we able to show these auto-save states in the content listing for when the content was last updated?

It is not clear that was actually a requirement. It feels like we don't have information of what the purpose of auto-save is.

Is it so that someone doesn't lose their work in XB if they forget to save or is so that content is save that acts like all other content, i.e, appears in lists.

If the put the Semi-collaborative editing section under Auto-save but maybe that was mistake. Maybe these are really 2 different requirements? Do they both belong in this issue? Is one more important in the near term the other?

From technical some pro/cons of 2 ways we could Auto-save

Saving the state of the front-end application to a tempstore

Pros

1. Quick, won't have the overhead of real entity revision
2. Does not require the entity to validatable?
3. Would allow another user to also load the same state of the front-end application to see the state of the front-end app?
4. Would not invoke hooks that happen on entity save. This would allow saving the state often without worrying about side effects caused by unknown implementations of these hooks(example sending an email)
5. Would not require Content Moderation to allow forward saved states
6. Would not require the entity to actual validate when in progress
7. Same saving mechanism might be able to be used when we need support config entities, such as content type templates

Cons

1. We have to invent a new way to save, saving forward changes of an entity that doesn't affect the published version is already a solved problem.
2. auto-save would not a be real revisions. could not easily appear in lists. To become a real revision another type of saving would need to happen
3. Because it won't use real revisions, any contrib modules or custom code that want to react to these saves would have to be customized for this new mechanism(this also could be seen as pro, if we don't want existing logic to be trigger on auto-saves)

Saving the entity in real revisions using a new Auto-save state with Content Moderation

Pros

1. Limited new code: Content Moderation already has a mechanism making changes to a content entity without affecting the published version
2. Content Moderation already allows multiple states so adding a new auto-save would be trivial
3. Auto-save state could be set-up to not create new revision each time, thus allowing frequent auto-saves without having many revision. But through existing hooks this behavior could probably changed by contrib/custom code
4. Real revisions would be shown in listing but also since they would have a different workflow state could be easily filtered out
5. Contrib/custom code would not need a new way to react to auto-saves vs other saves because same hooks would be fired

Cons

1. Unless we decorated the ModuleHandler service all the regular hooks be fired. Implementations of these hooks may not have be written with assumption that they would be invoked every 30 seconds(or however frequent we decide we need). Therefore they might not have been developed with the type of performance need and or user experience(say if a user was alerted each time). New or existing code could be adapted to filter out auto-save revisions but might have to be updated
2. The entity would be need to be validatable at every auto-save point. If we are going to use XB to create content and add/edit entity fields that entity might not be able to be saved unless all the required fields are filled. Also code may use the combo of 2 or more fields to determine if the entity is valid. This mean you could not auto-save until the entity is complete valid.

Workspaces also needs pro/cons but I don't have time right now

I could see us using a combo of both these approaches.

If we need just an auto-save feature right now we could implement with Content Moderation and an auto-save workflow state. If we don't need the features I listed under Semi-collaborative editing we could push this off until later. We could plan to implement this with something other than real entity saves but if we are able to somehow making this as responsive as people expect and solve the validation issue with real entity saves then we may be able to use the same method.

I haven't caught up on the comments in this issue, so this is just a drive-by idea, but I wanted to post it here in case it leads to any good discussion or helps spark other ideas.

The key problems with an auto-save saving an entity "for real" (whether as a draft revision or in a non-live workspace) as opposed to "just" in the temp store are:

1. It might be unexpected. Unlike with Google docs, Drupal content editors might not expect their stream-of-consciousness writing automatically becoming real draft revisions or real changes in the workspace that they're in.
2. The in-progress work being auto-saved might not be in a valid entity state (e.g., required fields might not be filled in yet).
3. A real entity save (even a draft revision or a save to a non-live workspace) can be slow. Especially if the site has contrib or custom code with hook implementations that call out to 3rd party web services.

So here's a possible idea for mitigating the above by combining Workspaces and Temp store:

• Every auto-save submission from the client we save to the temp store.
• We introduce an "Allow auto-saves" setting to Workspaces. This mitigates #1 above.
• If while in XB, the editor is also in a workspace that allows auto-saves, then when we get an auto-save submission from the client, in addition to saving to the temp store, we also save the entity for real, if:
  • It passes validation (mitigates #2 above).
  • We're not already in the process of saving it in a different request (i.e., we add a lock around a real entity save) (mitigates #3 above).

This would mean that even without the Workspaces module enabled, we still get auto-saving to the temp store. But with Workspaces enabled, and when in a workspace configured to allow auto-saves, we also get real entity saves happening for valid states and protected from stampedes without having to slow down the rate that we're able to save to the temp store.

tedbow → credited traviscarden → .

@effulgentsia interesting idea.

I do wonder if Workspaces is really a small thing for existing site to turn on. I think #17 kind of assume it is because otherwise you don't get "real" entity saves and the pros that come with it.

From @catch's comments it also makes Workspaces seem like such a low barrier that requiring for basic functionality seems like "no big deal". But when I try to find people that Workspaces on actual sites I haven't really found that many. Maybe it is just recently stable so that is why? I have found people that thought about using but then decided not to though.

It just seems like we already have the need "Saving a draft" that is mentioned in the summary. This seems to be a very good fit for Content Moderation and this seems to be a module many more have used in real site. It seems like it would also work well for auto-save.

Why would want to require Workspaces for very basic functionality such as auto-saving content. I know #17 proposes an idea where you get some form of auto-save without Workspaces and then an additional feature if you have Workspaces on. To me it just introduces yet another a decision a site builder has to make when deciding if they want to use XB. Or maybe if we don't require Workspaces, people will just get an auto-save feature that is suboptimal but they would never to get have to install Workspaces or another module to get the optimal auto-save functionality .Auto-save just seems like just such a basic piece of functionality that we should make the default experience be what we, or the product/ux XB folks, think is optimal. You should have to go out of your way to get the non-optimal functionality not the other way around. I know there might be different "optimal" functionality for different sites but I feel XB should have a very strong opinion on this.

What do we expect the default way to get XB to be? Is it just installing the XB module? Is it a recipe that has XB and other modules? Are we willing to make XB dependent on Workspaces, would that stop some people from using XB?

If default way to get XB is a recipe with Workspaces(or Content Moderation if deemed better for auto-save) but it is not an XB module dependency, how do we stop a site builder from uninstalling Workspaces?(assuming they didn't know they were changing the way auto-save in XB worked)? We could make an XB auto-save module(credit @travis.carden for idea) that depends on Workspaces that would be install with the default XB recipe

Content moderation has severe limitations - for example if media, taxonomy terms (via tags autocomplete) or other similar entities are created within experience builder while editing a node, it is not able to track their publishing state along with the node, so the media entity or taxonomy term will 'leak' into the live site.

I’ve rarely seen this as an actual problem on the websites I’ve worked on. Neither on different CMS that have same limitation, like Contentful.

@johnwebdev thanks for the input. My suspicion is that your experience is probably that same as many people but I have not done site builds for a long time so my experience is not current. But I think if we require Workspaces for the optimal Auto-Save experience, whether is actual module requirement or not, we would be adding a lot of complication to solve a problem that is a very real for some site but not for a majority of sites.

Working with Workspace does seems like it changes a lot things about working with Drupal site. I know this is for good reasons but not sure it would the problem all or most XB users have

1. Per @catch in #12, I think we may need to update the issue summary to replace every mention of with ? Although, is this only a problem on more complex sites, not on simpler ones, like @johnwebdev seems to imply in #20?
2. RE: validation + auto-saving by @catch in #12: I think we could do partial validation of an entity, like Form API's #limit_validation_errors or Field API's \Drupal\Core\Entity\EntityConstraintViolationList::filterByFields(). Although you're right that some field/entity DB schemas may simply not allow certain invalid values. But doesn't that mean that Content Moderation and Workspaces automatically have to be discarded as possible implementation methods? 🤔 Because autosaving should work from the very start, not only once the entity is in a valid state…
3. @lauriii in #15: RE: content listing → can you elaborate? Do you mean /admin/content? This could be solved using a new computed field on all entity types, or even just an additional computed property on an existing field type.
4. @effulgentsia in #17: very interesting … but how does that get around DB schema-level constraints that may not be met by an invalid entity?

re #22

I think we may need to update the issue summary to replace every mention of Content Moderation with Workspaces?

I think that would @catch's argument(just based on my reading of comments, no other info) and maybe that will be the way we will go. But I am not convinced yet, so could hold off making those changes to the summary.

I will try to post comment soon with more of the reasons not to do this

I'm going to try to process all the details here and update the issue summary.

But doesn't that mean that Content Moderation and Workspaces automatically have to be discarded as possible implementation methods? 🤔 Because autosaving should work from the very start, not only once the entity is in a valid state…

Yes, this is why I brought up autosave_form in #3462360-20: Partial preview updates: update preview of modified component only, not entire component tree → after going through much of the same thought processes that are happening in this issue in June - you just cannot do autosave as-such with entity revisions. There are other things you can do with entity revisions (like full site preview) but it is not the same thing, and the two can work together (i.e. autosave can preserve your edits, before you then save a new draft revision).

I would like to suggest that we do something like @effulgentsia's proposal in #17 but replacing Workspaces with Content Moderation.

Copy/paste from #17 with replacements
So here's a possible idea for mitigating the above by combining Content Moderation and Temp store:

• Every auto-save submission from the client we save to the temp store.
• We create a new 'auto-save'. state to workflows used on Content Entity types that XB(this could be tricky for existing sites)
• If while in XB, the editor is working, then when we get an auto-save submission from the client, in addition to saving to the temp store, we also save the entity for real, if:
  • It passes validation.
  • We're not already in the process of saving it in a different request (i.e., we add a lock around a real entity save)

If these are true then unless want to make Content Moderation a hard dependency of XB then

• If the site has Content Moderation installed and configure this entity type, We save to the new auto-save state
• If the site does not have Content Moderation installed or it is not configured for this entity type, but the entity type does have a published status and it is not currently published then we save a revision, still as unpublished.

In both cases we would need to decide whether we always want to create a new revision or want to more careful about keeping many revisions. Beside DB size, one reason to not to keep all auto-save revisions is the user might want to revert back to a previous state later, if we have an auto-save revision for every 30 seconds it could overwhelming to find a point to revert to.

If want to minimize the number of revisions we could save locally the revision id of the last auto-save and not create a new revision if most recent revision matches.

My reasoning is:

1. I understand that Workspaces can solve a real problem as @catch describes in #12 and @lauriii's comments #15
   

   I agree with @catch that Workspaces from UX perspective would likely to be closer to the ideal experience (in the long run).

   I think we should at least consider this as an option since we believe it could provide the best experience.

   I agree if implemented very well, the integration could provide the best experience and that XB should be very opinionated about the best UX.

   My concern is another goal of XB is to unify the page building ecosystem in Drupal. I know creating UX that is better than anything currently in Drupal will be part of how we unify, but I think if we make it too difficult for existing users to use XB or XB requires a too specific way of building sites then we risk the page building ecosystem remaining very fractured.

   @johnwebdev in #20 is an example of someone who hasn't seen the problem Workspaces solves (over Content Moderation) as an actual problem that needs to be solved for their sites. I think on the other end if someone does have the problem of entities leaking into the live site or wanting to publish a bunch of content entities in one batch will they for sure want to solve it in the way that Workspaces provides. There numerous contrib modules that handle content staging between actual Drupal environments. Do we want to try to lure current users who are using these solutions, which they may be heavily invested in, to use XB on their current or next site? Will it be harder if they have to also use Workspaces for XB to have a decent UX(including auto-save)?

   I know @effulgentsia proposal in #17 would not require Workspaces but I think it effectively requires it because a very basically functionality, auto-save, that users would expect to just work requires it.

2. The solution of using Content Moderation in this way would not rule out it working with Workspaces too for sites that need/want the multi-entity staging that it provides. We could adapt the case above where Content Moderation is not installed to always save an entity revision and not care about Publish state when within a Workspace.
3. We could make Drupal CMS offer an XB focused recipe that is even more opinionated than XB itself. In it we could require Workspaces(maybe through a "XB Full site preview" module using something like suggestion in #17). Other sites could also use this recipe. We could even make this promoted/recommended way of using XB.

   This would allow sites to not use the recipe if

   • The don't have the need for the Full Site preview
   • They are already using Workspaces and our prefered way with XB conflicts somehow with how they implemented it(including possible custom code)
   • They are already solving the multi-entity staging problem in another way such as using a dedicated staging environments and a contrib module such as Content Synchronization →

   But again we would recommend the recipe that Drupal CMS would also use as the default way to use XB.

Okay, I've fleshed out the requirements for the Auto-save feature, and this is on track to become the longest issue summary in history. 😬 (I blame d.o's CSS for the odd markup choices I made to make it more readable.) I plan to do the same for each of the other two features and then distill the discussion in the comments. Also, it's past 1:00am my time and too late for proofreading. Here's hoping for the best. 😛

@longwave, @lauriii, @catch and @hchonov discussed this issue at Drupalcon Barcelona. We agreed that we need a combination of solutions for the product requirements for autosave functionality - but the good news is that much of what we need to implement this already exists.

We should be able to use Autosave Form → to handle automatically saving XB data into tempstore. It provides an endpoint that XB should be able to talk to with little or no modification. It does not yet support displaying details of the autosaved state elsewhere in the CMS but this should not be difficult to add. It also currently uses per-user tempstore but converting it to (optionally) use shared tempstore should not be a problem.

This will allow us to save valid or invalid entities. We may want to extend it to attempt to validate the entity on autosave in order to indicate to the content creator whether the entity is capable of being saved or if there is an error that would prevent publishing.

We may be able to use Content Lock → to handle the non-concurrent editing problem when two users are using the XB interface for the same content item.

As XB may involve editing multiple entities (content or config) at once, we can use Workspaces from core to handle reviewing sets of changes across multiple entities before publishing them. We may want to implement #3095414: Consider adding the concept of temporary workspaces → to move Workspaces entirely behind the scenes so the content creator does not need any knowledge of workspaces - but advanced users will still be able to use full workspaces to e.g stage multiple sets of changes or go through longer review workflows.

We have ruled out Content Moderation as there is a longer term desire in core to phase out CM as it currently exists and replace it with a simple implementation based on Workspaces.

Updating the issue summary to reflect the elimination of Content Moderation as an option per #28.

...and again to updated the proposed resolution. I'll leave it assigned to me to continue fleshing out the distinct user stories for drafts and publishing.

@lauriii gave a very quick idea of how he envisaged XB editing to look like, and it looked very, very similar to how workspaces works already, which was both encouraging and surprising.

I had been assuming we'd want to hide the workspaces concept almost entirely for a default UX (which is the main missing piece from what it can already do compared to content moderation), but @lauriii said things like 'but what if I want to edit both the site menu and tweak a node from one page in XB and then publish both changes at once'. XB can't do that yet, but workspaces can already do this including config entity changes (in the wse_extras contrib module), so it is just a case of getting those things to meet.

The main UX difference was that the list of affected entities was shown right on the XB edit form, and it was possible to publish the changes directly from there without needing to go to the workspaces UI as such. We still might still want to hide the 'workspace' as a thing separate to the current set of edits itself (so still need #3095414: Consider adding the concept of temporary workspaces → ), but not the concept of reviewing/pubishing/staged changes - just localise it to the XB form when you don't have a separate review workflow. This all seems like less work/changes than I was thinking it would be - not that much of a jump at all.

@longwave thanks for the write up!

We should be able to use Autosave Form to handle automatically saving XB data into tempstore.

Right now the XB ui is not a Drupal Form so we would have to make it Drupal Form to take advantage of AutoSave.

Right now the front is doesn't use the Drupal form except for the side bar but not the complete tree and model values. In fact when in the UI and the sidebar is not open there is no <form> on the page at all.

From the discussions with @wim.leers it sounds like we pretty much have all we need in data currently passed from the client for the preview route to perform a tempstore and/or entity save, at least for the XB field.
I was just wondering if it would make more sense for the JS client to save/auto-save via an API call.
We may get some functionality for free if we use a drupal form but also there would probably be some work to get there instead of js api call from the react app

Just initial thoughts, but my worry with the front-end becoming too “drupal aware” is that we start limiting the people who can work on it. Or you start to need a combination of React Skills and Drupal knowledge(probably not novice either)
You will already need modern JS skills we can’t go back on that(and probably a good thing) but my guess is right now you don’t need a lot of drupal skills to work on the Front-end app.

Also have thoughts on Workspaces but will put that in another comment...

I talked with @lauriii, @wim leers, @traviscarden and @effulgentsia about the Workspaces requirement.

I brought up my idea from #19 again of making a recipe that would require Workspaces to provide UX we want with full site preview but not making Workspaces a depdency of XB itself.

Pretty much everyone in that list of folks was concerned about ever complexity it would take to support both that case where Workspaces was enable and when it was, which is a valid concern.

@lauriii suggested that we could make Workspaces a dependency of XB now and if it is determined later making Workspace a dependency of XB is big barrier for many sites using it we could change this decision before it is stable.

This seems like a reasonable idea to me. I still have my concerns about making Workspaces a dependency but

1. I have not built sites in a while so probably not the best person to judge how much of barrier it would be. I still think my points above are valid but it probably it is up to others who like @johnwebdev and others who are currently making site to express concern if they have any with having enable Workspaces . I can try to help the discussion with backend implications if they come up.
2. We are just at 0.1.0-alpha1 for XB so we have time to change this decision
3. Workspaces just became stable so it has a lot of time for improvement before we use it in a stable XB release which is many months away. This is bit of gamble, relying a recently stable module that has not been used by a lot of production sites but @lauriii said at Drupalcon Barcelona there was talk of being required for Drupal CMS would require Workspaces to have UX improvements that XB would also benefit from it.

If we do decide to depend on Workspaces I think it would great make it a dependency as soon as possible to make sure it the direction is as clear as possible. If we only make it dependency after we have spent months to make integration really nice that could shorten that time people to raise concerns before XB becomes stable. I assume the people how are following this issue or would be somehow "in the loop" to know are plans will be a very limited number of people.

If we intend that you should only use XB inside a workspaces then could simply:

1. add the dependency
2. add a check on our current demo xb/node/1 to see if they are in a Workspace.
3. If not in workspace and only 1 workspace exists automatically switch them to that workspace, give them a message why they were switched
4. If not in workspace and more workspaces exist redirect them to the workspaces listing page, give them message they have to they have switch to a workspace
5. If they don't have permission to switch to workspace, give them a message they currently demo doesn't support using the XB Demo outside of workspace and they don't have the permission

It doesn't have to be exactly like this but it would good to have follow-up like "Ensure XB can only be used in Workspace" ASAP if we make this decision that will force us to start using it. My guess is providing the optimal UX for Workspaces + XB will take a lot of work in XB, Workspace, and possibly Workspaces Extras. It would great if we could be practical with this part of module like we have with other parts, like the Canvas UI demo that doesn't allow saving, to get working with Workspaces in even rough way first. Though we shouldn't stop saving if you are in workspace.

Another concern I want to raise about relying on Workspaces is that if want to also be able to edit and stage Config changes as @catch mentioned in #31 we need to rely on Workspaces Extras → . While amateescu, the Workspaces core maintainer, is also maintainer of this and it is under active development which are really good signs, it doesn't actually have a published release and as far as I can tell only has 2 tests(functional only). If config support is must have we will probably need to factor in work on WSE or at least get a confirmation on the modules state.

This is bit of gamble, relying a recently stable module that has not been used by a lot of production sites

Workspaces is slightly different to a lot of experimental modules in core. It's been used in production on at least a handful of large production sites including at least one fortune 500 company for at least a couple of years or more. The last handful of patches that were required to get it to stable were also run on production for several years. The reason it took so long to get to stable was because the company that originally pushed towards getting it into core withdrew support for it more or less at the last minute, and then other companies started using it after that, but the core momentum was lost for some time in between. There were also issues with the initiative that extended the timeline, which I'm still bitter about, but too late to fix those now. Me and Fabianx were also running workspace's predecessor on production on Drupal 7 as early as 2015 (while the name is different, CPS, the architecture is almost identical).

All of this to say the data model and functionality is very stable, it just took a long time to be able to flip the core flag.

However this very long timeline has definitely affected wider adoption and I'm pretty sure there will be usability issues that will have to be fixed both for Drupal CMS and XB. But... both Drupal CMS and XB picking it early means there is time to identify and solve those issues starting from now, rather than in a year's time having run out of options with content_moderation and having to start later, so it should result in a much quicker timeline overall.

Another concern I want to raise about relying on Workspaces is that if want to also be able to edit and stage Config changes as @catch mentioned in #31 we need to rely on Workspaces Extras.

This is true, but the config staging is holistic (as is trash module support), so while there will be work involved to get config workspace support into core and stable at some point, it should be a lot less work than trying to implement config staging within XB without the 10+ years of work that has already gone into it. Trying to implement config moderation with content_moderation would be a complete non-starter.

It would great if we could be practical with this part of module like we have with other parts, like the Canvas UI demo that doesn't allow saving, to get working with Workspaces in even rough way first.

I think the easiest way to approach things would be to just make workspaces a requirement, and then add some basic configuration so that people installing XB don't need to set up any workflows etc. and live with exposing the workspaces concept for a while. Then hopefully some of the workspaces UX work can happen in parallel, and not necessarily directly in XB (although the final touches might need to be if it's in react).

Okay, my updates to the issue summary are complete. I think I've captured more-or-less everything. Assigning to @lauriii for input on the unconfirmed requirement details.

@lauriii, we've documented our current understanding of the requirements in the issue summary. We would like your confirmation so we can confidently finalize our architectural decisions. Every detail we've identified is marked with a colored circle indicating its status (confirmed or otherwise). (See the legend.) Basically, anything that isn't green needs confirmation--mostly from you.

Incidentally, the issue summary is now rather massive, and I could see it being extracted to its own issue for reference and discussion, if desired. I'll happily do that if asked.

The workspaces DrupalCon session is now online and even for me who helped build the initial workspaces implementation and reviewed much of the core work, I had no idea what was implemented in workspaces extras so it was useful to see it all pulled together into a demo https://www.youtube.com/watch?v=bgJ-oubGjbc

Couple of comments/questions on the issue summary:

And I am working on an entity in Experience Builder

The wording of these user stories assumes that the entity has already been explciitly saved (i.e. exists as an entity). autosave_form also handles the case where nothing has been explicitly saved - in this case when you go back to the node/add/* form it will tell you there's an unsaved draft to resume. I don't know if that is an XB requirement but it would be good to explicitly rule it in or out, currently it's not mentioned.

Then I am warned the entities I created will be deleted, and I am prompted to continue (Am I given the option to selectively delete them, or is it an all-or-none proposition?)

This could potentially be done with trash module which doesn't depend on workspaces but does integrate with it. It does add another layer of things but it implements soft delete + restore + purge. It can also be tackled by moving entities selectively into a different workspace so some can be deleted and some can be preserved.

A lot of the semi-collaborative editing section can be achieved in workspaces (e.g. you can send a link with a token to an anonymous user and they can see what you're working on in the workspace, or an auth user with permissions can browse workspaces)... but this does not include being able to see the editing UI live, which I think is what it's point at (?) - and I agree with the questions about discoverability - would they just be able to browser to the node/n/edit form, but then it would need to be a read-only version of that form.

Feature: Saving a draft

I think this is now just workspaces - except there's a question whether XB (or Drupal CMS) provides default workflows for that - i.e. a separate review stage doesn't make sense for a single-user site, even though they might want to save in-progress drafts and preview them sometimes.

Did we give any thought to CRDTs here?

CRDTs = Conflict-free replicated data types. Some interesting discussion on Drupal Slack.

Same Page Preview reuses NodePreviewController's method of using TempStore to persist the FormState of an unsaved node.

What's being discussed here appears to be decision to use workspaces and revisions instead of tempstore when building something with Experience Builder. Is presume the "something" here will be stored as an entity somehow, but maybe that something isn't node.

My question is, when we consider nodes, are we talking about modifying the underlying mechanism for storing unsaved nodes, so that they are previewed?

Also, when previewing nodes, is there the possibility of reusing Experience Builder's preview capabilities? In the XB future, does thinking about content as nodes even matter anymore?

What's being discussed here appears to be a decision to use workspaces and revisions instead of tempstore when building something with Experience Builder.

No not quite..

At the moment, the current state of discussions leads to something like:

When an item is explicitly saved -> new revision.

When an item is autosaved -> autosave_form (similar but not identical to temp store).

This is because an item that is autosaved may not be a valid entity - it could have required fields missing, so it can't be an entity revision in a useful way. Similarly on the node/add form, there is no entity yet to save a revision for at all.

Ah yes, incomplete data.

When we think about auto save, data that is incomplete and would fail validation, are solutions that persist data on the frontend able to be a candidate solution.

I'm not familiar with yjs or other solutions of that class. But even something as simple as JavaScript's local storage might provide a means to store complex data temporarily

Storage API
Indexed DB
And React is in the mix so, I guess Redux

All provide various advantages / use cases.

Do we have to auto save to the backend? Is there no way we could solve this on the frontend?

@cosmicdreams if we never autosave to the backend, then there is no ability to start something on desktop and resume it on phones. I use that feature on slack/gmail at least and even if there might be less of a use case for a website than an email draft, people will know it exists and potentially miss it if it's not there. Similarly if your laptop runs out of battery or browser updates etc. Collaborative editing will also have to save to some kind of backend even if it's different/evolves from the autosave one too.

@cosmicdreams there is a product requirement:

As a creator, I want to work collaboratively in real-time with other creators. This means being able to work on the same page simultaneously with other creators, and seeing changes made by them.

While full collaborative editing is some way off, autosaving changes on the server side means that other creators could still see a preview of something that a creator is working on without them having to explicitly publish it, in near-realtime.

The wording of these user stories assumes that the entity has already been explicitly saved (i.e. exists as an entity). autosave_form also handles the case where nothing has been explicitly saved ... I don't know if that is an XB requirement but it would be good to explicitly rule it in or out, currently it's not mentioned.

Good thinking, @catch. I assume it is not a requirement since, as far as I know, there's no way to use Experience Builder except on a piece of content that already exists.

I've updated the issue summary to include this assumption.

#46 Maybe laurii can confirm? Personally I think starting in XB when creating a new page would be better UX than first having to create and save a node before getting started.

A lot of the details in this issue (that assumption included) are pending confirmation from @lauriii. See the legend for interpreting the status markers.

Granting our assumptions, it seems like we're more or less in agreement on the proposed resolution at this point. (Tell me if I'm wrong.) I'll see if I can get @lauriii's eyes on the issue, and in the meantime, I'll find a good place in the codebase to document our architectural direction. Since the scope of this issue is research, that should resolve it, and we can create follow-ups to start on implementation.

Looked at this with @bostonjillian and @Renee Lund and got through the auto-save section.

🔵 When a given amount of time transpires (How much? Or should it be event-based?)

We want users to not have to think about this action, which means that it needs to be close to immediate (~few seconds). I guess based on that, it should probably be based on when changes are being made?

🟢 Scenario 5.1: Deleting created entities when abandoning progress

There should be a way to delete the draft by using a delete action but once the user has entered the content form and inserted any content there, we should retain the saved draft.

🔵 Scenario 6.1: Deleting created entities when abandoning progress

From UX perspective, there might be different expectation between the two examples given in the user story. Media library is often something that is considered to be separate from the content (i.e. page) that is being created. This is highlighted by the fact that it opens up in a modal dialog and has explicit steps for uploading and saving. This is different from the use case of Taxonomy Term because in that case you're just adding a tag inline using an autocomplete.

🟡 Auto-save only applies to saved entities. It doesn't make sense in any other context, e.g., an un-submitted node/add/* form, since (it is also assumed) Experience Builder can't even be opened except on an existing entity.

I don't think this is the right assumption. Content types that are using Experience Builder should open in XB even when creating content.

🟡 If a user wants to revert back to a specific point in history, that will be handled with revisions.

It should be possible to revert back to previous revisions and auto-save states. It would be okay if auto-save states get truncated into a single revision once content gets published.

🟡 Workspaces can be a dependency of Experience Builder. (Can it be a hard dependency for all behavior, or only a soft dependency for "enhanced" functionality? See Open questions.)

I think we need to work with the assumption that we depend on Workspaces for now. It doesn't seem that the UX of going with content moderation would meet the needs. I don't think we need to implement support for using XB without Workspaces now, but it would be great if we could build this in a way that we could reconsider this decision closer to the release. This way we could add a way to use XB without Workspaces later if needed.

🟡 What happens if the same user has the same node open in Experience Builder at the same time on two different computers? For example, what if a user goes back and forth between home and the office? Does one computer get locked out of editing and become "read only"? How do we avoid the two computers repeatedly overwriting each other's auto-saves?

We could do something similar to: https://www.youtube.com/watch?v=CsRBypaT_hM&t=27s. We could apply this both for the scenarios where the user is the same and where it's different.

🟡 What happens to one user's auto-save states if a node or revision is changed by another user?

Is this relevant if we do above ^?

It should be possible to revert back to previous revisions and auto-save states. It would be okay if auto-save states get truncated into a single revision once content gets published.

One extra question here - is this only when the content is actually published, or is it also when it gets sent for a workflow step like review? I think autosave_form would currently truncate drafts whenever the entity is explicitly saved. In the Barcelona meeting, we briefly discussed saving a new forward revision as part of the autosave when the entity is valid (or every so often when it's valid), this would provide a partial interim-history of autosaves which could more easily be preserved for longer (but still with some metadata noting they're implicit rather than explicit revision saves).

fwiw I think that aspect is something that could be experimented on once the initial implementation is up and running, feels like there are a couple of options and it will need to be built on a working foundation with some trial and error both in terms of UX and implementation. But also good to think ahead to it.

Re: #29

🟢 Scenario 5.1: Deleting created entities when abandoning progress

There should be a way to delete the draft by using a delete action but once the user has entered the content form and inserted any content there, we should retain the saved draft.

I don't understand. I assume "saved draft" means an auto-save state. Is that right? What do you mean by "content form"?

🔵 Scenario 6.1: Deleting created entities when abandoning progress

There might be different expectation between [Media entities and taxonomy terms]. [In] the use case of Taxonomy Term because… you're just adding a tag inline using an autocomplete.

But you can also configure Taxonomy term fields to "create referenced entities if they don't already exist". The point is that it’s possible to create separate entities from a node edit screen. Does the rest of the scenario look right?

🟡 Auto-save only applies to saved entities. It doesn't make sense in any other context, e.g., an un-submitted node/add/* form, since (it is also assumed) Experience Builder can't even be opened except on an existing entity.

I don't think this is the right assumption. Content types that are using Experience Builder should open in XB even when creating content.

Okay, so when you create an XB-enabled node, for example, there is no standard node/add form at all? You go straight to Experience Builder and that's the only way to interact with the entity? If so, that should be added to the assumptions. And we probably need a new scenario for editing the overall entity information--such as title, URL alias, and menu settings--from within Entity Builder. Would you agree?

🟡 If a user wants to revert back to a specific point in history, that will be handled with revisions.

It should be possible to revert back to previous revisions and auto-save states. It would be okay if auto-save states get truncated into a single revision once content gets published.

Wouldn't that mean that the "truncated" (meaning, squashed or merged, I assume) auto-save states are always identical to the list of revisions--making them effectively useless? Wouldn't that mean that every revision would have an identical auto-save state?

🟡 What happens if the same user has the same node open in Experience Builder at the same time on two different computers? For example, what if a user goes back and forth between home and the office? Does one computer get locked out of editing and become "read only"? How do we avoid the two computers repeatedly overwriting each other's auto-saves?

We could do something similar to: https://www.youtube.com/watch?v=CsRBypaT_hM&t=27s. We could apply this both for the scenarios where the user is the same and where it's different.

Given the workflow depicted in the video, I suppose that would mean a user on computer two would have to request control from computer one. And ignoring the logical challenges with that, it would mean that our auto-save states would no longer be associated with a user but rather with a browser session, right? Otherwise the concept of locking doesn't make any sense and we haven't solved the problem in question at all.

🟡 What happens to one user's auto-save states if a node or revision is changed by another user?

Now that I understand that only one user can be working on a single entity at a time, problem can't actually materialize, then as long as starting an editing session in Experience Builder locks the whole entity, this is irrelevant.

Wouldn't that mean that the "truncated" (meaning, squashed or merged, I assume) auto-save states are always identical to the list of revisions--making them effectively useless? Wouldn't that mean that every revision would have an identical auto-save state?

If autosaves are always newer than the latest revision, it's much easier to think about because you have revision 1, 2, 3, 4, 5 then autosave 1, 2, 3, 4, 5, not revision 1, autosave 1, revision 2, autosaves 2 etc..

Then to have interim checkpoints between explicit saves to potentially revert back to, the idea discussed of implicitly saving a revision in the background when the entity is valid comes into play (with metadata so we know that's what happened) - while this adds complexity, it would still result in two linear histories of the entity/form.

If all the above happens, then autosaves can be garbage collected every time an entity revision is saved (whether this is explicit from a button or implicit when the autosave state is a valid entity).

I don't understand. I assume "saved draft" means an auto-save state. Is that right? What do you mean by "content form"?

While the end user might not need to know the difference between an auto-save state and a forward revision, I think it would be very good to use the terms exclusively in this discussion and not mix and match 'draft' to mean both things.

@lauriii re

We could do something similar to: https://www.youtube.com/watch?v=CsRBypaT_hM&t=27s. We could apply this both for the scenarios where the user is the same and where it's different.

to be clear, since there is a lot going in that video, the only part you are talking about is requesting hand off control, of the XB edit, correct?

While the end user might not need to know the difference between an auto-save state and a forward revision, I think it would be very good to use the terms exclusively in this discussion and not mix and match 'draft' to mean both things.

I agree with this sentiment, but I think picking the right words is tricky. I think we need to have terminology for three states. Taking an implementation-perspective, we can call these states:

• an auto-saved state
• a non-live (is "non-live" the more accurate term than "forward"?) revision
• a live (is "live" the more accurate term than "published"?) revision

However, as the UX for this hasn't been fully designed yet, and the product requirements not even fully flushed out yet (though this issue is making good progress towards that), I wonder if it would help the requirements/design process to also have product/UX-oriented terminology for these states rather than jumping straight to the implementation-oriented terminology.

Perhaps something like:

• In-progress stuff (or some other term than "stuff", such as "work", "content", "edits", "changes", etc.)
• Reviewable stuff
• Published stuff

?

a non-live (is "non-live" the more accurate term than "forward"?) revision

To me it's not more accurate, because a 'non-live revision' includes past, previously-default, revisions too, and technically 'published' revision also includes non-live, previously default revisions, so we can't just use non-live and published. So we probably need to explicitly talk about four states :/

That would mean:

• old, previously default, revisions
• auto-saved states
• forward revisions (for review by ones-self or others)
• default revision

In-progress stuff (or some other term than "stuff", such as "work", "content", "edits", "changes", etc.)
Reviewable stuff
Published stuff

If we add 'old stuff' or 'published history' here too that probably covers it.

How about this?

It should be possible to revert back to previous revisions and auto-save states.

I strongly request that we descope "revert back to previous auto-save states" from the initial implementation of Save functionality and from the 0.2.0 milestone. I opened ✨ Implement time travel (undo/redo/revert) of auto-saved states across devices and users Active to discuss and work on just that feature in its own issue.

Did we give any thought to CRDTs here?

The issue in #58 proposes to use a CRDT library for implementing that issue, both to eventually support collaborative editing, and also because those are the libraries/algorithms that also implement the most mature, robust, and efficient storage of large histories.

I agree with #58

I think to make safer to not implement the reverting to auto-saves now but make it possible later we should be very clear that our auto-save implementation is entirely internal. We should be free to change it later no have to worry about BC. Especially before 1.0 and if we still haven't done ✨ Implement time travel (undo/redo/revert) of auto-saved states across devices and users Active by 1.0 we should continue to keep it internal so are free to support reverting later.

I agree that #3481006 would be a lot of work and it doesn't seem realistic to do it with all the product requirements we are trying to get done

I don't think I fully understand Rule 1: Users can view others' work "live" -> Scenario 1.2: Viewing someone else's in-progress work

🔵 And I select user #2's in-progress editing session (How?)
🔵 Then I will see the current state of user #2's in-progress work (How does Experience Builder know to start viewing someone else's work instead of starting a new, distinct editing session?)

Given 🟢 Rule 3: Only one user can edit an XB-enabled entity at a time and discussion we have had

I would assume that if

1. I am a user who has permission to use XB for an entity and see others work in progress in XB
2. Another user is using XB for node 1
3. I open XB for node 1

Then there would be no choice if I wanted to see the other user's progress in XB for node1 or not, I would automatically see their current work and when their auto-save happened my version would be updated.

To me it would make no sense for me start a "a new, distinct editing session" otherwise we have to figure out to merge the 2 sessions.

We have to work out "🟢 Rule 3: Auto-save states can be transferred" but think that would be where I would able take over the XB session for node 1 from the other user. I think after the auto-save state was transferred to me and I wanted a ""a new, distinct editing session"" I could either 1) clear the canvas or 2) revert to the last revision. But after the auto-save state is transferred to me then it would be no different that if I was the sole user editing XB node 1 and want to "start over".

I opened ✨ Lazily create draft revisions from valid auto-saved states on demand rather than on save or only on schedule Active as a child issue, with an idea for how we can make valid auto-saved states essentially equivalent (from a UX standpoint) to real draft revisions, but without the performance cost of doing a full entity save for every auto-save.

Chiming in as an early-ish adopter of content moderation. When we upgraded to Drupal 8, workspaces wasn't really a thing. We now have years of muscle memory built up around content moderation, having explained in depth to our users why it works the way it works as opposed to something like, say, the Oasis Workflow plugin for wordpress (which has been lauded above the Drupal tooling btw...).

In my experience, teaching the technical mechanics of a solution has been secondary to the "why doesn't this work like other workflow solutions that I am used to" barrier.

For us, it would be a very hard sell a switch to something completely different again with different justifications involved. If a motion to strong-arm use of a specific workflow tool is inevitable, I would urge looking at the most popular (by number of installs globally) workflow tools and to derive the end-user facing terminology from those tools versus Drupal implementation detail nomenclature.

Not sure if this is helpful at this stage of the game or not, but I wanted to get this point of view out there.

Re #63, it's very likely that for XB we'll build an XB-specific UI for listing, previewing, approving, and publishing draft content, even if we use Workspaces as the storage model. Your comment is great input for the design work being done on that. If the designs implemented for XB are well received, they can potentially make their way upstream to the core Workspaces UI eventually.

By using Workspaces as the storage model, if people don't like XB's listing/previewing/approving/publishing UI, they could potentially replace it with core's Workspaces UI, or any other UI built on top of Workspaces, such as even Content Moderation if/when that UI is rebuilt on top of Workspaces (see #28).

For XB, Workspaces is only viable if it can also handle config entities. I opened 🌱 Identify roadblocks to staging config in Workspaces (e.g., via wse_config) Active , so if anyone has knowledge to share there about wse_config, that knowledge would help us enormously in planning out our roadmap.