Auto-save code components

I was just on call with @balintbrews and @lauri regarding auto-saving Code Component. I just wanted to get my understanding down in regard to the usage of auto-saved components, because this would intersect with out parts of XB.

This is my understanding at least for 🌱 Milestone 0.2.0: Experience Builder-rendered nodes Active

1. If a code component is only in an auto-save state and has never been published, it will not be available in the component list in the library of the XB ui. Meaning you can't place an code component on the page/node if it is only in auto-save
2. if a code component has been published, it will be available in the component list in the library of the XB ui, it can be placed
3. if a code component has been published but the editor is currently making changes that are in auto-save, the version available in the component list in the library of the XB ui that can be placed and the version for instances that are already placed, is the published version and is not affected by the auto-save state. This auto-saved version of Code Component also would not affect any instances on page/nodes/global regions that are already been "Published"(in the XB process of "publish all")
4. for an already published version of a Code Component, if the editor has an auto-saved version, once the auto-saved Code Component is published all versions in both pending XB page/nodes/global regions, and already published XB page/nodes/global regions would use the updated version. There would no option to use the previous version

Thanks for the write-up, @tedbow. We’re looking for a slightly different workflow. I collaborated on this answer with @lauriii.

Code components can exist in two states. These are:

1. They only exist under "Code" on the Library tab. Not available to be added to pages, nodes, or global regions. However, they can be used to construct other components (e.g. button component could be imported to a card)
2. They are moved under "Components" on the Library tab. Available to be added to pages, nodes, other components, or global regions. No representation under "Code" anymore, the code component only exists under "Components".

Auto-save workflow

1. Changes are auto-saved and not visible in the published site.
2. Preview inside Experience Builder (returned by the /api/preview endpoint) is generated with the auto-saved changes taken into account.
3. Publishing makes the auto-saved changes public, it is built into the existing publishing process implemented in ✨ [PP-1] Implement the "Publish All" button Postponed .

This means that changes autosaved to components that are listed under "Code" or "Components" may include changes that need to be rendered inside the preview, whilst not being published to the publicly available site.

balintbrews → credited lauriii → .

I made issue to specifically deal with 📌 Code Components should render with their auto-saved state(if any) when rendered in the XB UI Active

Up until now, the XB team has been following a pseudo-scrum/pseudo-kanban process, but we're now shifting into more conventional scrum. We started a new 2-week sprint last Thursday (Jan 16). I'm tagging our current sprint's issues for visibility. This one I'm tagging with "spike" rather than "sprint", because our goal isn't to complete the implementation of it this sprint, but to refine its details.

Preview inside Experience Builder (returned by the /api/preview endpoint) is generated with the auto-saved changes taken into account.

Is this right? My understanding was that there were actually 3 states: not just "autosaved" and "published", but also one in between the two, let's call it "usable" for now (I'm sure we can come up with a better name later).

So let's say you already have a published component, and then you start making changes to the code, these changes would get "autosaved" for the purpose of the code component maintainer not losing their work, but would not actually be seen by content editors that are using the component, even within the XB preview. When the code component maintainer is satisfied with their changes, they would click a button to update the component that's in the component library. Now these changes are in the "usable" state, meaning this is now the code that's used for rendering within the XB preview, but it's not yet "published" to the live site. Publishing to the live site would happen as part of the "Publish All" process.

The above is what I understood the designs to be the last time I saw them, but it's possible my understanding is out of date. But I would find it odd as a component code maintainer to have all my stream-of-consciousness typing into the code editor getting automatically applied to what content editors are seeing. Seems like that would also be distracting to the content editors. If I'm removing 2 Tailwind classes and adding 3 others, seems like content editors should see the old design until the new design is ready, not all the interim steps in between (one TW class removed, then the other, then one of the 3 added, etc.).

If I'm understanding correctly, then I think it could be implemented as two separate autosave values. In other words, we could have one autosave key for KEY_FOR_COMPONENT.draft and another for KEY_FOR_COMPONENT.usable. The "Update Component" button would copy from .draft to .usable and the "Publish All" button would copy from .usable to the config entity. Yeah, maybe it's a bit of a misnomer for the .usable one to be in "autosave" storage, since it's not actually autosaved (it's only saved when a button is clicked), but it lets us reuse a lot of our existing autosave infrastructure and publishing code.

With this approach, everything in 📌 Code Components should render with their auto-saved state(if any) when rendered in the XB UI Active is still accurate, so long as it's based on the .usable key, not the .draft key.

If we don't like .draft and .usable as the names, another pair of names could be .scratch and .draft (where "draft" is what gets used for rendering content within the XB preview, and "scratch" is what gets autosaved as someone is typing within the code editor).

#10 touches upon the problem space I identified independently over at #3499931-6: [PP-1] HTTP API for code component config entities → — @effulgentsia essentially repeated what he wrote in #10 on a call just now.

(Sorry, Alex, I hadn't read this issue yet because it's not scheduled for this sprint.)

For a brief moment, what's in #10 indeed was the plan. However, given that we are aiming to move towards Workspaces based approach, it seemed that it would be better to simplify the workflow and to remove the step where changes would be autosaved and not visible within the workspace. The concept of requiring a manual step for changes to take effect within XB doesn't exist within XB outside of the code editor. Ideally we'd keep the experience there consistent with everything else, and not have this there as well. Therefore the current designs are basically taking the same approach to saving components and don't require a manual step for changes to take effect.

Ah, so what #13 means is that we can maybe not have a scratch vs draft distinction, and just do the same thing we do for nodes, pages, and page templates: only have two concepts:

• autosave = not an entity save = draft = what people see within XB including XB preview
• "Publish All" = entity save = published = what people see on the actual site (outside of XB)

What that means is that content editors who are editing within XB might see an interim state of a code component (e.g., not the old design and not the new design, but whatever the person editing the code component currently has in their code editor, see #10), but we're making the decision that that is okay at this point in XB's development, because once we have Workspaces we'll be able to solve it by having the code component editor and the content editor working in different workspaces, and until then we're not expecting early XB testers to be having a content editor and a code component editor working concurrently, and even if they are, then since they're just testing XB rather than using it in production, it's okay if the content editor is seeing interim states of a code component that's actively being worked on.

I'm okay with all that. However, there's still a wrinkle that at a given point in time there can be code in the code editor that is invalid in the sense of it doesn't compile to valid JS. For example, JSX tags might not be closed or other JS syntax errors. I think we'd still want to autosave this invalid JS code, so that if the component code editor's browser crashes and they restart it, they can still pick up where they left off. But then, what do we save into the compiled_js property as part of this autosave?

• Option 1: we save NULL (or something approximating NULL like an empty component that renders NULL) into the compiled_js property. This would then mean that content editors either see nothing within XB for that component or some placeholder message that we can insert explaining that the component's code is invalid.
• Option 2: we retain whatever the old value for compiled_js was. Meaning, the last valid compiled code. I think this would have the better UX of showing content editors something more useful, but maybe it's weird to save an autosave record where the compiled_js property isn't an accurate reflection of the source_code_js property?

#14 needs an answer from @lauriii.

I strongly prefer option 1, because it keeps internal consistency.

Option 2 seems better to me as an experience in some cases, especially without the Workspaces. However, I think it would be fine to go with Option 1 because it seems that it would be better to make it visible that the component is in an invalid state rather than let it error silently. Also as an experience, option 1 seems acceptable especially in the Workspaces enabled world.

Trying figure out sizing for this issue as far as what is needed for the back-end:

I see that ✨ Editing code components Active was just committed so there might be some concretes ideas of what the front-end expects from the backend now.

It seems like from discussion on ✨ HTTP API for code component config entities Active that we won't be implementing auto-save in in manner like content entities where the front will use something `\Drupal\experience_builder\Controller\ApiPreviewController` that returns a render markup and as a side effects does an auto-save, correct?

Instead we will be using a more direct method where the client will be sending up exactly the original and compiled js and css that it wants in the auto-save, correct?

I think if that is that case the auto-saving might be able to be a simpler controller wrapped around \Drupal\experience_builder\AutoSave\AutoSaveManager

Basically we would need a PATCH and GET(for reloading the editor) for the individual entity as saved in the AutoSaveManager

does that sound correct?

I have started on this

#18 ✨ Auto-save code components Active :

I see that #3499988: Editing code components was just committed so there might be some concretes ideas of what the front-end expects from the backend now.

Saving, or even dealing with any stored data was not part of that issue. We implemented creating (POST), renaming (PATCH), and deleting (DELETE) code components in ✨ Managing code components in library Active , but that's for the canonical source of the config entity, and not for the auto-saved version.

It seems like from discussion on ✨ HTTP API for code component config entities Active that we won't be implementing auto-save in in manner like content entities where the front will use something `\Drupal\experience_builder\Controller\ApiPreviewController` that returns a render markup and as a side effects does an auto-save, correct?

Instead we will be using a more direct method where the client will be sending up exactly the original and compiled js and css that it wants in the auto-save, correct?

Correct. 💯

Basically we would need a PATCH and GET(for reloading the editor) for the individual entity as saved in the AutoSaveManager

That is my understanding, too.

I thought about handling cleaning up code components auto-save state if the config entity was deleted but I realized that is problem across entity types so I created 🐛 ApiPublishAllController does not handle since deleted entities Active

Actually I still to test the openapi expected errors

added openapi and 403 tests

Very nice to see this working!

Two main remarks:

1. one about the current testing and the assurances it gives: https://git.drupalcode.org/project/experience_builder/-/merge_requests/5...
2. one about the implementation https://git.drupalcode.org/project/experience_builder/-/merge_requests/5...

@wim leers thanks for the review!

I am working on the need for test coverage changes https://git.drupalcode.org/project/experience_builder/-/merge_requests/5...

but pushed back/clarified on 2 other points. Assigning to you for the 2 points I disagreed but if want to wait on the test changes first thats fine

Working on the test changes I found this 🐛 XbConfigEntityHttpApiTest::testJavaScriptComponent doesn't change the component on the first PATCH request Active . would be good to get that little fix because we are updating that test function in this issue, so it is bit confusing. Because we would also want to ensure the PATCH doesn't affect the auto-saved version

re #29

I found another problem with the existing test coverage 🐛 XbConfigEntityHttpApiTest Active

#30: good catch. I've compiled a list of missed defects over at #3499931-35: HTTP API for code component config entities → 😬

Getting closer! https://git.drupalcode.org/project/experience_builder/-/merge_requests/5...

There are more problems with XbConfigEntityHttpApiTest and our openapi.yml see #3505224-6: XbConfigEntityHttpApiTest missing test coverage for individual GET requests for JavaScriptComponent + AssetLibrary →

I got sidetracked by #33 but did push this forward and address some of the points.

@wim leers if you want to wait till give this another self review that is fine too

The out-of-scope clarification follow-up we discussed and you captured in your comment (thanks 👍): 📌 Merge ApiPendingChangesController and ApiPublishAllController, clarify connection Active . (Can you please create it next time? 🙏)

I understand that this MR is slightly complicated by the pre-existing problems that were encountered: 🐛 XbConfigEntityHttpApiTest Active .

The 3 most important observations I encountered while reviewing:

1. But that does not mean we should do partial refactors like this that leave things in a more confusing state. Confusingness is subjective, but here it is objectively less consistent.
   I'm +1 to doing that, but only as a consistently applied pattern, which then requires its own issue.
2. I found the new ApiConfigAutoSaveControllersTest very confusing because it was both using a data provider and not: https://git.drupalcode.org/project/experience_builder/-/merge_requests/5... — fixed that too.
3. It was not clear what the intent of each test was, clarified that in https://git.drupalcode.org/project/experience_builder/-/merge_requests/5...

I fixed all clarity remarks I had, now I'd be comfortable landing this MR from a code quality POV.

The final bit that needs addressing: some missing test coverage + incompleteness of /openapi.yml: https://git.drupalcode.org/project/experience_builder/-/merge_requests/5...

FYI, per https://git.drupalcode.org/project/experience_builder/-/merge_requests/6... this is a blocker not only for ✨ Publishing code components Active , but also for 📌 Allow CMS Author to set site's homepage from navigation Postponed .

For #37.

I think this is ready for review now

Added some follow ups
📌 Introduce value objects for auto save entries, split up ApiAutoSaveController::post Active
📌 Consider adding AutoSaveData::asResponse Active

Automatically closed - issue fixed for 2 weeks with no activity.