Support a entity delete policy

Created on 17 May 2022, about 2 years ago
Updated 31 August 2023, 10 months ago

Problem/Motivation

When entities are deleted (or no longer exported using the filter criteria) on the remote website (server) there is currently no way for the client to know the entity needs to be deleted. A mechanism should be taught off to allow a client to also delete no longer available remote entities.

Also maybe a "delete" policy needs to be put in place too:
default => if remote delete/unavailable, delete on client
no_delete => if remote delete/unavailable, do not delete on client (break import status)
unpublish => if remote delete/unavailable, unpublish on client (break import status)

Steps to reproduce

Create a node on the remote site.
Setup a channel to expose this node.
Create a remote website on a client site.
Import the node on the client site.
Delete or make the node unavailable (channel filter criteria) on the remote site.

Proposed resolution

Client polls remote for each local entity that has import status. For each returned entity, act to delete policy.

Remaining tasks

* Figure out how we can poll the server efficiently with a list of locally imported entities.

* Create a delete policy mechanism (should the server also have a say in this? maybe a server wants to impose a delete policy so sensitive data is not replicated on clients unwanted).

* Figure out how we can visualise the no longer available entities, either in the pull form or in a separate tab.

Write tests.

User interface changes

a) The pull form should list local entities no longer found on the remote.
b) Provide a separate tab ("deleted entities") with a separate form.

API changes

Data model changes

✨ Feature request
Status

Active

Version

3.0

Component

Code

Created by

πŸ‡§πŸ‡ͺBelgium kriboogh

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • πŸ‡«πŸ‡·France Grimreaper France πŸ‡«πŸ‡·

    Hi,

    Thanks for the proposal.

    But entity deletion is out of the scope of Entity Share, I think it can be done in another contrib project but may be too business relative to provide a generic solution.

    The problem is for referenced entities imported with the main pulled entities. Entities could be referenced from anywhere, so how to decide if an entity should be deleted or not? If it is still used somewhere else you could get missing references and break other contents.

    About the user interface changes, I am more in favor of solution b) a new dedicated form.

    This makes me think of the following similar issue ✨ Unpublish already imported entities that are no more in the server channel Needs work .

  • πŸ‡¬πŸ‡§United Kingdom jollysolutions

    Could that be an option? Whether to delete the references or not? And if the reference was used else where would it not be reimported ie on the next cron?

  • πŸ‡ΊπŸ‡ΈUnited States Fool2

    I don't see why not. My concern is that entity_import_status does not track the reference in a meaningful way. So in order to do this (whether in a different contrib module or not) we would need to:

    • Figure out which references are attached to the deleted entity
    • Figure out if any other entities reference that entity (should it only count published?)
    • Perhaps add some metadata to the import status so it is clear what the current status is

    There are several conditions that will make this not-so-straightforward, especially for some of the more advanced plugins like entities embedded in a wysiwyg instead of populating a reference field. I don't have confidence that unpublishing or deleting references will be 100% reliable, perhaps things would have to be more deliberately configured.

  • πŸ‡¨πŸ‡­Switzerland Berdir Switzerland

    This could also be a feature we're interested in, need to discuss with the client. I think the combination of the configurable policy idea here and the work that was done in ✨ Support a entity delete policy Active .

    I agree that referenced entities are a bit more challenging, but I think updating them is already limited in a similar way, they are only updated if a referenced entity is updated I assume? In other words, the "it's not 100% reliably so we can't support delete" argument could also be applied to updating referenced entities or referenced entities in general :)

    The existing changes in that other issue do a query on the last update flag, if we combine that with a server lookup then it could be combined with an update-or-delete check on both "channeled" and referenced entities. FWIW, in our project, referenced entities is likely not a major concern, so I expect we could also live with a version that just does channeled entities.

    Also, the extra modules that have server -> client notifications to trigger updates could probably add support for deletes as well, or really just trigger it like any other update and the client will then figure out that it's gone.

  • πŸ‡¨πŸ‡­Switzerland Berdir Switzerland

    I had a look at updating ✨ Unpublish already imported entities that are no more in the server channel Needs work , and while I made some progress and making it configurable on that level is fairly easy (that's a cron setting there now, not import policy), I did run into some issues, notably reliably detecting when an entity really isn't around anymore when you use the SkipImported plugin which I guess everyone with a decent amount of shared entities does.

    A really scalable solution would probably require a special server-side resource (maybe a custom entity type, then we have jsonapi integration for free?) that listens on entity hooks of channeled and/or configurable entity types, and creates a track record for that. the client could then fetch (recent) entries in that resource and delete them locally. I suppose that could be a separate optional module but you'd again need two of those, a server and a client.

    Alternatively, I guess something could also be built on top of https://www.drupal.org/project/entity_share_websub β†’ , with a special notification, but that then relies on the fact that all such notifications are reliably distributed.

    Evaluating my options with my client now.

  • πŸ‡¨πŸ‡­Switzerland Berdir Switzerland

    I did implement a custom solution now with an entity type to track deletions, posted relevant parts of that in ✨ Unpublish already imported entities that are no more in the server channel Needs work (wanted to do it here, got a bit confused with the issues)

Production build 0.69.0 2024