Handling media items when their asset gets unavailable in Widen

Created on 24 April 2024, 7 months ago
Updated 18 September 2024, 2 months ago

After listening to the feedback from some Acquia DAM clients, the situation of handling earlier public assets becoming unavailable is a somewhat wider topic rather than putting an equality sign between their availability status in Widen and the published status field in Drupal.

First of all, it's important to understand that not the expiration date is the only factor that controls whether a DAM asset should be available outside of Widen or not. For example, if a Widen admin sets a future value for the Release date field of an asset (causing the asset to become unreleased immediately) for an earlier publicly available asset, then the Drupal users will be not informed about not using that piece of media. Therefore renaming this ticket to use the term “unavailable” instead of “expired”.

Looking at various real-life scenarios each considered valid, the module has to be prepared for the following use cases as well (assuming that a media item already exists in Drupal with a valid reference to an also existing asset UUID in Widen):

Also worth noting that some sort of effort has been made already to tackle the situation of expired assets: a red warning is displayed when the on-the-fly evaluation of expiration check results to false. For example, on the DAM-related media overview page (at /admin/content/dam-media):

There are two problems with this feature:

  1. This widget alter hook gets invoked in limited situations only. And even when it runs in its unpredictable manner, then does not store the results of its business logic at all. Therefore we can say it's a cosmetic band-aid only to handle these use cases.
  2. Takes into consideration the expiration date only. However, not the expiration date is the only factor that controls whether a DAM asset should be available outside of Widen or not. For example, if a Widen admin sets a future value for the Release date field of an asset (causing the asset to become unreleased immediately) for an earlier publicly available asset, then the Drupal users will be not informed about to not use that piece of media.

My proposed approach:

Original issue summary

Problem/Motivation

When assets expire in Acquia DAM(Widen collective), they are not automatically unpublished in the Drupal website, leading to outdated or unauthorized content being accessible to the public.

Steps to reproduce

  • Install Drupal.
  • Install and configure the Acquia DAM module.
  • Add an asset to Acquia DAM with a future expiry date.
  • Create a new content type including a media field linked to DAM.
  • Create a node, incorporating the new asset.
  • Modify the asset’s expiry date in DAM to a past date.
  • Navigate to the media page and trigger 'Check for update'.
  • Confirm the asset’s publication status.

Proposed resolution

When asset in DAM get expired it should be unpublished in Drupal.

Remaining tasks

  • Review
  • Merg
  • Release
📌 Task
Status

Postponed

Version

1.0

Component

Code

Created by

🇮🇳India rajeshreeputra Pune

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @rajeshreeputra
  • Status changed to Needs review 7 months ago
  • Pipeline finished with Failed
    7 months ago
    Total: 1562s
    #154993
  • Pipeline finished with Failed
    7 months ago
    Total: 1436s
    #154999
  • Assigned to Balu Ertl
  • Status changed to Needs work 7 months ago
  • 🇭🇺Hungary Balu Ertl Budapest 🇪🇺
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.1 & MySQL 5.7
    last update 7 months ago
    Composer require-dev failure
  • Pipeline finished with Failed
    7 months ago
    Total: 1449s
    #155556
  • Pipeline finished with Failed
    7 months ago
    #159943
  • Status changed to Needs review 7 months ago
  • 🇭🇺Hungary Balu Ertl Budapest 🇪🇺

    I separated all my code modifications into logical commits, so here not going into details on what and how was implemented. Instead, here are two annotated screenshots depicting some of the visually perceptible changes:

  • 🇮🇳India rajeshreeputra Pune

    @BaluErtl changes looks really awesome, +1 for this approach.

  • 🇭🇺Hungary Balu Ertl Budapest 🇪🇺
  • 🇭🇺Hungary Balu Ertl Budapest 🇪🇺

    Balu Ertl changed the visibility of the branch 3443055-assets-are-not to hidden.

  • Pipeline finished with Failed
    4 months ago
    Total: 169s
    #241868
  • Pipeline finished with Failed
    4 months ago
    Total: 1028s
    #244404
  • Pipeline finished with Failed
    4 months ago
    Total: 329s
    #244462
  • Pipeline finished with Failed
    4 months ago
    Total: 146s
    #244470
  • Pipeline finished with Failed
    4 months ago
    Total: 1035s
    #244482
  • Status changed to Postponed 4 months ago
  • 🇭🇺Hungary Balu Ertl Budapest 🇪🇺

    These improvements heavily rely on asset version comparison in order to be able to tell whether a remote DAM asset is newer or the the same as its local media entity counterpart in Drupal.

    As asset version management is being refactored in 🐛 Look for the "finalized" asset version not the latest modified Needs review , then 🐛 Asset versioning feature disabled results in a 403 error Postponed is also waiting down the line to get merged, therefore I suggest to mark this one as Postponed until they get landed first.

    Regarding the failing tests on this MR

    • PHP 8.1
      Some Drupal core D11 incompatibilites detected by Composer: “Your requirements could not be resolved to an installable set of packages.” (job output)
    • PHP 8.2 & 8.3
      Some other script failure at “step_script” stage:

      $ docker-php-ext-install sodium
      tar (child): /usr/src/php.tar.xz: Cannot open: No such file or directory
      tar (child): Error is not recoverable: exiting now
      tar: Child returned status 2
      tar: Error is not recoverable: exiting now
      

      (job output)

  • 🇭🇺Hungary Balu Ertl Budapest 🇪🇺
  • Pipeline finished with Failed
    4 months ago
    Total: 1101s
    #245573
  • 🇭🇺Hungary Balu Ertl Budapest 🇪🇺

    One idea for further improvement: make this DAM media listing page filterable for asset status sync feature.

    1. Research the approach defining a pseudo-field for the media entity to make each individual media item queriable about its own syncing status.
    2. If it proves to be possible then we move the current rendering logic into a Formatter dedicated to this pseudo-field.
    3. Then we could get rid of the current ViewsField plugin (its disadvantage is that makes sync status information available only in the context of Views but nowhere else in the entire application)
    4. Then simply add the pseudo-field to the DAM media listing page.
    5. At this point filtering on the View probably should work now but if not, then implementing a dedicated ViewsFilter plugin might be necessary.
  • 🇭🇺Hungary Balu Ertl Budapest 🇪🇺

    Both blockers got landed freeing up the status of this one:

    1. 🐛 Look for the "finalized" asset version not the latest modified Needs review
    2. 🐛 Asset versioning feature disabled results in a 403 error Postponed
  • 🇺🇸United States japerry KVUO

    After reviewing the screenshots in #6, I think the following answers most of the questions in #12....

    Summary
    Set corresponding Drupal media status to unpublished when the 'released_and_not_expired' is false. Also set the item to unpublished if there is a delete error code. In all other status transitions, do nothing.

    Answers and implementation details

    • Users should not have the option to publish/unpublish media items in Drupal when items are unavailable or deleted in Widen. Widen is the source of truth, and while Drupal can provide tighter restrictions (unpublishing assets in Drupal when an asset is published in Widen), the opposite is not possible.
      • - status should be set to unpublished during sync when widen shows deleted/expired (not simply in the form)
      • - The status switch should be greyed out, thus the red error message will never occur because its not possible to get to this state.
      • - The warning message can be simplified to: "This media item is unpublished due to being expired or deleted in Acquia DAM. Please [enable the asset in DAM] to use it in Drupal" --> [add a link the asset in widen]. If the user doesn't have access, thats fine.
    • The "Check status sync" screenshot can be simplified as well, since this issue should resolve any relevant mismatch in published status:
      • "Check Status Sync" --> Should say "DAM Status"
      • Use "Available" (with Green checkbox), "Expired" (Warning Box), "Deleted" (Red X), "Unknown" (? box)
    • Add a button link below the local tasks (that goes to a confirm deletion prompt), "Remove deleted assets"

    Make this DAM media listing page filterable for the asset status sync feature. Theoretical steps of implementation:

    At least for now, I don't think we need to do this. Simply adding a delete button should be fine. At some point it'd be good if this page had search or other filters, but thats not related to this issue (or before 1.1 release)

    Also might be worth researching the possibility of defining a custom entity operation. A “Sync media item status accordingly” (or similar wording) option could appear both on the drop-down buttons at the right end and also in the bulk operation select list. Performing this operation updates the publishment status of selected media items according to their availability status in Widen. Also leaves a trace in Watchdog for later reference

    This should occur with the regular sync operation that occurs during cron or the manually existing 'sync metadata' option.

    For example, if a Widen admin sets a future value for the Release date field of an asset (causing the asset to become unreleased immediately) for an earlier publicly available asset, then the Drupal users will be not informed about to not use that piece of media

    The only time we should change the publishing status in Drupal is when an item becomes expired or deleted. In all other cases, a user will need to manually republish the item in drupal. This protects items from unexpectedly becoming published in Drupal. If this becomes a problem, lets tackle that in another issue.

    When performing the 'bulk delete' operation, a query should be done against all unpublished media assets in drupal, which will make a call to widen to verify they were deleted. If so, delete the media item.

    Some things we don't need to worry about:
    1) Checking usage status. DAM overrides drupal when it comes to publish status. If its not available in the DAM, its not available regardless of Drupal's publish status. No need to add extra logic or warnings here.
    2) Base field storing the DAM status (I think) -- this would only be needed if we needed to match the published status, which we don't -- we only need an action to change Drupal's publish status based on whatever the current status is.

  • Pipeline finished with Success
    15 days ago
    #331029
  • Pipeline finished with Success
    15 days ago
    #331041
  • Pipeline finished with Success
    15 days ago
    #331069
  • Pipeline finished with Failed
    12 days ago
    Total: 1539s
    #333824
  • Pipeline finished with Success
    12 days ago
    Total: 1430s
    #333914
  • Pipeline finished with Failed
    11 days ago
    Total: 947s
    #334490
  • Pipeline finished with Failed
    11 days ago
    Total: 1153s
    #334496
  • Pipeline finished with Failed
    11 days ago
    Total: 1575s
    #334514
  • Pipeline finished with Failed
    11 days ago
    Total: 1715s
    #334597
  • 🇭🇺Hungary Balu Ertl Budapest 🇪🇺

    Crediting others for their contribution made on 📌 The cron job is repeatedly logging error messages. Active which proposal has been implemented on this ticket.

  • Pipeline finished with Failed
    9 days ago
    Total: 1456s
    #336573
  • Pipeline finished with Failed
    9 days ago
    Total: 2808s
    #336935
Production build 0.71.5 2024