Add a mapping target to media field

Created on 6 December 2017, over 7 years ago
Updated 19 June 2023, about 2 years ago

Hello,

I want to import datas from a rss feed to generate medias (facebook).
But, I don't see the "media target" in the settings of my feed type.
Do you think add this target in the futur?

Thanks.

Feature request
Status

Needs work

Version

3.0

Component

Code

Created by

🇫🇷France kumkum29

Live updates comments and jobs are added and updated live.
  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇨🇷Costa Rica Royden_CH

    Compatible with core 9.5.9 and 8.x-3.0-beta4

  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.5 + Environment: PHP 8.1 & pgsql-10.12
    last update about 2 years ago
    690 pass, 5 fail
  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.x + Environment: PHP 8.1 & MySQL 5.7
    last update about 2 years ago
    693 pass, 4 fail
  • 🇵🇱Poland besek

    #80 worked for me, however it saevd node title as image alt. I think it should be configurable so you can choose what values you want to save in title and alt fields of the image. Thanks

  • 🇵🇹Portugal jrochate

    I don't have standard media machine names, and the latest path WSODs.

    Drupal\Component\Plugin\Exception\PluginNotFoundException: The "media_base_image" plugin does not exist. Valid plugin IDs for Drupal\feeds\Plugin\Type\FeedsPluginManager are: file, datetime, number, media_file, uri, feeds_item, config_entity_reference, integer, temporary_target, daterange, email, timestamp, password, book, image, media_image, user_role, link, text, boolean, telephone, langcode, entity_reference, path, string, geofield_feeds_target, paragraphs in Drupal\Core\Plugin\DefaultPluginManager->doGetDefinition() (line 53

    and also

    Error: Class "Drupal\feeds\Feeds\Target\MediaFile" not found in Drupal\feeds\Entity\FeedType->getMappingTargets() (line 305 of /var/www/clients/client1/web39/web/repos/ccmar/web/modules/contrib/feeds/src/Entity/FeedType.php)

    it looks like we must use standard profile installation because the fields are hardcoded.

  • 🇺🇸United States amanire

    I've updated the patch in #80 for Drupal 10 compatibility. The corresponding feed had stopped importing and was throwing the following warning:

    Entity queries must explicitly set whether the query should be access checked or not. See Drupal\Core\Entity\Query\QueryInterface::accessCheck().

  • 🇺🇸United States amanire

    OK this patch now includes the explicit accessCheck for D10 compatibility.

  • 🇺🇸United States ed2908

    On Patches
    2928904-58.patch
    add_a_mapping_target_to_media_field-2928904-80.patch
    and
    add_a_mapping_target_to_media_field-2928904-86_0.patch

    I'm seeing a lot of:
    "The file, 917, failed to save because the extension, 17, is invalid.
    The file, 918, failed to save because the extension, 18, is invalid.
    The file, 920, failed to save because the extension, 20, is invalid."

    and a lot of "Image (field_media_image): This value should not be null."

    My mappings are:
    Feed field containing public://PATH TO IMAGE.jpeg -> Image (field_media_image)
    Feed field with file id -> Image (field_media_image): File ID
    Feed field with filename -> Name (name)

    My file IDs from the old site are 917, 918, 220 ect.
    Nowhere in the feed is 17, 18, 20. Certainly not being used as a file extension.

    Am I doing something completely wrong?

  • 🇧🇷Brazil isa.bel Balneário Camboriú

    I had some issues after updating to the latest stable version of the module, but using the patch from #84 worked for me!

  • 🇺🇸United States jesss

    Patch #86 is working for me!

  • 🇳🇱Netherlands djschoone Breda

    @jesss on which version did you apply? #86 on 3.0.0-beta4 gives me when visiting admin/structure/feeds/manage//mapping this error:

    Error: Call to a member function getSettings() on null in Drupal\feeds\Plugin\Type\Target\MediaTargetBase->__construct() (line 58 of modules/contrib/feeds/src/Plugin/Type/Target/MediaTargetBase.php).
    Drupal\feeds\Plugin\Type\Target\MediaTargetBase::create(Object, Array, 'media_image', Array) (Line: 21)
    Drupal\Core\Plugin\Factory\ContainerFactory->createInstance('media_image', Array) (Line: 22)
    Drupal\feeds\Plugin\Type\FeedsAnnotationFactory->createInstance('media_image', Array) (Line: 83)
    Drupal\Component\Plugin\PluginManagerBase->createInstance('media_image', Array) (Line: 515)
    Drupal\feeds\Entity\FeedType->getTargetPlugin(6) (Line: 375)
    Drupal\feeds\Form\MappingForm->buildRow(Array, Object, Array, 6) (Line: 160)
    Drupal\feeds\Form\MappingForm->buildForm(Array, Object, Object)
    call_user_func_array(Array, Array) (Line: 536)
    Drupal\Core\Form\FormBuilder->retrieveForm('feeds_mapping_form', Object) (Line: 283)
    Drupal\Core\Form\FormBuilder->buildForm(Object, Object) (Line: 73)
    Drupal\Core\Controller\FormController->getContentResult(Object, Object)
    call_user_func_array(Array, Array) (Line: 123)
    Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 627)
    Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 124)
    Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) (Line: 97)
    Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 181)
    Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 76)
    Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 58)
    Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 48)
    Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 28)
    Drupal\Core\StackMiddleware\ContentLength->handle(Object, 1, 1) (Line: 106)
    Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 85)
    Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 53)
    Asm89\Stack\Cors->handle(Object, 1, 1) (Line: 48)
    Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 51)
    Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 36)
    Drupal\Core\StackMiddleware\AjaxPageState->handle(Object, 1, 1) (Line: 51)
    Drupal\Core\StackMiddleware\StackedHttpKernel->handle(Object, 1, 1) (Line: 704)
    Drupal\Core\DrupalKernel->handle(Object) (Line: 19)
  • 🇺🇸United States jesss

    I'm on 3.0.0-beta4. Did you clear cache after applying the patch?

  • 🇳🇱Netherlands Nicasso

    I'm on 3.0.0-beta4 too and patch #86 does indeed generate an error when visiting: /admin/structure/feeds/manage/something/mapping

    A cache rebuild did not solve it unfortunately.

    Error: Call to a member function getSettings() on null in Drupal\feeds\Plugin\Type\Target\MediaTargetBase->__construct() (line 58 of modules/contrib/feeds/src/Plugin/Type/Target/MediaTargetBase.php).

  • First commit to issue fork.
  • Merge request !161#2928904 Added media target → (Open) created by jidrone
  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.5 + Environment: PHP 7.4 & MySQL 5.7
    last update over 1 year ago
    704 pass, 4 fail
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: PHP 8.2 & MySQL 8
    last update over 1 year ago
    704 pass, 4 fail
  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.5 + Environment: PHP 7.4 & MySQL 5.7
    last update over 1 year ago
    716 pass
  • Pipeline finished with Success
    over 1 year ago
    Total: 1354s
    #134995
  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.5 + Environment: PHP 7.4 & MySQL 5.7
    last update over 1 year ago
    716 pass
  • Status changed to Needs review over 1 year ago
  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.5 + Environment: PHP 7.4 & MySQL 5.7
    last update over 1 year ago
    708 pass, 4 fail
  • 🇨🇴Colombia jidrone

    Hello,

    The existing patch has the following issues:

    • The media targets provided are tightly coupled to other existing targets.
    • It supports only image and file media types, so if you have a media type with a different name, it will not work.
    • It assumes the source file of the media entity types always follows the pattern field_media_[media_type_id].
    • The comments from the maintainer in #79 Add a mapping target to media field Needs review where not addressed.

    This new MR, has the following approach:

    • As minimal as possible changes to existing targets.
    • All the code is in just one new target called Media for loose coupling.
    • Maintainer comments addressed to ensure it is only applicable when media module is enabled
    • It is only applicable when the media field has at least one media type that contains a source field of type file.
    • Detects the applicable media type depending on the file extension.

    I know there are still improvements to do and it will need tests, but this could be a better starting point.

  • The last submitted patch, 86: add_a_mapping_target_to_media_field-2928904-86.patch, failed testing. View results
    - codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

  • Pipeline finished with Success
    over 1 year ago
    Total: 1368s
    #135039
  • 🇳🇱Netherlands megachriz

    Thanks @jidrone! That looks like a big step in getting this issue done. 😃

    Who wants to make a start on writing the tests?

  • 🇺🇸United States irinaz

    @MegaChriz, I am wondering if we could ask GitHub copilot to write tests. I want to experiment and see what AI can do these days. I can provide code of the module the module and file " src/Feeds/Target/Media.php" ask a question "please write unit test for xxxx function". I wonder what information AI needs to write suggested unit tests.

  • 🇳🇱Netherlands megachriz

    @irinaz Nice idea to ask AI for writing tests! Maybe they can help a little on the unit tests. But I think we do also need Kernel tests here, that can check if media gets imported as expected. I wonder if AI would be able to figure out how to write tests for that. It would need to know how a Feeds import goes programmatically, it needs to know what data to feed and it needs to know how the imported media should look like.

  • 🇺🇸United States irinaz

    @MegaChriz, thanks! We probably already have examples for other Targets, correct? We would add examples of data with media, probably. I am quite interested in testing what we need to provide to AI and what can be automated.

  • Pipeline finished with Failed
    over 1 year ago
    Total: 47s
    #145344
  • Pipeline finished with Canceled
    over 1 year ago
    Total: 993s
    #145346
  • 🇺🇸United States irinaz

    Hi @megachirz and @jidrone,
    I did an experiment with writing kernel test using AI. I will try to run tests next week, but wanted to share some progress.

  • First commit to issue fork.
  • Pipeline finished with Failed
    10 months ago
    Total: 47s
    #317216
  • Pipeline finished with Success
    10 months ago
    Total: 289s
    #318344
  • 🇮🇹Italy Giuseppe87

    I have added an extra settings to the current patch, that let the user choose to save the new media with the name of the entity or the file.

    This could be useful, as it's common to have multiple entities with the same files, and it isn't ideal to have the media name given by the first entity imported.

    However being an extra\different approach I haven't updated the MR.

  • 🇮🇹Italy Giuseppe87

    Updated the patch of #104

  • Pipeline finished with Success
    9 months ago
    Total: 447s
    #319308
  • 🇳🇱Netherlands megachriz

    I've updated the issue summary and added a list of tasks remained to be done.

  • 🇳🇱Netherlands megachriz

    @batigolix and I looked at this issue yesterday at the DICTU Drupal Developers Day in Assen.
    @batigolix successfully imported a media image on a node. There were some errors, however:

    • When mapping to a media field first and then checkout the code from this issue he got an error for getSummary().
    • The first import test resulted into a SQL error related to language (language cannot be NULL). When changing the language on the target then to "English", the import went fine. It could be that there's an issue when creating a Media entity when the language is not specified. I did see that on his configuration the language on the target was set to an empty string, so perhaps that is related to the error.

    @batigolix is working on a test, but ran into the error "filename does not exist", which I did not have an explanation for right away.

  • 🇳🇱Netherlands hepabolu

    FYI I have tested this:

    - Drupal 10.3.10
    - PHP 8.3.14
    - Lando 3
    - Feeds 3.0.0-rc3
    - Feeds Tamper 2.0.0-beta4

    - Content type Publication with field_cover = Image
    - Media type Photos with field_image = Image and field_credits = Text
    - Content type Model with field_photos = entity reference to media type Photos

    - Feed import Publication, CSV upload, tamper format string to turn image filename into url
    - Feed import Photos, CSV upload, tamper format string to turn image filename into url
    - Feed import Model, CSV upload, figuring out how to set the reference to the imported Photos

    Without any patch:
    - the import of the image for Publication cover went fine
    - the import of the Photos failed on a missing name, but the files were added anyway with the tampered name as 'filename', adding the Name field didn't help

    With the patch from #105:
    - I set the language from Default to English
    - I explicitly set the Name field to a copy of the filename (in a separate CSV column)

    Import of the media type Photos went fine, filename was correct.

    I'll see if I can do more tests and report back.

  • 🇦🇺Australia max.valetov

    In #105 if (preg_match('/^Autocreate/', $line->__toString())) { __toString doesn't seem to be a valid method on render array.

  • 🇳🇱Netherlands megachriz

    In #105 if (preg_match('/^Autocreate/', $line->__toString())) { __toString doesn't seem to be a valid method on render array.

    @max.valetov Thanks for spotting! I think that the issue with the "Autocreate" option should be fixed in 🐛 Importing new files fails when "Autocreate entity" option is turned on. Active first. I think that one is easy to do.

  • 🇺🇸United States iknowbryan

    SYSTEM:
    DRUPAL 11.1.1
    PHP: 8.3.12

    GOAL: import news from an old site via CSV including the cover image via full URL.

    Installed #105 patch
    Now gives me the option to reference a media field with the FILE ID. GREAT.
    When I run the feed to import that news articles, all the info gets imported without errors. GREAT.
    When I edit one of the news items all the info is there including the cover image. GREAT.
    (Or so it seams because the thumbnail is generated.)
    I can also see the thumbnail in the media library grid.

    HOWEVER, when I go edit the media, the image field is empty. BOO.

    Any thoughts on how this could be??

  • 🇵🇹Portugal jrochate

    Same as #112, using Drupal 10.4 and PHP 8.4.

    When editing Media, the image it's no there, but it can be seen on files and on the related node.

  • 🇳🇱Netherlands megachriz

    @jrochate
    I added the task "When importing media images via a media field on a node, make sure that the image is visible on the media edit form." to the remaining task list. Does that summarize the issue well?

  • 🇵🇹Portugal jrochate

    hi @megachriz

    Yep, that nails it. Thanks.

  • 🇦🇺Australia max.valetov

    Hi @megachriz,

    @max.valetov Thanks for spotting! I think that the issue with the "Autocreate" option should be fixed in #2973170: Importing new files fails when "Autocreate entity" option is turned on. first. I think that one is easy to do.

    Not entirely sure what needs to be fixed/applied from that issue. Applying MR plain diff doesn't resolve the issue.

    (Latest v3 feeds with 105 patch applied)

  • First commit to issue fork.
  • Pipeline finished with Canceled
    5 months ago
    Total: 198s
    #439608
  • 🇭🇰Hong Kong hswong3i

    Since people already keep testing #105 Add a mapping target to media field Needs review and feedback directly, I give a merge to https://git.drupalcode.org/project/feeds/-/merge_requests/161 for simplify.

    Moreover, I merge changes for local file import logic from https://www.drupal.org/project/feeds/issues/2968671 File target: add support for local file path as source Needs work , else this 2 MR will get conflict and working together.

  • Pipeline finished with Success
    5 months ago
    Total: 434s
    #439611
  • 🇳🇱Netherlands megachriz

    @hswong3i
    I understand that people could need this and File target: add support for local file path as source Needs work together, but for keeping the issues focussed, it is better to keep them separated. Instead, maybe it is better to open a separate branch called '2928904-2968671-media-and-local-file-do-not-merge' for that.

    For people would like to see this issue to be resolved, we need to have automated tests - and there appears to be an issue with updating existing configuration. See Remaining tasks for details.

  • 🇳🇱Netherlands megachriz

    megachriz changed the visibility of the branch 2928904-add-a-mapping-test to hidden.

  • Status changed to Needs work 4 months ago
  • 🇳🇱Netherlands megachriz

    megachriz changed the visibility of the branch 2928904-add-a-mapping to hidden.

  • 🇳🇱Netherlands megachriz

    I made a start with writing the tests. The tests are currently failing though.

  • 🇦🇺Australia max.valetov

    Thanks @megachriz, applying MR161 works.

    We have an issue where on some sites (with the same feed type settings) we are getting an error "Media type could not be determined for [Media Title here]". If I am interpreting things correctly, it gets Title instead of a path so there's no file extension to match against and hence fails to determine the type.
    Do you know what could be causing this issue?

  • 🇳🇱Netherlands megachriz

    @max.valetov
    I did not take a deep dive into the Media target issue yet, mostly only worked on the tests so far.
    I do know that the source that you should map to the Media target must be an url using the http or https scheme.
    Other source types, like local path or not yet supported, but there is being worked on: File target: add support for local file path as source Needs work .
    The source url must contain an extension as well, but there is an issue open for supporting file urls that do not specify the extension: 💬 Failed to validate remote image with no file extension Needs work

    If you are not sure what value Feeds receives, I recommend to enable the Feeds Log module (included with Feeds). With that module you can see what items were processed and what the source item looked like at the time of processing.

  • 🇺🇸United States amanire

    Is it possible to map a source to populate the alt tags of new media items? I've been experimenting with this line of code in a debugger but it doesn't seem to be saving an alt value to the media item or node for me.

    Thank you for all of your hard work on this issue!

  • 🇺🇸United States jennypanighetti

    Tried to install patch from MR161 on feeds 3.0 and it failed to apply. Should this be run against dev?

  • 🇩🇪Germany kopfduenger Wiesbaden

    This patch builds on the patch #105 of giuseppe87 and fixes what was missing in order to attach podcast audio files (my use case) correctly to show up in media entities that were automatically created during a feed import.

    Specifically, it:
    • Modifies the dedicated Media.php plugin introduced before, and uses the already introduced File plugin to support entity reference fields targeting media entities.
    • Automatically selects the correct media bundle based on file extension, even when multiple media types are available.
    • Adds support for configuring whether the media entity name is derived from the referencing entity or the file name.
    • Ensures existing media entities referencing the same file are reused to avoid duplication.
    • Adds proper ownership handling (based on Feed author or fixed user ID).
    • Sets the media entity’s alt text and language properly.
    • Handles directory creation dynamically based on the media type’s file settings (uri_scheme, file_directory).
    • Improves logging and error handling via \Drupal::logger(), especially for download and media creation errors.
    • Strips query parameters from URLs to prevent extension mismatches.
    • Removes Feeds’ default autocreate option for clarity and to prevent conflicts with custom logic.
    • Updates File.php to support passing the file URL to getDestinationDirectory(), enabling per-media-type storage paths.

    The patch is intended for drupal/feeds version dev-3.x, commit 9f8db96, and is compatible with Drupal 10 and core Media module setups.

Production build 0.71.5 2024