Contrib.social

Parsing JSON - arrays

Open on Drupal.org →
Created on 12 October 2019, over 4 years ago
Updated 5 March 2024, 4 months ago

EDIT: added the yml and fixed the json data (was a copy/paste error)

Problem: Not able to get the data (id or Name) from the "Sets" array. Goal is to create 4 contents out of this data.

This is the source:

{
  "Time" : "2019-10-08",
  "Centers" : [ {
    "id" : "B01502",
    "CenterId" : 1502,
    "Sets" : [ {
      "id" : "B0150200",
      "Name" : "name_valuex"
    }, {
      "id" : "B0150201",
      "Name" : "name_valueu"
    } ]
  }, {
    "id" : "B01503",
    "CenterId" : 1503,
    "Sets" : [ {
      "id" : "B0150301",
      "Name" : "name_valueb"
    }, {
      "id" : "B0150302",
      "Name" : "name_valuea"
    } ]
  } ]
}

This is my YML

id: test_data
source:
  constants:
    TITLE_PREFIX: 'title prefix - '

  plugin: url
  data_fetcher_plugin: file
  data_parser_plugin: json

  urls:
    - /test.json
    
  item_selector: 1
  fields:
    - name: id
      label: 'id'
      selector: id
  ids:
    id:
      type: string
process:
  title: constants/TITLE_PREFIX
  field_presetid: id 
destination:
  plugin: 'entity:node'
  default_bundle: bundle_name

I have tried to put different values to "item_selector:" like /Centers/Sets or "0" etc.
Modules:
Migrate: Version: 8.7.8
Migrate Plus: Version: 8.x-4.2
Migrate Tools: Version: 8.x-4.4
Drupal: 8.7.8
PHP 7.2.22

✨ Feature request
Status

Needs review

Version

6.0

Component

Miscellaneous

Created by

🇫🇮Finland jukka792

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Comment 4 months ago →
    🇵🇪Peru marvil07
  • Merge request !87#3239180: Start a JSON subitems data parser → (Open) created by marvil07
  • Pipeline finished with Success
    4 months ago
    Total: 246s
    #111060
  • Issue was unassigned.
  • Status changed to Needs review 4 months ago
  • Comment 4 months ago →
    🇵🇪Peru marvil07

    It will be nice to have a way to walk through a set of items nested in all the item selector items.

    I have started one custom data parser plugin that does that.
    It basically extends the json data parser plugin, and then uses a given selector to unify its values across all the elements.

    E.g. the following the original example, the following source configuration bits will extract the four items.

    data_parser_plugin: json_subitem
item_selector: Centers
subitem_selector: Sets

    On the ideal world, jq pipelines can do the work once, but if the source endpoints cannot be changed, something like this data parser helps to produce a set that is sensible to walk.

    New commits on new 3087614-json-subitems topic branch and related new MR !87.

    - 3f81967 #3239180: Start a JSON subitems data parser
contrib.social Blog FAQ Discussions
Production build 0.69.0 2024