"fields" numeric keys don't correspond to columns

Created on 11 February 2022, almost 3 years ago
Updated 23 February 2023, over 1 year ago

Problem/Motivation

According to the documentation, each numeric key under "fields" should correspond to the 0-based index of a column:

  # Here we identify the columns of interest in the source file. 
  # Each numeric key is the 0-based index of the column. 
  # For each column, the key below is the field name assigned to 
  # the data on import, to be used in field mappings below. 
  # The label value is a user-friendly string for display by the 
  # migration UI.
  fields:
    0:
      name: id
      label: 'Unique Id'
    1:
      name: title
      label: 'Title'
    2:
      name: body
      label: 'Post body'

However, the value of the numeric key seems irrelevant. Instead, it always goes through each column of the CSV in order. Based on the comment "Here we identify the columns of interest in the source file.", it seems like we should be able to pick and choose which columns we'd like to use from a CSV, but it seems like the only way to do that is by defining every column up to the last column we need.

If my CSV looks like this:

id, title, body, author
0, Great Title, Some body text, James

and my migration is setup like this:

   0:
      name: id
      label: 'Unique Id'
    1:
      name: title
      label: 'Title'
    3:
      name: author
      label: 'Post author'

Then wherever "author" is used in the process section, the value of the column "body" from the CSV will be used instead.

Proposed resolution

Make each numeric key under "fields" correspond to a column in the CSV.

Remaining tasks

User interface changes

API changes

Data model changes

πŸ› Bug report
Status

Active

Version

3.0

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States Matthew.

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • I've just run into this issue also.

    Not a code fix, but simple solution. Defining the `fields` in the config is optional and not really needed if your CSV does have a header row.

    If you skip defining the fields you can refer to column names directly in the process section.

Production build 0.71.5 2024