Add http_status migrate_condition plugin

Created on 11 June 2024, 6 months ago
Updated 25 June 2024, 5 months ago

Problem/Motivation

I've always wondered if a condition that gets the http status code of a url would be a good idea. If the download process plugin gets a 404, some wierd things can happen. It might be nice to be able to dance around that a bit. There are probably good use cases.

Proposed resolution

Do it!

Examples:

 * 1. Skip the row if a file url gives 4xx or 5xx response.
 *
 * @code
 * process:
 *   _file_trouble:
 *     plugin: skip_on_condition
 *     method: row
 *     source: my_source_url
 *     condition: not:http_status
 * @endcode
 *
 * or simply skip processing a file field before download fails:
 *
 * @code
 * process:
 *   path_to_file:
 *     -
 *       plugin: skip_on_condition
 *       method: process
 *       source: source_url
 *       condition: not:http_status
 *     -
 *       plugin: download
 *       source:
 *         - source_url
 *         - destination_uri
 * @endcode
 *
 * 2. Store the teapot status.
 *
 * @code
 * process:
 *   field_is_a_teapot:
 *     plugin: evaluate_condition
 *     source: my_source_url
 *     condition:
 *       plugin: http_status
 *       code: 418
 * @endcode
 *
 * or equivalently
 *
 * @code
 * process:
 *   field_is_a_teapot:
 *     plugin: evaluate_condition
 *     source: my_source_url
 *     condition: http_status(418)
 * @endcode
 *
 * or even
 *
 * @code
 * process:
 *   field_is_a_teapot:
 *     plugin: evaluate_condition
 *     source: my_source_url
 *     condition:
 *       plugin: http_status
 *       code:
 *         - 418
 * @endcode

Remaining tasks

User interface changes

nope

API changes

http_status

Data model changes

nope

✨ Feature request
Status

Fixed

Version

2.2

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States danflanagan8 St. Louis, US

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Production build 0.71.5 2024