Contrib.social

JSON is invalid when multiple batches are needed

Open on Drupal.org →
Created on 18 June 2024, 8 months ago

Problem/Motivation

I still need to do more investigation, but it seems this module is producing invalid JSON when using it as an export format when multiple batches are needed for the export. We've seen it adding ending and beginning array markers back to back (][) in the output format, where the output should be one large array. This is probably because individual batches are JSON encoded and then concatenated. In order to produce a valid JSON file, we may need to do some post-processing on the resulting file.

Steps to reproduce

(Preliminary.. Like I said, we need to do more investigating, but wanted to get this "out there").

  • Create a data export display using JSON format and batches
  • Make sure you have enough content to need at least two batches (two content items with a batch of 1 may already be enough..?)
  • Run the export
  • See that the JSON file is invalid because it has concatenated multiple JSON arrays

Proposed resolution

Note sure yet. Probably some post-processing on the batch result to produce a valid JSON file. This might itself actually run into memory issues, which we were using data export to circumvent in the first place.

Remaining tasks

  • Investigate
  • Come up with a solid approach
  • Create merge request
  • Review
  • Merge

User interface changes

None.

API changes

None.

Data model changes

None.

🐛 Bug report
Status

Active

Version

1.4

Component

Code

Created by

🇳🇱Netherlands eelkeblok Netherlands 🇳🇱

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024