[1.0.x] Trucie (True Content Import)

Created on 16 April 2024, about 1 year ago
Updated 23 May 2024, 12 months ago

Trucie module provides content import from the spreadsheet file. Main features:

  • Supported formats: xlsx, xls, ods, csv.
  • Provides UI to configure the import (this can be exported to site config).
  • Provides basic processing rules for fields from the source file.
  • Supports ANY entity type including custom entities (not just nodes and taxonomy terms).
  • Provides events (similar to hooks) to make it possible to write custom data processing.
  • No dependencies.

There are number of modules available that provides content import from xlsx or csv, the main difference is that Trucie supports all entity types and not just the ones that comes from core (i.e. nodes and terms). Additionally, it's possible to configure basic processing rules for fields (like floatval or preg_replace).

This module is inspired by Migrate API and Feeds module. But with Migrate API you have to manually upload the source file to some directory on the server in order to do the content import, while with Trucie you can just select the file on the form to upload it.

It requires basic understanding of how entities are structured, so it's recommended to have a tech-savvy guy to configure the imports. But once it's set up any one can do the csv import at any time.

Project link

https://www.drupal.org/project/trucie →

📌 Task
Status

Fixed

Component

module

Created by

🇷🇺Russia i-grou

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @i-grou
  • 🇮🇳India vishal.kadam Mumbai

    Thank you for applying!

    Please read Review process for security advisory coverage: What to expect → for more details and Security advisory coverage application checklist → to understand what reviewers look for. Tips for ensuring a smooth review → gives some hints for a smoother review.

    The important notes are the following.

    • If you have not done it yet, you should run phpcs --standard=Drupal,DrupalPractice on the project, which alone fixes most of what reviewers would report.
    • For the time this application is open, only your commits are allowed.
    • The purpose of this application is giving you a new drupal.org role that allows you to opt projects into security advisory coverage, either projects you already created, or projects you will create. The project status won't be changed by this application and no other user will be able to opt projects into security advisory policy.
    • We only accept an application per user. If you change your mind about the project to use for this application, or it is necessary to use a different project for the application, please update the issue summary with the link to the correct project and the issue title with the project name and the branch to review.

    To the reviewers

    Please read How to review security advisory coverage applications → , Application workflow → , What to cover in an application review → , and Tools to use for reviews → .

    The important notes are the following.

    • It is preferable to wait for a Code Review Administrator before commenting on newly created applications. Code Review Administrators will do some preliminary checks that are necessary before any change on the project files is suggested.
    • Reviewers should show the output of a CLI tool → only once per application.
    • It may be best to have the applicant fix things before further review.

    For new reviewers, I would also suggest to first read In which way the issue queue for coverage applications is different from other project queues → .

  • 🇷🇺Russia i-grou

    Thanks @vishal.kadam. I've already read the checklist and ran phpcs. Hope it's fine, waiting for review. Will be glad to any feedback.

  • Status changed to Needs work about 1 year ago
  • 🇮🇳India vishal.kadam Mumbai

    1. feature/TRC-3 and feature/TRC-2 are wrong names for a branch and should be removed. Release branch names always end with the literal .x as described in Release branches → .

    2. Fix phpcs issues.

    phpcs --standard=Drupal,DrupalPractice --extensions=php,module,inc,install,test,profile,theme,css,info,txt,md,yml trucie/
    
    FILE: trucie/src/TrucieImporterInterface.php
    -----------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    -----------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    -----------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -----------------------------------------------------------------------------
    
    FILE: trucie/src/Form/TrucieImporterForm.php
    -------------------------------------------------------------------------------
    FOUND 3 ERRORS AFFECTING 3 LINES
    -------------------------------------------------------------------------------
       1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
     362 | ERROR | [ ] The array declaration extends to column 92 (the limit is 80). The array content should be split up over multiple lines
     419 | ERROR | [ ] The array declaration extends to column 115 (the limit is 80). The array content should be split up over multiple lines
    -------------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -------------------------------------------------------------------------------
    
    FILE: trucie/src/Entity/TrucieImporter.php
    -----------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    -----------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    -----------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -----------------------------------------------------------------------------
    
    FILE: trucie/src/Importer/BatchImporterFactory.php
    -------------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    -------------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    -------------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -------------------------------------------------------------------------------
    
    FILE: trucie/src/Importer/SpreadsheetBatchImporter.php
    -------------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    -------------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    -------------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -------------------------------------------------------------------------------
    
    FILE: trucie/src/Importer/CsvBatchImporter.php
    ------------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    ------------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    ------------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    ------------------------------------------------------------------------------
    
    FILE: trucie/src/Importer/BatchImporterInterface.php
    -------------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    -------------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    -------------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -------------------------------------------------------------------------------
    
    FILE: trucie/src/Importer/BatchImporterBase.php
    -------------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    -------------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    -------------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -------------------------------------------------------------------------------
    
    FILE: trucie/src/Importer/BatchImporterFactoryInterface.php
    -------------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    -------------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    -------------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -------------------------------------------------------------------------------
    
    FILE: trucie/src/Event/TrucieEvents.php
    -----------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    -----------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    -----------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -----------------------------------------------------------------------------
    
    FILE: trucie/src/Event/EntityEvent.php
    -----------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    -----------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    -----------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -----------------------------------------------------------------------------
    
    FILE: trucie/src/Event/RowEvent.php
    -----------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    -----------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    -----------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -----------------------------------------------------------------------------
    
    FILE: trucie/src/TrucieImporterListBuilder.php
    ------------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    ------------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    ------------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    ------------------------------------------------------------------------------
    
    FILE: trucie/src/EventSubscriber/ExampleEventSubscriber.php
    -------------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    -------------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    -------------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -------------------------------------------------------------------------------
    
    FILE: trucie/src/EventSubscriber/TrucieEventSubscriberAbstract.php
    -------------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    -------------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    -------------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -------------------------------------------------------------------------------
    
    FILE: trucie/src/Processor/TrucieImporterProcessorInterface.php
    -------------------------------------------------------------------------------
    FOUND 1 ERROR AFFECTING 1 LINE
    -------------------------------------------------------------------------------
     1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line
    -------------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -------------------------------------------------------------------------------
    
    FILE: trucie/src/Processor/TrucieImporterProcessor.php
    -------------------------------------------------------------------------------
    FOUND 1 ERROR AND 12 WARNINGS AFFECTING 13 LINES
    -------------------------------------------------------------------------------
       1 | ERROR   | [x] The PHP open tag must be followed by exactly one blank line
     174 | WARNING | [ ] Unused private method processExplode()
     189 | WARNING | [ ] Unused private method processImplode()
     204 | WARNING | [ ] Unused private method processFloatval()
     238 | WARNING | [ ] Unused private method processIntval()
     267 | WARNING | [ ] Unused private method processMbStrtolower()
     296 | WARNING | [ ] Unused private method processMbStrtoupper()
     325 | WARNING | [ ] Unused private method processUcfirst()
     354 | WARNING | [ ] Unused private method processPregReplace()
     383 | WARNING | [ ] Unused private method processStrtotime()
     412 | WARNING | [ ] Unused private method processDate()
     441 | WARNING | [ ] Unused private method processTrim()
     476 | WARNING | [ ] Unused private method processEntityLookup()
    -------------------------------------------------------------------------------
    PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
    -------------------------------------------------------------------------------

    3. FILE: assets/js/form_trucie_importer.js

    })(Drupal, drupalSettings);

    FILE: trucie.libraries.yml

      dependencies:
        - core/once

    Specify correct dependencies to avoid the unnecessary loading of files that are not needed and ensure all necessary dependencies are included for scripts to function properly.

    4. FILE: src/Event/EntityEvent.php

      /**
       * Constructs the class.
       *
       * @param string $importerId
       *   The related trucie importer id.
       * @param array $data
       *   The row data (from the source file).
       * @param \Drupal\Core\Entity\EntityInterface|null $entity
       *   The entity (generated from row data).
       */
      public function __construct(string $importerId, array &$data, ?EntityInterface $entity) {

    FILE: src/Event/RowEvent.php

      /**
       * Constructs the class.
       *
       * @param string $importerId
       *   The related trucie importer id.
       * @param array $data
       *   The row data (from the source file).
       */
      public function __construct(string $importerId, array &$data) {

    FILE: src/Form/TrucieImporterForm.php

      /**
       * Constructs a class.
       *
       * @param \Drupal\Core\Entity\EntityTypeManagerInterface $entityTypeManager
       *   The entity type manager.
       * @param \Drupal\Core\Entity\EntityTypeBundleInfoInterface $entityTypeBundleInfo
       *   The entity type bundle info.
       * @param \Drupal\trucie\Importer\BatchImporterFactoryInterface $importerFactory
       *   The batch importer factory.
       */
      public function __construct(

    FILE: src/Importer/BatchImporterBase.php

      /**
       * The class constructor.
       *
       * @param string $importerProcessorName
       *   The field values processor name.
       */
      public function __construct(string $importerProcessorName = '') {

    FILE: src/Importer/BatchImporterFactory.php

      /**
       * Constructs a class.
       *
       * @param \Symfony\Component\DependencyInjection\ContainerInterface $container
       *   The service container.
       */
      public function __construct(private ContainerInterface $container) {

    FILE: src/Processor/TrucieImporterProcessor.php

      /**
       * Constructs the class.
       *
       * @param \Drupal\Core\Entity\EntityTypeManagerInterface $entityTypeManager
       *   The entity type manager.
       */
      public function __construct(EntityTypeManagerInterface $entityTypeManager) {

    The documentation comment for constructors is not mandatory anymore, If it is given, the description must be Constructs a new [class name] object. where [class name] includes the class namespace.

    5. FILE: src/Form/TrucieImporterForm.php

      public function __construct(
        EntityTypeManagerInterface $entityTypeManager,
        EntityTypeBundleInfoInterface $entityTypeBundleInfo,
        BatchImporterFactoryInterface $importerFactory,
      ) {

    Drupal coding standards say function declarations are written on a single line, even if they exceed 80 characters.

  • 🇷🇺Russia i-grou

    Thanks @vishal.kadam for your review.

    1. feature/TRC-3 and feature/TRC-2

    1. feature/TRC-3 and feature/TRC-2 are wrong names for a branch and should be removed. Release branch names always end with the literal .x as described in Release branches.

    These branches are used to meet development rules used in our company. They have nothing to do with release, eventually they will be merged to 1.0.x branch and then removed. Here in the description I've mentioned that 1.0.x - is the branch we need to check. Or am I missing something?

    2. Fix phpcs issues

    1 | ERROR | [x] The PHP open tag must be followed by exactly one blank line

    Since we have a php 8 using the declare(strict_types=1) becomes a standard. I undestand that old standard was like this:

    <?php
    
    declare(strict_types=1);
    

    Maybe in the past the declare() directives were rarely used and it was totally fine to have a blank line after php tag.
    But things are changed, nowadays in all sorts of contrib packages I can see the shorten form:

    <?php declare(strict_types=1);
    

    As we have to add a declare(strict_types=1) line to every php file this shorten form makes sence and tend to become a new standard.
    As this does not affect any performance and is not a threat from security perspective I'd suggest to keep as is.

    FILE: trucie/src/Processor/TrucieImporterProcessor.php
    -------------------------------------------------------------------------------
    FOUND 1 ERROR AND 12 WARNINGS AFFECTING 13 LINES
    -------------------------------------------------------------------------------
     174 | WARNING | [ ] Unused private method processExplode()
     189 | WARNING | [ ] Unused private method processImplode()
     204 | WARNING | [ ] Unused private method processFloatval()
     238 | WARNING | [ ] Unused private method processIntval()
     267 | WARNING | [ ] Unused private method processMbStrtolower()
     296 | WARNING | [ ] Unused private method processMbStrtoupper()
     325 | WARNING | [ ] Unused private method processUcfirst()
     354 | WARNING | [ ] Unused private method processPregReplace()
     383 | WARNING | [ ] Unused private method processStrtotime()
     412 | WARNING | [ ] Unused private method processDate()
     441 | WARNING | [ ] Unused private method processTrim()
     476 | WARNING | [ ] Unused private method processEntityLookup()

    Actually these methods are used (see process()):

          ...
          $method = ($processor['type'] ?? '') ? 'process' . $this->makeMethodName($processor['type']) : '';
          if ($method && method_exists($this, $method)) {
            $value = $this->{$method}($value, $processor);
          }
    

    Maybe I could change private to protected or public, but this does not make sense as all these methods should not be available outside.
    So we can ignore these warnings.

    3. FILE: assets/js/form_trucie_importer.js

    })(Drupal, drupalSettings);

    Not sure I understand what exactly is wrong here. This is a typical way to do things in js, even some core js files has this.
    Could you give more explanation on that?

    FILE: trucie.libraries.yml

      dependencies:
        - core/once
    Specify correct dependencies to avoid the unnecessary loading of files that are not needed and ensure all necessary dependencies are included for scripts to function properly.

    Not sure I understand what exactly is wrong here. The once() js function coming from core is heavily used in this module, so I need this dependecny.
    Could you give more explanation on that?

    All other issues are fixed, check please.

  • 🇮🇳India vishal.kadam Mumbai

    FILE: assets/js/form_trucie_importer.js

    })(Drupal, drupalSettings);

    FILE: trucie.libraries.yml

      dependencies:
        - core/once

    Drupal and drupalSettings are used in the library, but they are not declared as dependencies in libraries.yml

    Rest looks fine to me.

  • 🇷🇺Russia i-grou

    Added new dependencies to trucie.libraries.yml, check now please.
    By the way is it enough to add just core/drupal as it already has core/drupalSettings as dependency?

  • 🇮🇳India vishal.kadam Mumbai

    Yes, you can add just core/drupal as it already has core/drupalSettings as dependency.

  • 🇷🇺Russia i-grou

    Done, check please.

  • 🇮🇳India vishal.kadam Mumbai

    Remember to change status, when the project is ready to be reviewed. In this queue, projects are only reviewed when the status is Needs review.

  • Status changed to Needs review about 1 year ago
  • 🇮🇳India vishal.kadam Mumbai

    Let’s wait for a Code Review Administrator to take a look and if everything goes fine, you will get the role.

  • Status changed to RTBC about 1 year ago
  • 🇮🇹Italy apaderno Brescia, 🇮🇹

    Thank you for your contribution!
    I updated your account so you can now opt into security advisory coverage for any project you created and every project you will create.

    These are some recommended readings to help you with maintainership:

    You can find more contributors chatting on Slack → or IRC → in #drupal-contribute. So, come hang out and stay involved → !

    Thank you for your patience with the review process. Anyone is welcome to participate in the review process. Please consider reviewing other projects that are pending review → . I encourage you to learn more about that process and join the group of reviewers.

    I thank also the dedicated reviewers as well.

  • Assigned to apaderno
  • Status changed to Fixed 12 months ago
  • 🇮🇹Italy apaderno Brescia, 🇮🇹
  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024