Replace array-based Schema API with a value object structure

Created on 31 December 2023, 8 months ago
Updated 8 May 2024, 4 months ago

Problem/Motivation

Spin off from ✨ Use events in Database Schema operations Active .

Today,

A Drupal schema definition is an array structure representing one or more tables and their related keys and indexes. A schema is defined by hook_schema(), which usually lives in a modulename.install file.

https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Core%21Database%...

Proposed resolution

Replace the current way of defining a table's structure via a nested array, with a structure of value objects defined in the Drupal\Core\Database\SchemaDefinition namespace.

This structure is immutable and database abstract (for example, directionally in the future the db-specific field type array key 'mysql_type', or the 'mysql_engine' will no longer be defined here).

A table currently represented like

    $schema = [
      'description' => 'The base table for configuration data.',
      'fields' => [
        'collection' => [
          'description' => 'Primary Key: Config object collection.',
          'type' => 'varchar_ascii',
          'length' => 255,
          'not null' => TRUE,
          'default' => '',
        ],
        'name' => [
          'description' => 'Primary Key: Config object name.',
          'type' => 'varchar_ascii',
          'length' => 255,
          'not null' => TRUE,
          'default' => '',
        ],
        'data' => [
          'description' => 'A serialized configuration object data.',
          'type' => 'blob',
          'not null' => FALSE,
          'size' => 'big',
        ],
      ],
      'primary key' => ['collection', 'name'],
    ];

Becomes, note the usage of named parameters (that in the future could allow some more flexibility when it comes to adding/changing/deprecating/removing function arguments):

    new Table(
      name: 'config',
      description: 'The base table for configuration data.',
      columns: [
        new Column(
          name: 'collection',
          description: 'Primary Key: Config object collection.',
          type: ColumnType::VarcharAscii,
          length: 255,
          notNull: TRUE,
          default: '',
        ),
        new Column(
          name: 'name',
          description: 'Primary Key: Config object name.',
          type: ColumnType::VarcharAscii,
          length: 255,
          notNull: TRUE,
          default: '',
        ),
        new Column(
          name: 'data',
          description: 'A serialized configuration object data.',
          type: ColumnType::Blob,
          notNull: FALSE,
          size: ColumnSize::Big,
        ),
      ],
      primaryKey: new PrimaryKey(['collection', 'name']),
    );

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

✨ Feature request
Status

Needs work

Version

11.0 🔥

Component
Database  →

Last updated less than a minute ago

  • Maintained by
  • 🇳🇱Netherlands @daffie
Created by

🇮🇹Italy mondrake 🇮🇹

Live updates comments and jobs are added and updated live.
  • Needs subsystem maintainer review

    It is used to alert the maintainer(s) of a particular core subsystem that an issue significantly impacts their subsystem, and their signoff is needed (see the governance policy draft for more information). Also, if you use this tag, make sure the issue component is set to the correct subsystem. If an issue significantly impacts more than one subsystem, use needs framework manager review instead.

  • Needs frontend framework manager review

    Used to alert the fron-tend framework manager core committer(s) that a front-end focused issue significantly impacts (or has the potential to impact) multiple subsystems or represents a significant change or addition in architecture or public APIs, and their signoff is needed (see the governance policy for more information). If an issue significantly impacts only one subsystem, use Needs subsystem maintainer review instead, and make sure the issue component is set to the correct subsystem.

Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @mondrake
  • Assigned to mondrake
  • 🇮🇹Italy mondrake 🇮🇹
  • Pipeline finished with Failed
    8 months ago
    Total: 162s
    #70495
  • Pipeline finished with Failed
    8 months ago
    Total: 511s
    #70496
  • Pipeline finished with Failed
    8 months ago
    #70511
  • Pipeline finished with Failed
    8 months ago
    Total: 519s
    #70524
  • Pipeline finished with Failed
    8 months ago
    #70543
  • Pipeline finished with Failed
    8 months ago
    Total: 180s
    #70547
  • Pipeline finished with Failed
    8 months ago
    Total: 672s
    #70556
  • Pipeline finished with Success
    8 months ago
    Total: 552s
    #70583
  • Pipeline finished with Failed
    8 months ago
    Total: 314s
    #70624
  • Pipeline finished with Failed
    8 months ago
    #70627
  • Pipeline finished with Success
    8 months ago
    #70630
  • Status changed to Needs review 8 months ago
  • 🇮🇹Italy mondrake 🇮🇹

    Here's a MR that tries to do the minimum possible:

    • add all the relevant value objects to the Drupal\Core\Database\SchemaDefinition namespace
    • adds a converter from the new API to the old API + a basic test, and makes adjustments so that we can use the new API in hook_schema() or other functions without impacting the database driver code (that will be huge, but follow ups). In practice the new API is converted to the old one on the fly.
    • added a couple of example conversions to demonstrate the new API, for system_schema() and Drupal\Core\Batch\BatchStorage::schemaDefinition(). I would leave conversions to followups, too, because it's a lot of work, especially for cases where the old array structure is built dynamically.

    Looking forward to reviews!

  • Pipeline finished with Success
    8 months ago
    Total: 1076s
    #70762
  • Status changed to Needs work 8 months ago
  • 🇮🇹Italy mondrake 🇮🇹

    Let’s change the trait to an abstract class.

  • Pipeline finished with Success
    8 months ago
    Total: 683s
    #71004
  • Pipeline finished with Success
    8 months ago
    Total: 1234s
    #71064
  • Pipeline finished with Failed
    8 months ago
    Total: 184s
    #71095
  • Issue was unassigned.
  • Status changed to Needs review 8 months ago
  • 🇮🇹Italy mondrake 🇮🇹

    For review. Invocation of hook_schema now return by default the array version of the schema definition, for BC. Code needs to explicitly opt-in to get Table objects instead or array shapes. In the future this will change but this allows to manage transition to the new approach.

  • Pipeline finished with Success
    8 months ago
    Total: 1029s
    #71100
  • Status changed to Needs work 8 months ago
  • 🇮🇹Italy mondrake 🇮🇹

    Adding an enum for column size.

  • 🇮🇹Italy mondrake 🇮🇹
  • Pipeline finished with Canceled
    8 months ago
    Total: 459s
    #71139
  • Status changed to Needs review 8 months ago
  • 🇮🇹Italy mondrake 🇮🇹
  • Pipeline finished with Success
    8 months ago
    Total: 8122s
    #71144
  • Pipeline finished with Success
    8 months ago
    Total: 939s
    #71408
  • 🇮🇹Italy mondrake 🇮🇹
  • Status changed to Needs work 6 months ago
  • The Needs Review Queue Bot → tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

    This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

    Consult the Drupal Contributor Guide → to find step-by-step guides for working with issues.

  • Pipeline finished with Failed
    6 months ago
    #110783
  • 🇺🇸United States bradjones1 Digital Nomad Life

    Worth noting we're doing something similar but not quite the same over at 📌 Adding GIN and GIST indexes to PostgreSQL databases RTBC .

    Per some feedback in Drupal Slack today, it sounds like the ideal state would be to use events a la ✨ Use events in Database Schema operations Active but we're postponed enough as it is on related issues. At the very least, this demonstrates how we can use objects in some circumstances inside of "ArrayPI" without breaking BC and enabling innovation.

Production build 0.71.5 2024