Create automated script for converting components to SDC

This work is in progress here:

https://github.com/qed42/twig-sdc-yaml-generator/blob/main/README.md

Akhil Babu → made their first commit to this issue’s fork.

The script checks the comment in twig files to determine the props and sots. These changes were required for the script to work.
components/03-organisms/webform/webform.twig

• The referenced_webform variable randers string value. But, it's type was documented as 'webform'. Changed this to string.

components/03-organisms/navigation/navigation.twig

• Corrected the spelling of 'boolean'.

components/02-molecules/video-player/video-player.twig

• Documented the properties of transcript_link

components/02-molecules/logo/logo.twig

• Adjusted the indentation.

I also tried to render 2 of the existing components and these are the results.

Snippet

• Updated paragraph--civictheme-snippet.html.twig to render the components using SDC. {% embed 'civictheme:snippet' %}{% endembed %}
• Created a new 'Page' content and added a 'Snippet' paragraph.
• The component gets rendered correctly

Thanks again for working thorough this. I haven’t had a chance to test yet. If there are questions, please ping in the slack channel

Progress Update

1. Handling of array type as per schema (not pushed to repo)
2. ntegrated include statements into the automated property generation process. (not pushed to repo)

{#
/**
 * @file
 * Banner component.
 *
 * Variables:
 * - content_top1: [string] Content slot.
 * - breadcrumb: [array] Breadcrumb links.
 * - content_top2: [string] Content slot.
 * - content_top3: [string] Content slot.
 * - content_middle: [string] Content slot.
 * - content: [string] Content slot.
 * - content_bottom: [string] Content slot.
 * - content_below: [string] Content slot.
 * - site_section: [string] Site section.
 * - title: [string] Title.
  *   ...
 */
#}

Example:

Consider the following Twig file :

{#
/**
* @file
* Banner component.
*
* Variables:
* - content_top1: [string] Content slot.
* - breadcrumb: [array] Breadcrumb links.
* - content_top2: [string] Content slot.
* - content_top3: [string] Content slot.
* - content_middle: [string] Content slot.
* - content: [string] Content slot.
* - content_bottom: [string] Content slot.
* - content_below: [string] Content slot.
* - site_section: [string] Site section.
* - title: [string] Title.
*/
#}
In this example, breadcrumb is a component rendered through an include statement in banner component:

{% if breadcrumb is not empty %}
  <div class="{% if featured_image is not empty %}col-xxs-12 col-m-6{% else %}col-xxs-12{% endif %}">
    {% include '@molecules/breadcrumb/breadcrumb.twig' with {
      theme: theme,
      links: breadcrumb.links,
      active_is_link: breadcrumb.active_is_link,
      modifier_class: 'ct-banner__breadcrumb',
    } only %}
  </div>
{% endif %}

In the comment section, breadcrumb is noted as an array. However, in the .component.yml file, we need to specify links as an array with properties under items.

The script can generate the following YAML, ensuring only the properties of breadcrumb used in the include statement are included (e.g., theme and other breadcrumb properties are excluded):

 breadcrumb:
      type: array
      title: Breadcrumb
      description: Breadcrumb links.
      items:
        links:
          type: array
          title: Links
          description: 'Array of link objects containing:'
          items:
            text:
              type: string
              title: Text
              description: Link text.
            url:
              type: string
              title: Url
              description: Link URL.
        active_is_link:
          type: boolean
          title: Active is link
          description: Show active element as a link.

The script now supports handling nested includes, such as an event card that includes a tag list, which in turn includes tag components.

Next steps

1. Test this functionality and make any necessary adjustments to match the schema accurately.
2. Update the repository.

I tested the converter in this issue:
https://www.drupal.org/project/demo_design_system/issues/3465318 📌 SDDS SDC Conversion Examples: Chip component [MANUAL] Active

My main thoughts so far is that I'd like the yml files to be more readable. Putting empty lines between the variables would be helpful.

Also I think we should always put double quotes around descriptions and titles:

here's a bit from the generated yml:

    size:
      type: string
      title: Size
      description: 'Chip size: large, regular, small.'
      default: regular
...
    is_selected:
      type: boolean
      title: Is selected
      description: Is the button selected (for filter chips).

Also in the conversion guide: https://www.drupal.org/community-initiatives/starshot-demo-design-system... →
It seems like we might want to only have titles for booleans like this:

    is_selected:
      type: boolean
      title: "Is the button selected (for filter chips)."

This is because for these checkboxes the title is the bit that's going to be displayed to the user and this makes more sense for that.

Also I tested ChatGPT to see if it could potentially be used along with a Python script to generate yml and it seems like it could.

I tested out the script in these issues:

https://www.drupal.org/project/demo_design_system/issues/3465333 📌 SDDS SDC Conversion Examples: Map component [MANUAL] Postponed
https://www.drupal.org/project/demo_design_system/issues/3465336 📌 SDDS SDC Conversion Examples: Banner component [MANUAL] Active

So far so good :)

Fyi, @jacobadeutsch has some good feedback here:

https://www.drupal.org/project/demo_design_system/issues/3466118#comment... 📌 Convert Starshot Tag component to SDC Needs review

It would be great if "status" and "group" defaults like the following so there are placeholders to edit:

# The human readable name.
name: Button

# Status can be: "experimental", "stable", "deprecated", "obsolete".
status: experimental

# Use this key to organize components together.
group: Default

Would also be cool if it created a simple README.md with something like the following. It could see which directory it's in to determine if it's an atom, molecule or organism.

Starshot Demo Design System Button

## Usage
This component is an atom which can be used on its own or within other other components.

Moving to needs work based on feedback above.

Updated script as per the feedback provided in comment 14 📌 Create automated script for converting components to SDC Fixed and 15 📌 Create automated script for converting components to SDC Fixed
https://github.com/qed42/twig-sdc-yaml-generator/tree/main

This update also supports automated prop generation for included components.

e.g tag component included in event card and they are available as prop for event card so now script will be able to generate below yml for it

   tags:
      type: array
      title: Tags
      description: Array of tags.
      items:
        type: string

Also for banner component breadcrumb is included and it's available as prop so yml will consist below

breadcrumb:
      type: array
      title: Breadcrumb
      description: Breadcrumb links.
      items:
        type: object
        properties:
          links:
            type: array
            title: Links
            description: 'Array of link objects containing:'
            items:
              type: object
              properties:
                text:
                  type: string
                  title: Text
                  description: Link text.
                url:
                  type: string
                  title: Url
                  description: Link URL.
          active_is_link:
            type: boolean
            title: Active is link
            description: Show active element as a link.

Next steps will check and work on feedback mentioned in comment 13 📌 Create automated script for converting components to SDC Fixed

Issue link where feedback was posted: https://www.drupal.org/project/demo_design_system/issues/3466118#comment... 📌 Convert Starshot Tag component to SDC Needs review
component name: Starshot Tag

@Kristen Pol @jacobadeutsch thanks for the feedback, please find details below

1. Booleans need to be changed to have their title as the description for the boolean. - Fixed
2. Should the defaults be displayed in the yml?
   I found example of default in D.O documentation so I have added it https://www.drupal.org/docs/develop/theming-drupal/using-single-director... → Annotated example component.yml →
3. The converter didn't create a enums where it should've
   can you please provide detail around this for which prop enums were not generated properly?

   For existing components we have list of enums hardcoded in script. there is no pattern available to retrieve the options/enums for prop from the comment section. if we are creating new components can we have them in some pattern so that script can retrieve and add them as a part of yml?

4. Not everything with a space in it has quotes around it, not sure if this is necessary or not
   Ran latest script this seems to be resolved, also we using package to generate yaml will check if there are any configs available for this.
5. it included a blank default for a boolean.. it seems like that it's getting that from line 27 of the twig. I deleted it because I wasn't seeing defaults for other bools, but this could be wrong
   Ran latest script did not see default added for boolean for given component

Attaching generated .component.yml file with new script for reference

Thanks! One thing is adding empty lines to make the file more readable.

I'll have Jacob review otherwise.

I've attached an instance of the enum not being generated from the OLD twig-sdc-yaml-generator below. The new script generated the enums fine.

    type:
      type: string
      title: Type
      description: 'Type: primary, secondary, tertiary'
      default: primary

If instead of generating based on a given list of hardcoded enums you wanted to create a text parser to generate them I think there is a standard procedure for writing the twigs that could make this possible:

 * - theme: [string] Theme: light, dark.
 * - type: [string] Type: primary, secondary, tertiary
 * - icon_placement: [string] Icon position: before, after.

* Not sure if a . at the end is standard, or if this twig has a typo and that's what caused the problem

I think this is standard? * - var_name: [string] Name: enum1, enum2, enum3, enum4, etc...

Maybe the second colon in the line could be used to detect a list of enums? I guess I might be missing other cases.. I don't remember if objects or arrays sometimes include a colon in their descriptions, and I guess if they did then a way around that could be checking the datatype first. Assuming all enum lists are of type string.

Other than that though I think the only notes I have are that spacing between the vars would be nice. Something like this:

name: tag

status: experimental

group: Atoms

props:

  type: object

  required:
  - content
  - url

  properties:

    theme:
      type: string
      title: Theme
      description: 'Theme: light, dark.'
      default: light
      enum:
      - light
      - dark

    type:
      type: string
      title: Type
      description: 'Type: primary, secondary, tertiary'
      default: primary
      enum:
      - primary
      - secondary
      - tertiary

    content:
      type: string
      title: Content
      description: Content.

Also group names are showing up as "default" right now. And the blank string bool defaults are still appearing but I assume you're on that.

Great work!!

Thanks for the feedback @jacobadeutsch! Looks very close!

Back to Anand and team :)

Progress update

1. Parse enums form the comments - Fixed
2. Also group names are showing up as "default" right now. - Fixed
3. Blank string bool defaults are still appearing - Fixed

Pending
spacing between the vars would be nice. - 60% of work here is done in case on nested properties formatting is not appearing properly will try to get this done by tomorrow.

Thanks 🙏

I've addressed the spacing between the variables. If there are any other formatting adjustments needed, please let me know.
Attached file for reference

This looks so much better! Yay :)

Let's call this done and future improvements can be in their own issues.

Kristen Pol → credited guptahemant → .

Adding credit based on project management work.

@anand.toshniwal93 If I've missed credits for people who worked on this on your team, send me their d.o names please.

The script is seriously awesome now, great job!

I'm working on creating the baseline ymls using it rn, and I did encounter a small problem:

    description_display:
      type: string
      title: Description display
      description: 'Description display: before, after, invisible. Default is before.
        Optional.'

      default: before
      enum:
      - before
      - after
      - invisible. Default is before. Optional



    message_type:
      type: string
      title: Message type
      description: 'Type of the message: error, warning, info, success. Default is
        error.'

      default: error
      enum:
      - error
      - warning
      - info
      - success. Default is error

I guess this is one thing I missed about the formatting of the twigs, there can be a default specified after the list. I think there should be at least a . at the end in these instances, so maybe first it can check for a . and then a \n just incase there isn't a . at the end? I'm not great with regex but I hope the solution is pretty simple. Not sure what happened with the indents either but idk if that will be a problem with the syntax.

Thanks @jacobadeutsch. Let's start a new issue with some follow up work for the script for next week.

Follow up feedback can be added here:

📌 Gather additional feedback when testing the SDC conversion script Active

Automatically closed - issue fixed for 2 weeks with no activity.