Add schema validation tests

Created on 3 December 2024, 8 months ago

Problem/Motivation

Currently we are doing a lot of automated testing, but one thing that is missing is to see if created configurations actually are following schemas.

Proposed resolution

Add schema validation tests to the pipeline

Remaining tasks

User interface changes

API changes

Data model changes

📌 Task
Status

Active

Version

1.0

Component

AI Core module

Created by

🇩🇪Germany marcus_johansson

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @marcus_johansson
  • 🇨🇭Switzerland bircher 🇨🇿
  • Assigned to bircher
  • 🇩🇪Germany marcus_johansson

    Since I had to fix some schemas when doing tests here, I think this becomes outdated if that is approved and merged: 📌 Create basic test so all modules can be installed Active

  • Issue was unassigned.
  • 🇨🇦Canada jibran Toronto, Canada

    I'll give it a crack. It will help me get up to speed with the codebase.

    Figure out how other projects does schema validation testing

    Normally, you create the config object programatically and validate it against the installed schema. Core has some helpful base classes for that. We can also have a base class for all submodules.

  • 🇨🇦Canada jibran Toronto, Canada

    Created a PR for just ai.settings for now. If we agree with the direction, then I'll create a META to track for all modules and convert this issue to focus on the base module.

    Remaining tasks:

    • Add tests for the remaining base module config.
  • Pipeline finished with Success
    1 day ago
    Total: 218s
    #559237
  • 🇩🇪Germany marcus_johansson

    Thank you @jibran - I think that is very good start and will cover the most important parts.

    Schema coverage, would be the next step, but that is less important for now.

  • Pipeline finished with Success
    about 21 hours ago
    #560047
  • 🇨🇦Canada jibran Toronto, Canada

    Pushed some minor CS fixes. If we are not going to fix another schema, then this MR is ready for review.

    @marcus_johansson, Sure, this makes sense. Do you think this MR is ready for now, or would you like to wait to merge it?

  • 🇩🇪Germany marcus_johansson

    Thanks Jibran - this is a good template for any test of a specific schema, however I think this will fail, since we have the pseudo types and since it doesn't actually save a value if its empty. I wrote another test you can try to implement to see if its failing. If its failing we would have to rework the submit function to also save empty values.

    My main issue that I was looking for was to find out when a schema is missing as well, since that is our biggest issue right now - but I can create a follow up issue for this. This is a good template for any test of a specific schema.

Production build 0.71.5 2024