SQLSTATE[42S02]: Base table or view not found error when used with CiviCRM entities and separate CiviCRM database

Created on 9 December 2024, 17 days ago

Problem/Motivation

I had a hunch this was related to my CiviCRM/Drupal websites having separate databases which I would like to keep separate due to the immense size of CiviCRM databases (Many GB in size).

My site has two databases; one for Drupal and another for CiviCRM. They are related via the civicrm_entity module which usually exposes all of the CiviCRM goodness to Drupal views and other Drupal mechanisms.

After several hours of R&D, I am certain that the Views Combine is stumbling when it attempts to render the CiviCRM data in a Drupal content view or trying to render Drupal content within a CiviCRM view.

My scenario is similar to that of @petednz who posted the related issue noted in this issue submission where in my use case, I have two Event Entities that I would like to list out: A Drupal content type called Events and the CiviCRM Events themselves however, I am receiving SQLSTATE errors when I attempt to pull the node content into a Drupal CiviCRM view or when I attempt to pull CiviCRM entities into a Drupal Content view.

I am confident that my setup of the Views Combine module is correct because I can combine different Drupal Content types in the same view without any trouble. It is only when I attempt to union Drupal entities with CiviCRM entities that the trouble occurs.

The site is the most recent Drupal 10 and the most recent CiviCRM.
Drupal Database: database_drupal
CiviCRM Database: database_civicrm

The Error is as follows (I have pasted the most useful info):

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'database_drupal.civicrm_event' doesn't exist: SELECT node_field_data.nid AS "nid"...

The error statement is correct, there is no civicrm_event table within the database named database_drupal because the civicrm_event table is within the database named database_civicrm.

The Views Combine module appears to be unaware of the source of the data it needs to query for CiviCRM entities.

In terms of the cause, it seems that when the Views Combine field is bootstrapped for rendering, it references all of the fields from the database of the parent view. This means that in a CiviCRM view, it tries to pull Drupal node field data from the CiviCRM database and when it is used within a Drupal content view, it tries to pull CiviCRM entity data from the Drupal database. This would be fine if the site had one massive database for everything rather than the distributed databases in my sites.

In terms of a solution, I'm looking at class ViewsCombiner and in particular the queries but I haven't come up with a solution yet.

🐛 Bug report
Status

Active

Version

1.0

Component

Code

Created by

🇨🇦Canada awasson

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @awasson
  • 🇨🇦Canada b_sharpe

    My guess is you're going to have problems with all sorts of queries if you're trying to join/union between separate DB's.

    Drupal technically can work this way, but only if the DB's are on the same server. Most managed hosting these days the DB server is different between DB's which means join/union is not natively supported without FEDERATED storage.

    Open to MR's here to solve, but just thought I'd point that out

  • 🇨🇦Canada awasson

    Hi Bryan,

    Yes, both databases are on the same server which is pretty standard for the Drupal/CiviCRM sites that I maintain. CiviCRM entities are made available to Drupal via a CiviCRM Entities module.

    I've been developing in CiviCRM since Drupal 6 and prior to Drupal 9/10 we used to add a hard coded DB array to the settings.php file to tell Views about the CiviCRM database. It was the form of:

    $databases['default']['default']['prefix']= [
    'default' => '',
    'civicrm_acl' => '`database_civicrm`.',
    'civicrm_acl_cache' => '`database_civicrm`.',
    'civicrm_acl_contact_cache' => '`database_civicrm`.',
    'civicrm_acl_entity_role' => '`database_civicrm`.',
    . . . More definitions . . .,
    ];

    I've opened up a conversation of the mattermost channel that handles the CiviCRM Entities module to see if there is a database api method that recognizes the database that is responsible for the appropriate entities.

    The actual error I get is as follows:

    SQLSTATE[42S02]: Base table or view not found: 1146 Table 'database_civicrm.civicrm_event' doesn't exist: SELECT node_field_data.nid AS "nid", NULL AS "civicrm_event_start_date", NULL AS "id", :view_combined_event_listing_block_1_combined_event_listing_block_1 AS "_view_id", NULL AS "_order_0" FROM "node_field_data" "node_field_data" WHERE ("node_field_data"."status" = :db_condition_placeholder_0) AND ("node_field_data"."type" IN (:db_condition_placeholder_1)) UNION SELECT node_field_data.nid AS "nid", NULL AS "civicrm_event_start_date", NULL AS "id", :view_drupal_event_listing_default_drupal_event_listing_default AS "_view_id", NULL AS "_order_0" FROM "node_field_data" "node_field_data" WHERE ("node_field_data"."status" = :db_condition_placeholder_2) AND ("node_field_data"."type" IN (:db_condition_placeholder_3)) UNION SELECT NULL AS "nid", civicrm_event.start_date AS "civicrm_event_start_date", civicrm_event.id AS "id", :view_events_listing_default_events_listing_default AS "_view_id", civicrm_event.start_date AS "_order_0" FROM "civicrm_event" "civicrm_event" WHERE (("civicrm_event"."is_active" = :db_condition_placeholder_4) AND ("civicrm_event"."is_public" = :db_condition_placeholder_5)) AND (((DATE_FORMAT((civicrm_event.start_date + INTERVAL -25200 SECOND), '%Y-%m-%d\T%H:%i:%s') >= DATE_FORMAT(('2024-12-10T07:00:00' + INTERVAL -25200 SECOND), '%Y-%m-%d\T%H:%i:%s'))) OR ((DATE_FORMAT((civicrm_event.end_date + INTERVAL -25200 SECOND), '%Y-%m-%d\T%H:%i:%s') >= DATE_FORMAT(('2024-12-10T07:00:00' + INTERVAL -25200 SECOND), '%Y-%m-%d\T%H:%i:%s')))) ORDER BY "_order_0" ASC LIMIT 11 OFFSET 0; Array ( [:db_condition_placeholder_0] => 1 [:db_condition_placeholder_1] => drupal_event [:view_combined_event_listing_block_1_combined_event_listing_block_1] => combined_event_listing:block_1 [:db_condition_placeholder_2] => 1 [:db_condition_placeholder_3] => drupal_event [:view_drupal_event_listing_default_drupal_event_listing_default] => drupal_event_listing:default [:db_condition_placeholder_4] => 1 [:db_condition_placeholder_5] => 1 [:view_events_listing_default_events_listing_default] => events_listing:default )

    If I can't find a solution that uses the database API, I'll likely just write a module and run the queries in code. Hopefully I can though because this would make a lot of Drupal/CiviCRM listings better; not just events but newsletters and other things that would benefit from being listed together.

    Cheers,
    Andrew

  • 🇨🇦Canada awasson

    Hi Bryan,
    The CiviCRM Entity module devs have indicated that it's unlikely I'll be able to resolve this issue without custom coding my own Views field plugins so I'm going to close this as works as designed and carry on.

    It's a very cool module though so I'll be using in again when I can. I may poke around and see if I can add a custom views field that works in tandem with it where I can override the database connection to assign the one for CiviCRM. I think that would work but it would probably be easier to just write a purpose built plugin for this one specific job. I've got Views field plugin boiler plate code I can borrow from a recent project that did something similar with a remote data warehouse.

    Thanks again for the help.

    Cheers,
    Andrew

Production build 0.71.5 2024