Marketo MA User Lead Sync Fails due to syncLead method posting to wrong endpoint

Created on 25 August 2022, about 2 years ago
Updated 23 January 2024, 10 months ago

Problem/Motivation

The Marketo MA User submodule in Marketo MA is unable to perform any lead syncs because the Marketo REST API endpoint that it's posting requests to is the wrong endpoint for a basic lead sync. Lead syncs can be triggered by user login, update, or creation.

When the REST API call is made it's posted to /rest/v1/leads/push.json, but requests made to this endpoint require a programName variable that will always be missing from an ordinary lead sync performed when a user is updated. This appears to be an undocumented change in the Marketo REST API as the documentation recommends using the push.json endpoint and lists the programName variable as optional.

Steps to reproduce

  1. Enable Marketo MA User
  2. Map User fields
  3. Choose to update leads on User Updates
  4. Edit any field in your user profile record that's mapped to Marketo

Nothing will happen in Marketo. The POST request to the API will fail with error code 1003. Further inspection will yield a message that the programName is required.

Proposed resolution

In MarketoMaApiClient the syncLead method should be updated to perform a SyncLeadRequest action instead of PushLeadToMarketoRequest. Doing this will allow us to bypass the required programName variable that is causing the request to be rejected by submitting to the /rest/v1/leads.json endpoint instead.

Optionally, a new pushLead method should also be created to post to the /rest/v1/post.json endpoint.

Remaining tasks

Someone more central to this project needs to take a good look at my patch and make sure it's not too divergent. Also, in a related area, conditional logic for when a Marketo REST API call should post to the leads.json endpoint or the push.json endpoint needs to be written in the Marketo MA Service class.

I'll make a merge request if a maintainer thinks that's a good idea.

πŸ› Bug report
Status

RTBC

Version

3.0

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States hyperlinked San Jose, CA

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Open on Drupal.org β†’
    Core: 10.1.4 + Environment: PHP 7.4 & MySQL 8
    last update 11 months ago
    Waiting for branch to pass
  • Pipeline finished with Failed
    11 months ago
    #68579
  • Issue was unassigned.
  • πŸ‡ΊπŸ‡ΈUnited States hyperlinked San Jose, CA

    I closed my merge request after realizing that more substantial adjustments would be needed, but these changes could have much farther reaching implications for this module that needs the guidance of someone closer to the source. For now, I'm going to post a patch and explain what I think needs to be done. This patch will work against 3.0 and the 3.0-dev releases.

    In Drupal\marketo_ma\Service\MarketoMaServiceApiClient the syncLead method actually corresponds to the push.json endpoint of the Marketo REST API. There is no method corresponding to the leads.json endpoint which is what syncLead should be aligned with. If it doesn't create havoc, syncLead needs to be renamed to pushLead and a new syncLead method should be created.

    Note: I will be referring to the REST API endpoints and not any Marketo MA method names to avoid any confusion.

    Why is this important? Originally the programName variable was optional for push.json. This made leads.json and push.json interchangeable for single lead actions. At some point, Marketo began throwing errors if you used push.json without a programName, but there doesn't appear to be any part of Marketo MA that's actually setting a programName properly.

    The only part of the code where programName is used is in the Webform handler for MarketoMA. You can enter the programName, but it's never used because the Webform Handler bypasses the updateLead methods that would post lead info to leads.json and push.json. Furthermore, the Marketo MA webform handler will collect programName, but if you use programName, you can supply a form ID. Without a form ID, the submission will fail from the Drupal side.

    In my attached patch, I've made it so that we have:
    syncLead -> leads.json
    pushLead -> push.json

    Test coverage and interface updates are included.

  • Open in Jenkins β†’ Open on Drupal.org β†’
    Core: 10.1.4 + Environment: PHP 8.1 & MySQL 5.7
    last update 11 months ago
    Patch Failed to Apply
  • πŸ‡ΊπŸ‡ΈUnited States hyperlinked San Jose, CA
  • πŸ‡ΊπŸ‡ΈUnited States hyperlinked San Jose, CA

    Updated patch with corrected patch indexing level.

  • πŸ‡ΊπŸ‡ΈUnited States hyperlinked San Jose, CA
  • Open in Jenkins β†’ Open on Drupal.org β†’
    Core: 10.1.4 + Environment: PHP 8.1 & MySQL 5.7
    last update 11 months ago
    Unable to generate test groups
  • πŸ‡ΊπŸ‡ΈUnited States neclimdul Houston, TX

    So my understanding:

    1. Documentation is bad. pushToMarketoUsingPOST requires a program name but doesn't document this requirement. It in fact documents the opposite programName (string, optional),
    2. With formid, program name is bypassed. Maybe not a huge deal but a secondary bug.
    3. Without formid, webform calls update lead. Without a program this fails because of #1.

    The solution is then to provide a push and sync method and choose the correct one based on the program logic?

    Marketo really made this so simple!

  • Pipeline finished with Failed
    10 months ago
    #81006
  • Status changed to Needs review 10 months ago
  • πŸ‡ΊπŸ‡ΈUnited States neclimdul Houston, TX

    Took a stab at automatically detecting if the program updating resource should be used. Best I can tell this is the right approach but the documentation is very confusing. Let me know if this help.

  • Pipeline finished with Running
    10 months ago
    #81421
  • Status changed to RTBC 10 months ago
  • πŸ‡ΊπŸ‡ΈUnited States hyperlinked San Jose, CA

    Thanks James! I've reviewed the latest commits and I can confirm that I'm able to have my lead data sent to Marketo when a user does something to trigger a lead sync that doesn't involve a program name.

    I can also confirm that a Webform submission with a form ID is properly submitted and received by Marketo.

    I can't confirm if everything that uses a program name is working properly as I've never used one of those, but I can confirm that the program name when supplied now affects the execution path of the API request whereas before, the program name was always discarded.

    BTW, I initially reviewed the wrong branch and I committed some similar changes to the 3.x branch. That commit can be ignored. Everything is in this one already. I'm bad at Gitlab.

Production build 0.71.5 2024