'Could not retrieve the remote URL' error message is vague

It seems like there isn't enough error logging here to really be able to debug what's going on.

Here's a patch to improve logging somewhat (although, maybe still not enough, depending what the exception message is :)

The Needs Review Queue Bot → tested this issue.

While you are making the above changes, we recommend that you convert this patch to a merge request → . Merge requests are preferred over patches. Be sure to hide the old patch files as well. (Converting an issue to a merge request without other contributions to the issue will not receive credit.)

Thanks mfb, have applied the patch.
Waiting for new errors.

Further update.

The patch adds to the error message.
Could not retrieve the remote URL (https://youtu.be/ZwEYpptNyek): Could not retrieve the oEmbed resource.

In a browser where I am not logged in I go the affected page with the embedded video.
Of course the video does not show.

I visit the same page in as a logged in user.
The video shows.

I flush the caches.

I return to the page In a browser where I am not logged in.
The video shows now whereas it did not show before.

Interesting caching issue looks like.

I do not drupal core caching enabled.
I have advaag disabled.

I am using redis and I am using cloudflare.

Any ideas?

There's hopefully another exception inside with a better error message? Here's another patch.

Contrib loggers that can show nested exceptions make debugging a lot easier than drupal core on its own.

Applied the patch.

The error message now reads.
Could not retrieve the remote URL (https://youtu.be/Fw6Rz2GAxc0): cURL error 28: Resolving timed out after 5000 milliseconds (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://www.youtube.com/oembed?url=https://youtu.be/Fw6Rz2GAxc0&maxwidth=560&maxheight=315
So this looks helpful.

Are there any quick fixes for this?
Thanks in advance.

As a non coder, this whole thing is pure voodoo.
As this seems to happen mostly for anon users and authenticated users see the video.

From here, it'll be up to you to figure out by talking to whoever is responsible for the network where your site is hosted. You now have an error you can show them - it should not take over 5000 milliseconds to resolve www.youtube.com - maybe the DNS server you are using is over-saturated, or caching needs to be enabled, or something like that.

This logging change might be worthwhile for other folks who are debugging this component, so I'll set the patch to "needs review"

This seems more like a support request ticket

Thanks for the good advice.
I talked to the server admin and he seems to have fixed this issue.
This is his explanation of what was going on.

so the issue on that video thing was that the DNS resolution _on the server_ was broken.

Our servers sometimes need to resolve DNS like anything else, and we had them pointed to Cloudflare public DNS (1.1.1.1) as well as Google Public DNS (8.8.8.8).

When the nodes are being saved the server is reaching out to Youtube to fetch some info, using cURL, and that is the error that we were seeing, that CURL call failing because DNS would not resolve for the youtube name.

Coincidentally (or not), Cloudflare.com was down last week due to DDoS attack (south Sudanese hackers claimed responsibility).

Both Cloudflare and Google are under constant DDoS attack (see https://www.zdnet.com/article/google-cloud-aws-and-cloudflare-report-lar...) .

...

This is the 2nd time we have received direct reports of the cloudflare DNS flaking out, so we have switched to another DNS provider.

We are in Canada so are using the CIRA CAnadian Shield DNS.

To fix this problem, replace the DNS on your server (/etc/resolv.conf) with a more reliable alternative (one from your ISP, or a local caching DNS server, etc). Unreliable DNS is the cause.

@smustgrave what do you think about the patch - this is rather difficult to debug unless the logging is improved.

So this could be repurposed with an updated title and issue summary.

New ticket opened up and this one is postponed on that.

Either or.

So revisting this and seems the request is to add additional logging information for possible future debugging or issues. On that idea +1 from me.

I have created the twig file based on my content type template/field/field--node--field-article-video--article.html.twig
with following code and
{#
/**
* @file
* Default template for a field.
*/
#}

{% if gallery_video_thumbURL %}

{% else %}

{% for item in items %}
{{ item.content }}
{% endfor %}

{% endif %}

In my theme file created function
function extractYouTubeVideoID($url) {
// Regular expression pattern to match YouTube video IDs
$pattern = '/(?<=watch\?v=|\/videos\/|embed\/|youtu.be\/|\/v\/|\/e\/|watch\?v%3D|\?v%3D)([\w-]{11})/i';

// Perform the regular expression match
preg_match($pattern, $url, $matches);

// Check if the video ID is found and return it, or return null if not found
return isset($matches[1]) ? $matches[1] : null;
}
function crystalrun_preprocess_field(&$variables) {
$entity = \Drupal::routeMatch()->getParameter('node');

if ($entity instanceof \Drupal\node\NodeInterface) { // Check if it is a node
$nid = $entity->id(); // You also have the node object right here$

$node = \Drupal\node\Entity\Node::load($nid);
$content_type= $node->getType();
/**********/
if ($content_type == "article"){
// check the field value exits or not.
$url = $node->field_article_video->getValue();
if (!empty($url))
{
$field_value = $node->field_article_video->entity->field_media_oembed_video->getValue()[0]['value'];
$videoID = extractYouTubeVideoID($field_value);
$variables['gallery_video_thumbURL'] = 'https://www.youtube.com/embed/'.$videoID ? 'https://www.youtube.com/embed/'.$videoID :'' ;
}else
{
$variables['gallery_video_thumbURL']="";
}
}
/*********/
}

Its absolutely working fine for me, please share you feedback and comment on the resolution, can be used with other contenttype simillary be creating twig file and handling the logic.

Can we have some tests of this new message?

I'm not super familiar with media module, so not sure this is the best place for it, but here is a hopefully a working test

Restoring status as test coverage is there https://git.drupalcode.org/issue/drupal-3402540/-/jobs/555070

Made a suggestion to improve the test.

Feedback appears to be addressed

Not eligible for backport as it changes a user-facing string.

Committed c43f7aa and pushed to 11.x. Thanks!

longwave → closed merge request !5989

Automatically closed - issue fixed for 2 weeks with no activity.