blocking => false not working

[peixotorms]

Summary

When I set blocking => false, the request is still blocking.

Given the following code sample

# make options
    $options = array(
        'timeout' => 60,
        'connect_timeout' => 10,
        'blocking' => false
    );

# request
$response = WpOrg\Requests\Requests::request($url, $headers, $data, POST, $options);

Unfortunately, if I set a short timeout like 1 second, the remote API call will not complete.
I don't need the response, only for it to run in the background while the script does something else.

I'd expect the following behaviour

I would expect it to return immediately, so the rest of the script can run.

Instead this happened

With a long API call lasting over 1 minute, the script hangs until it returns a reply.

Additional context

The way around this for non blocking requests is to use curl with CURLOPT_NOSIGNAL set to 1.
This let's me timeout early and the api call will complete properly.

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $endpoint);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($fields));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 1); // Set timeout to 1 second
curl_setopt($ch, CURLOPT_NOSIGNAL, 1); // Ignore all cURL signals to ensure script continues 
curl_exec($ch);
curl_close($ch);

Your environment

Environment	Answer
Operating system and version:	Ubuntu 22.04
PHP version	8.2 lsapi, openlitespeed
Requests version	v2.0.7

Tested against develop branch?

no

[schlessera]

@peixotorms I cannot replicate this. When using 'blocking' => false, Requests bails early on a request with an empty Response object. You can see this for example when checking that the 'status' in the response is false.

Can you provide more information or a piece of content that lets us replicate this?

[peixotorms]

Hi, it's been over 3 months, so I don't remember the exact case, but If I remember correctly, when you call an external url that takes for example 10 seconds to complete the output, it was also slowing down the response on the calling script, regardless of the blocking settings.

So script A calls script B with blocking = false, but script B needs 10 seconds to complete.
Script A would then take at least 10 seconds to finish as well.

Adding CURLOPT_NOSIGNAL when blocking = false, sorted it for me though.

I'll test this again against the latest version soon, but not right now.

Thanks

[emohamed]

@schlessera the problem is reproducible for me. It's apparent if you use an endpoint that takes longer to produce a response, like https://httpbin.org/delay/5:

$ wp eval --skip-wordpress '
    $start = microtime(true);
    $response = WpOrg\Requests\Requests::request( "https://httpbin.org/delay/5", [], [], "GET", [ "blocking" => false, "verify" => false ]  );
    print_r($response);
    echo "Request took: " . (microtime(true) - $start ) . " seconds";'

WpOrg\Requests\Response Object
(
    [body] =>
    [raw] =>
    [headers] => WpOrg\Requests\Response\Headers Object
        (
            [data:protected] => Array
                (
                )

        )

    [status_code] =>
    [protocol_version] =>
    [success] =>
    [redirects] => 0
    [url] =>
    [history] => Array
        (
        )

    [cookies] => WpOrg\Requests\Cookie\Jar Object
        (
            [cookies:protected] => Array
                (
                )

        )

)
Request took: 5.6029760837555 seconds

The response is indeed empty, however the library is awaiting for it.

[schlessera]

@peixotorms @emohamed Thanks, I can indeed replicate this.

Using CURLOPT_NOSIGNAL can introduce other problems, though, as this means that cURL skips registering any signal handlers. The most obvious issue there will be that DNS timeouts will not be caught and just go on indefinitely, unless you catch these yourself.

[westonruter]

Just seeing this issue as it was reported in Core Trac as #63547.

This could be a particularly bad performance problem for WordPress because WP Cron is spawned in what is supposed to be a non-blocking request:

'timeout'   => 0.01,
'blocking'  => false,

Since wp_cron() runs at the init action, which then schedules _wp_cron() to run at the wp_loaded action, meaning that if the loopback request takes longer than expected, every page load that spawns crawn will have a bad TTFB.

In particular, this comment in wp-cron.php file header is false:

The HTTP request to this file will not slow down the visitor who happens to visit when a scheduled cron event runs.

[westonruter]

Enabling the NOSIGNAL option did not fix the issue for me: https://core.trac.wordpress.org/ticket/63547#comment:6

[adamsilverstein]

Since wp_cron() runs at the init action

Could we move this to the shutdown action?

[adamsilverstein]

Since wp_cron() runs at the init action

Could we move this to the shutdown action?

Ah, I see @westonruter suggested the same on the Trac ticket.

[westonruter]

Another proposal to fix this issue reported by @TysonAndre (#264) and implemented by @siliconforks (#932) using CURL_VERSION_ASYNCHDNS.

(I still think it makes sense to move to shutdown regardless.)

[siliconforks]

Just to clarify: I don't think my PR (#932) would actually fix this specific issue (#826 - blocking => false not working). Basically I think there's two different issues (actually, three different issues) that may need to be addressed:

1. The issue that timeouts of less than 1 second are not currently allowed. That is specifically what my PR is attempting to address.

2. The issue that "blocking => false" does not really mean "not blocking". That is this issue (blocking => false not working #826) and my PR does not really do anything for that. (Note that the original code example above used 'timeout' => 60 and 'connect_timeout' => 10, both of which are far longer than one second, so my PR would not have any effect on that.)

3. And then there's a third issue, which is specific to WordPress - that it may be better to perform the cron request on shutdown rather than on init. That's something that maybe should be done regardless of what happens with the other two issues.

[westonruter]

3. And then there's a third issue, which is specific to WordPress - that it may be better to perform the cron request on shutdown rather than on init. That's something that maybe should be done regardless of what happens with the other two issues.

This was implemented in Core-63858.