Deadline exceeded while waiting for HTTP response about crmint HOT 5 CLOSED

Hi @benpowis,

AFAIK, there were no recent updates in BigQuery API.

Can you please try to use Google API Explorer to fetch the BigQuery job list for the project that is used in your data pipeline and let me know if it works for you? This might help to pinpoint the issue.

Thanks!

from crmint.

Hi @Slony

I ran the API explorer but I can't see any jobs relating to the CRMint pipelines.

Interestingly it appears some of the error jobs did actually run, but not all. I can share the pipeline privately if this helps?

Ben

from crmint.

Hi @Slony

This issue is now affecting multiple pipelines intermittently, sometimes they run just fine and sometimes I get this error, causing the job to fail.

Do you have any suggestions on how we can troubleshoot?

Thanks,
Ben

from crmint.

Could this error be related too https://stackoverflow.com/questions/14698119/httpexception-deadline-exceeded-while-waiting-for-http-response-from-url-dead ?

from crmint.

@benpowis thank you for investigating this issue and giving us a very useful hint about default 5 second timeout for URLFetch requests. The timeout has been increased to 300 seconds in PR#109.

from crmint.