Comments (4)
Ooooof, good catch. I'd say best bet is to merge the versioned docs into the release flow, but this can definitely be an issue in the future too... For simplicity i think we can just use the retry step we use for deploys to make it try to git pull, push several times
from crawlee.
Yeah, unfortunately, we need the version_docs
to run conditionally (only on major
/ minor
releases, not patch
). The retry step we use for deploys
sounds - although still a bit hacky - like what we need right now, do you have a link for that pls?
from crawlee.
Its the same step we use in I think docker cis on the docker repo to see if the module versions are published to npm yet. I can take a look at pr-ing this fix this week 👀
from crawlee.
I mean, if you know where to look, it would be nice if you could 🙏🏽 No worries if you don't find the time, it's quite low-prio and I'll revolve to it sooner or later :)
from crawlee.
Related Issues (20)
- Handle `Crawl-delay` directive in robots.txt
- RobotsFile.isAllowed returns false for allowed routes HOT 3
- Refactor `retireOnBlockedStatusCodes` to `isBlockedStatusCode` and move the retiring out of the `Session` class
- `AdaptivePlaywrightCrawler`: programmatically deciding when to render JS HOT 3
- Cheerio crawler going out of memory unexpectedly with lot of concatenated strings
- Support reading sitemap from a variable
- Trouble unzipping `sitemap.xml` (`zlib: incorrect header check`) HOT 5
- handledRequestCount from requestQueue.getInfo() after restart is 0 HOT 1
- Improve docs about request locking HOT 2
- Load more function for google maps
- Rename `Job` from statistics module to `RequestProcessingRecord`
- Replace spaces with tabs for indentation
- Consider porting over `ContextPipeline` from the Python counterpart
- Remove experimental containers in browser pool
- Remove version private field from browser pool
- Proxy authentication error HOT 1
- URLs rejected from file HOT 4
- Third-party proxy IP does not take effect
- Unify `RequestList` and `RequestProvider` interfaces and extract their "tandem" behavior from `BasicCrawler`
- Investigate a more explicit way of detecting retries in `ProxyConfiguration`
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from crawlee.