Comments (11)
Hmm, but with sdk v3 you should get the purge by default anyway, or not?
from apify-cli.
that's what i thought
but looking into the code, just a line before this diff above, the CLI sets CRAWLEE_PURGE_ON_START = '0'
, and as you cans see, it reverts that to the default 1 only if it detects crawlee, which it doesn't for my project.
from apify-cli.
I see, @vladfrangu do you remember why it's like that? Feel's wrong, I'd say we should remove that line.
It looks like now when you use the Apify CLI, it won't purge stuff by default even for crawlee projects and will require explicit --purge
flag? It shouldn't behave like that, it should be the exact opposite, purge by default for crawlee, and require --no-purge
to opt-out (which disables the auto purge via env var).
from apify-cli.
Maybe better to keep those changes for the upcoming v1 release as they are indeed breaking and could surprise users if it currently works in the opposite way as it should.
edit: on the other hand, we broke it just with the latest version ~month ago...
from apify-cli.
it should be the exact opposite, purge by default for crawlee, and require
--no-purge
to opt-out (which disables the auto purge via env var).
This is edging undefined behavior terrain very fast. I also don't see why pre-crawlee should be purge opt-in but post-crawlee should be purging opt-out.
I'd say lets clean up all of this logic in v1 (including for older sdks) -> always purge, opt out only. What I tried with that PR (but missed this case, which is easy to fix actually) is to make the crawlee behavior also opt in, like for old sdk.
from apify-cli.
I also don't see why pre-crawlee should be purge opt-in but post-crawlee should be purging opt-out.
Because that's what we changed on purpose with v3. In v2 you had to use the CLI to purge, with v3 we purge by default, and we wanted to keep that behavior as changing it is breaking.
always purge, opt out only
Yeah, that might be a good approach for v1. Thoughts @mvolfik?
from apify-cli.
always purge, opt out only
Yeah, that might be a good approach for v1. Thoughts @mvolfik?
If the --no-purge
flag is easily found, and you don't have to dig for some CRAWLEE_NO_PURGE
env var (this is a part of the reason I was opposed to this change with v3 when we spoke about this last time), let's go with it. A disadvantage is that the default behavior is destructive (deletes data), and I'm so used to running apify run -p
that I don't consider it a hassle anymore, but for common development workflow probably default=purge
makes more sense.
from apify-cli.
Nonetheless, unless the 1.0 is next week, can you please fix the original issue of this thread for 0.19? thanks
from apify-cli.
and I'm so used to running apify run -p
Yeah, and that's the root of the problem, people in Apify are used to this while newcomers are confused by it.
@vladfrangu let's fix this in v0.19 too please
If the --no-purge flag is easily found
How often do you actually want this behavior? I would say you want this only when testing resurrection of an actor, otherwise never? Which was one of the reasons we did the behaviour switch in the first place.
from apify-cli.
@mvolfik 0.19.3 is out, let us know how it works
from apify-cli.
Closing as the fix is now also in the master branch.
from apify-cli.
Related Issues (20)
- New release process
- `apify call` to support input passing HOT 2
- Add spec to shell autocomplete tool Fig HOT 2
- Interface for making "raw" API calls
- `apify run --purge` doesn't purge HOT 5
- apify push and call do not validate whether they are in an Actor folder HOT 3
- `actor:push-data` doesn't work in the Apify Platform HOT 4
- Better CLI API HOT 7
- Output from warnings doesn't go to `stderr` and breaks working `*nix` integrations even on otherwise non-breaking releases HOT 4
- Cryptic error message when `actor.json` is an invalid JSON
- Add in a flag to make commands output machine-readable results instead of human readable ones HOT 7
- Figure out a better auth file structure for future extensibility
- Experiment with Bun bundling for distributing the CLI as a single-file executable HOT 1
- Actor.getInput returns null HOT 12
- Version 0.20.0 installed via NPM fails to work HOT 3
- Clean up the purging logic for crawlee projects
- Support usage of `apify push` with monorepo projects
- Error: Could not send API token to CLI HOT 2
- .actor/actor.json not being found?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from apify-cli.