Comments (3)
If a user wants to do a GC at the end of the function, just add
gc()
at the end.
This is not trivial to do if you're calling a package function. You end up having to define a wrapper function that runs the real function, stores the result, calls gc()
, and finally returns the result. It's not hard, but it hurts readability a lot, and novice R users would probably never think of it. Compare:
bplapply(1:10, sqrt)
bplapply(1:10, function(x) { res <- sqrt(x); gc(); res; })
I suppose you could define a gc wrapper factory, something like:
gc_after <- function(FUN) {
function(...) {
res <- FUN(...)
gc()
res
}
}
bplapply(1:10, gc_after(sqrt))
Having the force.GC
argument is helpful (even if the default is false) because it signals to the user that this might be something useful for them to do (and in some cases it is useful, even if it's an ugly workaround). If we remove the argument, perhaps we should add something to the documentation mentioning that if the user encounters memory usage issues, they can force GC using any of the above methods?
from biocparallel.
I always use force.GC=FALSE
in practice, but I use this because I know force.GC
has performance penalties, and most times I do not need to do GC. For normal users, they may know what this option does but do not understand why they need it(in other words, they do not know when they should turn it off).
I do not use MulticoreParam a lot so this is not a problem for me, but I feel that force.GC
should not even be an option. If a user wants to do a GC at the end of the function, just add gc()
at the end. It would not take too much effort. Having this option not only complicates the structure of BiocParallal but also adds additional learning costs (which looks unnecessary). Therefore, instead of changing the default to force.GC=FALSE
, I suggest removing this option in our devel version to avoid this issue.
from biocparallel.
Thanks for the commentary.
I kept force.GC=
because there are common situations (users of package making use of bplapply()
) where it would be challenging to enforce garbage collection.
I changed the default, rationalizing that the first use is for speed gains, and that memory management is secondary and applies to a subset of problems.
from biocparallel.
Related Issues (20)
- Error in serialize(data, node$con, xdr = FALSE) : ignoring SIGPIPE signal HOT 5
- Nested parallellization question HOT 6
- stop.on.error = FALSE for DoParam doesn't work as expected HOT 2
- Troubles with bplapply within function (using SnowParam on Windows) HOT 6
- MulticoreParam bplapply unable to restart upon interrupt HOT 1
- Increase depth of traceback beyond `tryCatch()` for bp* functions - possible enhancement HOT 1
- The running time isn't reduced when using bplapply()? HOT 3
- BiocParallel errors HOT 4
- Handle worker abort better HOT 1
- move Rmpi to Enhances: HOT 1
- "foreach" %in% loadedNamespaces() instead of "package:foreach" %in% search()? HOT 1
- BiocParallel errors:could not find function ".OLD_read_block" HOT 11
- I meet a error when I use BiocParallel HOT 6
- BiocParallel : long vectors are not supported in .C() HOT 5
- BiocParallel for parallelization in BEER: Error and GPU Compatibility HOT 8
- BatchtoolsParam fails to propagate errors in bpiterate HOT 2
- Extremely minor: typo in docs? man/MulticoreParam-class.Rd HOT 1
- BiocParallel socketConnection error HOT 5
- strategy of tasks in MulticoreParam HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from biocparallel.