Comments (2)
To whom it may concern in the future.
My solution (workaround, to be more precise) is to override Runner and WandbLogger as follows:
class CustomRunner(dl.SupervisedRunner):
def collect_artifacts(self) -> Dict[str, str]:
artifacts = defaultdict(list)
for cb in self.callbacks.values():
if isinstance(cb, dl.CheckpointCallback):
artifacts['checkpoints'].extend(
[os.path.join(cb.logdir, f) for f in os.listdir(cb.logdir)
if re.search(r'.*(best|last).pth$', f)])
return artifacts
def log_artifacts(self):
artifacts = self.collect_artifacts()
for logger in self.loggers.values():
for tag, path_to_artifact in artifacts.items():
logger.log_artifact(tag, self,
path_to_artifact=path_to_artifact)
def on_experiment_end(self, runner: "IRunner"):
self.flush_log()
self.log_artifacts()
self.close_log()
self.engine.cleanup()
class CustomWandbLogger(WandbLogger):
def log_artifact(self, tag: str, runner: "IRunner", artifact: object = None,
path_to_artifact: Union[str, list] = None,
scope: str = None) -> None:
artifact = wandb.Artifact(f'{self.run.name}-{tag}', type='artifact')
if isinstance(path_to_artifact, list):
[artifact.add_file(p) for p in path_to_artifact]
elif os.path.isdir(path_to_artifact):
artifact.add_dir(path_to_artifact)
else:
artifact.add_file(path_to_artifact)
self.run.log_artifact(artifact)
from catalyst.
Hi @chekhovana ! Engineer from W&B here. I'm working on a fix for this. Thanks for your solution. It has been extremely helpful in building out a solution.
from catalyst.
Related Issues (20)
- Dependency `packaging` not specified — ModuleNotFoundError HOT 2
- os.environ["CUDA_VISIBLE_DEVICES"] = "" does not use CPU HOT 5
- comprehensive classification example HOT 3
- `runner.evaluate_loader` does not work with DataParallelEngine HOT 4
- Idea: offload all launch-related code Accelerate HOT 2
- Custom loader stages HOT 1
- Crashes on 2xT4 GPUs HOT 3
- utils.process_model_params HOT 2
- Multi Criterion Training HOT 4
- No utils.initialization file HOT 3
- DynamicBalanceClassSampler example does not use sampler HOT 1
- KWArg fp16 does not exist HOT 6
- Replace getters with properties HOT 1
- Bug in catalyst/callbacks/backward.py if the grad_clip_fn value is set. HOT 2
- Bug in catalyst/callbacks/backward.py if the grad_clip_fn value is set. HOT 1
- Bug in catalyst/callbacks/backward.py if the grad_clip_fn value is set. HOT 1
- `DataParallelEngine.prepare_model` missing `device_placement` kwarg. HOT 2
- Columns and DataType Not Explicitly Set on line 96 of report.py HOT 2
- Importing DistributedSamplerWrapper will invalidate the setting CUDA_VISIBLE_DEVICE. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from catalyst.