Comments (3)
@lediur Your analysis is most likely the correct one. If you're hitting the same storage account and container from up to 70 tasks at once, that can be potentially 70 x 3 x number of cores per node
requests to storage all at once. You may want to specify "blobxfer_extra_options": "--numworkers 2"
or something on the lower end. Mind sharing how big your data sizes and how many files are included in your output?
If you encounter this again, please post the exact blobxfer traceback, I can try to add some more custom retry handling for the specific error you are seeing from requests.
from batch-shipyard.
from batch-shipyard.
Closing, please re-open if necessary.
from batch-shipyard.
Related Issues (20)
- Singularity container creation failure due to failure to mount squashfs
- Pool resize with NC24rs_v3 fails to find PKEYS during nodeprep HOT 1
- Create a pool of nodes without Public IPs HOT 1
- Cryptography installation error in resource monitoring VM
- "Python: Command Not Found" in Ubuntu 20.04LTS During Node Preparation HOT 5
- Create Pypi package for Batch Shipyard HOT 1
- Merge task schema is missing completion option
- AAD with storage account in another subscription
- Prefetching images upon pool creation HOT 1
- TensorFlow-CPU quickstart issues
- Deployment of Standard_NC4as_T4_v3 fails if GPU drivers are specified HOT 2
- It can not start job by batch scheduler when VNET configured to use NAT gateway. HOT 1
- "Error response from daemon" occurred when pull shipyard image from default registry. HOT 1
- How can I set docker image name as a environment variable? HOT 1
- Support a dictionary to be applied to the task's "command" in a "custom" TaskFactory
- ERROR - Error response from daemon: Head "https://mcr.microsoft.com/v2/azure-batch/shipyard/manifests/3.9.1-cargo": dial tcp 204.79.197.219:443: i/o timeout HOT 2
- Use YAML anchors and aliases in job configuration files
- OpenMPI and Docker containers
- Batch Shipyard project discontinued? HOT 3
- This repo is missing important files
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from batch-shipyard.