Comments (1)
Hi @wsmoses,
The Lambdas are limited to 500 MB of disk space, and this thunk doesn't fit on a Lambda. To run this job to completion, you have two options:
- You can set up an EC2 instance as the overflow machine. The jobs that don't fit on Lambda can be sent to that machine for execution. As there are usually no more than a few thunks that don't fit on Lambda in each job (final stages of the build), you can get away with a single-core box, like an
m1.medium
instance (9ยข/hr).
Setting up the overflow machine is straightforward: after you building and installing gg
on the machine, run gg execute-server 0.0.0.0 <PORT>
. Now, run gg force
with a second engine:
gg force <TARGET> --jobs 1000 --engine lambda --jobs 1 --fallback-engine remote=<IP>:<PORT>
- You can run those thunks on your local machine. The only drawback is that now you have to download all the dependencies (all those objects listed in the thunk) back to your local machine, and depending on your network speed, this could hurt the performance.
To do this, run (make sure you have the latest version off the master
branch):
gg force <TARGET> --jobs 1000 --engine lambda --jobs 1 --fallback-engine local=mixed
Thank you for trying out gg
, and please let me know if there are any issues.
Best,
Sadjad
from gg.
Related Issues (20)
- If the file is already there, Alex gets mad.
- Build failure on Mac HOT 5
- OpenWhisk support
- New uploads required by intermediate thunks
- How to add model generation for a custom binary HOT 2
- .
- How to execute single file packaged python binaries HOT 20
- gg force --engine lambda failing after emptying bucket HOT 10
- Issue with running python binaries built with pyinstaller and nuitka on lambda HOT 2
- Issues with parallelizing thousands of tasks HOT 32
- Support CUDA HOT 6
- python sdk example fetch-deps.sh cannot work
- Build broken on RPM distros
- Building Chromium HOT 3
- Issues building mosh HOT 1
- Use OpenSSL for crypto HOT 1
- Issues when run example of Fibonacci HOT 1
- Error "gcc: unknown gcc flag: -mavx512vl" while building llvm
- Issues compiling FFmpeg HOT 3
- gg infer errors: unknown option: -plugin HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gg.