Comments (5)
The worker side of this was implemented in #143, we're waiting on an http implementation I threw together in #144 to use as a grpc endpoint for one or many workers, and #153 will enable this on the server
from bazel-buildfarm.
It is not currently possible to separate this right now, but for only the simple reason that we're using the same endpoint for the operation queue, the CAS, and the Action Cache.
Your use case is definitely sound, but I have a thought about providing the support for the different endpoint types in-worker: How about I just make 3 different configurable endpoint addresses for communication over grpc for each of the services, rather than trying to bake in client support for all manner of CAS/AC communication protocols (http, grpc, hazelcast, redis, memcached, mongo, etc).
I can also quite separately (doesn't even have to be in this repo) provide a grpc hosted server that can speak one of these (separate binaries for separate protos) and facilitate proxy/translation for either activity (CAS or AC).
I think it is an effective strategy to combat complexity to say 'the operation queue workers speak grpc and nothing else.'
from bazel-buildfarm.
Hi @werkt, I'm not sure if that was a comment directed at me.
What I was hoping for was to give a flag to the build farm worker to read/write to some cache backend.
I'm not sure how things work so I don't quite understand the 3 different endpoint configurations or the reason for a proxy.
Thanks!
from bazel-buildfarm.
I believe @werkt is saying that currently there are 3 different kinds of things, all handled on a single port by a single binary. If they were (or could be) on different ports, it would be far easier for others to add support for alternate implementation(s) of the functionality - whether it is a shim to use Bazel Remote Cache in place of the internal CAS, a custom AC backend, or what have you.
from bazel-buildfarm.
We've broken things out at this point, have at it!
from bazel-buildfarm.
Related Issues (20)
- Do deployments via Helm work as RBE ? HOT 5
- make execution-policy=as-nobody the default on Linux
- [ZstdCompression] Read request with offset incorrectly fails with not_found error and removes blob
- [Bazel CI] apt-get and integration tests are failing with Bazel@HEAD HOT 1
- Tips on learning buildfarm? HOT 2
- Remote builds stuck on Buildfarm that is deployed with Helm. HOT 6
- Implement Fetch 'Push' Service
- Support http_header: prefix in Fetch Qualifiers
- bazel.canonical_id unsupported in Asset Fetch API HOT 1
- ERROR: error running 'git fetch origin refs/heads/*:refs/remotes/origin/* refs/tags/*:refs/tags/*' while working with @build_buildfarm~: HOT 2
- Updated from java17 to java21 to use the new server and worker causes JVM to shutdown
- Fetch service does not properly handle missing content-length HOT 1
- Support Multiple Hashing Function
- Remote execution service executing processes locally HOT 4
- ERROR: Failed to query remote execution capabilities: UNAVAILABLE: io exception HOT 4
- Feasibility Analysis of Using Buildfarm for Large-Scale Development HOT 2
- Verbose Logging for Servers and Workers through Helm Chart
- Check logs during remote execution HOT 4
- Incorrect container port for Shard-worker in Helm template
- hardlinks in CAS leads to task failure in some cases HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bazel-buildfarm.