FAILED_PRECONDITION: A requested input (or the `Command` of the `Action`) was not found in the CAS. about bazel-buildfarm HOT 12 CLOSED

https://github.com/cdlee2/tools_remote-client is the canonical tool for interacting with those log files

from bazel-buildfarm.

Actually, I can see the problem from here: if you're operating with the default CAS size from examples/server.config.example, then you don't have enough storage to hold all of the inputs for your actions: your action inputs sum to over 1GiB for the action you posted. Buildfarm is evicting your existing content in order to store each of your missing blobs, and pushing out further needed content.

Your CAS must always be at least as big as your largest action, or it will never build successfully:

    # limit for CAS total content size in bytes
    cas_max_size_bytes: 1073741824 # 1024 * 1024 * 1024

I recommend setting that a bit higher

from bazel-buildfarm.

Much though I would like to debug your client's remote execution implementation, I am not responsible for this google.devtools.build of which you speak. I might recommend outputting some logging, perhaps with a --experimental_remote_grpc_log and attaching it here

from bazel-buildfarm.

$ bazel build --spawn_strategy=remote --genrule_strategy=remote --strategy=Javac=remote --strategy=Closure=remote --remote_executor=server:8980 --experimental_remote_grpc_log media_service reports success. Retrying with --experimental_remote_grpc_log=remote.log fails as expected.

This log won't contain any source, will it? I don't see any when opening with a text editor, but I want to make sure before attaching it 😃

from bazel-buildfarm.

I threw together a trivial LogEntry protobuf decoder, but it complains of invalid utf-8 - so it's safe to say that I'm doing something wrong.

from bazel-buildfarm.

There should be no source included in those logs, but there is the capacity for ActionResults to contain output artifacts, and I don't want to recommend that you expose those if you're concerned.

I've put up a branch of buildfarm at https://github.com/werkt/bazel-buildfarm/tree/missing-input-logging that will include the digests of the files/directories that are missing when execute is failing in that way. You can then use the remote_client to correlate the requests over those missing digests, and identify the activities performed and reduce the logging attachment required to present the issue.

from bazel-buildfarm.

remote.log
The log looked clean, so I uploaded it.

FWIW, I encountered this error with bazel 0.14.0 and with a version that includes an s3 patch. I'm not sure exactly what official version that most closely resembles though...

from bazel-buildfarm.

Give this a try with the missing-input-logging tree, as it will help identify what is going wrong with your actions.

from bazel-buildfarm.

Will do

from bazel-buildfarm.

Going to close this out, as that has to have been the issue, and buildfarm was responding somewhat appropriately (not crashing at least) - feel free to reopen if you have a feature request with requirements in mind.

from bazel-buildfarm.

Looks like you solved this minutes before I finally got around to rebuilding with your missing-input-logging branch 😀

If I understand correctly, the related changes are PRs #155, #156 which are merged. I'll edit the CAS size limit, rebuild, and report success/failure.

from bazel-buildfarm.

Success - thanks!

from bazel-buildfarm.