https://github.com/ocaml-opam/Camelus had this feature, it could be useful to ocaml-ci or opam-repo-ci to post a summary of the failures

Jobs are timing out after the tests run - example

Investigation shows that dune runtest does not exit cleanly:

❯ dune runtest
File "plugins/git/test/dune", line 1, characters 0-52:
1 | (mdx
2 |  (packages current_git)
3 |  (package current_git))
ocaml-mdx-test: internal error, uncaught exception:
                Unix.Unix_error(Unix.EINTR, "waitpid", "")

Background

Following the consideration of adding metrics to the different systems relying on OCurrent, it would be great to add some metrics to follow the energy consumption of the machine on which OCurrent systems are running. This would help us to quantify the cost of the different approaches we can use and consider the environmental impact of our technologies.

Summary

This could be achieved by writing a daemon that monitors the energy consumption from different sources. It could come from MRS registers on Intel x86_x64 and any similar technology for AMD, from any external source, a connected plug or an internal model that can compute the consumption.

In the end, those metrics would be exposed via Prometheus to be gather and displayed in a dashboard (as Grafana offers, for example).

When looking at the job graph, i have the following use-case:

1. lookup some error log on the userside website
2. open the corresponding subgraph (takes 30s already)
3. lookup some corresponding docker peek logs to rebuild (takes 20s)
4. lookup some more logs on the userside website
5. the page reloads and i have to wait 30s again...
6. leave it to reload, go back to it and it in a reload cycle once again

I don't personally need the automatic refresh feature added in #277 at all.
Is there a way to disable it?

This is useful to have a quick access to this for jobs that are stalling, or that will overload the cluster.
On opam-repo-ci there is too much going on on the admin UI for it to be usable/enjoyable.

The GitHub API call for getting an installation's repositories only returns one page at a time (seems to be about 30 items). We should fetch the remaining pages. But it would also be good to have some limits in place, in case someone enables "all repositories" by mistake...

Multiple projects require additional application-specific information to be associated with OCurrent jobs. This should not be included in the base OCurrent library in order to keep it a minimal, application-agnostic core, but could instead be included as a plugin.

We require need to add information that maps directly to individual jobs, such as commit hash, server that the job is running on, as well as information that relates jobs together, such as the commit history for a branch. This must be stored in a database table for persistence. To facilitate the changing of the schema, a DB migration system must also be present.

OCaml-CI already has these systems implemented for itself, but with many other projects also requesting it (listed below), extracting out the shared functionality would be ideal to avoid repeated code.

• Opam-repo-CI needs jobs to be searchable by commit hash and server that the OCluster job is running on.
• OCurrent-deployer needs a history of previous builds — ocurrent/ocurrent-deployer#190.
• Multicoretests-CI needs to be able to search jobs by commit hash and store the running times of jobs.

(The list of requesting projects and required fields is not exhaustive.)

A recent article "�[31m"?! ANSI Terminal security in 2023 and finding 10 CVEs exposed flaws in terminal emulators when they parse escape sequences. A related potential security issue could occur if a http client "tricks" the server into logging raw, unescaped data, through for example a GET request (see the case of python -m http.server). We should ensure that ocurrent is not affected by this class of problems, and if so, escape user-submitted data that may be logged.

Hello,
Thanks for this tool - it's going to be really helpful for our Tezos benchmarking project, and the github app interface is really sweet.

That said, the documentation around the Github App usage is lacking. There are many details left as an exercise to the user, and I feel quite exercised in that respect. I'm currently hung up on what URL to paste into the configuration of my Github App. That information is obviously encoded in the example; however, this is my first ocurrent project, and I'm unfamiliar with the syntax. I cannot find the documentation for routes any where on the API docs site. Thus the meaning of ... /? nil is mysterious to me. I'm sure if I were an Emacs user Merlin could find the docstrings for me, but on VS Code nothings coming up via ocamllsp, at least not in the current-bench project.

Any of the following would be a big help to future users (ranked by preference)

• A detailed walkthrough that makes no assumptions of familiarity with Github Apps or Ocurrent
• Detailed annotations in the example
• Documentation somewhere on the docs site.

Hi, I'd like to use obuilder_spec to describe an image and be able to locally build it using Current_docker.

For now it's complicated because Current_docker.Default.build expects a ('Contents of Dockerfile.t), but Obuilder_spec.Docker.dockerfile_of_spec returns the dockerfile as a string. So I actually have to write down the content to a file using Current_fs.save and then use 'File of Fpath.t in the current_docker build.

Do you think it would be possible to have the conversion from an obuilder spec to a Dockerfile.t to simplify the process ?

It would be nice if occurent can eventually match this Vercel/netlify feature:

Are there plans to provide a hosted instance of this, similar to https://man.sr.ht/builds.sr.ht/ ?

When working with a lot of commits, it can get quite annoying to not have the commit messages attached to them. This is handled by Current_github, but not by current_git.
I'd be happy to implement this myself, but I don't understand how I'd go and do that.

Current.Job provides two functions to log output:

val write : t -> string -> unit
(** [write t data] appends [data] to the log. *)

val log : t -> ( 'a, Stdlib.Format.formatter, unit, unit ) format4 -> 'a
(** [log t fmt] appends a formatted message to the log, with a newline added at the end. *)

the logs are then streamed as text to the web interface.

I think it would be nicer if the logs were streamed with metadata, e.g. the build step where they were produced, to let the end interface better chose how to display them instead for streaming pre-formatted logs.
In the web interface, this would allow to have drop downs to display/hide the logs of each build step, highlight the label, allow selecting and copying (useful for Dockerfiles ocurrent/ocaml-ci#585).

In particular, no helpful error message is printed (just the alt text for the image):

Initally opened here.

When testing with ocurrent/ocaml-ci#429, the Show line numbers and allow jumping to specific lines in jobs logs #309 does loose the line numbers at certain points when streaming the logs.

022-12-12 17:02.08 current.github [WARNING] GitHub returned errors: 
                                               {
                                                 "errors": [
                                                   {
                                                     "message": "Parse error on \"\\xE2\" (error) at [13, 15]",
                                                     "locations": [
                                                       {
                                                         "line": 13,
                                                         "column": 15
                                                       }
                                                     ]
                                                   }
                                                 ]
                                               }

This is because of 8f9b075 which replaces the needed ... in the GraphQL requests. I don't really see the point of this commit personally so I would suggest just reverting it entirely.

I added a non-public repo to the CI Github app (which github happily let's you do) and it shows up in the UI. Unfortunately, all builds fail and when attempting to look at the details the page fails with an HTTP 500 page that doesn't tell me anything on why it failed.

@kit-ty-kate confirmed my suspicion that this is due to the repo being non-public: https://ocaml.ci.dev:8100/job/2023-04-27/130807-git-fetch-a5ebdd

I don't know what the policy about handling non-public repos is, but it would be nice to at least have an error message that says it is failing because the repo is not public and to have it work I'd need to switch it to be public.

In the current state of the webhook function, the user cannot influence the way Current_github has to deal with the events it receives from GitHub. We should provide a way to give an event handler to make it possible to decide how to handle them from a user perspective.

This issue has been brought up a few times (ocurrent/ocluster#179, ocurrent/ocaml-ci#323) but I thought I would open it on the repository where the fix/feature would have to start!

Currently, current_git does not support checking out private repositories. For now, I'll limit the discussion to Github repos. There's a few ways off of the top of my head you could go about trying to fix this:

1. Passing in a Github token to be placed in the URL when doing the fetch. Something like this code with some overriding of the pp_cmd to not leak the token in the logs. This works until your submodules are also private.
2. Setup either the main account on the machine you are running to have access to Github or setup some user on the machine with access and run git as them. This doesn't feel great and probably will have permission errors.
3. There might be some sort of thing we can with git-credentials and a token to overcome the submodule problem, I'm not sure.

Any other ideas?

Currently when something fails during a Current_docker.pull its output will stay a failure until the next scheduled pull. For this particular command we probably don't want that. Here is my proposal to improve this case:

1. If a pull fails, then just take its previous value.
2. If there was no previous value, try again a minute later, then every hour if it still fails.
3. Unless the service just started and the failed previous value is to be taken from the disk cache, in which case, try right away, then every hour if it still fails.

See ocurrent/ocaml-ci#422 for detailed list.

All the changes made will be in ocurrent proper and can be shared between all pipelines using ocurrent.

I'm not sure whether or not I built the test case right or if I missed something but this code:

let () = Random.self_init ()

module Rand = struct
  type t = No_context

  let id = "rand"

  module Key = struct
    type t = int

    let digest = string_of_int
  end

  module Value = struct
    type t = int

    let marshal = string_of_int
    let unmarshal = int_of_string
  end

  let build No_context job _key =
    let open Lwt.Infix in
    Current.Job.start ~level:Current.Level.Harmless job >>= fun () ->
    print_endline "build triggered";
    Lwt.return (Ok (Random.int 100))

  let pp = Fmt.int

  let auto_cancel = true
end

module Rand_cache = Current_cache.Make(Rand)

let schedule = Current_cache.Schedule.v ~valid_for:(Duration.of_sec 1) ()

let () =
  let v = Rand_cache.get ~schedule No_context 0 in
  Lwt_main.run @@
  Current.Engine.thread @@
  Current.Engine.create @@ fun () ->
  let open Current.Syntax in
  let+ _ : int =
    Current.component "test" |>
    let> () = Current.return () in
    Current_incr.of_cc @@
    Current_incr.read v @@ fun (v, job) ->
    begin match v with
    | Error (`Active `Ready) -> print_endline "ready"
    | Error (`Active `Running) -> print_endline "running"
    | Error (`Msg m) -> print_endline ("error: "^m)
    | Ok v -> print_endline (string_of_int v)
    end;
    Current_incr.write ~eq:(fun _ _ -> false) (v, job)
  in
  ()

doesn't end up being an infinit loop printing random numbers between 0 and 100 like I thought it would be, but only stopped after the second try:

$ dune exec ./test.exe
ready                
6
# waits 1 second
build triggered
27
# waits forever
^C

Have I missed something?

side-note: I could've used the Current.state function instead of the low-level functions from Current_incr, I'm not sure why I did, but it does the same thing anyway.

CI for Eio is failing. The analyse step finds an opam file in the ocaml-uring submodule, which no longer exists:

2022-04-29 10:50.10: New job: Analyse
2022-04-29 10:50.10: Waiting for resource in pool analyse
2022-04-29 10:57.01: Got resource from pool analyse
2022-04-29 10:57.01: Checking out commit 6f6b2eee. To reproduce:
                       git clone --recursive "https://github.com/ocaml-multicore/eio.git" && cd "eio" && git fetch origin "refs/pull/212/head" && git reset --hard 6f6b2eee
2022-04-29 10:57.01: Exec: "cp" "-a" "--" "/var/lib/ocurrent/var/git/eio.git-2b23594b7c63b62fff46d7f80bbb69077b881604076ad7869a499d8832a61ef7/.git" 
                           "/tmp/git-checkout15a32123"
2022-04-29 10:57.01: Exec: "git" "-C" "/tmp/git-checkout15a32123" "submodule" 
                           "update" "--recursive"
Submodule path 'ocaml-uring': checked out '2ea06dbb9169964d328a36696907750ed6e85398'
Submodule path 'ocaml-uring/vendor/liburing': checked out 'ccfe96eb3dca697e58f1dc42f82fb79f01a948b1'
2022-04-29 10:57.02: Exec: "git" "-C" "/tmp/git-checkout15a32123" "reset" 
                           "--hard" "-q" "6f6b2eee4f1ee42697825309d3ac4b2e65da4eef"
warning: unable to rmdir 'ocaml-uring': Directory not empty
2022-04-29 10:57.02: Exec: "git" "-C" "/tmp/git-checkout15a32123" "submodule" 
                           "sync"
2022-04-29 10:57.02: Exec: "git" "-C" "/tmp/git-checkout15a32123" "submodule" 
                           "update" "--recursive" "--init"
2022-04-29 10:57.03: Exec: "find" "." "-maxdepth" "3" "-name" "*.opam"
...
2022-04-29 10:57.13: Results:
                       {
                         "opam_files": [
                           "eio.opam", "eio_linux.opam", "eio_luv.opam",
                           "eio_main.opam", "ocaml-uring/uring.opam"
                         ],

I'm not sure how it got into this state. OCurrent probably needs some unit-tests with submodules.

In any case, doing the submodule update before the reset is likely a mistake. Copying the code in OCluster might help.

Following https://github.com/ocaml/dune/blob/main/.ocamlformat

The disk store directory is currently at $PWD/var/.

It seems better to respect XDG_DATA_DIRS as a location for this cache, to avoid polluting the top-level directories of projects using OCurrent and needing to ignore var/ everywhere.

The Dune source code contains an internal XDG library that could be used as a starting point.

to do, to do, to do to do to do to do to do

A job can indicate whether it should be auto-cancelled when no longer needed, or whether it should run to completion anyway.

However, we should probably always auto-cancel if the job hasn't started yet (e.g. waiting for confirmation).

I am learning the examples in the doc. Both auth and build_matrix raise

File "plugins/gitlab/api.ml", line 216, characters 39-63:
216 |     Project.Commit.commits ~project_id:project.project_short_id ~ref_name:project.project_short_default_branch ()

Error: This expression has type int64 but an expression was expected of type
         int

It could be fixed by the following lines though I don't know the exact reason yet.

(* 216  *) 
Project.Commit.commits ~project_id:(Int64.to_int project.project_short_id) ~ref_name:project.project_short_default_branch ()

(* 403 *)
  { Commit_id.repo; id = `PR (Int64.to_int pr); hash; committed_date }

I am using OCaml 4.13.1 and the source cloned from the current repo.

Had this repo pinned to upstream rather than the current release and I think there's a bug in the name of the repository used when getting the default ref:

The repo parameter is passed name but that is now the name of the default branch and not the name of the repository itself. Noticed this because when trying to fetch something it was fetching https://github.com/patricoferris/main.git

Auto-retry failed operations after some suitable timeout for errors such as:

Error: GitHub query for realworldocaml/book failed: Unix.Unix_error(Unix.ETIMEDOUT, "connect", "")

Also, provide some way to restart them manually in the web UI (though redelivering a recent webhook delivery may unblock things as a work-around).

Hello,

sorry if this is the wrong repository to report such a question, I simply do not know where the right one would be (and https://hub.docker.com/r/ocaml/opam/ doesn't mention any).

On https://hub.docker.com/r/ocaml/opam/ it states: "There are also some alternate OCaml configurations such as with flambda enabled or with frame pointers enabled." -- but I do not understand which images have flambda enabled (or frame pointers, or no flat float array). I'm using the fedora-35 4.14 line.

So, my question is: are fedora 35 flambda containers available somewhere? Would it be possible to add such?

Thanks for all your hard work.

It seems that Docker BuildKit frequently fails with missing output. We should fix this by either:

1. Fixing BuildKit so it doesn't fail, or
2. Allowing the log analyser to mark a build as needing to be retried and retrying it automatically some number of times.

I'm not sure where to post this exactly so I'll post this here for now.

In the effort of making a new CI the idea came up to make https://github.com/AltGr/Camelus/ an integrated part of it (as a module). In theory this should need only a bit of rewiring of the connexion with github but I'm not sure of the details, @talex5 might have a better idea.

cc @AltGr @thomasblanc @avsm

2021-09-14 11:09.25: New job: Analyse
2021-09-14 11:09.25: Checking out commit 80b2602f. To reproduce:
                       git clone --recursive "https://github.com/ocaml/opam-repository.git" && cd "opam-repository" && git fetch origin "refs/pull/19505/head" && git reset --hard 80b2602f
2021-09-14 11:09.25: Exec: "cp" "-a" "--" "/var/lib/ocurrent/var/git/opam-repository.git-a0bc4b41ddb868605dc1500002c27cfa79faf4389fe53b6754817ea16b48d458/.git" 
                           "/tmp/git-checkout1b9f9275"
cp: cannot stat '/var/lib/ocurrent/var/git/opam-repository.git-a0bc4b41ddb868605dc1500002c27cfa79faf4389fe53b6754817ea16b48d458/.git': No such file or directory
2021-09-14 11:09.28: Job failed: Command "cp" "-a" "--" "/var/lib/ocurrent/var/git/opam-repository.git-a0bc4b41ddb868605dc1500002c27cfa79faf4389fe53b6754817ea16b48d458/.git" 
"/tmp/git-checkout1b9f9275" exited with status 1

I've fixed it by restarting the job fetching the master commit. I'm guessing this came from some kind of job removing git commits when they are no space left or something or the sort?

Requesting the pipeline.svg on OCaml Docs CI results in a Stack Overflow.

Docker now gives this warning:

WARNING: experimental flag squash is removed with BuildKit. You should squash inside build using a multi-stage Dockerfile for efficiency

I suppose its usage in ocurrent, docker-base-images, should be removed.

When attempting to use a previously-built container with Docker.build or Docker.run, we should check that it still exists. If not, invalidate the cache entry so it gets built again.

(the Git plugin already does something similar in with_checkout)

For some build types, the logs can take up an excessive amount of disk space. Most logs compress well, and compressing a whole directory of similar logs should work even better. Logs are already organised in a separate directory for each day, so we can just compress the whole directory at the end of the day.

Also, these compressed directories should eventually be deleted.

There is one slight problem: logs are organised by their start date and might not be complete when it's time to compress. Instead, put in-progress logs in a temporary directory and copy them to their final located when done, using the end date instead.

The compressed archive needs to be indexed so that we can fetch archived log data quickly.

@kit-ty-kate has done something similar for opam-health-check and might have more ideas.

The Docker terms fail with a somewhat cryptic error message if docker is not available on the host system. Thanks to @gs0510 for pointing this out.

2019-11-27 16:59.40: New job: "docker" "pull" "ocaml/opam2:ubuntu-19.04"
2019-11-27 16:59.40: Exec: "docker" "pull" "ocaml/opam2:ubuntu-19.04"
2019-11-27 16:59.40: Job failed: Command "docker" "pull" "ocaml/opam2:ubuntu-19.04" exited with status 127

It would be nice to have a more helpful "try installing Docker" message here. There may be similar issues with Current_git stages.

The optional parameter gref of Current_git.clone is set to master by default. As more and more repositories switch to other default branch names (such as main), I believe this optional parameter could have a better behavior when it's not set, by automatically figuring out the default branch of the repository instead of using master.

• Provide a sort/filter on the jobs page
• Link back to the pipeline a job belongs to
• Refresh UI on changes to job state
• Rebuild all jobs or rebuild all failed jobs.

See https://ci.ocamllabs.io:8100/jobs

I've been using ocurrent for a little over a month now, and found it incredibly useful and helpful (thank you all!).

One thing I'd quite like to do, but been unable to figure out, is pass additional arguments to docker build. More specifically, I'd like to impose a CPU and memory limit on builds (via the --cpus and -m flags).

I'd be happy to put together a PR which adds a ?build_args argument to S.DOCKER.build (similar to run's run_args), I just wanted to get people's opinions on this, and check there wasn't an existing approach I was missing?