Documentation

This is a Work In Progress. Thank you for your patience as we update it with detailed documentation covering each of the OCaml-CI pages.

OCaml-CI is a Continuous Integration tool for OCaml projects. It helps the development process by automatically testing code against several versions of the OCaml compiler and various operating systems.

Continuous Integration (CI) is an automated testing process used to monitor changes to a central repository of code. A CI performs a series of automated steps (or jobs), e.g., building, testing, and deploying code. Developers can confidently and regularly integrate code into the central repository, relying on the CI system's automated build to detect and fix problems early.

OCaml-CI uses metadata from the project’s opam and dune files to work out what to build, and it also uses caching to make builds fast. It takes the information in the project's opam files to automatically test against multiple OCaml versions and OS platforms.

In Getting Started, you installed the the OCaml-CI app, which follows this pattern when deployed:

• First, it retrieves the list of organisations that have installed it.
• For each organisation, it gets the list of repositories to check.
• For each repository, it gets the branches and PRs to check.
• For each target, it fetches the head commit, generates a Dockerfile, and builds it.

The generated Dockerfile first adds all the *.opam files found in the project, then uses opam to install all the dependencies, and finally adds the rest of the source files. This means that rebuilds are often very fast, because Docker will reuse the previously cached build step as long as the opam files don’t change.

A key point is that OCaml-CI will run an opam solve for all of the dependencies in all the*.opam files together.

For example, given a project with the following opam files:

 bondi.opam
 bondi-lwt.opam
 bondi-async.opam
 

OCaml-CI will run an opam solve for all of the dependencies in these three opam files. So, if you depend on OCaml 5 in bondi-lwt.opam, then a build will only happen on that version of OCaml, even if the other two packages could run on earlier versions of OCaml.

The following terms are used across OCaml-CI, so it's best familiarise yourself with them before reading through this documentation.

Git forge : A service that hosts Git repositories. Currently GitHub and GitLab are supported.

Project : Used loosely to describe software written in OCaml.

Organisation : An organisation that owns projects that they want to build. This typically corresponds to an account on a Git forge.

Repository : The Git repository that houses the project.

Build : A collection of steps that correspond to the complete set of actions taken by OCaml-CI when it is run against a project. The build status of a branch or pull request usually corresponds to the outcome of the build of the latest commit of that branch or pull request.

Pipeline : An automated series of actions (steps) necessary for a build.

Ref : Git branches that exist within the repository. There is thus the concept of a default ref (typically main or master) and those that correspond to pull requests or merge requests.

Step : A step or a job is a unit of work. For example, building a project on platform-x-with-compiler-version-y-with-opam-version-z or linting a project via OCamlFormat. The first step of a build is the analysis step which plans the rest of the steps of the build.

Running time : The running time of a step is the duration for which a step runs. That is, the time elapsed between the point where it starts running and the point where it stops.

The build run time is the sum of the time taken by the analysis step and that of the longest running step of the build. It represents the 'wall clock' time of the build

The index page of OCaml-CI lists all organisations that OCaml-CI knows about, across both GitHub and GitLab. The logo of the Git forge is used to indicate the source of the organisation. Note that if no repositories are registered with OCaml-CI, the organisation will not be listed.

A filter is provided to select between GitHub or GitLab. Search is also available.

To see its repositories, click on the organisation.

The Repositories Page

Along with a listing of an organisation's known repositories, this page shows an overview and some key metrics of each repository's default branch.

Salient information regarding each repository

Each repository is shown with a snapshot of information relevant to the builds of the repository's default branch

1. The status of the latest build
2. The commit of the latest build
3. The date and time of the latest build
4. A graph depicting build history
5. The average time that a build takes to complete
6. The reliability of the build
7. The frequency of builds

Build history

The chart visualises the last 15 builds that have run on the default branch. The height of each bar reflects the build's running time, and its status is represented by its colour. The tooltip on hover provides more information about the build, and clicking on a bar will take you to the corresponding build page.

Average running time

The average running time of all the default branch's completed builds. (See the documentation of the Build Page for a precise definition of running time).

Reliability

The percentage of builds of the default branch that passed.

Frequency

The number of builds of the default branch per week.

To see all the Git branches of the repository, click on the row.

The Refs Page

The refs page (of a repository) shows all the repository's Git branches built on OCaml-CI. Each row corresponds to a ref, showing information about its latest build. Refs are separated into three groups: the default branch of the repository, branches that do not correspond to pull or merge requests, and branches that do.

Salient information regarding each ref

Each ref is shown with a snapshot of information relevant to its latest build

1. The status of the latest build
2. The name of the ref
3. The message corresponding to the latest commit
4. The latest commit
5. The pull or merge request that the ref corresponds to
6. The date and time of the latest build
7. The running time of the latest build

To see the latest build of a ref, click on the row.

The Build Page

The build page is typically the most important page of a CI. It shows the results of running the CI on a commit and thus helps determine the repository's health following the additional changes introduced by the commit. It shows the overall status as well as an overview of the steps that constitute the build. When examining a commit on GitHub or GitLab, clicking on the commit's build status will bring you to this page.

Salient information regarding a build

1. The status of the build
2. The commit that the build corresponds to
3. The date and time that the build was created
4. The branch (or branches) that the commit belongs to
5. The 'wall clock' run time of the build

Build history button

To see the history of builds for the ref, click this button.

Running time for a build

A build's running time is defined to be the sum of the time taken by the analysis step and that of the longest running step of the build. It represents the 'wall clock' or elapsed time of the build.

Cancelling a running build

The 'Cancel' button allows the cancellation of a build that is running or that has not yet started.

Rebuilding

A build that has stopped or finished can be rebuilt.

There are two options available on clicking the 'Rebuild' button. 'Rebuild all' rebuilds all steps (except for analysis). 'Rebuild failed' only rebuilds the failing steps.

Steps

The table of steps shown on the build page provides summary information of each build step. Each table row corresponds to a step; it shows the name of the step, its build status, the date and time created, the time spent in an enqueued state, and its running time.

To see the details of a step, including its logs, click on the row.

The Step Page

The step page shows the results of running a step on OCaml-CI. If a step has failed, typically the step's page will help understand and diagnose the cause of the failure. The logs displayed on this page contain details of exactly what OCaml-CI did to execute the step and will contain information such as stack traces and/or error messages to help identify the issue.

Salient information regarding the step

The step page has the following key pieces of information:

1. The build status is indicated as a green tick for Passed, a red cross for Failed, a yellow circle for Running, and a grey circle for Unknown.
2. The title describes the step - e.g., platform-x-with-compiler-version-y-with-opam-version-z or linting a project.
3. There are timestamps to indicate when the step was created and when it finished.
4. The amount of time that the step was enqueued (before it started running) appears between these timestamps.
5. The specific commit that was built is hyperlinked to the commit on the relevant Git forge.
6. The time it took to run the step appears on the right of the top matter and refers to the total time the step ran.

The logs

Logs are generated during the build process. To refer to a specific portion of the logs, click on the first line of interest, hold the Shift key, and click on the last line of interest. This will result in highlighting a block of lines. To copy a step page's URL that highlights exactly these lines, click on the button that appears in the bottom left of the highlighted block.

Steps to reproduce

When the logs have finished streaming, the section of the logs that contain code for reproducing the build are extracted to a section above the logs. This section is closed by default, but it can be opened by clicking on 'Steps to Reproduce.' The code in this section can be copied to the clipboard by clicking on the 'Copy code' button and executed as a shell script.

Rebuilding and cancelling a step

When a step is running, it can be cancelled by clicking on the Cancel button.

Similarly, when a step has finished, it can be rebuilt by clicking on the Rebuild button.

The History Page

The history page shows the build history of a ref. It shows every commit of that branch built by OCaml-CI. Each row that appears on this page corresponds to the latest build of a commit, and clicking on a row takes you to the build page of that commit. Each row identifies the commit, its build status, the time at which the build started, and the running time of the build.