By Daniel Hines
Esy is an indepensible part of our workflow at Marigold. Its Nix-inspired sandboxed builds give us a lot of flexibility in sourcing and building and our codebases, which is essential as we utilize the yet-to-be-released OCaml multicore and package things for more exotic environments Windows and OSX.
My experience with has been that Esy really is the easiest, most “works-out-of-the-box” way to get started with an OCaml project. You’re not likely to run into difficult-to-debug issues if you stick to the main path, which has been well-polished by the Esy maintainers.
When I first saw the error, I didn’t know where it was coming from or how to begin troubleshooting it. Yet with some help from my coworkers, we solved it quite easily. Along the way, I learned a few tricks about Esy, the GCC toolchain, and how to debug issues with native code. In this article, I’ll share them with you in a simplified Esy project.
Because the topics of GCC, Linux binaries, etc. are quite deep, I cannot cover them here with the detail they deserve. Instead, I will link to good entrypoints into these topics and show how they applied to a particular OCaml toolchain issue.
Learning how to create a new Esy project is a valuable thing in itself, but it can be tricky if you’re new to it, so we’ll review the process here.
Dune requires at least a single empty opam file, so create one.[^1]!
Install our dependencies, build the project.
If you’re new to the OCaml stack, all this can be a bit overwhelming, The relationship between Esy, Dune and OPAM is not always clear. For this reason, I highly recommend the reading getting started tutorial, which explains Esy’s sandboxing process.
Suppose we want to automate the use of our new web server in some way. In my case, I was working on the Deku sidechain, which has a simple shell script to automate bringing up a cluster of nodes running on different ports for development.
However, if you try to execute this binary, you'll get our shared library error:
This kind of error hints that something can't find a C library it expects. To solve it, we'll need to understand a bit about how binaries work on Linux.
You’ll see the file is a symbolic link. It’s actually a link to a link, so let’s follow it recursively:
We can see that the file is an executable structured in the the Executable and Linkable Format (ELF). ELF is a format for providing the kernel with hints about how a binary should be executed, and is the standard format used many Unix-like systems.
The details of the ELF format are beyond what I can cover here - I recommend this brief guide for an overview of the structure. Additionally, this guide covers many of the commands we’ll use today in more detail.
This will list all the system libraries. You can add to this list by installing libraries with your system package manager; however, you can only install one version at a time. This is not convenient for development, where you may want to experiment with many versions a library in different packages (even different versions of the same package).
You can see that several libraries are linked to the Esy sandbox versions instead of my system-installed versions. This could have caused problems if there two versions were incompatible.
Our web server works again[^2]!
But it would be even nicer if our binary had no runtime dependencies beyond the basics provided by Linux. We can achieve this through the process of static linking. Static linking is analogous to what Webpack and Rollup do in the frontend world:the program and all its required libraries are bundled together into a single file.
However, statically linking Ocaml code with Esy is a bit involved. We’ll leave the process for a follow-up post.
Have fun compiling!