Open recursion, in the context of object-oriented programming (OOP), refers to the ability of a method on an object to call another method on the same object (“self”), with the implementation of the second method not being fixed. So, for example:
class Greeter: def greet(self): return 'Hello, ' + self.addressee() def addressee(self): return 'World' g = Greeter() print(g.greet())
Hello, World
Now, if we derive a subclass from Greeter and override one of its
methods:
class Greeter: def greet(self): return 'Hello, ' + self.addressee() def addressee(self): return 'World' class NameGreeter(Greeter): def __init__(self, name): self.name = name def addressee(self): return self.name g = NameGreeter('Bob') print(g.greet())
Hello, Bob
So, overriding the addressee method changed the behavior of the
greet method. In other words, the greet method from the parent
class called the addressee method of the subclass, even though that
method doesn’t even “know” that the NameGreeter class exists.
One salient use case for open recursion is in Abstract Syntax Tree (AST) transformers. This arises in the land of OCaml preprocessors, where syntax extensions are implemented by defining a class whose methods correspond to the mapping functions over different kinds of OCaml AST nodes.
The object system supports open recursion, which allows a transformer to be defined by inheriting from a default base class, whose methods are no-ops (they return the input AST node). Overriding just the method for the type of AST node the syntax extension is interested in works correctly because the inherited methods will call the new implementations instead.
You can read more about how open recursion facilitates writing AST transformers, and more generally about writing ppx’s: see whitequark’s excellent blog post.
Here’s the above Python code translated into OCaml:
class greeter = object (self) method greet = "Hello, " ^ self#addressee method addressee = "World" end class name_greeter name = object (self) inherit greeter as super method! addressee = name end let g = new name_greeter "Bob" let () = print_endline g#greet
Hello, Bob
Note the use of method! instead of method in the subclass. Akin
to the @Override annotation in Java, it asks the compiler to check
that you are overriding an existing method. If not, the compiler will
issue a warning, in case you have misspelled the method name, for
example.
Now, there’s a common feeling in the OCaml community that the OOP part of the language is largely subsumed by the module system, which is a powerful and versatile abstraction tool. One of the main claims that OOP still has over modules is that open recursion is not possible with modules. For example, the analogous code written using modules does not have the desired behavior:
1: module Greeter = struct 2: type t = unit 3: 4: let create () = () 5: let addressee _t = "World" 6: let greet _t = "Hello, " ^ addressee _t 7: end 8: 9: module Name_greeter = struct 10: type t = { name : string } 11: include (Greeter : module type of Greeter with type t := Greeter.t) 12: let create ~name = { name } 13: let addressee t = t.name 14: end 15: 16: let g = Name_greeter.create ~name:"Bob" 17: let () = print_endline (Name_greeter.greet g)
Hello, World
Because the definition of greet refers to the lexical binding of
addressee in force at the location of the definition, i.e., line
5. That means that overriding the definition of
addressee has no effect except on the behavior of calling
addressee itself.
It turns out we can accomplish open recursion using the module system, but we need to structure our types a bit differently than using the straightforward classes-to-modules translation.
First, we define a Greeter signature for the overall signature of
the class. Then, add the base “class” Greeter as a recursive
functor that takes a module of its own output type as input. That is
to say,
module type Greeter = sig type t type ctor val create : ctor -> t val addressee : t -> string val greet : t -> string end module Greeter (Self : Greeter) : Greeter with type t = Self.t and type ctor = Self.ctor = struct type t = Self.t type ctor = Self.ctor let create = Self.create let addressee _t = "World" let greet t = "Hello, " ^ Self.addressee t end
And now we can define the Greeter “class” as the Greeter functor
instantiated on itself:
module rec G : (Greeter with type t = unit and type ctor = unit) = Greeter (struct type t = unit type ctor = unit include (G : Greeter with type t := unit and type ctor := unit) let create () = () end) let () = let g = G.create () in print_endline (G.greet g) ;;
Hello, World
And now, we define the “subclass” Name_greeter:
type 'super name_greeter = { super : 'super ; name : string } module Name_greeter (Self : Greeter) : Greeter with type t = Self.t and type ctor = Self.ctor = struct type t = Self.t type ctor = Self.ctor module rec Super : (Greeter with type t := t and type ctor := ctor) = Greeter (Self) let create = Self.create let addressee = Self.addressee let greet (t : t) = Super.greet t end module rec NG : (Greeter with type t = unit name_greeter and type ctor = unit * string) = Name_greeter (struct include NG let create (super, name) = { super; name } let addressee t = t.name end) let g = NG.create ((), "Bob") let () = print_endline (NG.greet g)
Which prints:
Hello, Bob
Success! But at what cost…
Regular expressions are a popular and powerful tool for manipulating, validating, and parsing strings. You can use regular expressions to extract phone numbers, validate email addresses, and inflict pain and suffering to future maintainers of your code (including you!).
But regular expressions are a lot more powerful than you think! This post will describe an example of a way you can use regexps to perform computation, in particular, determining whether a number is prime 1. (This post assumes familiarity with basic regular expression concepts. If you’re not acquainted with regexps, a decent introduction to them lives here.)
I recently came across a regexp-based test for prime numbers2, which is what inspired this blog post.
In Perl:
sub is_prime { my ($number) = @_; return (1 x $number) !~ m/\A (?: 1? | (11+?) (?> \1+ ) ) \z/xms; } print join ' ', grep { is_prime($_) } 0..100;
The above program produces a list of primes up to 100, as expected:
2 3 5 7 11 13 17 19 23 29 31 37 41 43 47 53 59 61 67 71 73 79 83 89 97
How the hell does a regular expression, which simply matches some string pattern, calculate whether or not a number is prime? Let’s dissect the code, and we’ll see that it’s equivalent to a naive trial division algorithm.
Here’s a simplified and annotated version of the above code.
sub is_prime { my ($number) = @_; return (1 x $number) # a string with '1' repeated $number times !~ # negated regexp match operator m{ \A # beginning of string (?: # non-capturing group 1? # zero or one '1' | (11+) # two or more '1's \1+ # one or more repetitions of previous group ) \z # end of string }xms; # eXtended syntax, allows comments and spaces } print join ' ', grep { is_prime($_) } 0..100;
First, we construct a string consisting of $number occurrences of the digit
1. We could have selected any other character instead of 1; what’s
important is that the string has length $number.
The regexp itself matches any string of 1’s whose length is zero, one, or a
composite number. Therefore, we use Perl’s !~ operator, which returns a true
value if the input string does not match the pattern. That’ll tell us
whether or not $number is prime.
\A and \z specify that the match operator should return true only when
the entire string fits the pattern, instead of the default behavior, which is
to return true if any substring of the input fits the pattern.
1? is straightforward; it matches exactly zero or one occurrences of 1.
(11+) matches two or more occurrences of 1. Because of the parentheses,
it also captures the matching substring, which can then be referred to
later in the pattern using the \1 syntax (1 because it’s the first
capturing group).
\1+ then matches one or more occurrences of whatever was matched by
(11+?).
The length of the substring captured by (11+?) acts like a trial divisor.
The left-hand side of the alternation bar | is fairly simple. It takes care
of the edge cases of 0 and 1, which are not prime or composite.
(11+) acts as a trial divisor. It matches a substring of two or more 1’s,
capturing however many it matched into group 1. The length of that substring
is divided into the rest of the string.
\1+ tries to match a substring whose length is some whole number multiple
of the trial divisor. If it reaches exactly the end of the string (of length
$number), then $number must be divisible by the trial divisor.
In total, the right-hand side of the alternation bar | ends up matching two
or more occurrences of the trial divisor string. Therefore, if it matches the
whole string, then $number must be composite.
Let’s see how the regexp would work on a couple of example inputs.
'1' !~ m/\A (?: 1? | (11+) \1+ ) \z/xms;
The string '1', as well as the empty string, matches the left side of the
alternation bar, 1?. This is the special case (neither prime nor
composite).
'111111111' !~ m/\A (?: 1? | (11+) \1+ ) \z/xms;
We know the left-hand side of the alternation bar won’t match, so let’s
focus on the right-hand side: (11+)\1+.
For the capture group, (11+), the regexp engine will match it with the
first two 1’s, then the first three, the first four, etc. For each prefix
of the string matched by (11+), the engine will attempt to match the
remainder of the string against \1+.
When the engine attempts to match the pattern with (11+) corresponding to
11, it will then try to match the rest of the string with (11)+, i.e.,
some positive even number of 1’s. It will not be able to do so; after
consuming the first 8 characters of the input, the regexp engine will not be
able to match another occurrence of (11). However, it won’t be at the end
of the string yet, so the \z assertion fails and the match must backtrack.
When the engine attempts to match the pattern with (11+) corresponding to
111, it will try to match the remaining 6 characters against (111)+, and
will succeed because 3 divides 6 evenly. Therefore, the regular expression
matches and the !~ operator returns a false value, telling us that the
number 9 is composite.
'1111111' !~ m/\A (?: 1? | (11+) \1+ ) \z/xms;
Again, we’ll focus on the right-hand side of the alternation bar.
(11+) matches prefixes of length 2 or greater. For each of these
“divisors”, the engine will attempt to match one or more additional
occurrences of the prefix until the end of the string. Since none of (2, 3,
…, 6) divide evenly into 7, they cannot match the rest of the pattern
successfully.
How about the prefix of length 7, i.e., the entire string being matched by
(11+)? Well, since \1+ requires at least one more occurrence of the
captured group to appear before the end of the string, the engine will
reject the input even though 7 divides itself evenly.
Let’s return to the original version of the code:
sub is_prime { my ($number) = @_; return (1 x $number) !~ m/\A (?: 1? | (11+?) (?> \1+ ) ) \z/xms; } print join ' ', grep { is_prime($_) } 0..100;
In (11+?), the ? modifies the + quantifier, causing it to become
“non-greedy”. This means the regexp engine will try to match as few
occurrences of the preceding pattern as possible. With this modifier, the
engine will try using a divisor of two, then three, then four, and so on
increasing, rather than decreasing from $number down to two (which is the
default behavior for +). Since a composite number is much more likely to be
divisible by 2 than some large divisor, it is much more cost-effective to
start small and work upwards.
(?> ) prevents the subpattern contained within from backtracking. For
example, in (?> \1+ ), the regexp engine will match as many occurrences of
\1 as possible. If it fails to match the remainder of the pattern, it won’t
retry the match with fewer occurrences, since that would be futile–it
definitely won’t reach the end of the string (\z) after matching fewer
characters than before!
With these performance enhancements in place, the regexp-based primality test enjoys the speed of a sensible implementation of a naïve trial-division algorithm, if not a more sophisticated primality testing algorithm.
Regular expressions are more powerful than you think. In fact, they’re
strictly more powerful than what mathematicians formally define as “regular
expressions”, since they include features like backreferences (\1) and
look-ahead/-behind ((?=)).
That being said, never use regular expressions to test primality.
In previous parts, we:
Now, it’s time to put everything together. Our final plugin will provide us a way to execute brainfuck code in a buffer without leaving Emacs, and with the benefit of the type safety and performance of OCaml.
First, we need to modify our jbuild file from part 1.
1: (jbuild_version 1) 2: 3: (executables 4: ((names (plugin)) 5: (libraries (bf_lib ecaml)) 6: (preprocess (pps (ppx_here))))) 7: 8: (rule (copy plugin.exe ecaml-bf.so)) 9: 10: (alias 11: ((name plugin) 12: (deps (ecaml-bf.so)))) 13: 14: (alias 15: ((name runtest) 16: (deps ((alias plugin))) 17: (action (run emacs -Q -L . --batch --eval "(require 'ecaml-bf)"))))
Only the executables rule was modified, adding bf_lib (our interpreter
library) to the list of required libraries and adding a preprocess rule for
ppx_here, which provides the [%here] syntax we’ll be using with defun.
To begin with, we’re just going to allow Emacs to evaluate brainfuck code
through our interpreter, non-interactively (i.e., from Lisp code). We’ll
define a function named bf-eval that just wraps our library code:
1: open Ecaml 2: 3: let eval_program program input = 4: let program = Value.to_utf8_bytes_exn program in 5: let input = Value.to_utf8_bytes_exn input in 6: Bf_lib.Program.run' ~program ~input 7: |> Value.of_utf8_bytes 8: ;; 9: 10: let () = 11: defun [%here] (Symbol.intern "bf-eval") 12: ~docstring:"evaluate [program], a brainfuck program, given [input]" 13: ~args:[ Symbol.intern "program" 14: ; Symbol.intern "input" ] 15: (function 16: | [| program; input |] -> eval_program program input 17: | _ -> invalid_arg "wrong arity") 18: ;;
I split up the definition into two parts. eval_program wraps
Bf_lib.Program.run' and does the work of converting Emacs values (in this
case, strings) into OCaml values and vice versa.
The second part calls defun to register the function with Emacs, providing
a docstring and argument names and checking the number of arguments passed.
We can try running it:
jbuilder build @plugin emacs -Q -L _build/default/src --batch --eval "(require 'ecaml-bf)" \ --eval '(print (bf-eval ",+." "A"))' # B
The program ,+. simply inputs a byte, increments it, and prints the result,
so it transforms "A" into "B".
If we load the plugin into a normal (non-batch mode) Emacs, we wouldn’t be able to call the function as an M-x command, since we haven’t marked it as an interactive command yet.
Next, let’s write a function that evaluates the brainfuck code in the current
buffer. bf-eval-buffer will be an interactive command, so it can be called
via M-x bf-eval-buffer and will prompt the user for a
string to serve as the input to the program.
20: let eval_current_buffer input = 21: let program = 22: Current_buffer.contents () 23: |> Text.to_utf8_bytes 24: in 25: let input = Value.to_utf8_bytes_exn input in 26: Bf_lib.Program.run' ~program ~input 27: ;; 28: 29: let () = 30: defun [%here] (Symbol.intern "bf-eval-buffer") 31: ~docstring:"evaluate the current buffer as brainfuck code" 32: ~interactive:"sInput: " 33: ~args:[ Symbol.intern "input" ] 34: (function 35: | [| input |] -> 36: let output = eval_current_buffer input in 37: messagef "Output: %s" output; 38: Value.nil 39: | _ -> invalid_arg "wrong arity") 40: ;;
The argument to interactive, "sInput: ", causes the command to prompt the
user for a string as its first argument (the minibuffer will contain the
prompt Input:).
Next, we’ll define a major mode for this new feature, as well as a key binding so that users can easily access the command without using M-x.
42: let mode_name = Symbol.intern "bf-mode" 43: 44: let () = 45: (* define [bf-mode] as a major mode *) 46: let mode = 47: Major_mode.define_derived_mode ~parent:Major_mode.fundamental 48: [%here] 49: ~change_command:mode_name 50: ~docstring:"Major mode for interacting with brainfuck code." 51: ~initialize:ignore 52: ~mode_line:"brainfuck" 53: in 54: (* bind [C-x C-e] to [bf-eval-buffer] *) 55: let keymap = Major_mode.keymap mode in 56: Keymap.define_key keymap (Key_sequence.create_exn "C-x C-e") 57: (Command (Command.of_value_exn (Value.intern "bf-eval-buffer"))) 58: ;;
Our major mode doesn’t need anything to be initialized, so we simply pass
ignore to that argument. change_command determines the name of the mode,
i.e., the name of the command that sets the major mode, in this case,
bf-mode. docstring provides the documentation that is displayed by
describe-mode (C-h m), while mode_line sets the
display name of the mode that appears, unsurprisingly, in the mode line.
Additionally, we bind the sequence C-x C-e to
bf-eval-buffer in the keymap of our major mode, so that when the mode is
activated, pressing that key sequence will trigger the command
bf-eval=buffer.
bf-mode
Ecaml’s Auto_mode_alist module provides a clean interface for manipulating
the auto-mode-alist variable, which determines how Emacs decides which
major mode to use when you open a file, like starting c-mode when you open
a file named foo.c. We’ll register bf-mode for *.b files:
60: let () = 61: (* automatically start [bf-mode] upon opening a [*.b] file *) 62: Auto_mode_alist.add Auto_mode_alist.Entry.( 63: [ { delete_suffix_and_recur = false 64: ; filename_match = Regexp.of_pattern "\\.b\\'" 65: ; function_ = Some mode_name } ]) 66: ;;
The delete_suffix_and_recur field allows an entry to defer to another mode
when the file may have more than one extension.1 filename_match
specifies the regular expression that determines which filenames activate the
major mode. Finally, function_ specifies what mode to activate when the
file name matches filename_match2.
jbuilder build @plugin
emacs -Q -L _build/default/src --eval "(require 'ecaml-bf)"
Try it out for yourself!
All of the code’s on GitHub. Bug reports and patches welcome.
For example, you could have an entry that is activated by opening a file
with a .gz suffix, such as foo.c.gz. It might decompress the file and then
recursively activate the correct major mode based on the remainder of the
filename.
It actually doesn’t have to be a mode change command; it can just be any function.
Let’s write the core of our plugin, a brainfuck interpreter. We’ll make it a library so it can be easily compiled into our plugin later, but this step actually doesn’t involve Emacs or Ecaml at all.
If you’re not super interested in the interpreter code and just want to get to the Emacs/Ecaml part, skim the interface briefly and then move on to the next post.
Brainfuck is an esoteric programming language created in 1993 by Urban Müller, and notable for its extreme minimalism.
– Wikipedia
(The rest of this section is also based off of the brainfuck Wikipedia page).
The language has only eight commands, each denoted by a single character. All commands operate on a single, infinite array of zero-initialized bytes. There is also a single data pointer that points to some position in the array, plus an instruction pointer that keeps track of where we are in the program.
The commands are shown in the table below. Any characters in the source file other than the 8 commands are treated as comments and ignored.
(“Array element” refers to the element currently pointed to by the data pointer.)
| character | meaning |
|---|---|
> |
increment the data pointer |
< |
decrement the data pointer |
+ |
increment the array element |
- |
decrement the array element |
. |
print out the array element (byte) |
, |
read into the array element (byte) |
[ |
skip to ] if byte is zero |
] |
jump back to [ if byte is nonzero |
Reading and printing are based on bytes. For example, if the data pointer
currently points to an array element of (decimal) value 97, and the program
executes command ., the byte 0x61 will be printed, and it will show up in
your terminal as LATIN SMALL LETTER A.
If you’re familiar with C, you may find the following equivalence between brainfuck and C helpful.
We’ll assume the following main():
int main() { uint8_t memory[QUITE_BIG] = {0}; uint8_t *ptr = &memory[0]; // brainfuck code goes here }
| brainfuck | C |
|---|---|
> |
++ptr; |
< |
--ptr; |
+ |
++*ptr; |
- |
--*ptr; |
. |
putchar(*ptr); |
, |
*ptr = getchar(); |
[ |
while (*ptr) { |
] |
} |
For our simple implementation, we’ll assume that our array has a size of 106 bytes (1MB), and that the data pointer starts on the leftmost side of the array.
Let’s set up our workspace…
mkdir -p ~/ecaml-bf/lib/bf_lang cd ~/ecaml-bf/lib/bf_lang touch bf_lang.ml{,i} vi jbuild
I’m using Jane Street’s Core_kernel standard library overlay since I’m
familiar with its idioms, and it’ll already be installed since Ecaml depends
on it. Here’s our jbuild:
(jbuild_version 1) (library ((name bf_lib) (synopsis "An interpreter library for brainfuck.") (libraries (core_kernel))))
Here’s what our library’s interface will look like:
1: open! Core_kernel 2: 3: module Input : sig 4: (** Stateful input source. *) 5: type t 6: 7: val of_string : string -> t 8: 9: val of_chan : In_channel.t -> t 10: end 11: 12: module Output : sig 13: (** Stateful output sink. *) 14: type t 15: 16: val of_buffer : Buffer.t -> t 17: 18: val of_chan : Out_channel.t -> t 19: end 20: 21: module Memory : sig 22: (** A big byte array. *) 23: type t = Bigstring.t 24: 25: (** [create_fresh n] creates a zero-initialized memory of size [n] bytes. *) 26: val create_fresh : int -> t 27: end 28: 29: module Program : sig 30: (** An immutable representation of a brainfuck program. *) 31: type t 32: 33: (** "Compile" a program from its source code. *) 34: val parse : Input.t -> t 35: 36: (** [run] mutates [memory], consumes [input], and writes to [output]. *) 37: val run 38: : t 39: -> memory : Memory.t 40: -> input : Input.t 41: -> output : Output.t 42: -> unit 43: 44: (** A simple interface for [run]. *) 45: val run' 46: : program : string 47: -> input : string 48: -> string (* output *) 49: end
Input.t and Output.t represent possible sources of input and output for
our programs (and for the source of the program text). The of_* functions
convert other types into things our program can read from. For example, the
return value of Input.of_string s returns a value that keeps track of how
far into the string we’ve read.
Memory.t represents the working memory of the programs, i.e., a very large
byte array.
Finally, Program, the most exciting module of our interpreter, can be used
in one of two ways:
run, the lower-level interface, gives the user full control over the
input and output source of the program. It also allows the user to parse
the program just once and reuse the result multiple times.run', a simplified interface, simply accepts strings as input and
returns a string as output.
Memory needs to be a large array of bytes. While this sounds like a job for
type bytes, we’ll actually use Bigstring.t, which doesn’t have the same
size limitations1 as bytes.
Bigstring.t is a type alias for (char, Bigarray.int8_unsigned_elt,
Bigarray.c_layout) Bigarray.Array1.t from the bigarray library. You can
think of it as basically being a pointer to a big contiguous area of
malloc()’d memory.
We’ll also add a new function for creating a zero-initialized memory of a
given length. Bigstring.create by itself just allocates the memory, but
doesn’t initialize it, so it could have arbitrary contents.
35: module Memory = struct 36: type t = Bigstring.t 37: 38: let create_fresh n = 39: let memory = Bigstring.create n in 40: Bigarray.Array1.fill memory '\000'; 41: memory 42: ;; 43: end
Since we want our library to be fairly general-purpose, we’ll accept input
and produce output to a generic interface, with adapters for common
input/output sources/sinks such as in_channel or Buffer.t.
Input will be a closure that returns a char option (it returns None at
the end of input). We’ll also define an iter function that repeatedly calls
its argument with the next character from the input until the input is
exhausted.
3: module Input = struct 4: (* [t]'s return None at the end of input. *) 5: type t = unit -> char option 6: 7: let of_string str = 8: let i = ref 0 in 9: fun () -> 10: if !i >= String.length str 11: then None 12: else ( 13: let char = str.[ !i ] in 14: incr i; 15: Some char) 16: ;; 17: 18: let of_chan chan = fun () -> In_channel.input_char chan 19: 20: let rec iter t ~f = 21: match t () with 22: | None -> () 23: | Some c -> f c; iter t ~f 24: ;; 25: end
Output will be represented as a function that accepts a char. It’s a bit
simpler since we don’t have to keep track of any state (Buffer and
Out_channel do it for us).
27: module Output = struct 28: type t = char -> unit 29: 30: let of_buffer buf = fun char -> Buffer.add_char buf char 31: 32: let of_chan chan = Pervasives.output_char chan 33: end
(of_buffer could have been written as simply Buffer.add_char, and
similarly for of_chan, but I wrote them out all the way2 for clarity.)
Writing the lexer is straightforward. We can simply scan each character and
produce either a command token or None to ignore a non-command character.
46: module Command = struct 47: type t = 48: | Left 49: | Right 50: | Decrement 51: | Increment 52: | Input 53: | Output 54: | Loop_begin 55: | Loop_end 56: 57: let of_char = function 58: | '<' -> Some Left 59: | '>' -> Some Right 60: | '-' -> Some Decrement 61: | '+' -> Some Increment 62: | ',' -> Some Input 63: | '.' -> Some Output 64: | '[' -> Some Loop_begin 65: | ']' -> Some Loop_end 66: | _ -> None 67: ;; 68: end
A brainfuck program is simply a sequence of commands. Since the looping commands require the interpreter to jump to the matching bracket, we’ll record the positions of corresponding left and right brackets in a jump table. This is a hash table that maps the position of a bracket to the position of its partner, and vice versa.
70: type t = 71: { commands : Command.t array (* just the sequence of commands *) 72: ; jump_table : int Int.Table.t } (* maps matching brackets' positions to 73: each others' *)
The only job of the parser is to determine the positions of matching pairs of brackets. It should croak if it finds unmatched bracket pairs in the program text. We’ll define two functions to do that:
75: let extra_left = invalid_argf "unmatched left bracket at command index %d" 76: let extra_right = invalid_argf "unmatched right bracket at command index %d"
invalid_argf is a printf-like function; in this case, extra_left and
extra_right have types int -> unit -> 'a, i.e., they accept the current
position and a unit value, raising an exception with the location of a syntax
error. (“Command index” means the index of the character in program text
representing the unmatched bracket in question, if we ignore all non-command
characters.)
Here’s the whole parser:
78: let parse source_code = 79: (* a stack *) 80: let open_brackets = ref [] in 81: let commands = ref [] in 82: let current_pos = ref 0 in 83: let jump_table = Int.Table.create () in 84: let add_command cmd = 85: commands := cmd :: !commands; 86: incr current_pos 87: in 88: let push_left_bracket () = 89: open_brackets := !current_pos :: !open_brackets; 90: add_command Command.Loop_begin 91: in 92: let pop_left_bracket () = 93: match !open_brackets with 94: | [] -> extra_right !current_pos () 95: | hd::tl -> 96: Hashtbl.add_exn jump_table ~key:!current_pos ~data:hd; 97: Hashtbl.add_exn jump_table ~key:hd ~data:!current_pos; 98: open_brackets := tl; 99: add_command Command.Loop_end 100: in 101: Input.iter source_code ~f:(fun char -> 102: match Command.of_char char with 103: (* ignore non-command characters *) 104: | None -> () 105: 106: (* regular commands just get appended *) 107: | Some (Left | Right | Decrement | Increment | Input | Output as cmd) 108: -> add_command cmd 109: 110: (* record loop beginning positions *) 111: | Some Loop_begin -> push_left_bracket () 112: 113: (* record entries in jump table *) 114: | Some Loop_end -> pop_left_bracket ()); 115: 116: (* check for any remaining open left brackets *) 117: (match !open_brackets with 118: | [] -> () 119: | hd :: _ -> extra_left hd ()); 120: { commands = Array.of_list_rev !commands 121: ; jump_table } 122: ;;
Most of the code is fairly straightforward; most of the code is dedicated to matching bracket pairs, and other commands are just passed straight through.
The runner steps through the program text, executing commands on the given memory, input, and output. It basically simulates the abstract state machine that brainfuck code is meant to run on.
124: let run t ~memory ~input ~output = 125: (* get/set bytes in memory; bytes are unsigned, wrap around on overflow *) 126: let get pos = Bigstring.get_uint8 memory ~pos in 127: let set pos x = Bigstring.set_uint8 memory ~pos (x % 256) in 128: 129: (* we can actually always jump one past the matching bracket *) 130: let jump pc = 1 + Hashtbl.find_exn t.jump_table pc in 131: 132: (* pc :: program counter 133: pos :: data pointer *) 134: let rec loop ~pc ~pos = 135: match t.commands.( pc ) with 136: | Left -> loop ~pc:(pc + 1) ~pos:(pos - 1) 137: | Right -> loop ~pc:(pc + 1) ~pos:(pos + 1) 138: 139: (* parenthesized for clarity *) 140: | Decrement -> set pos ((get pos) - 1); loop ~pc:(pc + 1) ~pos 141: | Increment -> set pos ((get pos) + 1); loop ~pc:(pc + 1) ~pos 142: 143: | Input -> 144: (match input () with 145: | None -> () (* do nothing if we run [,] at EOF *) 146: | Some c -> set pos (Char.to_int c)); 147: loop ~pc:(pc + 1) ~pos 148: 149: | Output -> 150: output (Char.of_int_exn (get pos)); 151: loop ~pc:(pc + 1) ~pos 152: 153: | Loop_begin -> 154: if get pos = 0 155: then (loop ~pc:(jump pc) ~pos) 156: else (loop ~pc:(pc + 1) ~pos) 157: 158: | Loop_end -> 159: if get pos <> 0 160: then (loop ~pc:(jump pc) ~pos) 161: else (loop ~pc:(pc + 1) ~pos) 162: 163: | exception _ -> () (* reached end of commands, so we terminate *) 164: in 165: loop ~pc:0 ~pos:0 166: ;;
A couple decisions we’ve made above about edge cases:
, (input) command when we have no remaining input,
we simply ignore it (i.e., do nothing). Other common choices including
setting the current cell to 0 or -1.
We also define a simplified interface for running programs. This’ll be less
awkward to use for simple cases. create_memory allocates an array of 106
bytes and fills the buffer with 0x00’s. run' is simply a wrapper around
run that creates a fresh memory and accepts and returns everything as
string’s.
69: let create_memory () = 70: let memory_size = 1_000_000 in 71: let memory = Bigstring.create memory_size in 72: Bigarray.Array1.fill memory '\000'; 73: memory 74: ;;
168: let run' ~program ~input = 169: let t = parse (Input.of_string program) in 170: let buffer = Buffer.create 16 in 171: let memory = create_memory () in 172: let input = Input.of_string input in 173: let output = Output.of_buffer buffer in 174: run t ~memory ~input ~output; 175: Buffer.contents buffer 176: ;;
Let’s try out our interpreter. jbuilder has an awesome utop command which
starts a toplevel3 with access to the library we’ve just built (--dev
enables stricter compilation flags).
# chdir to project root jbuilder utop lib/bf_lib --dev
utop # Bf_lib.Program.run' ~input:"" ~program:"++++++++[>++++[>++>+++>+++>+<<<<-]>+>+>->>+[<]<-]>>.>---.+++++++..+++.>>.<-.<.+++.------.--------.>>+.>++.";; - : string = "Hello World!\n"
Awesome. We have a working brainfuck interpreter library! See Wikipedia for more example programs to try out.
In the next post, we’ll see how to use Ecaml’s API to interact with various components of Emacs, such as buffers, files, key bindings, and more. Stay tuned!
The maximum length of a string or bytes is given in OCaml by
Sys.max_string_length. On 64-bit systems, it’s 144115188075855863, which is
surely big enough, but on 32-bit systems it’s 16777211 (about 16 MB), and we
might want to give a program a bigger memory than that in the future.
Or, if you prefer, eta-expanded.
OCaml’s term for a REPL (Read-Eval-Print Loop), i.e., interactive prompt.
Similar to python or irb.
This time, we’ll take a look at the Ecaml modules used to interact with different components of Emacs. These modules define type-safe interfaces that represent buffers, font faces, and various Elisp data structures like hash tables and vectors.
This post reflects the state of the Ecaml library as of version v0.9.115.24+69 (git commit 25a9f825). It is, of course, being actively developed and improved upon, and some details may change between when this post was written and now, when you are reading it.
This post won’t cover everything in the Ecaml library, since it’s quite big1, but it should help give you a general idea for how it’s organized. Most of the individual functions correspond to one or two Elisp functions for which more information can be found by reading the corresponding section of the Emacs Lisp manual.
For example, Merlin reports the following type and documentation comment for
Ecaml.Buffer.find, which links to the corresponding Emacs Manual section:
(* [find ~name] returns the live buffer whose name is [name], if any. [(describe-function 'get-buffer)]. *) val find : name:string -> Buffer.t option
Evaluating (describe-function 'get-buffer), either manually or by invoking
describe-function interactively (C-h f or
SPC h d f for Spacemacs) brings you to the corresponding
documentation:
get-buffer is a built-in function in ‘C source code’.
(get-buffer BUFFER-OR-NAME)
Return the buffer named BUFFER-OR-NAME. BUFFER-OR-NAME must be either a string or a buffer. If BUFFER-OR-NAME is a string and there is no buffer with that name, return nil. If BUFFER-OR-NAME is a buffer, return it as given.
The correspondence between the behavior of Ecaml.Buffer.find and
get-buffer should be apparent: find returns Some buffer if there exists
a buffer with the given name, otherwise None.
Ecaml module
The top-level module of the Ecaml library is named, unsurprisingly, Ecaml.
It contains all of the other modules for interacting with more specific areas
of Emacs functionality, as well as some commonly used functions directly in
the Ecaml module:
module Ecaml : sig module Advice = Ecaml__.Advice (* snip *) module Working_directory = Ecaml__.Working_directory val defadvice : ?docstring:string -> ?position:Advice.Position.t -> Lexing.position -> advice_name:Advice.Name.t -> for_function:Symbol.t -> (args:Value.t list -> inner:(Value.t list -> Value.t) -> Value.t) -> unit val defun : (Symbol.t -> Function.Fn.t -> unit) Function.with_spec val defcustom : Lexing.position -> Symbol.t -> Ecaml.Customization.Type.t -> docstring:string -> group:Ecaml.Customization.Group.t -> standard_value:Value.t -> unit val defvar : Lexing.position -> Symbol.t -> Value.t -> docstring:string -> unit val define_derived_mode : ?parent:Ecaml.Major_mode.t -> Lexing.position -> change_command:Symbol.t -> docstring:string -> initialize:(unit -> unit) -> mode_line:string -> Ecaml.Major_mode.t val inhibit_messages : (unit -> 'a) -> 'a val message : string -> unit val message_s : Core_kernel.Sexp.t -> unit val messagef : ('a, unit, string, unit) Core_kernel.format4 -> 'a val provide : Symbol.t -> unit val inhibit_read_only : (unit -> 'a) -> 'a end
Let’s go through a couple of the most important functions directly under
Ecaml:
defun
defun allows you to define a named function, visible to any Elisp code, but
whose body consists of an implementation in OCaml.
Its full signature, with the Function.with_spec type alias spelled out, is:
val defun : ?docstring:string -> ?interactive:string -> ?optional_args:Symbol.t list -> ?rest_arg:Symbol.t -> Lexing.position -> args:Symbol.t list -> Symbol.t -> Function.Fn.t -> unit
That’s a lot to unpack, so we’ll take the arguments one at a time:
docstringdescribe-function when you do C-h f in
Emacs, so they’re very useful for functions that are
intended to be invoked by a user and not a program.interactiveinteractive is a special form in Elisp that turns a
function into a command. It’s really complicated so I
shan’t describe it here, but it basically does some magic
so that you can prompt the user for input and your
function receives that input as arguments.optional_args and rest_argdescribe-function so you should still pick good names.Lexing.position
defun requires a Lexing.position argument, which
describes where in your OCaml code the function was defined. This shows
up in the describe-function output so that you can easily figure out
where the OCaml implementation lives.
ppx_here, a syntax extension, provides a handy shortcut for specifying
the current source location: simply write [%here] where you need a
Lexing.position.2, 3
argsoptional_args and rest_arg, just provides names for the
formal parameters for your function. See above.Symbol.tfoo, any Elisp code
that contains, say, (foo), will call your function.Function.Fn.tEcaml.Value.t array ->
Ecaml.Value.t, i.e., it should accept some number of arguments and
return a valid Elisp value. Unfortunately, there’s no way for the
type-checker to guarantee that you’ll get the right number of arguments
(this is basically a Lisp function, after all), so you will have to
handle any arity errors.
Here’s an example of a simple function being defined using defun.
open Ecaml let () = defun [%here] (Symbol.intern "say-hello") ~optional_args:[ Symbol.intern "name" ] ~args:[] (function | [| name |] -> let name = if Value.is_nil name then "World" else (Value.to_utf8_bytes_exn name) in Value.of_utf8_bytes ("Hello, " ^ name ^ "!") | _ -> invalid_arg "wrong arity") ;; let () = provide (Symbol.intern "ecaml-bf")
The way Elisp treats optional arguments is not, as you might guess, that the
function might receive zero or one argument, but rather that the name
argument might be nil if no argument was provided. We check that to
determine whether to use a default value, and then return an appropriate
greeting.
We can test it like so:
alias eb="emacs -Q -L _build/default/src --batch" eb --eval "(require 'ecaml-bf)" --eval '(print (say-hello))' # "Loaded Ecaml." # # "Hello, World!" eb --eval "(require 'ecaml-bf)" --eval '(print (say-hello "Emacs"))' # "Loaded Ecaml." # # "Hello, Emacs!"
If you’re familiar with Elisp, it might be helpful to see how corresponding Ecaml and Elisp code compare, so here’s the above example written in Elisp.
(defun say-hello (&optional name) (let ((name (or name "World"))) (concat "Hello, " name "!")))
Okay, so it’s a lot shorter :sweat:. But that’s okay, because most of the
code we write won’t be just tons of boilerplate wrapping trivial OCaml
functions. Instead, the point of Ecaml is to let OCaml do the heavy lifting,
so we only need to define the interface for our plugin using defun and
then we can hack on whatever interesting functionality we want to provide.
defvar and defcustom
These functions allow you to define Elisp variables4, similarly to what
defun does for functions. The difference between them is that defcustom
defines a customizable variable. The Elisp manual, section 14.3:
“Customizable variables”, also called “user options”, are global Lisp variables whose values can be set through the Customize interface. Unlike other global variables, which are defined with ‘defvar’ (*note Defining Variables::), customizable variables are defined using the ‘defcustom’ macro. In addition to calling ‘defvar’ as a subroutine, ‘defcustom’ states how the variable should be displayed in the Customize interface, the values it is allowed to take, etc.
So, what does that mean for your plugin? Well, basically it just means that the variable can be set and queried interactively by the user using the Emacs Customize interface.
message and co.val message : string -> unit val message_s : Core_kernel.Sexp.t -> unit val messagef : ('a, unit, string, unit) Core_kernel.format4 -> 'a
The next three functions are simply various ways of printing messages to the echo area. The echo area is the tiny area at the very bottom of the Emacs window, where messages will appear, such as “(No changes needed to be saved)” when you try to save a file you already just saved.
The Elisp function message displays a new message in the echo area.
Messages are also appended to the special *Messages* buffer, so you can
view that buffer to see any messages you may have accidentally dismissed.
message and message_s accept a string and a Sexp.t5,
respectively. (Sexp.t is from Jane Street’s sexplib.)
messagef also prints to the echo area but it accepts a formatting string
and arguments the same way printf does, e.g.,
let () = messagef "the answer is %d" 42
provide
provide registers a feature with Emacs, basically telling Emacs that the
plugin was successfully loaded. This is needed in order to load the plugin
using require (or Emacs will complain). It also prevents Emacs from trying
to load the same plugin again, since require checks to see if its argument
has been registered as a feature before loading a plugin. Call provide when
your plugin is finished setting up.
What follows is a listing of the current modules in Ecaml (as of version v0.9.115.24+69), each with a brief summary of the functionality provided within.
Many of the modules correspond one-to-one with concepts in Emacs better-documented in the manual than I can do here. Module names are linked to the appropriate section of the Emacs or Emacs Lisp manuals.
auto-mode-alist, which determines
how Emacs decides what major mode to open a file in, based on the
filename.make-backup-files, which controls whether
Emacs will make backup files (like foo.c~) when you edit files.message and co. print their messages here, at the bottom of
Emacs’s frame. Also allows you to temporarily inhibit messages in the
echo area (but they’re still logged to *Messages*).find-function jumps to the source code where a function
is defined, given its name. It’s not built-in as part of
Emacs, but rather is defined by the Find Func package.Form contains functions specifically related
to treating Lisp values as code (e.g., eval and quote).Function.Fn.t is the common signature
for all OCaml functions that are to be called from within Emacs.grep from within Emacs. grep is run
asynchronously, and the results are collected in the *grep* buffer.before-save-hook to
remove trailing whitespace from a file you’re editing before you save it.Input_event.modifiers to determine whether the
Control key was held down while a key was pressed.execute-kbd-macro.path,
which allows you to examine the load path, which is a list of directories
where Emacs will look for the file name you pass to load.Best described by an excerpt from the doc comments:
(* [update_emacs_with_entries] updates [load-history] with the information supplied to [add_entry], which make it possible to, within Emacs, jump from a symbol defined by Ecaml to the Ecaml source. *)
Used behind-the-scenes by defcustom, defun, defvar,
and so on.
intern and read. Normally there is only one, stored in
the variable obarray, so there is only one symbol with a given name.Point contains
functions for setting the point and searching forward for a given string,
for example.ispell, or
Merlin. This also includes the Emacs server.Q.K) and ampersand-symbols like &optional (module Q.A). These are
all stored here to avoid unnecessarily allocating OCaml data structures
to refer to these symbols whenever they’re needed.Current_buffer, but for Emacs’s “windows”.
Identifiers in Elisp are converted to symbols, which are
“interned”. This means that two symbols with the same name9 are
physically the same object, and so can be compared using pointer
arithmetic. Module Symbol also contains functions for calling Elisp
functions from OCaml, with convenience functions for different arities.
For example, Symbol.funcall1 accepts a symbol, which should
denote the name of a function, and a single argument, which is passed to
the function. Functions in this module whose names end in _i ignore the
return value and return unit instead of Value.t. These are useful
since a lot of Elisp functions with side effects don’t return anything
useful.
Most of the other modules' functionality are built on top of
Symbol and its function-calling functions.
Value.t. Contains a lot of type predicates that allow you
to test the type of an arbitrary value.Working_directory.within runs a function with the current working
directory set to a given value.That was quite a few modules. We’ll use some of them next week in building our interpreter plugin. Catch you next time.
Plus, I have no idea what many of the individual pieces do.
ppx_jane, Jane Street’s set of ppx rewriters, includes ppx_here, so
if you already have that installed, you can use it to provide [%here]. You’ll
need to add the following to your jbuild file:
(jbuild_version 1) (executables ((names (main)) (libraries (ecaml)) (preprocess (pps (ppx_jane))))) ; add this line
If you’re not keen on using syntax extensions, you can use OCaml’s
built-in __POS__ macro. Unfortunately, this macro returns a tuple instead of a
record, but since all the fields are in the right order, you can do Obj.magic
__POS__ to get a Lexing.position. But don’t tell anyone I told you that!
Elisp is a Lisp-2, meaning that functions and variables live in two different and non-overlapping namespaces. That’s why defining functions and variables uses different mechanisms (unlike in, say, OCaml!).
See? Interfacing OCaml with Lisp isn’t that weird after all!
In fact, this was one of the key original motivations for Ecaml.
Surprisingly, faces in Elisp live in a completely separate namespace from variables and functions. So perhaps Elisp is a Lisp-3?
For example, the grouping operator is \( ... \), not ( ... ). Plain
old parentheses just match literal parentheses in the target string.
in the same obarray.
Why on Earth would I write such a thing when there’s literally a Watch Later feature built into YouTube itself? Well, I guess it must be my weird consumption habits.
At one point I had over 2800 videos on my YouTube Watch Later playlist, which became super unwieldy. YouTube playlists only load 100 videos at a time, and my Watch Later list was in a totally arbitrary order and completely unsearchable.
The main modules are App::WatchLater, which controls most of the database and
command-line options, and App::WatchLater::YouTube, which is an interface to
the YouTube Data API.
The script watch-later maintains a SQLite database of videos, including the
video ID, title, and channel name and ID. It also keeps track of which have been
watched and which have not. (More fields may be added in the future, e.g., video
duration.)
Then, using the SQLite command-line tool, it is easy to query for videos I’ve bookmarked from a given channel or with a given keyword in the title, so that I can watch something appropriate for the mood I’m in.
Since the YouTube Data API no longer allows access to the user’s WL (Watch
Later) playlist items, it was not possible for me to directly retrieve all of
the videos in my queue and then clear out my queue.
Instead, I had to write another script to scrape the playlist page for video IDs, 100 at a time, and then another bit of JavaScript to simulate clicking the “remove video from playlist” button for those 100 videos. Lather, rinse, repeat.
Now, just 2420 to go. If only I had the willpower to take care of the root cause…
]]>There was recently a thread on the OCaml Discourse forum about sample Emacs plugins using the Ecaml library. Since the library doesn’t seem to have been getting that much use, and since I have some personal interest1 in seeing it used more, I decided to help by writing a Getting Started guide. Here goes.
This is part 1 of a series (next). We’ll see why you might use Ecaml to write a plugin, and get it set up and running with a simple Hello World. All of the code for this series is available on GitHub.
(This guide has only been tested on macOS 10.12, but probably works on other Unices like Linux or OS X. Unfortunately I have no idea about Windows.)
Emacs is a great text editor operating system habitat. Emacs’s power and
popularity arguably stem from its ability to be endlessly customized and
enhanced by plugins (i.e., extensions, packages, modules)—for example, I’m
writing this post in org-mode while editing its YAML metadata (in the same
file) in yaml-mode. These two editing modes are able to play nicely in the
same buffer thanks to a package called mmm-mode (“multiple major modes”…
mode).
In this series, we’re going to write an interpreter for a minimalist Turing-complete programming language. We want to be able to run the code we’re editing directly within Emacs, instead of switching to our shell and running it from there, so we’ll embed our interpreter as a plugin in Emacs.
In the past, Emacs plugins have all been written in a language called Emacs Lisp (or Elisp), which, as its name suggests, is a Lisp. Elisp is a perfectly nice language, especially for writing the sort of code you might find in a plugin, but it has a couple of drawbacks. The main one I’m interested in is speed.
Ya see, Emacs’s Lisp interpreter ain’t exactly the fastest. A naïve speed comparison between Elisp and Ruby, for dramatic effect:
(benchmark-run (print (loop for i below 1000000 sum i))) ;; 499999500000 ;; (0.389676 0 0.0)
time ruby -e 'p (0...1000000).reduce(&:+)' # 499999500000 # ruby -e 'p (0...1000000).reduce(&:+)' 0.17s user 0.03s system 81% cpu 0.236 total
Both of these snippets just iterate from 0 to 999,999, adding up
The Elisp code takes quite a bit longer to add up the natural numbers less
than one million, and that’s including the startup time of the ruby
interpreter! Now, to be fair, when compiled, the above Elisp code only takes
around 0.125 seconds, but that’s still substantially slower than compiled
languages like C or Java (or OCaml).
Emacs 25 added the ability to load native code plugins. Native code refers to code that runs directly on the processor, instead of being interpreted by another program, like Emacs’s Lisp interpreter or Ruby’s virtual machine.
There’s quite a good introduction to the basics of writing native plugins
here, but the gist of it is that Emacs exposes a small C API which you can
write modules against. Compiling them to shared objects (i.e., dynamically
linked libraries, .so’s, .dll’s, etc.) allows Emacs to dynamically load
the plugin at runtime.
OCaml is a statically-typed functional programming language with a great deal of versatility, contrary to what you might have first expected when you just read the words “statically-typed” and “functional”. Of course, since you’re reading this blog post, I assume you are familiar with OCaml and won’t belabor singing its praises.
I’ll just point out that OCaml has a quite nice way to interface with C code using its foreign function interface (FFI), which means we can use OCaml to write native plugins for Emacs, as long as we have some library to provide the interface between the Emacs C API and our OCaml plugin. That’s where Ecaml comes in.
Ecaml is an open-source OCaml library for writing native Emacs plugins, available on OPAM or Jane Street’s GitHub page. We’ll use Ecaml to write our Emacs plugin, in OCaml instead of Elisp.
mkdir ~/ecaml-bf cd ~/ecaml-bf git init # or hg, if you prefer touch jbuild-workspace
We’ll use the development version of Ecaml, since the version in the OPAM repository is significantly outdated. Jane Street hosts an OPAM repo with the development versions of their open-source libraries, so let’s add the repo to OPAM:
opam switch 4.05.0
opam repo add janestreet-bleeding https://ocaml.janestreet.com/opam-repository
opam install ecaml
We’ll start with Hello World for now (src/plugin.ml):
let () = Ecaml.message "Hello from OCaml"; Ecaml.provide (Ecaml.Symbol.intern "ecaml-bf") ;;
Our simple plugin just writes a message to the echo area (or standard error
if Emacs is run in --batch mode). Emacs plugins need to provide the name
of the plugin in order to be loaded successfully by require, so we do that
just before our plugin exits.
Note that OCaml’s module initialization (i.e., the stuff that runs when you compile and execute a normal OCaml program) serves as the code that is run when the plugin is loaded. In order to extend Emacs’s behavior while running, we’ll need to define some Elisp functions and key bindings to call those functions, later. For now, let’s just try our plugin out.
Unfortunately, I haven’t yet been able to figure out a way to get jbuilder
to build shared objects. However, taking advantage of the fact that
executable files and shared objects have basically the same file format
(e.g., Mach-O or ELF), we can just pretend that the compiled executable is a
shared object. It’s fine, trust me. Probably.
We’ll set up our jbuild file like so:
(jbuild_version 1) (executables ((names (plugin)) (libraries (ecaml)))) (rule (copy plugin.exe ecaml-bf.so)) (alias ((name plugin) (deps (ecaml-bf.so)))) (alias ((name runtest) (deps ((alias plugin))) (action (run emacs -Q -L . --batch --eval "(require 'ecaml-bf)"))))
Note the copy rule which simply copies the ocamlopt-produced native
executable file to another name ending in .so. This is the filename
extension that Emacs will look for when we try to load our plugin.
We’ll also define a runtest action that will run Emacs in batch mode and
require our plugin. (I added the -Q flag to skip user initialization,
etc. No need to load all of my Spacemacs config just to run a small test in
batch mode!). Let’s try it out:
jbuilder build @plugin 2>/dev/null && jbuilder runtest # Hello from OCaml # "Loaded Ecaml."
As you can see, the plugin printed out our message to standard error. It also
printed out a message from Ecaml itself. Normally these messages just end up in
your *Messages* buffer, so it’s easy to check to see that they’re there.
Great! We successfully compiled and loaded our plugin into Emacs, without
writing a lick of Elisp (except for the (require 'ecaml-bf)). Next time we’ll
start working on our interpreter. Stay tuned!
I wrote much of an earlier version of Ecaml2 (e.g., the C stubs and low-level OCaml interface) during my internship at Jane Street in the summer of 2016 under the tremendous guidance of my mentor Stephen Weeks. It’s a great place to work, and I recommend anyone interested in their internship program to definitely check it out and/or apply.
Of course, it has since been much expanded, rewritten, and improved upon by people much more experienced than me.