OCaml Planet

October 19, 2014

Shayne Fletcher

Tail-recursion

Tail-recursion

Stack overflow refers to a condition in the execution of a computer program whereby the stack pointer exceeds the address space allocated for the stack. The usual result of "blowing the stack" is swift and brutal abnormal termination of the program.

The amount of memory allocated by the operating system for a given program's stack is finite and generally little can be done by the programmer to influence the amount that will be made available. The best the programmer can really do is to use what's given wisely.

We can get a sense of the limits of the stack in practical terms with a program like the following.


let rec range s e =
if s >= e then []
else s :: range (s + 1) e

let rec loop i =
let n = 2.0 ** (i |> float_of_int) |> int_of_float in
try
let _ = range 0 n in
loop (i + 1)
with
| Stack_overflow ->
Printf.printf "Stack overflow at i = %d, n = %d\n" i n

let () = loop 0
range is a function that produces the half-open range $\left[s, e\right)$ - the ordered sequence $\left\{s, s + 1, s + 2, \dots, e - 2, e - 1\right\}$. Note that range is defined in terms of itself, that is, it is a recursive function. The idea is to use it in an unbounded loop to build sequences of increasing lengths of powers of $2$ : ${2^0, 2^1, 2^2, \dots}$. We set it off and when we encounter stack overflow, terminate the program gracefully reporting on the power of $2$ found to give rise to the condition. In my case I found that I was able to make $\approx 2^{19} = 524,288$ recursive calls to range before the stack limit was breached. That's very limiting. For realistic programs, one would hope to be able to produce sequences of lengths much greater than that!

What can be done? The answer lies in the definition of range and that thing called tail-recursion. Specifically, range is not tail-recursive. To be a tail-recursive function, the last thing the function needs do is to make a recursive call to itself. That's not the case for range as written as the recursive call to itself is the second to last thing it does before it returns (the last thing it does is to 'cons' a value onto the list returned by the recursive call).

Why being tail-recursive is helpful is that tail-calls can be implemented by the compiler without requiring the addition of a new "stack frame" to the stack. Instead, the current frame can be replaced in setting up the tail-call being modified as necessary and effectively the recursive call is made to be a simple jump. This is called tail-call elimination and its effect is to allow tail-recursive functions to circumvent stack overflow conditions.

Here's a new definition for range, this time implemented with tail-calls.

let range s e =
let rec aux acc s e =
if s >= e then acc
else aux (s :: acc) (s + 1) e
in List.rev (aux [] s e)
With this definition for range I find I can build sequences of length up to around $\approx 2^{26} = 67,108,864$ elements long without any sign of stack overflow which is a huge improvement! At around this point though, my sequence building capabilities start to be limited by the amount of physical memory present on my PC but that's a different story entirely.

by Shayne Fletcher (noreply@blogger.com) at October 19, 2014 05:39 PM

October 17, 2014

Erik de Castro Lopo

Haskell : A neat trick for GHCi

Just found a really nice little hack that makes working in the GHC interactive REPL a little easier and more convenient. First of all, I added the following line to my ~/.ghci file.

  :set -DGHC_INTERACTIVE

All that line does is define a GHC_INTERACTIVE pre-processor symbol.

Then in a file that I want to load into the REPL, I need to add this to the top of the file:

  {-# LANGUAGE CPP #-}

and then in the file I can do things like:

  #ifdef GHC_INTERACTIVE
  import Data.Aeson.Encode.Pretty

  prettyPrint :: Value -> IO ()
  prettyPrint = LBS.putStrLn . encodePretty
  #endif

In this particular case, I'm working with some relatively large chunks of JSON and its useful to be able to pretty print them when I'm the REPL, but I have no need for that function when I compile that module into my project.

October 17, 2014 10:16 PM

October 16, 2014

WODI

OCaml 4.02.1 released

Windows binary builds for OCaml 4.02.1 are now available. Download links for the 32-bit and 64-bit build can be found in the download section.

You can upgrade from an existing installation of OCaml 4.02.0 with

godi_upgrade

or from source code with:

godi_update
godi_perform -rebuild -newer

The OCaml 4.02.0 builds are not longer maintained. If you still want to continue to use the binary builds for this OCaml version (or revert back to it), you have to change the repository address at /opt/wodi(32|64)/etc/godi.conf from

GODI_BINPKG_SERVER=http://dl.arirux.de/7/binaries${MINGW_WORDSIZE}/

to

GODI_BINPKG_SERVER=http://dl.arirux.de/7.old/binaries${MINGW_WORDSIZE}/

October 16, 2014 10:00 PM

Jane Street

What the interns have wrought: RPC_parallel and Core_profiler

We're in the midst of intern hiring season, and so we get a lot of questions about what it's like to be an intern at Jane Street. One of the things people most want to know is what kind of projects they might work on as an intern.

That's of course hard to answer precisely, since we don't yet know what next summer's intern projects are going to be. But you can get a sense by seeing some of what interns have done in the past. To that end, I thought I'd describe a couple of intern projects that were done this past summer.

Rpc_parallel

Rpc_parallel is a new library written by Todd Lubin (who will be returning to Jane Street full-time next fall), aimed at making parallel programming in OCaml simpler and safer.

Writing programs that take advantage of multiple cores, and even of multiple computers, is of course a necessity. In OCaml, such parallelism is typically achieved by writing multiple single-threaded programs that use message passing to coordinate.

We have lots of parallel message-passing systems, but while these share a lot of infrastructure (notably, the Async_rpc library), we've never really settled on a lightweight way to construct complete parallel programs. Instead, each such program tends to have its own little process coordination framework.

A while back, we wrote a library called Async_parallel, (described here). Async_parallel does a lot of things right -- in particular, it makes it easy to spin up and manage new processes and distribute work between them.

But Async_parallel has a problem. It is based on OCaml's marshal facility, which allows you to serialize arbitrary data, including functions, between processes. This sounds great, but it has a dark side: marshalling arbitrary functions turns out to be error prone. In particular, in order to send a function over the wire, you also have to send a copy of any data that that function implicitly depends on.

Unfortunately, it's hard to know what kind of data is hidden behind a function, which can cause a few different forms of trouble: it might send much more data than you expect, it might fail unexpectedly if it hits one of the forms of data that can't be marshalled, or it might lead to crazy and hard-to-predict behavior if some of the data required by the function is meant to be mutable shared state.

Instead, we wanted a library that was more typeful and constrained in terms of what was sent over the wire. This pushed us towards a design where, at the cost of some extra verbosity, we explicitly declare the types of data that is sent. In exchange, we get a system that is easier to understand and debug.

One of the great things about Rpc_parallel is how fast it came together. Todd got it into a sufficiently good state by the middle of the summer that he was able to use it for his other projects (interns typically have at least two major projects over the course of the summer).

Rpc_parallel also benefitted from some world-hopping collaboration. Interns spend at least a week in an office other than their home office, and Todd ended up visiting Hong Kong. While there, he ended up spending a lot of time talking and working with Chengqi Song, who had a lot of experience with Async_parallel. Out of those discussions came a complete redesign and rewrite of the library, factoring out the core primitives for coordinating across multiple processes, and making the overall library simpler and more general.

By the end of the summer, a few other people picked up and starting using it for other projects, and last week, it was released open source, so you can take a look at it yourself on github.

Core_profiler

Profiling is surprisingly hard, and as such it's perhaps unsurprising that there are lots of ways of doing it. If you want to understand the cost of an individual operation, you might want a micro-benchmarking library like Haskell's Criterion, or our own Core_bench. If you're trying to understand properties of a whole application, like which lines of code it's spending its time on, or how many cache-misses are occurring, you might want to use something like Linux's perf tools, which use CPU-level counters to efficiently gather profiling statistics from your program.

Another useful technique is for the programmer to modify the source to add explicit probes that keep track of when certain program points are reached, and write out that information to a log that can be analyzed after the fact.

Daniel Richman (who will be returning for an internship next summer) worked along with Roshan James (formerly an intern himself, now fulltime) on a library called Core_profiler, which aims to make the use of such probes easy and efficient. Efficiency matters quite a bit, because if logging a probe takes more time than the thing you're trying to measure, you basically can't extract reliable data. Keeping the overhead small, therefore, is a must.

Accordingly, a lot of Daniel's time was spent thinking very carefully about how to write the probes in a way that would only minimally disrupt the execution of the program. He started a simple but less efficient library, called Stats_reporting, which took about 60ns and two words of allocation per probe, and started machining it down from there.

The first step was to avoid all allocation, which meant we could no longer use bin-prot, our standard binary serialization technique, since bin-prot requires that you allocate an OCaml object representing the data to be serialized. So they moved to using an internal library called Protogen for generating zero-allocation serialization code.

That brought us down to about 30ns, and zero allocations. We then decided to try out writing our own hand-rolled binary protocol, so we could have a yet-more-closely optimized binary layout. That brought us down to about 20-25ns. The next hack was to customize the binary format yet more by packing multiple values into a single, 63-bit OCaml integer. That, plus some cleverness to make sure that writes were word-aligned, brought the cost of a simple probe down to 12ns. In addition to the design changes, Daniel also spent a lot of time carefully examining the assembly code, to make sure that there were no surprises from the code generator.

We're pretty happy with the end result. We think that Core_profiler probes are now a good bit cheaper than the dynamic probes you can insert using a system like DTrace, and are in any case the best tool we have for tracking the performance of even relatively quick code-paths. And, as a nice bonus to all this optimization, the offline processing got about twice as fast as a side effect of the runtime improvements.


There's more to be said about both of these projects, and about the many other projects that were done this summer. If you're interested in applying, you can do so here:

http://janestreet.com

You don't need to know anything about finance or functional programming to apply. But if you come, you'll learn a ton about both by the time you're done.

by Yaron Minsky at October 16, 2014 12:48 PM

Andrej Bauer

TEDx “Zeroes”

I spoke at TEDx University of Ljubljana. The topic was how programming influences various aspects of life. I showed the audence how a bit of simple programming can reveal the beauty of mathematics. Taking John Baez’s The Bauty of Roots as an inspiration, I drew a very large image (20000 by 17500 pixels) of all roots of all polynomials of degree at most 26 whose coefficients are $-1$ or $1$. That’s 268.435.452 polynomials and 6.979.321.752 roots. It is two degrees more than Sam Derbyshire’s image,  so consider the race to be on! Who can give me 30 degrees?

The code

The zeroes GitHub repository contains the code.

There really is not much to show. The C program which computes the zeroes uses the GNU Scientific Library routines for zero finding and is just 105 lines long. It generates a PPM image which I then processed with ImageMagick and ffmpeg. The real work was in the image processing and the composition of movies. I wrote a helper Python program that lets me create floyovers the big image, and I became somewhat of an expert in the use of ImageMagick.

The code also contains a Python program for generating a variation of the picture in which roots of lower degrees are represented by big circles. I did not show any of this in the TEDx talk but it is available on Github.

Oh, and a piece of Mathematica code that generates the zeroes fits into a tweet.

The videos

The “Zeroes” Vimeo album contains the animations. The ones I showed in the TEDx talk are in Full HD (1920 by 1080). There is also a lower resolution animation of how zeroes move around when we change the coefficients. Here is one of the movies, but you really should watch it in Full HD to see all the details.

<iframe allowfullscreen="" frameborder="0" height="281" src="http://player.vimeo.com/video/109062970?title=0&amp;byline=0&amp;portrait=0" width="500"></iframe>

The pictures

  • The computed image zeroes.ppm.gz (125 MB) at 20000 by 17500 pixels is stored in the PPM format. The picture is dull gray, and is not meant to be viewed directly.
  • The official image zeroes26.png (287 MB) at 20000 by 175000 pixels in orange tones. Beware,  it can bring an image viewing program to its knees.
  • I computed tons of closeups to generate the movies. Here are the beginnings of each animation available at Vimeo, and measly 1920 by 1080 pixels each (click on them).
    • The whole image: zoom
    • Zoom at $i$: arc
    • Zoom at $(1 + \sqrt{3} i)/2$:tofringe
    • Zoom at $1.4 i$:fringe
    • Zoom at $3 e^{7 i \pi/24}/2$:tozero
    • Zoom at $1$:unzoom

by Andrej Bauer at October 16, 2014 07:01 AM

October 15, 2014

OCamlCore Forge News

OCaml Forge maintenance

Recently the OCaml forge has required a lot of hardware reboots. The server is probably near end of life and I need to upgrade the whole infrastructure to a recent server. Rackspace Cloud, as part of their developer support program, is kindly providing a new host for the forge. You may encounter a few more problems in the coming weeks, due to this migration. Ping me if anything is utterly wrong (sylvain ... le-gall.net).

by Sylvain Le Gall at October 15, 2014 06:58 AM

October 14, 2014

OCaml Platform

Binary distribution with 0install

0install provides an easy way to distribute binaries to users, complementing OPAM's support for source distribution.

The traditional way to distribute binaries is to make separate packages for recent versions of the more popular Linux distributions (RPMs, Debs, PKGBUILDs, etc), a .pkg for OS X, a setup.exe Windows, etc, plus some generic binaries for users of other systems or who lack root privileges. This requires a considerable amount of work, and expertise with many different package formats. 0install provides a single format that supports all platforms, meaning that you only need to do the packaging work once (though you should still test on multiple platforms).

Example: OPAM binaries

You can install a binary release of OPAM on most systems by first installing the "0install" package from your distribution ("zeroinstall-injector" on older distributions) and then adding OPAM.

  1. Get 0install. For the major Linux distributions:

    $ yaourt -S zeroinstall-injector             # Arch Linux
    $ sudo apt-get install zeroinstall-injector  # Debian, Ubuntu, etc
    $ sudo yum install 0install                  # Fedora
    $ sudo yum install zeroinstall-injector      # Red Hat
    $ sudo zypper install zeroinstall-injector   # OpenSUSE

    For OS X: ZeroInstall.pkg

    0install also has Windows support, but there is currently no Windows binary of OPAM so I didn't publish one with 0install either.

  2. Install OPAM:

    $ 0install add opam http://tools.ocaml.org/opam.xml
    $ opam --version
    1.1.1

    If you already have an opam command but want to try the 0install version anyway, just give it a different name (e.g. 0install add 0opam http://tools.ocaml.org/opam.xml to create a 0opam command).

0install add will open a window if a display is available, or show progress on the console if not:

If you want to use the console in all cases, use --console.

0install identifies each package with a full URI rather than with a short name. Here, we are telling 0install to create a local command called opam that runs the program http://tools.ocaml.org/opam.xml.

0install keeps each package it installs in a separate directory where it won't conflict with anything. You can see where it put OPAM with the "show" command:

$ 0install show opam
- URI: http://tools.ocaml.org/opam.xml
  Version: 1.1.1
  Path: /home/test/.cache/0install.net/implementations/sha256new_RUOX6PWGDCHH5TDNEDRHQJ54YZZ4TSAGBB5AEBRNOKSHM3N7XORA
  
  - URI: http://repo.roscidus.com/utils/aspcud
    Version: 1.9.0
    Path: /home/test/.cache/0install.net/implementations/sha1new=5f838f78e489dabc2e2965ba100f14ae8350cbce
  
  - URI: http://repo.roscidus.com/utils/curl
    Version: 7.32.0-10.20
    Path: (package:rpm:curl:7.32.0-10.20:x86_64)

OPAM depends on two other programs: aspcud provides a better solver than the internal one and curl is used to download OPAM packages (it also generally needs gcc, m4 and patch, but I started with just the ones people are likely to be missing). In this case, 0install has satisfied the curl dependency using an official Fedora package, but needed to install aspcud using 0install. On Arch Linux, it can use distribution packages for both. 0install will install any required distribution packages using PackageKit, which will prompt the user for permission according to its policy.

You can upgrade (or downgrade) the package by adding a version constraint. By default, 0install prefers the "stable" version of a program:

$ 0install update opam
A later version (http://tools.ocaml.org/opam.xml 1.2.0-pre4) exists but was not selected.
Using 1.1.1 instead.
To select "testing" versions, use:
0install config help_with_testing True

You could do as it suggests and tell it to prefer testing versions globally, or you can add a version constraint if you just want to affect this one program:

$ 0install update opam --not-before=1.2.0-pre4
http://tools.ocaml.org/opam.xml: 1.1.1 -> 1.2.0-pre4

You can also specify an upper bound (--before) or a fixed version (--version) if you prefer. You can control the versions of dependencies with --version-for. By the way, 0install supports tab-completion everywhere: it can do completion on the sub-command (update), the application name (opam), the option (--not-before) and even the list of available versions!

Finally, if an upgrade stops a program from working then you can use whatchanged to see the latest changes:

$ 0install whatchanged opam
Last checked    : 2014-08-26 11:00
Last update     : 2014-08-26
Previous update : 2014-07-03

http://tools.ocaml.org/opam.xml: 1.1.1 -> 1.2.0-pre4

To run using the previous selections, use:
0install run /home/tal/.config/0install.net/apps/opam/selections-2014-07-03.xml

Note: this has a granularity of a day, so you won't see any changes if you're following along, since you didn't have it installed yesterday.

The package metadata

If you visit http://tools.ocaml.org/opam.xml you should see a normal-looking web-page describing the package. If you view the source in your browser, you'll see that it's actually an XML document with a stylesheet providing the formatting. Here's an extract:

<group license="OSI Approved :: GNU Lesser General Public License (LGPL)">
  <group>
    <requires interface="http://repo.roscidus.com/utils/aspcud"
              importance="recommended">
      <executable-in-var name="OPAMEXTERNALSOLVER"/>
    </requires>

    <requires interface="http://repo.roscidus.com/utils/curl"
              importance="recommended">
      <executable-in-var name="OPAMCURL"/>
    </requires>

    <command name="run" path="opam"/>

    <implementation arch="Linux-x86_64"
                    id="sha1new=6e16ff6ee58e39c9ebbed2fb6c6b6cc437b624a4"
                    released="2014-04-17"
                    stability="stable"
                    version="1.1.1">
      <manifest-digest sha256new="RUOX6PWGDCHH5TDNEDRHQJ54YZZ4TSAGBB5AEBRNOKSHM3N7XORA"/>
      <archive href="http://test.roscidus.com/archives/opam-Linux-x86_64-1.1.1.tgz"
               size="1476315"/>
    </implementation>

This says that the 64-bit Linux binary for OPAM 1.1.1 is available at the given URL and, when unpacked, has the given SHA256 digest. It can be run by executing the opam binary within the archive. It depends on aspcud and curl, which live in other repositories (ideally, these would be the official project sites for these programs, but currently they are provided by a third party). In both cases, we tell OPAM about the chosen version by setting an environment variable. The <group> elements avoid repeating the same information for multiple versions. curl is marked as recommended because, while 0install supports most distribution package managers, if it can't find a curl package then it's more likely that it failed to find the curl package than that the platform doesn't have curl.

Lower down there is a more complex entry saying how to build from source, which provides a way to generate more binaries, and the XML is followed by a GPG signature block (formatted as an XML comment so that the document is still valid XML).

Security

When you use a program for the first time, 0install downloads the signing GPG key and checks it with the key information service. If this service knows the key, it saves it as a trusted key for that site. If not, it prompts you to confirm. In future, it will check that all updates from that site are signed with the same key, prompting you if not (much like ssh does).

If you would like to see the key information hints rather than having them approved automatically, use 0install config auto_approve_keys false or turn off "Automatic approval for new feeds" in the GUI, and untrust the key (right-click on it for a menu):

You will then see prompts like this when using a new site for the first time:

Making packages

Ideally, OPAM's own Git repository would contain an XML file describing its build and runtime dependencies (curl and aspcud in this case) and how to build binaries from it. We would then generate the XML for releases from it automatically using tools such as 0release. However, when trying out 0install you may prefer to package up an existing binary release, and this is what I did for OPAM.

The simplest case is that the binary is in the current directory. In this case, the XML just describes its dependencies and how to run it, but not how to download the program. You can create a template XML file using 0template (or just write it yourself):

$ 0install add 0template http://0install.net/tools/0template.xml
$ 0template opam.xml
'opam.xml' does not exist; creating new template.

As it ends with .xml, not .xml.template, I assume you want a feed for
a local project (e.g. a Git checkout). If you want a template for
publishing existing releases, use opam.xml.template instead.

Does your program need to be compiled before it can be used?

1) Generate a source template (e.g. for compiling C source code)
2) Generate a binary template (e.g. for a pre-compiled binary or script)
> 2

Writing opam.xml

Filling in the blanks, we get:

<?xml version="1.0"?>
<interface xmlns="http://zero-install.sourceforge.net/2004/injector/interface">
  <name>OPAM</name>
  <summary>OCaml package manager</summary>

  <description>
    OPAM is an open-source package manager edited by OCamlPro.
    It supports multiple simultaneous compiler installations, flexible
    package constraints, and a Git-friendly development workflow.
  </description>

  <homepage>https://opam.ocaml.org/</homepage>

  <feed-for interface="http://tools.ocaml.org/opam.xml"/>

  <group license="OSI Approved :: GNU Lesser General Public License (LGPL)">
    <requires importance="recommended" interface="http://repo.roscidus.com/utils/aspcud">
      <executable-in-var name="OPAMEXTERNALSOLVER"/>
    </requires>

    <requires importance="recommended" interface="http://repo.roscidus.com/utils/curl">
      <executable-in-var name="OPAMCURL"/>
    </requires>

    <command name="run" path="opam"/>

    <implementation arch="Linux-x86_64" id="." local-path="." version="1.1.1"/>
  </group>
</interface>

This is almost the same as the XML above, except that we specify local-path="." rather than giving a URL and digest (see the Feed file format specification for details). The <feed-for> says where we will eventually host the public list of all versions.

You can test your XML locally like this:

$ 0install run opam.xml --version
1.1.1

$ 0install select opam.xml          
- URI: /tmp/opam/opam.xml
  Version: 1.1.1
  Path: /tmp/opam
  
  - URI: http://repo.roscidus.com/utils/aspcud
    Version: 1.9.0-1
    Path: (package:arch:aspcud:1.9.0-1:x86_64)
  
  - URI: http://repo.roscidus.com/utils/curl
    Version: 7.37.1-1
    Path: (package:arch:curl:7.37.1-1:x86_64)

Now we need a way to generate similar XML for released archives on the web. Rename opam.xml to opam.xml.template and change the <implementation> to:

<implementation arch="{arch}" version="{version}">
  <manifest-digest/>
  <archive href="http://example.com/archives/opam-{arch}-{version}.tgz"/>
</implementation>

If the archive is already published somewhere, you can use the full URL in the <archive>. If you're making a new release locally, just put the archive in the same directory as the XML and give the leaf only (href="opam-{arch}-{version}.tgz").

You can now run 0template on the template XML to generate the XML with the hashes, sizes, etc, filled in:

$ 0template opam.xml.template version=1.1.1 arch=Linux-x86_64
Writing opam-1.1.1.xml

This generates:

<group license="OSI Approved :: GNU Lesser General Public License (LGPL)">
  <requires importance="recommended" interface="http://repo.roscidus.com/utils/aspcud">
    <executable-in-var name="OPAMEXTERNALSOLVER"/>
  </requires>

  <requires importance="recommended" interface="http://repo.roscidus.com/utils/curl">
    <executable-in-var name="OPAMCURL"/>
  </requires>

  <command name="run" path="opam"/>

  <implementation arch="Linux-x86_64"
                  id="sha1new=6e16ff6ee58e39c9ebbed2fb6c6b6cc437b624a4"
                  released="2014-08-26"
                  version="1.1.1">
    <manifest-digest sha256new="RUOX6PWGDCHH5TDNEDRHQJ54YZZ4TSAGBB5AEBRNOKSHM3N7XORA"/>
    <archive href="http://example.com/archives/opam-Linux-x86_64-1.1.1.tgz" size="1476315"/>
  </implementation>
</group>

You can test this as before:

$ 0install run opam-1.1.1.xml --version
1.1.1

Finally, you can submit the XML to a repository (which is easy to host yourself) using the 0repo tool:

$ 0install add 0repo http://0install.net/tools/0repo.xml
[ ... configure your repository ... ]
$ 0repo add opam-1.1.1.xml

0repo will merge the XML into the repository's master list of versions, upload the archive (or test the URL, if already uploaded), commit the final XML to Git, and push the XML to your web server. There are simpler ways to get the signed XML, e.g. using 0publish-gui for a graphical UI, but if you're going to release more than one version of one program then the automation (and sanity checking) you get from 0repo is usually worth it. In my case, I configured 0repo to push the signed XML to a GitHub Pages repository.

There are plenty of ways to extend this. For the OPAM 1.2 release, instead of adding the official binaries one by one, I used 0template to make a template for the source code, added the 1.2 source release, and used that to generate the binaries (mainly because I wanted to build in an older docker container so the binaries would work on more systems). For my own software, I commit an XML file saying how to build it to my Git repository and let 0release handle the whole release process (from tagging the Git repository, to building the binaries in various VMs, to publishing the archives and the final signed XML). In the future, we hope to integrate this with OPAM so that source and binary releases can happen together.

Extending 0install

0install is easy to script. It has a stable command line interface (and a new JSON API too). There is also an OCaml API, but this is not yet stable (it's still rather Pythonic, as the software was recently ported from Python).

For example, I have used 0install to manage Xen images of Mirage unikernels. This command shows the latest binary of the mir-hello unikernel for Xen (downloading it first if needed):

$ 0install download --os=Xen http://test.roscidus.com/mir-console.xml --show
- URI: http://test.roscidus.com/mir-console.xml
  Version: 0.2
  Path: /home/tal/.cache/0install.net/implementations/sha256new_EY2FDE4ECMNCXSRUZ3BSGTJQMFXE2U6C634PBDJKOBUU3SWD5GDA

Summary

0install is a mature cross-platform system for managing software that is included in the repositories of most Linux distributions and is also available for Windows, OS X and Unix. It is useful to distribute binary executables in cases where users shouldn't have to compile from source. It supports GPG signature checking, automatic updates, pinned versions and parallel installations of multiple versions. A single package format is sufficient for all platforms (you still need to create separate binary archives for e.g. OS X and Linux, of course).

0install is a decentralised system, meaning that there is no central repository. Packages are named by URI and the metadata is downloaded directly from the named repository. There are some extra services (such as the default mirror service, the search service and the key information service), but these are all optional.

Using 0install to get OPAM means that all platforms can be supported without the need to package separately for each one, and users who don't wish to install as root still get signature checking, dependency handling and automatic updates. We hope that 0install will make it easier for you to distribute binaries of your own applications.

My talk at OCaml 2014 (video, slides) gives more information about 0install and its conversion to OCaml.

by Thomas Leonard at October 14, 2014 12:00 AM

October 10, 2014

Functional Jobs

Functional Software Engineer at Cake Solutions Ltd (Full-time)

At Cake Solutions we work with our customers to build high-quality, scalable and resilient software systems using the latest technology. As a software engineer, you'll not only be able to write good, maintainable software, but also stay at the forefront of technology and advocate a principled approach to software engineering. You'll get the opportunity to work on a wide range of interesting projects for our clients, using Java, Scala, Play, and Akka.

What to expect:

To begin with, you will take part in a 2 week Typesafe certified workshop style training course where you will get introduced to Scala, Akka and Play.

You can expect a lively and challenging environment with very interesting problems to solve. We are happy to train and mentor the right people; the important thing is to have a bright mind and the motivation to question, explore and learn. Having published work, on Github or elsewhere, is extremely helpful. CVs should focus on what you really know, not what you've seen once.

If you are a graduate, we are not expecting commercial experience, but we want to see evidence of hobby / university engineering. We do, however, expect you to be independent and to have good understanding of the principles of software engineering.

You will also get flexible working hours and gym membership.

Skills & Requirements As a software engineer at Cake Solutions, you should:

-- Have a good understanding of Java and the JVM. Experience with Scala is a plus.

-- Know how to use UNIX or Linux.

-- Know how to apply object-oriented and functional programming styles to real-world software engineering.

-- Have experience with at least one database system and be aware of the wider database landscape (relational, document, key/value, graph, ...).

-Understand modern software development practices, such as testing, continuous integration and producing maintainable code.

Advantages include:

-- Open-source contributions.

-- Modern web development experience.

-- An understanding of asynchronous and non-blocking principles.

-- Experience in writing multi-threaded software.

-- More detailed knowledge of strongly-typed functional programming (e.g. Scala, Haskell, OCaml).

-- Even more specifically, experience with Akka or Play.

About Cake Solutions Ltd Cake Solutions architects, implements and maintains modern and scalable software, which includes server-side, rich browser applications and mobile development. Alongside the software engineering and delivery, Cake Solutions provides mentoring and training services. Whatever scale of system you ask us to develop, we will deliver the entire solution, not just lines of code. We appreciate the importance of good testing,Continuous Integration and delivery, and DevOps. We motivate, mentor and guide entire teams through modern software engineering. This enables us to deliver not just software, but to transform the way organisations think about and execute software delivery.

The core members of our team are published authors and experienced speakers. Our teams have extensive experience in designing and implementing event-driven, resilient, responsive and scalable systems. We use modern programming languages such as Scala, Java, C++, Objective-C, and JavaScript to implement the systems' components. We make the most of messaging infrastructures, modern DBMSs, and all other services that make up today's enterprise systems. Automated provisioning, testing, integration and delivery allow us to release high quality systems safely and predictably. The mentoring through continuous improvement at all levels of the project work gives our clients the insight and flexibility they expect.

We rely on open source software in our day-to-day development; it gives us access to very high quality code, allows us to make improvements if we need to, and provides access to excellent source of inspiration and talent. We give back to the open source community by contributing to the open source projects we use, and by publishing our own open source projects. The team have contributed various Typesafe Activator templates, and shared their expertise with Akka and Scala in Akka Patterns and Akka Extras. Outside of the Typesafe stack, we have contributed to Tru-strap, and OpenCV. The team members have also created open source projects that scratch our own itch: we have Reactive Monitor, Specs2 Spring and Scalad.

Get information on how to apply for this position.

October 10, 2014 12:02 PM