OCamlcore Planet

John Whitington: Compiling Code Under OCaml and F# (Part Two)

Fri, 04 Jul 2008 14:02:53 +0000

Twenty thousand lines of CamlPDF and cpdf later, here are some numbers:

Occasions on which conditional compilation is required: 22
Compilation warnings with fsc –no-warn 62: 15
Time taken: 22 hours

The current executable appears to be about 8 times slower than OCaml native compilation, but I haven’t examined this enough to know how much we might be able to improve upon it.

I’m planning to clean up the code to see how much of the conditional compilation we can get into a single Utility module.

What’s next is to repackage the command line tools as an API for .NET users. I know very little about this topic, so it’s going to require quite a bit of effort before I’m willing to put it on sale and support it.

If you’re familiar with packaging up libraries for .NET and would like to beta test, drop me a line in the comments, or via our website.

David Teller: Extrapol update

Fri, 04 Jul 2008 11:09:01 +0000

A quick work regarding the current status of Extrapol and its release.

Development of Extrapol progresses. With our current set of sample, Extrapol works flawlessly. We’re now adding features, improving error reporting and de-hard-wiring the model of the C standard library from the tool and moving it towards an external configuration file as well as progressively moving towards larger and more realistic samples. Development will come to an abrupt (and temporary) halt at the end of this week, though, due to personal matters (i.e. I’m getting married).

The release planned for next week, on the other hand, is canceled. As the research field of applied security is very competitive, and after careful discussion with the rest of my research team, we have decided to only release a version of Extrapol after the scientific content has been accepted for publication in a conference or journal. At the request of one of the institutes which founds this research, I will also refrain from posting detailed information on the theory and algorithms behind Extrapol, until these are cleared by the institute and accepted for publication. Without entering the details, Extrapol is expected to serve in critical infrastructures, which explains the need for clearance.

However, rest assured that there will be a release and it will be open-source (presumably licenced under a combination of MIT and LGPL). The only question is when — and this probably won’t happen before November.

David Baelde: Enigmes

noreply@blogger.com (mrpingouin) — Thu, 03 Jul 2008 13:30:18 +0000

Quand on pose une énigme mathématique, à moitié formalisée, on finit toujours par avoir une réponse type: "on enlève le bandeau qu'on a sur les yeux", "on casse la gueule au bourreau", etc. D'où l'idée de formaliser vraiment, sous forme d'un exercice de programmation... (qui a dit "maniaque" ?)

Un groupe de n prisonniers se trouve en mauvaise posture. Ils vont être mis en file indienne, chaque prisonnier voyant seulement les prisonniers devant lui. Puis on leur mettra sur la tête un chapeau bleu, blanc ou rouge, qu'ils ne peuvent voir. Un bourreau procède alors comme suit: en partant du dernier prisonnier il demande "Quelle est la couleur de ton chapeau ?", et le prisonnier n'a le droit qu'à une réponse, qui doit être une couleur. A la fin, les prisonniers qui ont deviné la couleur de leur chapeau sont épargnés, les autres exécutés. Les prisonniers peuvent se concerter avant l'épreuve sur la stratégie à adopter. Quelle est la meilleure stratégie ?

Voici maintenant l'énoncé complètement formalisé. Un prisonnier est représenté par une fonction qui prend deux listes de couleurs (celles qu'il voit devant lui, celles qui ont été prononcées comme réponses derrière lui), et renvoie une couleur (sa réponse). Dans la liste des réponses précédentes, la réponse du dernier de la file est la dernière de la liste. Trouver une fonction prisonniers qui à un entier n associe une liste de n fonctions/prisonniers telle que le nombre de morts calculé par (execution (prisonniers n)) soit le plus petit possible dans le pire cas. La fonction d'exécution est la suivante:

type couleur = Bleu | Blanc | Rouge
type prisonnier = couleur list -> couleur list -> couleur

let random_couleur () =
  let n = Random.int 3 in
    if n = 0 then Bleu else if n = 1 then Blanc else Rouge

let execution (prisonniers : prisonnier list) =
  let prisonniers =
    List.map (fun p -> p, random_couleur ()) prisonniers
  in
  let rec morts avant = function
    | (p,c)::l ->
        let reponse = p (List.map snd l) avant in
          (if reponse = c then 0 else 1) +
          (morts (reponse::avant) l)
    | [] -> 0
  in
    morts [] prisonniers

Amusez-vous bien, et postez vos réponses dans le langage de votre choix...

Edit: une autre énigme beaucoup plus étonnante, que Florent et Julien m'ont racontée. Vous avez devant vous sur une table 7 pièces côté pile et 19 pièces côté face. Mais vos yeux sont bandés (et vous portez des moufles) de sorte que vous ne pouvez savoir l'état d'une pièce. Tout ce que vous pouvez faire c'est en retourner certaines, autant que vous voulez, et les déplacer. Il vous faut les séparer en deux groupes qui contiennent le même nombre de pièces côté pile.

La solution sera une fonction qui prend le nombre de piles et le nombre de faces (je vous aide bien en généralisant) et qui renvoie deux tableaux de taille piles+faces: le premier indique les indices des pièces choisies, le second indique les pièces qu'il faut retourner. La fonction sera passée à la fonction de test suivantes:

(** Perfect Fisher-Yates shuffle
  * (http://www.nist.gov/dads/HTML/fisherYatesShuffle.html). *)
let randomize a =
  let permute i j =
    let tmp = a.(i) in
      a.(i) <- a.(j);
      a.(j) <- tmp
  in
  let l = Array.length a in
    if l>=2 then
      for i=0 to l-2 do
        permute i (i + Random.int (l-i))
      done

let test nb_piles nb_faces joueur =
  let choix,retourne = joueur nb_piles nb_faces in
  let n = nb_piles + nb_faces in
  let a = Array.init n (fun i -> i<nb_piles) in
  (* Nombre de piles à gauche - nombre de piles à droite *)
  let d = ref 0 in
    randomize a ;
    for i = 0 to n-1 do
      if retourne.(i) then a.(i) <- not a.(i) ;
      d := (if choix.(i) then (+) else (-)) !d (if a.(i) then 1 else 0)
    done ;
    !d = 0

A suivre: les solutions commentées...

David Teller: Improving exception-management in OCaml

Wed, 02 Jul 2008 14:46:28 +0000

Short version

Catch me if you can is a small library for OCaml 3.10. The latest release is version 0.2, which you may find here. This library improves management of errors in OCaml. It is released under the LGPL licence. It has been written by David Teller, Arnaud Spiwack, Till Varoquaux and Gabriel Scherer.

Long version

As all languages of the ML family — and most modern languages indeed — OCaml permits the management of exceptional situations using exceptions. This mechanism lets programmer register protected sections of code, as well as exception handlers to handle any exception which may be raised during the execution of a protected section. Whenever an exception is raised, the protected section of code is immediately stopped and the corresponding exception handler is executed instead. In addition, exceptions may convey some information regarding the nature of the exceptional circumstance.

In OCaml, the mechanism is fast, it’s convenient and it’s type-safe, much like the rest of the language (barring any type-unsafe interaction with C). However, a few things are missing. If we consider the rest of the language, exceptions are both heavyweight and clumsy: each exception must be declared before being used and there’s no way to introduce a polymorphic type parameter in the exception. In addition, languages such as Java offer to important features missing in OCaml: automatic case coverage and exception hierarchies. While a nice tool exists to provide case coverage for exceptions in OCaml, this tool is complex and unfortunately unmaintained.

Catch me if you can offers an alternative mechanism, comparable to ML exceptions, to handle errors. In comparison with OCaml’s native exception mechanism, this library adds:

automatic inference of exceptions (i.e. no need to declare your exceptions, unless you want to)
more flexible exceptions (i.e. exceptions may have polymorphic type parameters, constraints, etc.)
hierarchies (i.e. an IOException is a sub-case of Exception and a super-case of NetworkException)
case coverage (i.e. the compiler can tell you if you forgot a case or sometimes if you wrote useless ones)
conditional success handlers (i.e. do something with the result in case of success)
conditional success-and-failure handlers (i.e. “finally”).

To attain this, we replace the mechanism of exceptions by an error monad, we replace exception constructors with polymorphic variants and we introduce a dose of syntactic sugar.

Examples

Expression evaluator

Let’s write a simple expression evaluator for the following set of expressions:

type expr =
  | Value of float
  | Div     of expr * expr
  | Add    of expr * expr
  | Mult   of expr * expr
  | Subs  of expr * expr

These may be evaluated using the following function:

let rec eval = function
 | Value x    -> x
 | Add (x,y) -> eval x +. eval y
 | Mult(x,y)  -> eval x *. eval y
 | Div(x,y)   -> eval x /. eval y
 | Subs(x,y) -> eval x -. eval y

Of course, this function is bound to fail in case of division by zero. While this is expected, there is nothing in the source code — much less in the type of the function — to let us know which exception will be raised in case of division by zero.

An alternative would be to add manual error checking, as follows:

type ('a, 'b) result =
 | Ok of 'a
 | Error of 'b

let rec eval = function
 | Value x    -> OK x
 | Add (x,y) -> (match eval x with
                      | Error e -> Error e
                      | Ok x' ->  match eval y with
                               | Error e -> Error e
                               | Ok y'    -> Ok (x' +. y'))
 | Mult (x,y) -> (match eval x with
                      | Error e -> Error e
                      | Ok x' ->  match eval y with
                               | Error e -> Error e
                               | Ok y'    -> Ok (x' *. y'))
 | Div (x,y) -> (match eval x with
                      | Error e -> Error e
                      | Ok x' ->  match eval y with
                               | Error e -> Error e
                               | Ok y'    -> if y' = 0. then Error "Division by zero"
                                                else              Ok (x' /. y'))
 | Subs (x,y) -> (match eval x with
                      | Error e -> Error e
                      | Ok x' ->  match eval y with
                               | Error e -> Error e
                               | Ok y'    -> Ok (x' -. y'))
(*eval : expr -> (float, string) result*)

After this transformation, the type of the exception appears in the type of eval — here, we used strings, but anything else would have been fine. Of course, the downside is that this is unreadable. Well, what about the following ?

let rec eval = function
 | Value x    -> return x
 | Add (x,y) -> perform with module Error
                        x' <-- eval x;
                        y' <-- eval y;
                        return (x' +. y' 
 | Mult (x,y) -> perform with module Error
                        x' <– eval x;
                        y' <– eval y;
                        return (x' *. y' 
 | Div (x,y) -> perform with module Error
                        x' <– eval x;
                        y' <– eval y;
                        if y'=0. then throw "Division by zero"
                        else            return (x' /. y' 
 | Subs (x,y) -> perform with module Error
                        x' <– eval x;
                        y' <– eval y;
                        return (x' -. y' 
(*eval : expr -> (float, string) result*)

This extract uses [our customized version of] Pa_monad (included in the package), which brings syntactic support for monads. While this is more verbose than the original version, it’s also safer, insofar as we can guarantee that exceptions won’t remain uncaught.

Still too long? Then what about using the appropriate operators?

open Error.Operators
let rec eval = function
 | Value x    -> x
 | Add (x,y) -> eval x +. eval y
 | Mult(x,y) -> eval x *. eval y
 | Div(x,y)   -> eval x /. eval y
 | Subs(x,y) -> eval x -. eval y

Except for the module opening, that’s the same thing as our first listing. Just with the added safety.

Throwing, catching and hierarchies

By the way, the type of the result is

(*eval : expr -> (float, [> `Arithmetic of (unit, [> `Div_by_zero of (unit, _) exc ]) exc ]) result*)

That is, eval may either succeed and return a float or fail and return an arithmetic exception, which also turns out to be a division by zero. That’s classes of exceptions.

With our syntactic sugar, raising such an exception is done by

throw (exception Arithmetic (); Div_by_zero ())

Note that we could have put some content instead of (). Note that exceptions are typed as they appear in the code and don’t need to be declared (if you wonder, polymorphic variants are involved in this).

Of course, various kinds of exceptions may be combined, as in the following extract:

match ... with
| 1 -> throw (exception Arithmetic (); Div_by_zero ())
| 2 -> throw (exception Arithmetic (); Overflow "by gosh !&quot 
| 3 -> throw (exception IO file_descr)
| …
(*
('a, [> `Arithmetic of (unit, [> `Div_by_zero of (unit, _) exc
                                     |   `Overflow of (string, _)   exc]) exc
     |   `IO of (int, [`> ]) exc ]) result*)
*)

While the type of the expression is difficult to read, catching is easy

attempt ... with
 | val s -> (*success*)
 | Arithmetic (); Div_by_zero () -> (*Division by zero*)
 | Arithmetic (); _                   -> (*Other arithmetic*)
 | IO _                                  -> (*Some IO stuff *)
 | finally _                             -> (*Don't forget to close the door*)

This extract introduces three keywords:

attempt is our replacement for try
val is used to pattern-match against the result of a successful evaluation
finally is used to pattern-match against the final result, whether this result was obtained after a successful evaluation or after an exception was raised and handled.

Unbreaking tail-recursion

The following extract is wrong:

let line_count filename =
  let rec loop file count =
  try
    ignore (input_line file);
    loop file (count + 1)
  with
    End_of_file -> count
  in
    loop (open_file filename) 0

Don’t get me wrong, it will compile and run. The problem is that it’s not tail-recursive. In other words, it will be much slower and much more memory-consuming than if exceptions had been ignored. Why ? Because exception End_of_file may have been raised from the next call to loop, so the recursive call cannot be optimized into a non-recursive call. Of course, exceptions can’t be ignored in this extract, as they are required to determine when to stop reading the file. Now, a simple transformation would make the problem go away :

let line_count filename =
  let rec loop file count =
    let should_continue =
    try
      ignore (input_line file);
      true
    with End_of_file -> false
  in
    if should_continue then loop file (count + 1)
    else                    count
  in
    loop (open_file filename) 0

Well, the transformation is simple, but it’s annoying and hard to read. What’s even more annoying is that it’s quite common. With Catch me if you can, we would rather write the following:

let input_line2 x = Error.legacy input_line x

let line_count filename =
  let rec loop file count =
    attempt input_line2 file with
      | val _ -> loop file (count + 1)
      | _     -> count
  in
  loop (open_file filename) 0

In this extract, legacy is a simple manner of wrapping an existing, one-argument, function and convert it to our new exception style. It’s not quite as good as wrapping the function manually and giving it an actual exception, but it’s better than nothing.

All in all, the resulting function line_count is shorter, easier to read, takes less memory and is also faster than the original.

What about performance?

Now, that’s a complex question. Short answer: there’s an acceptable performance loss. False short answer: we wrote a paper on that subject.

Matías Giovannini: Sorting out Sorting

noreply@blogger.com (Matías Giovannini) — Wed, 02 Jul 2008 12:17:40 +0000

I believe that refactoring in the functional milieu is as important as it is in the object-oriented world. There's usually room for further improvement of one's definitions. Often, it is no more than the trivial application of selective inlining of definitions that clarify code that turned out more cluttered and opaque that it could have been, possibly from working at a remove from the problem

The Caml Humps: Ooauth 0.1

Wed, 02 Jul 2008 07:52:56 +0000

an implementation of the OAuth 1.0 protocol for OCaml, as well as an OCaml binding to Skydeck web API (providing access to your cell phone call log).

Matías Giovannini: Bitonic Sorting

noreply@blogger.com (Matías Giovannini) — Wed, 02 Jul 2008 00:25:44 +0000

The riffle shuffle I've written about is not a mere card trick. It is a basic building block in the art of devising parallel networks for efficient use of SIMD computers. A typical application is the so-called bitonic sort. An elementary introduction to the algorithm with animations might make its working clear, but I find Knuth's exposition transparent: Let us say that a sequence 〈z1, … zp〉 of

Skydeck: Skydeck API and OAuth for OCaml

Tue, 01 Jul 2008 19:37:02 +0000

The Skydeck API had only been out a few days when someone began a Ruby binding for it. We’re thrilled that there’s been so much interest already, and look forward to some great applications built against our API.

Of course, we like OCaml here, so we are pleased to announce the release of an OCaml binding to the API. You can find it on our developer downloads page.

As part of our API design we decided to use OAuth, so you can allow applications to access your data without giving away your Skydeck username and password. We are also pleased to announce the release of ooauth, an OCaml implementation of OAuth.

Ooauth implements both the consumer and service provider parts of OAuth (we use it in our server and in the API binding above), so you can use it to implement your own API or to consume the many APIs that use OAuth, such as the Google Data APIs. It is also available on the developer downloads page.

Caml Weekly News: Caml Weekly News, 01 Jul 2008

Tue, 01 Jul 2008 12:00:00 +0000

patterns v0.4 / ocamlhackers.ning.com is open / OCaml PLEAC reaches 70%

Ocamlcore Forge News: EThreads version 1.0.1 released

Mon, 30 Jun 2008 08:56:54 +0000

The first early version of EThreads has been released. EThreads extends threads library. - threadless ivar and mvar - several usefull modules such as mailbox, broadcast, timeout, rpc.