.NET and CoffeeScript: comparing Jurassic, Jint, and IronJS
Recently I went looking for ways to write a .NET desktop app that could compile CoffeeScript to JavaScript. There are already several NuGet packages for exactly this, but most of them look like they’re tightly bound to ASP.NET. So I struck out on my own, following the general steps in “CoffeeDemo – A Simple Demo of IronJS, using CoffeeScript”. The main CoffeeScript compiler is written in CoffeeScript, but they also provide one written in JavaScript, so my basic outline was:
- Instantiate a JavaScript engine.
- Tell the JavaScript engine to run coffee-script.js. This creates the CoffeeScript object and its compile method.
- Tell the JavaScript engine to call CoffeeScript.compile, and pass a string containing the CoffeeScript code to compile.
But my first attempt ran slower than I’d hoped for. (Well, coffee-script.js is 163 KB, and that’s the minified version! So I guess it does have a lot to do.)
I decided to find out whether I could do better: I tried several different JavaScript-in-.NET implementations, to see which one would perform the best. I tested Jurassic, Jint, and IronJS. My results are below, along with the C# code in case anyone is interested in seeing the minor differences between the APIs.
In all three cases, the coffeeCompiler parameter contains the 1.1.2 version of coffee-script.js, as downloaded from GitHub; and the input parameter contains a one-line CoffeeScript script:
alert "Hello world!"
Jurassic
Jurassic dynamically compiles JavaScript to CLR code at runtime, so you take a performance hit the first time you run some JS, but it should be pretty fast after that. Jurassic is available via NuGet.
private void CompileCoffeeScriptUsingJurassic( string coffeeCompiler, string input) { Console.WriteLine("Jurassic"); var stopwatch = Stopwatch.StartNew(); Console.WriteLine(stopwatch.Elapsed + ": Creating engine"); var engine = new ScriptEngine(); Console.WriteLine(stopwatch.Elapsed + ": Parsing coffee-script.js"); engine.Execute(coffeeCompiler); Console.WriteLine(stopwatch.Elapsed + ": Adding compile wrapper"); engine.Execute("var compile = function (src) " + "{ return CoffeeScript.compile(src, { bare: true }); };"); Console.WriteLine(stopwatch.Elapsed + ": Compiling CoffeeScript input"); var output = engine.CallGlobalFunction("compile", input); Console.WriteLine(stopwatch.Elapsed + ": Done"); Console.WriteLine("Output:"); Console.WriteLine(output); Console.WriteLine(); }
Jint
Jint is a JavaScript interpreter. It’s not available through NuGet yet, but it’s a single DLL.
private void CompileCoffeeScriptUsingJint( string coffeeCompiler, string input) { Console.WriteLine("Jint"); var stopwatch = Stopwatch.StartNew(); Console.WriteLine(stopwatch.Elapsed + ": Creating engine"); var engine = new JintEngine(); Console.WriteLine(stopwatch.Elapsed + ": Parsing coffee-script.js"); engine.Run(coffeeCompiler); Console.WriteLine(stopwatch.Elapsed + ": Adding compile wrapper"); engine.Run("var compile = function (src) " + "{ return CoffeeScript.compile(src, { bare: true }); };"); Console.WriteLine(stopwatch.Elapsed + ": Compiling CoffeeScript input"); object output = null; try { output = engine.CallFunction("compile", input); } catch (JsException ex) { Console.WriteLine("ERROR: " + ex.Value); } Console.WriteLine(stopwatch.Elapsed + ": Done"); Console.WriteLine("Output:"); Console.WriteLine(output); Console.WriteLine(); }
IronJS
IronJS is based on the DLR, so it seemed like it might strike a great balance between upfront compile time and runtime — after all, that’s what the DLR is all about.
IronJS is available through NuGet — there’s both an IronJS.Core (standalone) and an IronJS (depends on IronJS.Core), with nothing to explain the difference between the two; but at least for this code, you only need IronJS.Core.
private void CompileCoffeeScriptUsingIronJs( string coffeeCompiler, string input) { Console.WriteLine("IronJS"); var stopwatch = Stopwatch.StartNew(); Console.WriteLine(stopwatch.Elapsed + ": Creating engine"); var engine = new CSharp.Context(); Console.WriteLine(stopwatch.Elapsed + ": Parsing coffee-script.js"); engine.Execute(coffeeCompiler); Console.WriteLine(stopwatch.Elapsed + ": Adding compile wrapper"); engine.Execute("var compile = function (src) " + "{ return CoffeeScript.compile(src, { bare: true }); };"); Console.WriteLine(stopwatch.Elapsed + ": Fetching compile wrapper"); var compile = engine.GetGlobalAs<FunctionObject>("compile"); Console.WriteLine(stopwatch.Elapsed + ": Compiling CoffeeScript input"); var result = compile.Call(engine.Globals, input); var output = IronJS.TypeConverter.ToString(result); Console.WriteLine(stopwatch.Elapsed + ": Done"); Console.WriteLine("Output:"); Console.WriteLine(output); Console.WriteLine(); }
The results
Jurassic
00:00:00.0000063: Creating engine
00:00:00.1540545: Parsing coffee-script.js
00:00:03.4346969: Adding compile wrapper
00:00:03.4408860: Compiling CoffeeScript input
00:00:05.3466983: Done
Output:
alert("Hello world!");
Jint
00:00:00.0000019: Creating engine
00:00:00.2617049: Parsing coffee-script.js
00:00:02.5270733: Adding compile wrapper
00:00:02.5295317: Compiling CoffeeScript input
ERROR: Parse error on line 2: Unexpected 'STRING'
00:00:02.5832895: Done
Output:
IronJS
00:00:00.0000019: Creating engine
00:00:00.2282421: Parsing coffee-script.js
00:00:55.5590620: Adding compile wrapper
00:00:55.5629230: Fetching compile wrapper
00:00:55.5642908: Compiling CoffeeScript input
00:01:17.8580574: Done
Output:
alert("Hello world!");
Jint wasn’t up to the task — it got a weird error when trying to call CoffeeScript.compile. I played with this a bit, and found that it would work if I passed an empty string, but give errors with non-blank CoffeeScript to compile; sometimes a string error like above, sometimes a weird error about multiline comments. It’s too bad, because Jint shows a lot of promise, speed-wise. I don’t know what the problem is; the error didn’t give me much to go on, and I’m not terribly motivated to pursue the problem when the other libraries work. (I did write this up in their bugtracker, though — it’s issue #6928.)
I was surprised that IronJS was so much slower than the others — about 20x slower than Jint at running coffee-script.js, and about 10x slower than Jurassic. This is especially puzzling since the article I based my code on mentions a “compilation lag”. To me, 55 seconds is hardly “lag”!
The winner here (and coincidentally the first one I tried) is Jurassic — so the performance that disappointed me is also the best I’m likely to get. On my laptop, you take about a 3.5-second penalty to compile coffee-script.js, and then another two seconds to run CoffeeScript.compile on a one-line script.
I did find that subsequent calls to CoffeeScript.compile were nearly instantaneous with all three libraries. So Jurassic’s 2 seconds is probably due to the JIT compiler running for the first time on that runtime-generated code. Not sure what to make of the 20 seconds for IronJS; is the DLR just that big?
October 17th, 2011 at 9:10 am
You might want to take a look at this Jurassic discussion.
http://jurassic.codeplex.com/discussions/272661
It explains how to save the generated CIL to an assembly. This way you can generate the CoffeeScript assembly once, then use the compiled assembly for subsequent loads.
October 17th, 2011 at 8:30 pm
Hey, awesome! I’ll have to play with that. Thanks!
October 20th, 2011 at 2:43 am
I’m not sure a single line script is a very good benchmark, but did you take a look at SassAndCoffee?
https://github.com/xpaulbettsx/SassAndCoffee
It hosts V8 to do the compilation and is considerably faster afaik, especially for more complex scripts.
October 20th, 2011 at 1:19 pm
Of course, using the Google V8 engine from .NET code is far, far faster than any of the managed options when you’re compiling mid-size to large CoffeeScript files. In my case, compiling six particular *.coffee files took 11 seconds with Jurassic, and 0.4 seconds with V8. I wrote this desktop app…
http://coffeemonitor.codeplex.com/
…using V8 integration code I borrowed (with permission) from this Visual Studio plugin project:
https://github.com/xpaulbettsx/SassAndCoffee
October 21st, 2011 at 12:46 am
V8 blows all of those out of the water – you’re free to reuse SassAndCoffee.Core, which already implements CoffeeScript compilation in a dead-simple library. Check it out!
October 21st, 2011 at 7:50 am
Nice writeup. I went through a similar exercise recently when writing some code to run jslint/jshint from a command line: https://github.com/jamietre/SharpLinter
I had exactly the same problems you did. IronJS more or less worked but was slow as molasses (though I think I still had problems getting data in & out which seems unnecessarily complicated). I couldn’t get Jint to run JSLINT at all. I didn’t actually try Jurassic (I don’t think I knew of its existence) so now I am curious about it.
I ended up using the technique that Luke Page does in his visual studio extension: the Noesis javascript.net project: http://javascriptdotnet.codeplex.com/
It is basically a wrapper google’s V8 engine. It’s fast as nuts and works perfectly. Once you get the dependencies configured correctly, it’s also the easiest to integrate of any of them. The downside is that it depends on a 32 bit windows dll, and so your project is limited to 32 bit targets but no big deal.
October 21st, 2011 at 10:17 pm
Lots of votes for V8! I’ll have to look into it.
SassAndCoffee.Core says it does all kinds of ASP.NET stuff, which is why I didn’t look at it the first time around — I’m writing a desktop app, so I don’t want ASP.NET dependencies (which would require the full .NET Framework, instead of just the Client Profile). But I wonder if the description on NuGet might be lying. Given that there’s a SassAndCoffee.Core, and a SassAndCoffee.AspNet that depends on it, maybe the Core really will work in the Client Profile, even though the description says it depends on ASP.NET. I’ll dig into it and see if I can find out.
October 22nd, 2011 at 12:49 am
So it looks like the description is wrong — SassAndCoffee.Core doesn’t require ASP.NET, even though its description says it does.
However, it doesn’t seem to expose any options, and it looks like it hard-codes the ‘bare’ option to true, which is not what I want. (See my question on StackOverflow.)
I’ve also seen some weirdness where, if I use SassAndCoffee’s CoffeeScriptCompiler in my code, and I run my app under the debugger, the app never exits. It looks like JavaScriptBasedCompiler’s static constructor is starting a thread that isn’t marked as IsBackground and that never gets stopped. (Oddly enough, it seems to exit just fine when I’m not under the debugger.)
October 25th, 2011 at 10:41 am
This is John Gietzen, a contributor to IronJS.
We are looking into our performance problems now. However, we have noticed that compiling for 64-bit and for 32-bit have significantly different performance profiles, and I implore you to try out a few different modes before condemning IronJS.
Again, we acknowledge that this needs to be fixed, but in the meantime, try out a different CPU profile.
October 25th, 2011 at 3:43 pm
Interesting. Is it 32-bit that’s currently slow in IronJS, or 64-bit? I’m using Visual Studio 2010, which creates 32-bit projects by default, and I’d expect that’s what you would normally target.
October 25th, 2011 at 7:21 pm
I’m the author of Jurassic. It’s really nice to see Jurassic coming out on top
Saying that, I actually recommend v8 unless you need the zero-dependency, medium trust, or Silverlight capabilities that Jurassic provides. v8 smokes Jurassic and there’s no way I can compete, performance wise, with a team of Google engineers.
October 26th, 2011 at 5:56 pm
I’ll definitely look into JavaScript.NET, which wraps V8. The only reason I didn’t include it this time is that it’s not on NuGet.
December 20th, 2011 at 10:55 pm
So I have to ask, I have been looking at JINT, IronJS, and saw Javascript.Net a while ago when I first started looking for a JS .Net implementation. I am building a framework to allow clients/users to write custom code to do some business logic. Something I noticed is that Javsacript.Net hasn’t been active for over a year, while JINT and IronJS seem to be a lot more active. I haven’t looked at Jurassic as of yet, but I probably will, but any thoughts on stale code and whatnot?
December 24th, 2011 at 10:34 am
Chris, I haven’t used any of these projects that much, so I don’t have much experience to share with you — although if you’re using it for custom business logic, Jint has great sandboxing support that might interest you.