Is new JIT-ed programming language in windows 8 metro/winRT possible?

Question

Lets say I want to create a new programming language or use some language not supported by MS (Haskell, Java... whatever) but want to be able to code against windows 8 metro/winRT.

I know that all apps in Metro UI are sandboxed.
I know that I can program in native C++, so I assume that I can do it also in C or assembly.
But:

1. Is it possible to create a JIT that will produce assembly code on the fly - like CLR JIT- without breaking sandbox restrictions?
2. Assuming that I will use only API allowed in Metro sandbox is it possible to not use XAML as UI interface? - Can I use Direct2D/DirectX directly?

Answer 1

Here's a trick I found to gain access to these missing APIs (VirtualAlloc, VirtualProtect), which are required in order to implement a jitter (Disclaimer: This is my blog).

By playing with unsafe C# code, you can by bypass the restrictions imposed on .NET reflection and invoke the internal P/Invoke method Win32Native.GetProcAddress. Once you have that function, you automatically get access to any Win32 function, including VirtualAlloc.

Answer 2

It's worth noting that even if you managed to get a JIT working, perhaps by emitting CLR bytecode instead of native machine code maybe, or even if you just implemented an interpreter without a JIT, you could only ever run code that was included in your original app package. If you downloaded script/bytecode/etc... from outside the application and tried to run/JIT/Interpret it in a "local context" (i.e. with direct access to WinRT libraries), you would (with possible exception - see note below) be in violation of section 3.9 of the app certification requirements:

3.9 All app logic must originate from, and reside in, your app package You app must not attempt to change or extend the packaged content through any form of dynamic inclusion of code or data that changes how the application interacts with the Windows Runtime, or behaves with regard to Store policy. It is not permissible, for example, to download a remote script and subsequently execute that script in the local context of your app package.

The tricky wording here is in this "local context" sentence. It is not clear what that means exactly. For example, if you have a web browser control with a portal open to a website (or an iframe in an html app that runs in the "web context"), that web browser/iframe is "running" JITed javascript code, but it will run in a different "context", and doesn't have direct access to WinRT apis. It's not clear if this exception is only for the IE browser control in XAML or the iframe w/web context in html apps, or whether you could implement your own "web context" in your interpreter or dynamic runtime.

That being said, you are allowed to pre-compile any kind of application code as a way to get that code into an app and certified, similar to how it's done on iOS, except that you have the option of pre-compiling to .NET, native machine code, or javascript.

Answer 3

1) Apart from browsers that could have access to some restricted API (and could be able to integrate their JIT, but afaik, authorized APIs for Browsers are not yet published), standard Win8 Metro App will not have any access to functions like "VirtualAlloc/VirtualProtect" (used to create/change read/write/executable memory pages): It means that JIT developed in C++ under a Windows 8 Metro App won't pass certification . You can already check your application with the certification toolkit available with Visual Studio 11 Beta.

Someone could say "I'm going to hack the PE section to force read+write+executable" section without having to use VirtualAlloc/VirtualProtect functions, but unfortunately, this hack won't work either, as you are forced to compile a Windows 8 Metro exe with the option /NXCOMPAT:YES, which means that It is enabling "Data Execution Prevention" (DEP).

Another one could be tempted to generate DLL on the fly and load them from the disk using the new "LoadPackagedLibrary" but this function is actually locked if the DLL was not part of the original deployment (actually, I have not been able to make it working properly)

The only JIT that is available is the .NET JIT. All .NET dynamic languages like IronPython/IronRuby (if they are updated for Win8 Metro) that are using the DLR (Dynamic Language Runtime in .NET) or even the reflection Emit will be jitted. So if you target .NET CLR bytecode, you can have your code jitted.

IKVM.Net for example, that is able to run Java code inside a .NET CLR could be able to run under a Windows 8 Metro App (yet at the condition that It will be refactored to use only certified Win8 Metro API).

2) Yes, it is possible to write a Direct2D/Direct3D11 application without using XAML.

Check Windows 8 Metro samples http://code.msdn.microsoft.com/windowsapps/Windows-8-Modern-Style-App-Samples. For example, most Direct2D samples are not using XAML.

The CoreWindow class that is a lower level API to the Metro Window system is the entry point for a pure Direct2D/Direct3D11 application.

Answer 4

As near as I can tell, it is way too early to make the call on this. I personally don't know how to write a jitter without using VirtualProtect(), the core winapi function that lets you turn a chunk of memory with machine code generated by the jitter into executable code.

There are a number of native winapi functions available to a WinRT app. The list of blessed system functions is available here. The memory related apis are pretty limited, VirtualQuery is the only one in the list that comes close.

So how do the current language projections do it? Let's have a look. The CLR has a projection, it gets loaded into any Metro app you write in a managed language like C#. Running dumpbin.exe /imports on c:\windows\microsoft.net\framework\v4.0.30319\clr.dll generates a pretty big list of dependencies on Windows DLLs. A snippet from that dump:

Dump of file clr.dll

File Type: DLL

  Section contains the following imports:

    KERNEL32.dll
...

              430 RaiseException
              581 VirtualAlloc
              584 VirtualFree
              589 VirtualQuery
              587 VirtualProtect          <=== here!
              339 HeapDestroy
              336 HeapAlloc
              342 HeapValidate
              540 SleepEx
              547 SwitchToThread
              ... etc

Another language projection is for javascript, implemented in the "Chakra" engine. Hard to figure out exactly what DLL implements that engine, it's just a code name. Running a sample Javascript project with unmanaged debugging enabled reveals "jscript9.dll" loaded. Let's do dumpbin.exe /imports on this one:

  ....
  6898F4D5    10D DebugBreak
  6891FDA1    55E TerminateProcess
  6898EF9E    57E UnhandledExceptionFilter
  6891FD58    43C RaiseException
  68903BB7    59E VirtualProtect                 <=== here!
  6A218590    366 InterlockedPushEntrySList
  6A2185A9    365 InterlockedPopEntrySList
  6A2195AA    35C InitializeSListHead
  689026F9    598 VirtualAlloc
  68902852    59B VirtualFree
  6890603E    4A2 ResetWriteWatch
  ...etc

Well, it is there. It would have to be. Trouble is, right now you can't call this function. It certainly wouldn't pass scrutiny from the Store validator.

This needs to stew, at least until the real WinRT becomes available, the one that runs on ARM cores. Not just the one that runs on top of Win32 that you now have running in the Windows 8 Consumer Preview. And can readily take advantage of existing winapi functions, not just the trimmed list. That's going to be around the end of the year, probably. Real hardware won't be in your hands until summer of next year.

Answer 5

Yes, it is possible to write a projection onto WinRT for other programming languages. It's even encouraged. A conference called Lang.Next was held on the Microsoft campus earlier this month, and it was all about language design. One of the sessions was specifically on this topic, and you might want to watch it (I have): http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012/The-Windows-Runtime and read the comments on the page also. Let me quote one:

Martyn made it clear that we not only want language designers and implementers to add WinRT to their list of target platforms for their language and toolchains, but we will help and advise.

So get started! :-)