Linux binfmt_misc

Thanks to special feature of Linux kernel, it is possible to register custom binary formats as executable. If specific binary format was registered though binfmt_misc interface and containing file has executable atribute set, kernel will execute such file using specified interpreter/emulator.

To learn more about this feature you can visit following pages:

• http://en.wikipedia.org/wiki/Binfmt_misc - short informative article with some examples
• http://www.kernel.org/doc/Documentation/binfmt_misc.txt - more in-depth documentation

Support for this has to be compiled into the kernel and can be verified following way:

# check whether there is support for this feature
zcat /proc/config.gz | grep -i binfmt_misc

# should output:
CONFIG_BINFMT_MISC=y

If enabled, place where in the root directory tree this interface is exposed, is distribution and init system specific. You can use findmnt tool to locate mount point in question:

# locate the binfmt_misc control interface
findmnt binfmt_misc

# sample output:
TARGET                   SOURCE      FSTYPE      OPTIONS
/proc/sys/fs/binfmt_misc binfmt_misc binfmt_misc rw,relatime

According to http://repo.or.cz/w/luajit-2.0.git/blob/HEAD:/src/lj_bcdump.h LuaJIT bytecode magic bytes are 0x1b, 0x4cx, 0x4a and fourth byte designates LuaJIT version. Header starts at offset 0 and default LuaJIT runtime should handle version checking itself.

/* Bytecode dump header. */
#define BCDUMP_HEAD1            0x1b
#define BCDUMP_HEAD2            0x4c
#define BCDUMP_HEAD3            0x4a

In our example, we can thus register LuaJIT bytecode as executable format following way:

# as root, we write following string to control file
# don't forget paths may be different on your distribution!
echo ":luajit:M::\x1b\x4c\x4a::/usr/bin/luajit:" > /proc/sys/fs/binfmt_misc/register

#we can verify that registration was sucessful with ls command:
ls -al /proc/sys/fs/binfmt_misc

# sample output:
luajit	register  status

With this setup, we can now run bytecode directly as normal binaries:

#  Create some bytecode
luajit -be "print('hello world through binfmt_misc interface')" binfmt_test
# mark file as executable
chmod +x binfmt_test     
# testing run:
./binfmt_test

# final output:
hello world through binfmt_misc interface

As we can see, this specific Linux feature allows it to treat LuaJIT compiled bytecode as first class executable format. Number of possible applications is endless.

For example, as bytecode can load faster than pure lua code, often or early invoked scripts can be precompiled into bytecode and even stored on initial system ramdisk, along with luajit and other required dependencies. This allows us to achieve maximum efficiency and speed of invocation even without the need of shell, which might be important in constrained environments like live CDs and bootable USB keys or even embedded systems.

One could also create utility programs and scripts which can seem indistinguishable from compiled applications to normal user, without giving up benefits of fast development turnaroud and flexibility of lua.