Embedding and AOT

[glyh]

I am trying to embedding janet into Zig, and I succeeded with embedding tutorial.

However, there's an overhead using the dostring/dobytes that I want to avoid. Is it possible to call compile functions in zig's compile time, so I can have bytes code embedded in my executable instead of having source code being compiled multiple times whenever I try to run the executable?

Thank you!

[CosmicToast]

Disclaimer: I'm still relatively new to Janet specifics, so what I say below might not be perfectly accurate.

You can do whatever dobytes does: https://github.com/janet-lang/janet/blob/master/src/core/run.c#L30
The short version is to parse using janet_parser_*, then compile individual forms using janet_compile.

Those forms can then be used at runtime by putting them into a fiber and janet_continueing it.

This all, of course, depends on you knowing exactly what you want to run ahead of time.
It may also interact poorly with imports and environments (you'll need to handle those either at runtime or build time somehow), though the approach of creating the fiber at runtime means that you can set the environment to whatever you want.

[sogaiu]

On a side note, if you haven't checked out @greenfork's jzignet yet, may be it's worth a look. There is specific mention of embedding in the README:

You can:

Embed Janet programs into Zig

There is also: https://hg.sr.ht/~paulsnar/zig-janet which is apparently based on the above (at least according to this).

...and there is this project which seems to be written in Zig with some Janet bits inside. No clue about specifics though (^^;

[glyh]

@sogaiu I did check jzignet, it's not compiling on my version of zig so I bascially rewrite my own. I don't think they have a solution for my problem, tho.

[omasanori]

I think #999 would answer some of your questions. Also, here are some code snippets.

janet/src/boot/boot.janet

Lines 4126 to 4136 in 4ff81a5

	(def image
	(let [env-pairs (pairs (env-lookup root-env))
	essential-pairs (filter (fn [[k v]] (or (cfunction? v) (abstract? v))) env-pairs)
	lookup (table ;(mapcat identity essential-pairs))
	reverse-lookup (invert lookup)]
	# Check no duplicate values
	(def temp @{})
	(eachp [k v] lookup
	(if (in temp v) (errorf "duplicate value: %v" v))
	(put temp v k))
	(marshal root-env reverse-lookup)))

janet/src/boot/boot.janet

Lines 4229 to 4237 in 4ff81a5

	(print "static const unsigned char janet_core_image_bytes[] = {")
	(loop [line :in (partition 16 image)]
	(prin " ")
	(each b line
	(prinf "0x%.2X, " b))
	(print))
	(print " 0\n};\n")
	(print "const unsigned char *janet_core_image = janet_core_image_bytes;")
	(print "size_t janet_core_image_size = sizeof(janet_core_image_bytes);"))

janet/src/core/corelib.c

Lines 1254 to 1264 in 4ff81a5

	Janet marsh_out = janet_unmarshal(
	janet_core_image,
	janet_core_image_size,
	0,
	dict,
	NULL);
	/* Memoize */
	janet_gcroot(marsh_out);
	JanetTable *env = janet_unwrap_table(marsh_out);
	janet_vm.core_env = env;

[omasanori]

Searching janet_core_image in the amalgamated janet.c may also be helpful.

[iacore]

One way is to run (make-image (curenv)) and load the image at runtime. Modules imported will be cached by module/cache, and will be in the image.

cfunction can't be included this way by default. You can modify janet_core_env() (add cfunction to it) before make-image.