typescript has to be installed globally and have tsc in path

[reconbot]

When using a typescript project, typescript has to be installed globally and your shell has to have tsc in it's path. This isn't mentioned in the docs or via the doctor command.

[phoddie]

(These notes are more for Nick than Francis, but more voices on the optimal behavior here can only help.)

@reconbot raises a good point. The TypeScript compiler is an example of a dependency that applies to some Moddable SDK projects, but not all. The fontbm tool is another. We don't want to require every developer to install these tools, but we don't want developers to puzzle through build failures when they are missing.

In theory recursively processing the project manifest can identify these dependencies. Processing the manifest is an adventure. I recently updated mcconfig to output a flat version of the project manifest to manifest_flat.json. That is a start but not enough to easily know, for example, if there are TypeScript files or use of fontbm.

I did some more hacking on mcconfig tonight to add that information to manifest_flat.json. Just by checking manifest.["x-files-info"].ts.length a tool can know if aa project uses TypeScript. Here's a simple example of output for helloworld on macOS with main.js changed to main.ts and manifest_typings.json included:

manifest_flat.json

{
	"config": {
		"format": "RGB565LE",
		"rotation": 0
	},
	"creation": {
		"keys": {
			"initial": 32,
			"incremental": 32,
			"name": 53,
			"symbol": 3
		},
		"parser": {
			"buffer": 1024,
			"table": 17
		},
		"main": "main",
		"chunk": {
			"initial": 32768,
			"incremental": 1024
		},
		"heap": {
			"initial": 2048,
			"incremental": 64
		},
		"stack": 384,
		"static": 0
	},
	"defines": {
		
	},
	"data": {
		
	},
	"modules": {
		"*": [
			"/Users/hoddie/Projects/moddable/build/simulators/modules/screen",
			"/Users/hoddie/Projects/moddable/modules/base/time/*",
			"/Users/hoddie/Projects/moddable/modules/base/time/mac/*",
			"/Users/hoddie/Projects/moddable/modules/base/timer/*",
			"/Users/hoddie/Projects/moddable/modules/base/timer/mac/*",
			"/Users/hoddie/Projects/moddable/modules/base/timer/modTimer",
			"/Users/hoddie/Projects/moddable/modules/files/resource/*",
			"/Users/hoddie/Projects/moddable/modules/base/instrumentation/modInstrumentation",
			"/Users/hoddie/Projects/moddable/typings/*",
			"/Users/hoddie/Projects/moddable/examples/helloworld/main"
		],
		"commodetto/*": [
			"/Users/hoddie/Projects/moddable/typings/commodetto/*"
		],
		"dns/*": [
			"/Users/hoddie/Projects/moddable/typings/dns/*"
		],
		"mc/*": [
			"/Users/hoddie/Projects/moddable/typings/mc/*"
		],
		"pins/*": [
			"/Users/hoddie/Projects/moddable/typings/pins/*"
		],
		"piu/*": [
			"/Users/hoddie/Projects/moddable/typings/piu/*"
		],
		"text/*": [
			"/Users/hoddie/Projects/moddable/typings/text/*"
		],
		"embedded:provider/*": [
			"/Users/hoddie/Projects/moddable/typings/embedded_provider/*"
		],
		"embedded:io/*": [
			"/Users/hoddie/Projects/moddable/typings/embedded_io/*"
		],
		"embedded:io/socket/*": [
			"/Users/hoddie/Projects/moddable/typings/embedded_io/socket/*"
		]
	},
	"resources": {
		
	},
	"ble": {
		
	},
	"recipes": {
		
	},
	"preload": [
		"time",
		"timer",
		"Resource"
	],
	"strip": "*",
	"commonjs": [
		
	],
	"errors": [
		
	],
	"warnings": [
		
	],
	"run": {
		
	},
	"typescript": {
		"compiler": "tsc",
		"tsconfig": {
			"compilerOptions": {
				"types": [
					"/Users/hoddie/Projects/moddable/xs/includes/xs",
					"/Users/hoddie/Projects/moddable/typings/global"
				]
			}
		}
	},
	"x-files-info": {
		"bleServices": [
			
		],
		"bmpFont": [
			
		],
		"bmpMask": [
			
		],
		"c": [
			{
				"target": "screen.c.o",
				"source": "/Users/hoddie/Projects/moddable/build/simulators/modules/screen.c",
				"recipe": null
			},
			{
				"target": "modTime.c.o",
				"source": "/Users/hoddie/Projects/moddable/modules/base/time/mac/modTime.c",
				"recipe": null
			},
			{
				"target": "modTimer.c.o",
				"source": "/Users/hoddie/Projects/moddable/modules/base/timer/modTimer.c",
				"recipe": null
			},
			{
				"target": "timer.c.o",
				"source": "/Users/hoddie/Projects/moddable/modules/base/timer/mac/timer.c",
				"recipe": null
			},
			{
				"target": "Resource.c.o",
				"source": "/Users/hoddie/Projects/moddable/modules/files/resource/Resource.c",
				"recipe": null
			},
			{
				"target": "modInstrumentation.c.o",
				"source": "/Users/hoddie/Projects/moddable/modules/base/instrumentation/modInstrumentation.c",
				"recipe": null
			}
		],
		"cdv": [
			
		],
		"clut": [
			
		],
		"data": [
			
		],
		"dts": [
			{
				"target": "Resource.d",
				"source": "/Users/hoddie/Projects/moddable/typings/Resource.d.ts"
			},
			{
				"target": "base64.d",
				"source": "/Users/hoddie/Projects/moddable/typings/base64.d.ts"
			},
			{
				"target": "ble.d",
				"source": "/Users/hoddie/Projects/moddable/typings/ble.d.ts"
			},
			{
				"target": "console.d",
				"source": "/Users/hoddie/Projects/moddable/typings/console.d.ts"
			},
			{
				"target": "crc.d",
				"source": "/Users/hoddie/Projects/moddable/typings/crc.d.ts"
			},
			{
				"target": "debug.d",
				"source": "/Users/hoddie/Projects/moddable/typings/debug.d.ts"
			},
			{
				"target": "deepEqual.d",
				"source": "/Users/hoddie/Projects/moddable/typings/deepEqual.d.ts"
			},
			{
				"target": "dns.d",
				"source": "/Users/hoddie/Projects/moddable/typings/dns.d.ts"
			},
			{
				"target": "file.d",
				"source": "/Users/hoddie/Projects/moddable/typings/file.d.ts"
			},
			{
				"target": "flash.d",
				"source": "/Users/hoddie/Projects/moddable/typings/flash.d.ts"
			},
			{
				"target": "global.d",
				"source": "/Users/hoddie/Projects/moddable/typings/global.d.ts"
			},
			{
				"target": "hex.d",
				"source": "/Users/hoddie/Projects/moddable/typings/hex.d.ts"
			},
			{
				"target": "http.d",
				"source": "/Users/hoddie/Projects/moddable/typings/http.d.ts"
			},
			{
				"target": "instrumentation.d",
				"source": "/Users/hoddie/Projects/moddable/typings/instrumentation.d.ts"
			},
			{
				"target": "mdns.d",
				"source": "/Users/hoddie/Projects/moddable/typings/mdns.d.ts"
			},
			{
				"target": "modules.d",
				"source": "/Users/hoddie/Projects/moddable/typings/modules.d.ts"
			},
			{
				"target": "net.d",
				"source": "/Users/hoddie/Projects/moddable/typings/net.d.ts"
			},
			{
				"target": "ping.d",
				"source": "/Users/hoddie/Projects/moddable/typings/ping.d.ts"
			},
			{
				"target": "preference.d",
				"source": "/Users/hoddie/Projects/moddable/typings/preference.d.ts"
			},
			{
				"target": "securesocket.d",
				"source": "/Users/hoddie/Projects/moddable/typings/securesocket.d.ts"
			},
			{
				"target": "sntp.d",
				"source": "/Users/hoddie/Projects/moddable/typings/sntp.d.ts"
			},
			{
				"target": "socket.d",
				"source": "/Users/hoddie/Projects/moddable/typings/socket.d.ts"
			},
			{
				"target": "structuredClone.d",
				"source": "/Users/hoddie/Projects/moddable/typings/structuredClone.d.ts"
			},
			{
				"target": "telnet.d",
				"source": "/Users/hoddie/Projects/moddable/typings/telnet.d.ts"
			},
			{
				"target": "time.d",
				"source": "/Users/hoddie/Projects/moddable/typings/time.d.ts"
			},
			{
				"target": "timer.d",
				"source": "/Users/hoddie/Projects/moddable/typings/timer.d.ts"
			},
			{
				"target": "uuid.d",
				"source": "/Users/hoddie/Projects/moddable/typings/uuid.d.ts"
			},
			{
				"target": "websocket.d",
				"source": "/Users/hoddie/Projects/moddable/typings/websocket.d.ts"
			},
			{
				"target": "wifi.d",
				"source": "/Users/hoddie/Projects/moddable/typings/wifi.d.ts"
			},
			{
				"target": "worker.d",
				"source": "/Users/hoddie/Projects/moddable/typings/worker.d.ts"
			},
			{
				"target": "zip.d",
				"source": "/Users/hoddie/Projects/moddable/typings/zip.d.ts"
			},
			{
				"target": "commodetto/Bitmap.d",
				"source": "/Users/hoddie/Projects/moddable/typings/commodetto/Bitmap.d.ts"
			},
			{
				"target": "commodetto/PixelsOut.d",
				"source": "/Users/hoddie/Projects/moddable/typings/commodetto/PixelsOut.d.ts"
			},
			{
				"target": "commodetto/Poco.d",
				"source": "/Users/hoddie/Projects/moddable/typings/commodetto/Poco.d.ts"
			},
			{
				"target": "commodetto/index.d",
				"source": "/Users/hoddie/Projects/moddable/typings/commodetto/index.d.ts"
			},
			{
				"target": "commodetto/outline.d",
				"source": "/Users/hoddie/Projects/moddable/typings/commodetto/outline.d.ts"
			},
			{
				"target": "commodetto/parseBMF.d",
				"source": "/Users/hoddie/Projects/moddable/typings/commodetto/parseBMF.d.ts"
			},
			{
				"target": "commodetto/parseBMP.d",
				"source": "/Users/hoddie/Projects/moddable/typings/commodetto/parseBMP.d.ts"
			},
			{
				"target": "commodetto/parseRLE.d",
				"source": "/Users/hoddie/Projects/moddable/typings/commodetto/parseRLE.d.ts"
			},
			{
				"target": "commodetto/readJPEG.d",
				"source": "/Users/hoddie/Projects/moddable/typings/commodetto/readJPEG.d.ts"
			},
			{
				"target": "dns/index.d",
				"source": "/Users/hoddie/Projects/moddable/typings/dns/index.d.ts"
			},
			{
				"target": "dns/parser.d",
				"source": "/Users/hoddie/Projects/moddable/typings/dns/parser.d.ts"
			},
			{
				"target": "dns/serializer.d",
				"source": "/Users/hoddie/Projects/moddable/typings/dns/serializer.d.ts"
			},
			{
				"target": "dns/server.d",
				"source": "/Users/hoddie/Projects/moddable/typings/dns/server.d.ts"
			},
			{
				"target": "mc/config.d",
				"source": "/Users/hoddie/Projects/moddable/typings/mc/config.d.ts"
			},
			{
				"target": "mc/index.d",
				"source": "/Users/hoddie/Projects/moddable/typings/mc/index.d.ts"
			},
			{
				"target": "pins/analog.d",
				"source": "/Users/hoddie/Projects/moddable/typings/pins/analog.d.ts"
			},
			{
				"target": "pins/audioin.d",
				"source": "/Users/hoddie/Projects/moddable/typings/pins/audioin.d.ts"
			},
			{
				"target": "pins/audioout.d",
				"source": "/Users/hoddie/Projects/moddable/typings/pins/audioout.d.ts"
			},
			{
				"target": "pins/digital.d",
				"source": "/Users/hoddie/Projects/moddable/typings/pins/digital.d.ts"
			},
			{
				"target": "pins/i2c.d",
				"source": "/Users/hoddie/Projects/moddable/typings/pins/i2c.d.ts"
			},
			{
				"target": "pins/index.d",
				"source": "/Users/hoddie/Projects/moddable/typings/pins/index.d.ts"
			},
			{
				"target": "pins/pwm.d",
				"source": "/Users/hoddie/Projects/moddable/typings/pins/pwm.d.ts"
			},
			{
				"target": "pins/rmt.d",
				"source": "/Users/hoddie/Projects/moddable/typings/pins/rmt.d.ts"
			},
			{
				"target": "pins/servo.d",
				"source": "/Users/hoddie/Projects/moddable/typings/pins/servo.d.ts"
			},
			{
				"target": "pins/smbus.d",
				"source": "/Users/hoddie/Projects/moddable/typings/pins/smbus.d.ts"
			},
			{
				"target": "piu/CombTransition.d",
				"source": "/Users/hoddie/Projects/moddable/typings/piu/CombTransition.d.ts"
			},
			{
				"target": "piu/MC.d",
				"source": "/Users/hoddie/Projects/moddable/typings/piu/MC.d.ts"
			},
			{
				"target": "piu/Sound.d",
				"source": "/Users/hoddie/Projects/moddable/typings/piu/Sound.d.ts"
			},
			{
				"target": "piu/Timeline.d",
				"source": "/Users/hoddie/Projects/moddable/typings/piu/Timeline.d.ts"
			},
			{
				"target": "piu/WipeTransition.d",
				"source": "/Users/hoddie/Projects/moddable/typings/piu/WipeTransition.d.ts"
			},
			{
				"target": "piu/index.d",
				"source": "/Users/hoddie/Projects/moddable/typings/piu/index.d.ts"
			},
			{
				"target": "piu/shape.d",
				"source": "/Users/hoddie/Projects/moddable/typings/piu/shape.d.ts"
			},
			{
				"target": "text/decoder.d",
				"source": "/Users/hoddie/Projects/moddable/typings/text/decoder.d.ts"
			},
			{
				"target": "text/encoder.d",
				"source": "/Users/hoddie/Projects/moddable/typings/text/encoder.d.ts"
			},
			{
				"target": "text/index.d",
				"source": "/Users/hoddie/Projects/moddable/typings/text/index.d.ts"
			},
			{
				"target": "~.embedded/provider/builtin.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_provider/builtin.d.ts"
			},
			{
				"target": "~.embedded/provider/index.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_provider/index.d.ts"
			},
			{
				"target": "~.embedded/io/_common.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/_common.d.ts"
			},
			{
				"target": "~.embedded/io/analog.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/analog.d.ts"
			},
			{
				"target": "~.embedded/io/digital.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/digital.d.ts"
			},
			{
				"target": "~.embedded/io/digitalbank.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/digitalbank.d.ts"
			},
			{
				"target": "~.embedded/io/i2c.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/i2c.d.ts"
			},
			{
				"target": "~.embedded/io/index.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/index.d.ts"
			},
			{
				"target": "~.embedded/io/pulsecount.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/pulsecount.d.ts"
			},
			{
				"target": "~.embedded/io/pwm.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/pwm.d.ts"
			},
			{
				"target": "~.embedded/io/serial.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/serial.d.ts"
			},
			{
				"target": "~.embedded/io/smbus.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/smbus.d.ts"
			},
			{
				"target": "~.embedded/io/spi.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/spi.d.ts"
			},
			{
				"target": "~.embedded/io/system.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/system.d.ts"
			},
			{
				"target": "~.embedded/io/socket/listener.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/socket/listener.d.ts"
			},
			{
				"target": "~.embedded/io/socket/tcp.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/socket/tcp.d.ts"
			},
			{
				"target": "~.embedded/io/socket/udp.d",
				"source": "/Users/hoddie/Projects/moddable/typings/embedded_io/socket/udp.d.ts"
			}
		],
		"h": [
			"/Users/hoddie/Projects/moddable/build/simulators/modules/screen.h",
			"/Users/hoddie/Projects/moddable/modules/base/timer/modTimer.h",
			"/Users/hoddie/Projects/moddable/modules/base/instrumentation/modInstrumentation.h"
		],
		"image": [
			
		],
		"js": [
			{
				"target": "screen.xsb",
				"source": "/Users/hoddie/Projects/moddable/build/simulators/modules/screen.js",
				"preload": false
			},
			{
				"target": "time.xsb",
				"source": "/Users/hoddie/Projects/moddable/modules/base/time/time.js",
				"preload": true
			},
			{
				"target": "timer.xsb",
				"source": "/Users/hoddie/Projects/moddable/modules/base/timer/timer.js",
				"preload": true
			},
			{
				"target": "Resource.xsb",
				"source": "/Users/hoddie/Projects/moddable/modules/files/resource/Resource.js",
				"preload": true
			},
			{
				"source": "/Users/hoddie/Projects/moddable/build/tmp/mac/mc/debug/helloworld/mc.config.js",
				"target": "mc/config.xsb"
			}
		],
		"nodered2mcu": [
			
		],
		"resources": [
			
		],
		"sound": [
			
		],
		"string": [
			
		],
		"ts": [
			{
				"target": "main.xsb",
				"source": "/Users/hoddie/Projects/moddable/examples/helloworld/main.ts",
				"preload": false
			}
		]
	}
}

With this in place, xs-dev could use mcconfig -d (without the -m) to process the manifest. It can check the dependencies. Once those are up-to-date, it can either execute make on the resulting makefile or just run mcconfig -d -m (maybe that's fast enough).

Here's the patch to mcconfig.js to output all the file information. It goes at line 1139:

mcconfig.js patch

		var file = new DefinesFile(this.tmpPath + this.slash + "mc.defines.h", this);
		file.generate(this);

		const flat = {
			...this.manifest,
			["x-files-info"]: {
				bleServices: this.bleServicesFiles.slice(),
//				bmpAlpha: this.bmpAlphaFiles.slice(),
//				bmpColor: this.bmpColorFiles.slice(),
				bmpFont: this.bmpFontFiles.slice(),
				bmpMask: this.bmpMaskFiles.slice(),
				c: this.cFiles.slice(),
				cdv: this.cdvFiles.slice(),
				clut: this.clutFiles.slice(),
				data: this.dataFiles.slice(),
				dts: this.dtsFiles.slice(),
				h: this.hFiles.slice(),
				image: this.imageFiles.slice(),
				js: this.jsFiles.slice(),
				nodered2mcu: this.nodered2mcuFiles.slice(),
				resources: this.resourcesFiles.slice(),
				sound: this.soundFiles.slice(),
				string: this.stringFiles.slice(),
				ts: this.tsFiles.slice(),
			}
		};

		this.writeFileString(this.tmpPath + this.slash + "manifest_flat.json", JSON.stringify(flat, undefined, "\t")); 

[HipsterBrown]

@phoddie Does the typescript.compiler field refer to the binary that is called when processing the project's TypeScript files? Could that be a path to a binary within the project's node_modules directory?

With this in place, xs-dev could use mcconfig -d (without the -m) to process the manifest. It can check the dependencies.

I wonder if there is also a path towards making mcconfig.js an import-able ES module for xs-dev when available, rather than shelling out for this info. That is probably a longer term refactor and collaboration, but I want to surface the idea for future conversations around a smoother developer experience.

For the manifest_flat.json approach, xs-dev could hash & cache that file to make subsequent runs quicker when the project manifest doesn't change.

[phoddie]

Does the typescript.compiler field refer to the binary that is called when processing the project's TypeScript files? Could that be a path to a binary within the project's node_modules directory?

Yes, that appears to be the case.

I wonder if there is also a path towards making mcconfig.js an import-able ES module for xs-dev when available, rather than shelling out for this info. That is probably a longer term refactor and collaboration, but I want to surface the idea for future conversations around a smoother developer experience.

Should be possible. I'm open to exploring that. The implementation of mcconfig / mcrun is somewhat tightly coupled to XS, however, as it relies on the native tools module. The same could be done in Node too. Still, mcconfig runs very quickly (the build takes time, but the processing is nearly instant) so performance isn't a clear motivation.

For the manifest_flat.json approach, xs-dev could hash & cache that file to make subsequent runs quicker when the project manifest doesn't change.

Very true. That would certainly let you avoid unnecessary checks in the environment (e.g. is tsc available). But it is the generated makefile that actually decides which files have changed and, consequently, what needs to be rebuilt.