[Idea] Use external components in Toolpad

[Janpot]

Use external components in Toolpad

Toolpad brings a lightweight code editor to create custom components. But the appeal of pro-code is that users are free in choosing their tooling, editors, libraries. We should give users back this freedom by allowing them to build custom components outside of Toolpad, in their local environment. Toolpad should be able to import such components and allow for visually composing UI with them.

Proposal

The user needs to create some sort of manifest that describes the content of their library

{
  "components": {
    "Map": {
      "file": "./src/Map.ts",
      "description": "Google Maps component"
    },
    "Chart": {
      "file": "./src/Chart.ts"
    },
    "Rating": {
      "file": "./src/Rating.ts"
    }
  }
}

The components themselves are created with createComponent as we have them now.

Create a CLI tool that has the following commands:

toolpad build

• It inspects the manifest
• Bundles all of the components and their dependencies
• creates a folder with all built files + creates a runtime manifest that maps components to their file
• build target is ESM. All dependencies are bundled, except for the ones available in toolpad. e.g. react, @mui/material, ...

toolpad pack

This builds and packs the library into a zip file. The user can upload this zipfile to Toolpad to make the components available.

considerations

• Ideally, once supplied in Toolpad, the manifest and files are downloaded once and stored on the instance forever. The user will have to refresh or download the library again to update. We should keep each version we download around for easy rollback.
• In addition to uploading the library, we can think of allowing users to host it and supply a url

[bharatkashyap]

The components themselves are created with createComponent as we have them now.

Does this mean that the user has to manually define props in argType definition syntax as we have them now?

[Janpot]

Yes, that will probably always be the case as we can't express in the typesystem everything Toolpad needs to know about components. In the future we can think about extracting a default interface during the build when the user omits them.

[bharatkashyap]

This builds and packs the library into a zip file. The user can upload this zipfile to Toolpad to make the components available.

This seems like a lot of additional manual effort for low reward. If the upside is simply being able to keep the code on my local filesystem, why not allow users to sync custom component code from their GitHub repositories? Something like this:

[Janpot]

This still is one part of the equation. The components still need to be in a format that Toolpad can understand. meaning:

• with all dependencies bundled
• in commonjs format
• with react externalized (and react-dom, @mui/*, ...)

We currenlty made it "sort of" work for ESM code that doesn't import react by using import, but we don't have any way to supply the toolpad react instance to this code.

[apedroferreira]

Not sure what other alternatives there are, but I've used https://rollupjs.org before and it would probably work well for building.

[Janpot]

personally I think esbuild would make sense these days. but that's an implementation detail in the end

[apedroferreira]

Yeah true, also there must be newer tools these days that I haven't used, esbuild looks great.

[oliviertassinari]

I think that this GitHub discussion should also cover one important aspect of the experience:
What does it look like with local development? It feels that a build command only will create friction when the component imported needs to be "fixed". I can only think of two types of solutions:

1. A local running server, that exposes the component. Something like it feels https://www.builder.io/c/docs/integrating-builder-pages is doing. I imagine using the same method as https://docs.plasmic.app/learn/nextjs-quickstart/#framework=nextjs, https://www.makeswift.com/docs/guides/manual-setup. You have something running locally on port http://0.0.0.0:3000/, and the tool displays it on the webpage.
2. A local running server, that is self-sufficient. Something like it feels https://docs.airplane.dev/getting-started/views#develop-your-view-locally is doing.
3. A local running server, in watch mode. Maybe it's the same as 1. anything that allows display on demand the components.

[Janpot]

1. […] You have something running locally on port http://0.0.0.0:3000/, and the tool displays it on the webpage.

I think that will run into all kinds of cross origin issues, mismatching protocols to start with. We'll likely have to setup some tunneling logic similar to ngrok to make the build output available to a Toolpad server.

[oliviertassinari]

Related to #390