Developing Labs Prototypes Repo

The lay of the land

This repository is configured as a TypeScript monorepo, using Node's built-in workspace capability.

Each item within the project lives as an npm package, though not all packages are actively published to npm.

Packages

Package Name	NPM	Description
Core
`@google-labs/breadboard`		The core Breadboard library
`@breadboard-ai/build`		Library for defining boards and node types
`@breadboard-ai/visual-editor`	Unpublished	The source code for the Breadboard Visual Editor
Kits
`@google-labs/agent-kit`		Breadboard kit for building agent-like experiences
`@google-labs/core-kit`		Breadboard kit for foundational board operations like `map` and `invoke`
`@google-labs/gemini-kit`		Breadboard kit for working with the Gemini API
`@google-labs/google-drive-kit`		Breadboard kit for working with Google Drive
`@google-labs/json-kit`		Breadboard kit for working with JSON in boards
`@google-labs/template-kit`		Breadboard kit to help with templating
`@google-labs/palm-kit`		Breadboard kit for working with the PaLM API in boards
`@exadev/breadboard-kits`		A variety of utilities, general purpose nodes, and kits specific scenarios
Tools & Support Libraries
`@google-labs/breadboard-cli`		Command-line tool for generating, running, and debugging boards
`@breadboard-ai/import`		Library for importing boards from an OpenAPI spec
Internal/Experiments
`@breadboard-ai/manifest`	Unpublished	An internal tool that generates a JSON list of board files
`@breadboard-ai/python-wasm`	Unpublished	Implementation of runPython component using WASM (Web Assembly)
`@google-labs/breadboard-python`	Unpublished	A library that allows developers to define Breadboards with Python
`@google-labs/breadboard-schema`	Unpublished	An internal tool that generates a JSON schema from the Breadboard TypeScript types
`@google-labs/breadboard-website`	Unpublished	The documentation website for Breadboard
`@google-labs/connection-server`	Unpublished	Used for handling connections to third party services
`@google-labs/discovery-types`	Unpublished	Library which generates TypeScript declarations from the PaLM API Discovery Document
`@google-labs/node-nursery-web`	Unpublished	A place for experimenting with board nodes that aren't yet ready for their own package (web specific)
`@google-labs/node-nursery`	Unpublished	A place for experimenting with board nodes that aren't yet ready for their own package (general)
`@google-labs/node-proxy-server`	Unpublished	Library that allows running nodes remotely

Building a package

We use Wireit as the build tool for the monorepo.

All packages within the monorepo have a similar structure:


├── package.json
├── src
│   └── index.ts
├────── <dir>
│        ├── <file>.ts
│
│        ...
├── tests
│   └── <file>.ts
│
│   ...

Project source files go into the src directory, while tests go into tests.

The TypeScript build is configured to produce a dist directory in the root of each package. This is the directory that is published to npm.

Getting started

After cloning the repo:

Install all of the dependencies for all of the packages in the monorepo:

npm i

To build the whole project:

npm run build

To run all tests:

npm run test

Most of the time, you will likely want to bring up the Breadboard Web UI. To do so, run the w command:

npm run w

To start a doc site server:

npm run d

To start a board server and a Breadboard Web UI:

npm run s

Occasionally, there will be changes that will require a full rebuild with installing new packages, etc.

When you find your repo in a funky state, use npm run clean. It will delete all build artifacts and bring the repo back to the pristine state, allowing you to restart with npm i and all those things.

Caution

If you have any local files that you have saved in the tree, they will be deleted.

npm run clean

Creating a new package

To create a new package:

1️⃣ Copy a template project as a new directory under packages. For example, if the name of your new prototyping project is awesome-ai-game, you would run something like this:

cp -rf templates/blank packages/awesome-ai-game

2️⃣ Replace the following placeholders:

{{name}} -- specify the name of the package. It usually matches the name of the directory you just created, like awesome-ai-game.
{{description}} -- describe the nature of the project in one sentence.

3️⃣ Add project as a reference to the tsconfig.json. For example, for awesome-ai-game:

{
  "extends": "@google-labs/tsconfig/base.json",
  "files": [],
  "references": [{ "path": "packages/awesome-ai-game" }]
}

4️⃣ Verify that you have the right setup. Run npm i and npm run build and make sure that the new package shows up in the build log. Then try to run it:

cd packages/awesome-ai-game
node .

You should see code goes here as output.

5️⃣ Build whatever it is you've dreamed up!

The new package is configured as a private package by default. If you would like to publish it to npm, flip the private flag to true in package.json.

Working on your project

If everything is set up well, you will spend most of your time tinkering inside of your package.

We recommend opening VSCode in the package directory to reduce the amount of clutter. When you run npm run inside of your package directory, it will automatically scope the build to only dependencies in your package.

To build your package:

npm run build

To test your package:

npm test

You can add more commands to package.json and invoke them using npm run <command>.

To add a new dependency for your package, just run npm i <package-name> in your package working directory.

When you need to refer to other packages in the monorepo, you will need to do a bit of manual wiring.

In your project's package.json edit the contents of dependencies (or devDependencies) to add the entry for the package on which you want this package to depend. For example, if we're adding a dependency on the package called "@google-labs/ai-game-engine" that also resides in this monorepo, we will do:

"dependencies": {
  "@google-labs/ai-game-engine": "*",
}

Testing

Out of the box, the project template is configured to use ava for testing. You can pick a different test framework. Just make sure to configure your package.json to point to it, so that npm can run it.

Cleaning stuff

Sometimes, TypeScript Compiler or Wireit (or both!) gets confused, and the only way forward is to clean up the build artifacts and start over. To do that, run:

npm run clean

Source Code Headers

Every file containing source code must include copyright and license information. This includes any JS/CSS files that you might be serving out to browsers. (This is to help well-intentioned people avoid accidental copying that doesn't comply with the license.)

/**
 * @license
 * Copyright 2023 Google LLC
 * SPDX-License-Identifier: Apache-2.0
 */

TypeScript Developer Happy Path

This section describes that we generally use within the Breadboard team, and it's probably what will give you the most comfortable developer experience in this repo.

VSCode

Get VSCode. There are tons of other editors, but VSCode has likely the most seamless TypeScript integration.

Install “ESLint” and “Prettier” extensions. TypeScript support is built-in by default.

“ESLint” – provides live commentary on the quality of your code.
“Prettier” – will make your code look pretty automatically.

Tweak the settings to your liking. The most important one is to format-on-save, so that you never need to worry about formatting TypeScript ever again. Here is what we have:

"editor.rulers": [80] <-- Draws a nice ruler at 80 columns
"editor.fontFamily": "Fira Code", <-- Fira Code support
"editor.fontLigatures": true, <-- make pretty arrows with Fira Code
"editor.defaultFormatter": "esbenp.prettier-vscode"
"editor.formatOnSave": true, <-- format with Prettier on save

Use the built-in terminal (Ctrl+`). For convenience, split the TERMINAL and PROBLEMS tabs horizontally.

This setup creates a really nice separation of the workspace: the top part is where we write code, and the bottom part is where we see if it works. As we type, the problems come and go in the bottom-right window. When we are ready to try running my code, we switch to the terminal and run it from there.

Because TypeScript is built-in, TypeScript errors will show up live in the PROBLEMS window as well, which is super-convenient. Learn keyboard shortcuts. Ctrl+P (Cmd+P) and Ctrl+Shift+P (Cmd+Shift+P) are likely the most important ones.

Occasionally, VSCode’s built-in TypeScript machinery gets into a confused state. For this purpose, there’s a “TypeScript: Restart TS Server“ command available via Cmd/Ctrl+Shift+P. We can also use the “Developer: Reload Windows“ command to flush out the gremlins.

Workflow

The dev cycle is:

Open the directory of the package (or several of them) in VSCode
Write some code
Make ESLint and TypeScript live-compiler happy (no errors show up in the PROBLEMS window)
Run npm run build to build the code. For packages like the Visual Editor we generally use npm run dev, which often provides a live reload feature in the browser.
Run your code with node . or whatever is the right way to run it. For convenience, create an npm script to combine building and running.
Go to the “Write some code” step.

Build system

This is a monorepo, which in Node.js vernacular means that it is a flat list of npm packages that are all hosted in the same git repository.

The main reason we need to run npm run build is because in the monorepo, we need to compute the dependency graph before compiling TypeScript to Javascript, and that is not something that comes standard with the TypeScript compiler.

Front-end

Vite is currently brought up in the visual-editor package. Use it as a template for other front-end TypeScript packages.

Sending PRs

This repo protects the main branch, which means all changes must go through a GitHub PR. This enforces that all tests pass and packages builds before any change lands, and provides an opportunity for code review.

Tip

The GitHub CLI makes it easy to send PRs by typing gh pr create. You can use the --fill or -f flag to automatically populate the title and description from your commits. See the create command documentation for more information.

Changesets

This repo uses Changesets to ease the burden of releasing of NPM packages. The benefits are that it publishes multiple packages at once, understands the dependencies between all packages in the monorepo, automatically updates the package.json and CHANGELOG.md files, and automatically creates release tags.

Tip

If you need to publish NPM packages, see the Publishing NPM packages section below.

After sending a PR, you may receive a comment from changeset-bot that looks like this:

This bot is telling you that your PR does not contain a Changeset file. Changeset files are how Changesets understands which packages need to be released at any given time, along with the kind of version bump that is needed for them.

The easiest way to create a Changeset file for your PR is to run this command from the root of the monorepo:

npx changeset

This command will prompt you with an interactive list of packages. Select the packages that the PR affects and indicate whether the changes are semver major (breaking), minor (new features), or patch (bug fixes).

Note

If your change only affects unpublished packages, then you can safely skip adding a changeset file and ignore the bot.

Then just push the generated changeset file to your PR!

Publishing NPM packages

To publish an NPM package, you have to be a Googler. This is unlikely to change in the future. Having said that, here are the steps to publish a package.

At the root of the repository, ensure you are synchronized to the tip of main and create a new release branch.
```
git checkout main
git pull
git checkout -b release
```
Use the Changesets version command to find all packages that need releasing and automatically update their package.json and CHANGELOG.md files. Note that Changesets automatically bumps the semver constraints for dependent packages when needed, so there is no need to manually edit any package.json files.
```
npx changeset version
```
[!NOTE] Sometimes, the command may fail with a fairly cryptic error like "TypeError: Cannot destructure property 'packageJson' of 'undefined' as it is undefined.". This typically means that a package was renamed sometime between the last release and now. To fix, look through the .md files in the ./.changeset directory, and remove all lines that reference the package by its old name.
Check what is planned to be published by looking at the latest commit which Changesets created in the previous step. Make sure it looks reasonable, and send a PR with the changes so that others can see what will be published.
```
git show
gh pr create -f # or git push if you don't have the gh tools
```
Sign in to NPM:
```
npm adduser
```
Generate a token for the Google NPM release proxy registry. Running the command below will open a browser window. Select 24 hour temporary token after which the command should exit by itself.
```
npm login --registry https://wombat-dressing-room.appspot.com
```
Wait for the PR from step 3 to pass CI.
Use the Changesets publish command to publish all changes and generate release tags (e.g. @google-labs/breadboard@0.8.0).
```
npx changeset publish
```
Push the release tags added in step 7 to GitHub so that they are associated with the commit from step 2.
```
git push --follow-tags
```
Merge the PR from step 3.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DEVELOPING.md

DEVELOPING.md

Developing Labs Prototypes Repo

The lay of the land

Packages

Building a package

Getting started

Creating a new package

Working on your project

Testing

Cleaning stuff

Source Code Headers

TypeScript Developer Happy Path

VSCode

Workflow

Build system

Front-end

Sending PRs

Changesets

Publishing NPM packages

Files

DEVELOPING.md

Latest commit

History

DEVELOPING.md

File metadata and controls

Developing Labs Prototypes Repo

The lay of the land

Packages

Building a package

Getting started

Creating a new package

Working on your project

Testing

Cleaning stuff

Source Code Headers

TypeScript Developer Happy Path

VSCode

Workflow

Build system

Front-end

Sending PRs

Changesets

Publishing NPM packages