Add process guide for migrating packages into the core monorepo

docs/package-migration-process-guide.md

@@ -0,0 +1,213 @@
+# Package Migration Process Guide
+
+This document outlines the process for migrating a MetaMask library into the core monorepo. The migration target is assumed to comply with the requirements defined by [`metamask-module-template`](https://github.com/MetaMask/metamask-module-template) and [`module-lint`](https://github.com/MetaMask/module-lint).
+
+## Phase A: **Preparation** in the _Source Repo_
+
+### **[PR#1]** 1. Add the following migration notice to the README
+
+```markdown
+<table><tr><td><p align="center"><b>⚠️ PLEASE READ ⚠️</b></p><p align="center">This package is currently being migrated to our <a href="https://github.com/MetaMask/core"><code>core</code></a> monorepo. Please do not make any commits to this repository while this migration is taking place, as they will not be transferred over. Also, please re-open PRs that are under active development in the core repo.</p></td></tr></table>
+```
+
+- [Example PR](https://github.com/MetaMask/eth-json-rpc-provider/pull/38)
+
+### 2. Disable `dependabot` dependency upgrades
+
+- [Disable dependabot alerts](https://docs.github.com/en/code-security/dependabot/dependabot-alerts/configuring-dependabot-alerts#enabling-or-disabling-dependabot-alerts-for-a-repository) for the repo.
+- Delete the source repo's `.github/dependabot.yml` file.
+- Contact a [**maintainer**](https://github.com/orgs/MetaMask/teams/engineering?query=role%3Amaintainer) to perform this step.
+
+### 3. Add the source repo to the ZenHub workspace repo filter so that its issues/PRs show up on the board
+
+### **[PR#2]** 4. Align dependency versions and TypeScript, ESLint, Prettier configurations with the core monorepo
+
+- If the dependency versions of the migration target are ahead of core, consider updating the core dependencies first.
+- Apply the configurations of the core monorepo to the source repo files.
+  - Preserve any TypeScript compiler flags that are enabled in the source repo but not in core.
+- Resolve all errors or issues resulting from these changes.
+- [Example PR](https://github.com/MetaMask/eth-json-rpc-provider/pull/28)
+
+### **[PR#3]** 5. Review the `metamask-module-template`, and add any missing files or elements (e.g. LICENSE)
+
+- [Example PR](https://github.com/MetaMask/eth-json-rpc-provider/pull/24)
+
+### **[PR#4]** 6. If applicable, rename the migration target package so that it is prepended by the `@metamask/` namespace
+
+- Modify the "name" field in `package.json`.
+- Update the title of the README.md.
+- Add a CHANGELOG entry for the rename.
+
+### **[PR#5]** 7. Create a new release of the migration target from the source repo
+
+- All subsequent releases of the migration target will be made from the core monorepo.
+- [Example PR](https://github.com/MetaMask/eth-json-rpc-provider/pull/29)
+
+## Phase B: **Staging** from the core monorepo's `merged-packages/` directory
+
+### **[PR#6]** 1. Migrate the source repo's git history into the `merged-packages/` temporary directory in core
+
+1. [Install `git-filter-repo`](https://github.com/newren/git-filter-repo/blob/main/INSTALL.md). This tool is like Git's `filter-branch` command, but much easier to use and much less error-prone.
+2. Navigate to a temporary directory: `cd /tmp`
+3. Clone the repo for the library you want to migrate: `git clone https://github.com/MetaMask/<repo-name>` (e.g. `<repo-name>` = `utils`). Do **NOT** use an existing clone.
+4. `cd` into the newly cloned repo.
+5. Run `git filter-repo --to-subdirectory-filter merged-packages/<package-name>` (e.g. `<package-name>` = `utils`). This will rewrite all history to move all files to `merged-packages/<package-name>`. (This is why we're making a fresh clone in a temporary directory — this action is irreversible.)
+6. `cd` to wherever you keep the `core` repo on your machine.
+7. Add the library as a remote: `git remote add <package-name> /tmp/<package-name>`.
+8. Fetch history: `git fetch <package-name> --no-tags`
+9. Make a new branch: `git checkout -b migrate-<package-name>`
+10. Merge the library into the monorepo: `git merge --allow-unrelated-histories <package-name>/<primary-branch>` (e.g. `<primary-branch>` = `main`)
+11. Open a pull request in the core repo that reflects the above changes.
+
+- DO NOT rebase the `migrate-<package-name>` branch.
+- Ensure that superfluous merge commits don't pollute the migrated git history. If necessary, replace the PR branch with a cleaned-up post-migration commit history by rerunning the git history migration steps before merging the PR.
+  - Coordinate with the team to minimize the time that this PR stays open.
+  - For further context on this point: https://github.com/MetaMask/core/pull/1804#issuecomment-1771445829
+- Contact a [**maintainer**](https://github.com/orgs/MetaMask/teams/engineering?query=role%3Amaintainer) to temporarily enable merge commits into main.
+- Merge PR **without squashing** to preserve the migrated git commit history.
+- [Example PR](https://github.com/MetaMask/core/pull/1872)
+
+### 2. Port tags
+
+- Use the `scripts/migrate-tags.sh` tool to port the source repo's release tags into the migrated git history in core, following [these instructions](./migrate-tags.md).
+- Port the tags locally with the correct package name prefixes.
+- Create a fork of the core monorepo for testing.
+- Push the ported tags to a fork of the core monorepo for testing. **Do not run the script against `MetaMask/core` before testing it on a fork.**
+- Verify that the tag diff links in CHANGELOG are working.
+- The diff between last tag before migration and first one after will always include the entire history of the monorepo.
+- Push the ported tags to the core repo.
+- Verify that the tag diff links in CHANGELOG are working.
+- Identify and create tags for release commits that were untagged in the original repo.
+
+### **[PR#7]** 3. Remove files and directories that will be replaced by files in the monorepo root directory
+
+- **Remove**: `.github/`, `.git*`, `scripts/`, `.depcheckrc.json`, `.yarn/`, `.yarnrc.yml`, `yarn.lock`, `.editorconfig`, `.eslint*`, `.prettier*`, `.nvm*`.
+- **Keep**: `src/`, `tests/`, `CHANGELOG.md`, `LICENSE`, `package.json`, `README.md`, `jest.config.js`, `tsconfig*.json`, `typedoc.json`
+- [Example PR](https://github.com/MetaMask/core/pull/1764)
+
+### **[PR#8]** 4. Replace config files
+
+- Update `tsconfig*.json`, `typedoc.json`, `jest.config.js` to extend from corresponding files in root. Copy contents of corresponding files in other non-root packages.
+- Keep TypeScript compiler flags and compilation target.
+- Add tsconfig reference paths for non-root packages that are upstream dependencies of the migration target.
+- Keep Jest coverage threshold values.
+- Add `deepmerge` as a devDependency.
+- [Example PR](https://github.com/MetaMask/core/pull/1765)
+
+### **[PR#9]** 5. Align dependencies and build scripts with monorepo
+
+- Remove redundant build scripts that are already listed in the root package.json (including `prepack`)
+- Identify validator fixes for CHANGELOG by navigating to `merged-packages/<package-name>`, running `../../scripts/validate-changelog.sh @metamask/<package-name>`, and applying the diffs.
+  - If the package has been renamed or needs to be renamed with the `@metamask/` namespace, supply two arguments: `versionBeforePackageRename`, `tagPrefixBeforePackageRename`.
+  - See https://github.com/MetaMask/auto-changelog#validate
+- Remove redundant dependencies that are already listed in the root package.json.
+  - Exception: do not remove TypeScript.
+- Align dependency versions with other non-root packages.
+  - If migration target dependency version is ahead: Decrement to match monorepo packages.
+  - If it's behind:
+    - Bump if it's an internal dependency (i.e. the dependency is another sub-package in the monorepo).
+    - If it's external, bump only if there are no resulting breaking changes that need to be resolved.
+- [Example PR](https://github.com/MetaMask/core/pull/1766)
+
+### **[PR#10]** 6. Add exception for non-MIT license
+
+- If the migration target uses a non-MIT license, add exception entries in the root `constraints.pro` file.
+  - Add 2 rules to license section of `constraints.pro`: Exclude (`\=`) from MIT rule and include in ISC rule (`==`)
+- Make sure the new rule doesn't break any of the existing package.json files by running `yarn constraints`.
+- [Example PR](https://github.com/MetaMask/core/pull/1888)
+
+### **[PR#11]** 7. Update the README to reflect its new status as a non-root package in the monorepo
+
+- Preserve package intro sentence/paragraph.
+- Add/modify "Installation" section.
+- Preserve "Usage" section.
+- Remove "Test", "Build" and other instructions on common development tasks.
+- Add "Contributing" section.
+
+## **[PR#12]** Phase C: **Integration** into the core monorepo's `packages/` directory
+
+- The following steps of "Phase C" need to happen in a single PR.
+- Coordinate with the team to minimize the time that this PR stays open to avoid merge conflicts with the main branch from accumulating.
+- [Example PR](https://github.com/MetaMask/core/pull/1738)
+
+### 1. Move the migration target directory from `merged-packages/` into `packages/`
+
+- Run `yarn install` in the root directory.
+- Check that all tests are passing in migration target by running `yarn workspace @metamask/<package-name> test`.
+
+### 2. Update downstream repos
+
+- **IMPORTANT** Add reference paths for the migration target in the root `tsconfig.json` and `tsconfig.build.json` files.
+  - This step is essential to avoid build failure for the migration target during release workflow.
+- Add tsconfig reference paths for the migration target in downstream packages.
+- Bump the migration target version in downstream packages and root.
+  - Notes on why this version bump needs to happen as part of this PR (import module shadowing): https://github.com/MetaMask/core/pull/1738#discussion_r1357554901
+
+### 3. Linter fixes
+
+- Apply yarn constraints fixes to migration target package.json file: `yarn constraints --fix`.
+- Add the `changelog:validate` build script to the package.json file.
+
+### 4. Resolve or TODO downstream errors
+
+- If introducing the migration target breaks any downstream repos:
+  - Resolve simple errors as part of this PR.
+  - Mark and ignore complex/blocked errors using `@ts-expect-error TODO:` annotations.
+- Create a separate issue for resolving the marked errors as soon as the migration is completed.
+  - e.g. https://github.com/MetaMask/core/issues/1823
+
+### 5. Record changes made to any core package in its CHANGELOG, under the `## [Unreleased]` heading
+
+- CHANGELOG entries should be recorded in the migration target's downstream packages for version bumps to the migration target's current release.
+- [Example PR](https://github.com/MetaMask/core/pull/2003/files): this step can be performed either as a part of Phase C, or in a separate, subsequent PR.
+
+### 6. Finalize merge
+
+- Check that all tests are passing in all subpackages of core and CI.
+- Double-check that dependency version bumps or other changes made to main while the PR was open are correctly merged and reflected.
+
+## Phase D: Clean-up and Release
+
+### Source repo
+
+1. Transfer open issues from the source repo into the core monorepo using GitHub's `Transfer issue` feature (prepend the title with the package name: `[<package-name>]`).
+
+2. For open PRs in the source repo, lock conversation (do not provide a reason), and leave a comment requesting that authors reopen the PR in core with a link pointing to the discussion in the original PR. For important PRs, manually migrate into core or create tickets for follow-up.
+
+```markdown
+This library has now been migrated into the [core monorepo](https://github.com/metamask/core/). This PR has been locked and this repo will be archived shortly. Going forward, releases of this library will only include changes made in the core repo.
+
+- Please push this branch to core and open a new PR there.
+- Optionally, add a link pointing to the discussion in this PR to provide context.
+```
+
+3. **[PR#13]** Leave a note in the source repo's README announcing the migration and pointing to core.
+
+```html
+<table>
+  <tr>
+    <td>
+      <p align="center"><b>⚠️ PLEASE READ ⚠️</b></p>
+      <p align="center">
+        This package has been migrated to our
+        <a href="https://github.com/MetaMask/core"><code>core</code></a>
+        monorepo, and this repository has been archived. Please note that all
+        future development and feature releases will take place in the
+        <a href="https://github.com/MetaMask/core"><code>core</code></a>
+        repository.
+      </p>
+    </td>
+  </tr>
+</table>
+```
+
+4. Archive the source repo to prevent any changes from being pushed to it going forward. Contact a [**maintainer**](https://github.com/orgs/MetaMask/teams/engineering?query=role%3Amaintainer) to perform this step.
+
+### Core
+
+1. **[PR#14]** Add migration target to the list of packages in the README as well as the dependency graph in the README (the latter can be updated automatically by running `yarn generate-dependency-graph`).
+2. Fix downstream errors that were marked with `@ts-expect-error TODO:` during the migration process.
+
+- If possible, perform this step before the first post-migration release of the migrated package.
+
+3. **[PR#15]** Use the `yarn create-release-branch` tool to publish a release of core with a new version for the migrated package and any updated downstream packages.