Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial version of package:extension_discovery #129

Merged
merged 6 commits into from
Aug 2, 2023

Conversation

jonasfj
Copy link
Member

@jonasfj jonasfj commented Jul 20, 2023

Establishing a convention for how allow other packages to provide extensions for your package.

And providing a findExtensions method that searches all packages in package_config.json for extension/<targetPackage>/config.json.

Also caching detected extensions in .dart_tool/extension_discovery/<targetPackage>.json and comparing with timestamps of package_config.json and config.json (for relative path-dependencies, which are considered mutable).

I'm open to bikeshedding:

  • Choosing a different place than extension/<targetPackage>/config.json
  • Using YAML files instead of config.json (it's a bit slower and more error-prone, but probably fine)
  • Finding a better package name 🤣

But:

  • JSON is simple and fast (for advanced setups, an extension could have it's own additional config file)
  • A new top-level directory extension/ seems fitting (none of existing folders in package layout convention seems fitting).
    • It's singular like the other names.
    • We could consider documenting.
  • Any package name will do 🙈

@kenzieschmoll, thoughts?

@devoncarew, wdyt about this package living here? I think that means it'll be pulled in to the Dart SDK via DEPS.

I think it's mostly useful for development tools as we have no code-generation hooks that can easily leverage this convention (yet, who knows someday, maybe).

@kenzieschmoll
Copy link
Contributor

kenzieschmoll commented Jul 20, 2023

DBCs:

  • I like the name extension_discovery
  • extension/<targetPackage>/config.json: what are the possible options for <targetPackage>? [devtools, analyzer, flutter_tools, others?]. What would happen if a package author includes a top level directory that is not one of the known tooling platforms (extension/foo/config.json)? Would this be a no-op or is there a use case for this?
    • EDIT: I read the README, and I think this is clear now. Allowing for any arbitrary <targetPackage> makes this package flexible enough to be used for pub package extensions as well as devtools extensions, analyzer plugins, etc.

Comment on lines +28 to +30
It is the responsibility package that can be extended to validate extensions,
decide when they should be enabled, and documenting how such extensions are
created.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add here that it is also the responsibility of the package that will be extended to detect when the cache has been invalidated so that they can call findExtensions again to get an updated set of extensions?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If people want to watch the file-system for changes, then I think it makes sense to build it in here.

You'd need to watch .dart_tool/package_config.json and config.json for every mutable extension.

I imagine that devtools will load extensions when a debugging instance is launched. If I do dart pub get after launching my app, can I expect hotreloading of all dependencies to work?

I don't mind adding some watch logic.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding some watch logic would be great. In the short term, we can probably get by with having a way for the user to manually refresh the list of extensions from the extensions settings menu and by refreshing the list of available extensions on each hot reload or restart event. This of course would only be possible for Flutter. If there was a running dart CLI app that DevTools was connected to, the app would need to be killed, restarted, and reconnected to DevTools to pick up any changes that happened from running dart pub get.

I imagine that devtools will load extensions when a debugging instance is launched. If I do dart pub get after launching my app, can I expect hotreloading of all dependencies to work?

@christopherfujino do you know if all the dependencies are reloaded on hot reload? If we can expect this to work, DevTools can automatically refresh the list of extensions on hot reload / restart.

Comment on lines 57 to 61
// Find extensions for the "hello_world" package.
// WARNING: This only works when running in JIT-mode, if running in AOT-mode
// you must supply the `packageConfig` argument, and have a local
// `.dart_tool/package_config.json` and `$PUB_CACHE`.
final extensions = await findExtensions('hello_world');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you provide an example of what this function call would look like in AOT mode when the packageConfig is specified?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure it easy to make a good example for that.

It's kind of like: findExtensions('hello_world', packageConfig: Uri.parse('/home/my_user/workspace/myapp/.dart_tool/package_config.json'));

Where /home/my_user/workspace/myapp is the place I did my source checkout and ran dart pub get.


So in the context of devtools, you'd probably run findExtensions('hello_world', packageConfig: Uri.parse('path/to/the/users/workspace/.dart_tool/package_config.json')).
Where path/to/the/users/workspace/ is the workspace/app for which the user is running devtools.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was mostly an attempt at pointing to the "Runtime limitations" section.

Comment on lines +167 to +168
When writing an extension it is strongly encouraged to have a dependency
constraint on the package being extended. This ensures that the extending
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just noting that this will not apply for DevTools extensions since we are not shipped as a pub package. Not sure if there are other "non-package" use cases for extensions, or if DevTools is the exception.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's recommendation because it really depends on the use-case.

In many cases if you're writing an extension to a package, then you're implementing an interface from the package, so then it's also natural to have a dependency on it.

If you wanted to have to ability to do, you could ask devtools extension packages to have a dependency on devtools_ext and then bump that package when you break the extension interface.
But it's also fair that you just detect that an extension isn't compatible and thus shouldn't be loaded.

In you use-case, not loading the extension is probably better than having a version conflict.

@devoncarew
Copy link
Member

@devoncarew, wdyt about this package living here? I think that means it'll be pulled in to the Dart SDK via DEPS.

No issue w/ hosting it here, but we will want to start w/ an issue in order to nail down things like who's the primary maintainer, what team is backing them up, what the initial SLO will be, the anticipated final SLO, ... go/create-dart-package-bug.

Comment on lines +204 to +205
If your deployment target a compiled Flutter application or AOT-compiled
executable, then you will have to create some code/asset-generation.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one caveat is that a profile mode flutter app is AOT compiled but findExtensions would still work if given the location of this app's package_config.json

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I just think it's important to point out that if you are writing a Flutter app, you can't really use findExtensions and have it magically working on your phone.
Because obviously the package_config.json and all the source code for the dependencies installed in pub-cache, aren't available on your phone, at-least not without some serious hacks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I think most of my comments are because I'm reading from the perspective of having another tool (DevTools, which is running on the user's local machine) call findExtensions on a separate app (also running on the user's machine), and the warnings you are heading are when an app would be trying to call findExtensions itself.

@kenzieschmoll
Copy link
Contributor

Left some comments. Overall, looks great!

@jonasfj
Copy link
Member Author

jonasfj commented Jul 21, 2023

what team is backing them up, what the initial SLO will be, the anticipated final SLO, ... go/create-dart-package-bug.

Filed b/292190799

@jonasfj
Copy link
Member Author

jonasfj commented Jul 26, 2023

Note to self:

  • Document what kind of Uri should be passed for packageConfig (hint, probably absolute file Uri)
  • Throw ArgumentError if packageConfig parameter is not an absolute file Uri.

Comment on lines +8 to +15
dev_dependencies:
dart_flutter_team_lints: ^1.0.0
lints: ^2.0.0
test: ^1.21.0
test_descriptor: ^2.0.1

environment:
sdk: ^3.0.0
Copy link
Contributor

@kenzieschmoll kenzieschmoll Aug 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do all these dependencies already exist in the Dart SDK? or would they be acceptable to bring in? package:extension_discovery will be depended upon by package:devtools_shared, which is a dep of package:dds and therefore the Dart SDK.

Copy link
Contributor

@kenzieschmoll kenzieschmoll Aug 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see lints_rev, test_rev, and test_descriptor_rev in DEPS, but I do not see dart_flutter_team_lints.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's in there :) It comes in as part of the dart-lang/ecosystem repo dep.

description: >-
Discovery of packages that provide extensions for your package.
version: 1.0.0-wip
publish_to: none
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be removed before landing?

Copy link
Contributor

@kenzieschmoll kenzieschmoll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple questions around publishing and more importantly, about dependencies since a DEP for package:extension_discovery will need to be added to the Dart SDK in order for DevTools to use this package (https://github.com/dart-lang/sdk/wiki/Adding-and-Updating-Dependencies).

I checked this code out though and verified that package:extension_discovery is WAI for the DevTools extensions use case: flutter/devtools#6139.

LGTM overall.

@jonasfj jonasfj merged commit af3fc99 into dart-lang:main Aug 2, 2023
4 checks passed
@jonasfj jonasfj deleted the extension_discovery branch August 2, 2023 11:45
@jonasfj
Copy link
Member Author

jonasfj commented Aug 2, 2023

Merged, published: https://pub.dev/packages/extension_discovery and transferred to tools.dart.dev.

@kenzieschmoll
Copy link
Contributor

@jonasfj we will need to also add this package to the SDK DEPS (https://github.com/dart-lang/sdk/wiki/Adding-and-Updating-Dependencies) and roll into google3. Is that something you can do?

@kenzieschmoll
Copy link
Contributor

Actually, the tools repo is already in DEPS, so we'll just need to bump the hash. That PR is here: https://dart-review.googlesource.com/c/sdk/+/317804.

However, we will still need to setup a google3 mirror for this package.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants