-
Notifications
You must be signed in to change notification settings - Fork 339
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature request: "proxy" packages that support direct editing of files without rebuild #1748
Comments
Also, just to be clear, if we all agree this is a good feature, my team can implement it and submit a PR for it. |
I suppose I'd have a couple questions. Is the intent that the entire package is treated as rooted-at the source? It's very common for packages to only sort of... Actually become their structured whole when built into their install folder. Basically, I see this as only useful for python packages, where the source structure often/usually (but not always) is identical to the target structure (even then, it can be.. Not that uncommon for there to be differences, or extra things introduced or renamed or whatever in the build process). I ask because.. Maybe it makes sense to hang this feature on a mechanism that, perhaps, matches up with a "very clean way to express a packages whose source is identical to its target". For instance, I'm wary of developers using this, and then somehow expecting that their "not identical source" doesn't act like their "the way it usually builds and installs" target. This is poorly named, but I'm wondering if an attribute like "easy_copy" could express a type of package whose build command is an automatic "equivalent to rsync -larv" style copy, and that automatically enables or disables such a feature, to basically marry the idea that "only if your source and target would be identical, can you reasonably expect to use this in a non-side-effect-ensured-way". Of course, I can see reasons why power users may not like that, but I've seen a lot of developer misunderstandings around this kind of idea, and while the feature is interesting, I think it may need some kind of guardrail, even if it's just a warning that "You're using a proxied package, results may vary depending on your setup", or something to that effect. |
It's most useful for packages that follow this paradigm, but as I've shown in my example package above, you could use the
I considered something like this as well, but I don't want to hamstring power users. Here's a compromise:
I'm not sure if a build_system is the right way to handle this but it seems like an easy way for a package to opt-in to this behavior. I also considered |
Hi all,
I think this topic has probably come up at every studio that has ever used Rez. I think adoption would be faster, and developer workflows would be simpler and safer if Rez supported this concept natively. Let me know what you think.
Goal
As a Rez developer I want a way for local rez packages to reference live files, so that I can rapidly iterate without needing to rebuild after every change. Files that I might want to edit in-place include interpreted source code (python, shell scripts), configuration files, and the package.py files themselves.
Motivation
In a typical Rez studio environment, there are many rez packages which do not require compilation as part of the build process. The build commands for these packages may simply copy some folders and files to the install location. Some Rez packages are used to bundle together particular packages and versions, and so the package.py file is the sole file that requires editing and installing.
In these situations, it is a drain on developer time to rebuild packages each time that a file is edited.
Proposal
Add a new
--proxy
flag torez build
: when a package is built usingrez build --proxy --install
Rez will install a special proxypackage.py
that redirects to the sourcepackage.py
This proxy package will be installed to the local packages path, so that it will be found by the normal Rez discovery mechanism
It will have the bare minimum of attributes (such as name and version), along with two new attributes
proxied_package
: path to the package.py that produced itproxied_root
: (optional) path which should override the root variable within commands. This can be provided viarez-build --proxy --proxied-root=...
, otherwise it is taken from the source package’sdefault_proxied_root
if present.When Rez resolves a proxy
package.py
, it will load the source (aka "proxied")package.py
indicated by theproxied_package
attribute. If present, it will inject the value ofproxied_root
as theroot
variable within thecommands
context. This allows packages which have a similar source and release folder layouts to "just work".There will also be a new context variable called
is_proxied
which will be set toTrue
when a package is loaded via a proxy stub; this can be used to add conditional logic to the commands context.For safety, proxy installs should not be supported by
rez release
Various commands which list packages in a resolve should indicate when a local package is proxied
Example files
Here's an example package.py that uses this feature:
Here's the "proxy" package that would be installed into the local packages path:
Rejected ideas
Add machinery inside package.py files
We could use the very flexible package.py files to emulate this behavior. One idea was to modify our build command to install a
.redirect.json
file alongside thepackage.py
, then when thecommands
runs we would check for the existence of this file, and change the behavior.Cons:
Checking for the presence of the
.redirect.json
file adds overhead even when a package is not in editable mode.The
package.py
itself must be re-installed after editing for changes to take place (unless we also use a symlink to the package.py which adds further complexity and room for failure).Since Rez itself is not aware of this concept, when Rez prints a resolve it will not indicate whether the package is a "proxy" pacakge or not. This can be confusing and dangerous for developers
Use
pip install --editable
Cons:
The
package.py
itself must be re-installed after editing for changes to take place (unless we also use a symlink to the package.py which adds further complexity and room for failure).Since Rez itself is not aware of this concept, when Rez prints a resolve it will not indicate whether the package is a "proxy" pacakge or not. This can be confusing and dangerous for developers
Only works for Python projects
The text was updated successfully, but these errors were encountered: