Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sdk): support mounting sim.Container to state directories #6295

Merged
merged 3 commits into from
Apr 22, 2024

Conversation

Chriscbr
Copy link
Contributor

@Chriscbr Chriscbr commented Apr 22, 2024

To support retaining the state of sim.Container across separate Wing Console sessions, this PR adds the capability to reference the resource's state directory through the WING_STATE_DIR environment variable. This will be used by the postgres winglib to persist state through a mounted volume:

let container = new sim.Container(
  name: "postgres",
  image: image,
  env: {
    POSTGRES_PASSWORD: "password"
  },
  volumes: ["$WING_STATE_DIR/pgdata:/var/lib/postgresql/data"],
  containerPort: 5432
);

Related to #6284

Checklist

  • Title matches Winglang's style guide
  • Description explains motivation and solution
  • Tests added (always)
  • Docs updated (only required for features)
  • Added pr/e2e-full label if this feature requires end-to-end testing

By submitting this pull request, I confirm that my contribution is made under the terms of the Wing Cloud Contribution License.

@Chriscbr Chriscbr requested a review from a team as a code owner April 22, 2024 17:15
Copy link

Thanks for opening this pull request! 🎉
Please consult the contributing guidelines for details on how to contribute to this project.
If you need any assistence, don't hesitate to ping the relevant owner over Slack.

Topic Owner
Wing SDK and utility APIs @chriscbr
Wing Console @ainvoner, @skyrpex, @polamoros
JSON, structs, primitives and collections @hasanaburayyan
Platforms and plugins @hasanaburayyan
Frontend resources (website, react, etc) @tsuf239
Language design @chriscbr
VSCode extension and language server @markmcculloh
Compiler architecture, inflights, lifting @yoav-steinberg
Wing Testing Framework @tsuf239
Wing CLI @markmcculloh
Build system, dev environment, releases @markmcculloh
Library Ecosystem @chriscbr
Documentation @hasanaburayyan
SDK test suite @tsuf239
Examples @hasanaburayyan
Wing Playground @eladcon

@Chriscbr Chriscbr changed the title feat(sim): support mounting sim.Container to state directories feat(sdk): support mounting sim.Container to state directories Apr 22, 2024
@monadabot
Copy link
Contributor

monadabot commented Apr 22, 2024

Console preview environment is available at https://wing-console-pr-6295.fly.dev 🚀

Last Updated (UTC) 2024-04-22 17:49

@monadabot
Copy link
Contributor

monadabot commented Apr 22, 2024

Benchmarks

Comparison to Baseline ⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜
Benchmark Before After Change
version 59ms±0.68 59ms±0.68 0ms (-0.83%)⬜
functions_1.test.w -t sim 398ms±5.03 395ms±4.8 -3ms (-0.75%)⬜
functions_1.test.w -t tf-aws 820ms±3.62 816ms±5.21 -4ms (-0.49%)⬜
jsii_big.test.w -t sim 2802ms±16.33 2793ms±19.09 -9ms (-0.33%)⬜
jsii_big.test.w -t tf-aws 2996ms±15.92 2990ms±13.62 -6ms (-0.21%)⬜
functions_10.test.w -t sim 488ms±5.56 490ms±7.54 +2ms (+0.34%)⬜
functions_10.test.w -t tf-aws 2008ms±12.31 2012ms±14.33 +3ms (+0.17%)⬜
jsii_small.test.w -t sim 368ms±3.19 369ms±5.26 +1ms (+0.21%)⬜
jsii_small.test.w -t tf-aws 599ms±4.08 607ms±9.29 +8ms (+1.26%)⬜
empty.test.w -t sim 359ms±3.67 363ms±5.4 +3ms (+0.91%)⬜
empty.test.w -t tf-aws 589ms±1.48 596ms±4.99 +7ms (+1.26%)⬜
hello_world.test.w -t sim 391ms±4.02 395ms±3.52 +4ms (+0.94%)⬜
hello_world.test.w -t tf-aws 1504ms±6.29 1508ms±6.06 +4ms (+0.29%)⬜

⬜ Within 1.5 standard deviations
🟩 Faster, Above 1.5 standard deviations
🟥 Slower, Above 1.5 standard deviations

Benchmarks may vary outside of normal expectations, especially when running in GitHub Actions CI.

Results
name mean min max moe sd
version 59ms 57ms 60ms 1ms 1ms
functions_1.test.w -t sim 395ms 389ms 408ms 5ms 7ms
functions_1.test.w -t tf-aws 816ms 803ms 826ms 5ms 7ms
jsii_big.test.w -t sim 2793ms 2763ms 2864ms 19ms 27ms
jsii_big.test.w -t tf-aws 2990ms 2966ms 3017ms 14ms 19ms
functions_10.test.w -t sim 490ms 462ms 498ms 8ms 11ms
functions_10.test.w -t tf-aws 2012ms 1973ms 2045ms 14ms 20ms
jsii_small.test.w -t sim 369ms 358ms 379ms 5ms 7ms
jsii_small.test.w -t tf-aws 607ms 586ms 635ms 9ms 13ms
empty.test.w -t sim 363ms 353ms 378ms 5ms 8ms
empty.test.w -t tf-aws 596ms 587ms 610ms 5ms 7ms
hello_world.test.w -t sim 395ms 385ms 401ms 4ms 5ms
hello_world.test.w -t tf-aws 1508ms 1494ms 1521ms 6ms 8ms
Last Updated (UTC) 2024-04-22 17:54

Copy link
Contributor

mergify bot commented Apr 22, 2024

Thanks for contributing, @Chriscbr! This PR will now be added to the merge queue, or immediately merged if rybickic/container-state is up-to-date with main and the queue is empty.

@mergify mergify bot merged commit ec7e177 into main Apr 22, 2024
16 checks passed
@mergify mergify bot deleted the rybickic/container-state branch April 22, 2024 17:55
@monadabot
Copy link
Contributor

Congrats! 🚀 This was released in Wing 0.71.9.

mergify bot pushed a commit that referenced this pull request Apr 22, 2024
Normalize the code within the SDK so that the paths to various simulator files are stored as absolute paths.

This is causing an issue where the feature added in #6295 only works when running `wing it` and not in `wing test` because `wing test` initializes the simulator with a relative path.

## Checklist

- [x] Title matches [Winglang's style guide](https://www.winglang.io/contributing/start-here/pull_requests#how-are-pull-request-titles-formatted)
- [x] Description explains motivation and solution
- [ ] Tests added (always)
- [ ] Docs updated (only required for features)
- [ ] Added `pr/e2e-full` label if this feature requires end-to-end testing

*By submitting this pull request, I confirm that my contribution is made under the terms of the [Wing Cloud Contribution License](https://github.com/winglang/wing/blob/main/CONTRIBUTION_LICENSE.md)*.
Copy link
Contributor

@eladb eladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good job

mergify bot pushed a commit that referenced this pull request Apr 25, 2024
If a container mounts a volume to the state directory as described in #6295, it can sometimes result in permission errors when the CLI is cleaning up files after `wing test` because the mounted directory may have different permissions.

```
EACCES: permission denied, scandir 'target/test/mount.test.wsim/.state/c892787d9270cdccc06c34c95f7f9738be92b19134'
```

This error only seems to happen on CI for me. I'm not sure of the cause but I'm guessing that in CI, docker runs in a different (rootless maybe?) mode than on macOS.

Unblocks winglang/winglibs#207

## Checklist

- [ ] Title matches [Winglang's style guide](https://www.winglang.io/contributing/start-here/pull_requests#how-are-pull-request-titles-formatted)
- [ ] Description explains motivation and solution
- [ ] Tests added (always)
- [ ] Docs updated (only required for features)
- [ ] Added `pr/e2e-full` label if this feature requires end-to-end testing

*By submitting this pull request, I confirm that my contribution is made under the terms of the [Wing Cloud Contribution License](https://github.com/winglang/wing/blob/main/CONTRIBUTION_LICENSE.md)*.
mergify bot pushed a commit that referenced this pull request Apr 26, 2024
Another pass at solving #6284

The approach taken in #6295 where we allow users to bind to the state directory is problematic because docker sometimes changes the permissions of directories on the user's file system after mounting them. This was causing `wing test` to fail (fixed in #6320) and `wing pack` to fail as well.

This PR reworks the API so that data can be persisted via named volumes. Specifically, an anonymous docker volume can be specified, and Wing will automatically name it and reuse it across Wing Console runs. The change should fix the previous issues since named volumes are managed by docker.

The main downside of this approach is that deleting your `target` directory or the state files in `.state/` will not clean everything up all resources/files created by Wing from your machine, as Docker volumes may still exist. Eventually we can provide some facilities for automatically cleaning these, but we're leaving it intentionally out of scope until this general approach has been validated.

## Checklist

- [x] Title matches [Winglang's style guide](https://www.winglang.io/contributing/start-here/pull_requests#how-are-pull-request-titles-formatted)
- [x] Description explains motivation and solution
- [ ] Tests added (always)
- [ ] Docs updated (only required for features)
- [ ] Added `pr/e2e-full` label if this feature requires end-to-end testing

*By submitting this pull request, I confirm that my contribution is made under the terms of the [Wing Cloud Contribution License](https://github.com/winglang/wing/blob/main/CONTRIBUTION_LICENSE.md)*.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants