Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: Drop redis_socketio service #1382

Merged
merged 2 commits into from
Aug 7, 2023
Merged

refactor: Drop redis_socketio service #1382

merged 2 commits into from
Aug 7, 2023

Conversation

ankush
Copy link
Member

@ankush ankush commented Nov 2, 2022 •
edited
Loading

Summary:

  • After this change bench wont generate config for redis_socketio
    instance.
  • Instead we set the same port as redis_cache service for use in
    socketio. i.e. both will share same instance.

Code changes:

  • Config will have this difference:
- "redis_socketio": "redis://localhost:12000",
- "redis_queue": "redis://localhost:11000",
+ "redis_socketio": "redis://localhost:11000",
+ "redis_queue": "redis://localhost:11000",
  • supervisord/systemd wont start redis_socketio service
  • for development setups: redis_socketio service is removed from procfile.

Why?

  • Currently this redis instance is used only to ship messages from python
    to nodejs server... which is largely stateless (only pub/sub channels)
  • This change reduces 1 running process. Less resources for this ~= more
    resources for other running things.

All dependent deployment projects will likely have to refactor their code to drop this service, shouldn't be too much work.
Example: Frappe cloud (already done), Frappe Docker.

@ankush
refactor!: Reuse redis_cache instace for socketio
a172b14 
WARNING: Just a POC. You're free to modify your setup to test this out.

Summary:
- After this change bench wont generate config for redis_socketio
  instance.
- Instead we set the same port as redis_cache service for use in
  socketio. i.e. both will share same instance.

Code changes:

- Config will have this difference:

```diff
- "redis_socketio": "redis://localhost:12000",
- "redis_cache": "redis://localhost:13000",
+ "redis_socketio": "redis://localhost:13000",
+ "redis_cache": "redis://localhost:13000",
```

- supervisord/systemd wont start redis_socketio service
- for development setups: redis_socketio service is removed from procfile.

Why?

- Currently this redis instance is used only to ship messages from python
to nodejs server... which is largely stateless (only pub/sub channels)
- This change reduces 1 running process. Less resources for this ~= more
  resources for other running things.

TODO:
- [ ] decicisions - should we even do this? this should be optinal?
- [ ] refactor progressbar in frappe
- [ ] Tests - manual, FC, docker, automated
- [ ] "migration" plan

All dependent deployment projects will likely have to refactor their code to drop this service, shouldn't be too much work.
Example: Frappe cloud, Frappe Docker.
@sonarcloud
Copy link

sonarcloud bot commented Nov 2, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@ankush ankush changed the title refactor!: Reuse redis_cache instace for socketio refactor!: Drop redis_socketio service Nov 2, 2022
@ankush
Copy link
Member Author

ankush commented Nov 21, 2022 •
edited
Loading

not finding time to finish it, also 5-10MB memory saving at max 😔

You can achieve this in your deployments just by doing the config change that I mentioned in description.

@ankush ankush closed this Nov 21, 2022
@ankush ankush changed the title refactor!: Drop redis_socketio service refactor: Drop redis_socketio service Aug 7, 2023
@ankush ankush reopened this Aug 7, 2023
@ankush ankush marked this pull request as ready for review August 7, 2023 06:56
@sonarcloud
Copy link

sonarcloud bot commented Aug 7, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
1.1% 1.1% Duplication

@ankush ankush merged commit 87efb75 into frappe:develop Aug 7, 2023
13 checks passed
@ankush ankush deleted the bye_redis_socketio branch August 7, 2023 07:25
@github-actions
Copy link

github-actions bot commented Aug 8, 2023

🎉 This PR is included in version 5.17.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@SaiFi0102
Copy link
Contributor

Good idea, but could have been backwards compatible

@ankush
Copy link
Member Author

ankush commented Sep 5, 2023

@SaiFi0102 this is backward compatible AFAIK, unless you need 3rd service running for something else?

@SaiFi0102
Copy link
Contributor

I did a bench setup supervisor after migrating and old instance to v14. Since common_site_config.json was configured for port 12000... the bench got stuck until I found out about this change. So I'm guessing others can also fall into this issue.

@ankush ankush mentioned this pull request Sep 5, 2023
Merged
@ankush
Copy link
Member Author

ankush commented Sep 5, 2023

@SaiFi0102 #1481 this should fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
released
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ankush @SaiFi0102