[WIP] hosting the sync-server with the desktop app (POC)

[MikesGlitch]

The goal

Free server hosting via the desktop app. We'd run the sync server inside of the desktop app, expose it to the internet via ngrok (or similar) and provide a QR code/url for the user to connect their phone to the server.

We could provide an option to start actual-server when the computer starts up. If we did this the server would be alive whenever the computer is on. If the user downloads the pwa to their phone they could budget even when the server is offline - it would sync when the computer is turned back on.

Todos:

• Get the sync server working in the app
• Expose the sync server using ngrok
• Test this as my daily driver
• Package the app and make it fully functional
• Prevent error message "We had problems syncing your changes. Please report this as a bug by [opening a Github issue]" when the server is offline. I think it's because it hits ngrok first - ngrok can't connect to tunnel.
• Fix server status appearing as offline when it finishes starting after web client loads [Bug]: says "server offline" when it actually came back #3128
• Automatically set the server URL when the user uses the Electron Server. Update the UI to say that we're connected to "Local Server" or something instead of the server URL.
• Socket hangup issue - when closing and opening the budget fresh when connected to ngrok server url.
  • I switched to udinci and it was still happening. It seems related to NGROK - I don't get it when I connect directly to the server.
  • When I connect via the web I don't get it at all.
  • This error can be avoided by connecting direct to localhost via electron and it doesn't occur on web/pwa. It's avoiding the error but not fixing it - but that may be satisfactory because it's better to connect to the server via localhost on electron for better offline functionality

Server Log: TypeError: fetch failed
    at node:internal/deps/undici/undici:12345:11
    at async sM (C:\Users\PandaMan\AppData\Local\Programs\desktop-electron\resources\app.asar\build\loot-core\lib-dist\electron\bundle.desktop.js:115:5067)
    at async _R.subscribe-get-user (C:\Users\PandaMan\AppData\Local\Programs\desktop-electron\resources\app.asar\build\loot-core\lib-dist\electron\bundle.desktop.js:295:5421) {
  cause: SocketError: other side closed
      at TLSSocket.onSocketEnd (node:internal/deps/undici/undici:8903:26)
      at TLSSocket.emit (node:events:530:35)
      at endReadableNT (node:internal/streams/readable:1696:12)
      at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
    code: 'UND_ERR_SOCKET',
    socket: {
      localAddress: '192.168.0.22',
      localPort: 60333,
      remoteAddress: '3.125.209.94',
      remotePort: 443,
      remoteFamily: 'IPv4',
      timeout: undefined,
      bytesWritten: 293,
      bytesRead: 0
    }
  }
}

• Make progress towards getting the sync server in the main Actual repo
• Figure out why I'm getting 401's after a day or so - haven't had this in a while. Maybe it's to do with me wiping my data while testing. tokens get saved to db so that's a possibility.
• Change the server port to something else so it doesn't clash with local dev testing (not 5006)
• Add global-store.json to the backup export? I corrupted it somehow and the file got wiped. It's a better backup
• Allow the server to run on https within localhost - this will require setting up a certificate - this can be optional - setup a server config ui?
  • Make the electron app manage the SSL so that users don't have to set it up
• Build the UI
  • on server setup page when electron, push people towards using this functionality - still allow external server functionality
  • a settings area/modal to allow server configuration for initial setup and edits - Should probably live on file management page and under the server status dropdown.
  • a start and stop functionality for the server under the server status dropdown
• Tidy the code
• Test on
  • Window
  • Linux
  • Mac

Known issues

• server /info endpoint isn't returning data - it means we can't see what version the server is on. I think it's to do with it not finding the package.json file. We're using an external package for it.

Next: Figure out the best way to bring actual-server into the actual repo

Expose the running server to the internet

Manged to get it working with ngrok. Ngrok's big and they give you a free domain. It works a treat. I haven't tested it in anger, I'm sure it will disconnect me after an hour or so so we may need to add logic to reconnect.

Thoughts on bundling the actual-server code

• If actual-server was in the actual repo we could bundle using electron-builder. The package size seemed good when I tried that. This seems like the right way forward for all kinds of reasons - some not related to electron - type sharing, debugging, simlicity etc.

Old thoughts:

• Actual-server bundle is huge - consider using webpack or https://github.com/vercel/ncc . Webpack seems promising, they have targets for electron and node.
• BetterSqlite3 is complaining that it was compiled for server on a different version of node than electron. We'll also run into the arch issue so we'll need to fully make that external in the actual-server bundle and use the one we have in electron.
• If we bundle, the bundle should only be used for electron (if we want to bundle for normal use it will be a separate bundle)

Server sync notes

• We could warn the client against syncing if the client version is greater than the server version.
  • This means we should never have out of sync migrations
  • We'd always know the server version & client version at this point - we would have already migrated but we wont sync it with the server unless it is safe to do so.
  • Downside - if it takes 2 weeks to update the server the user is unable to sync until it is updated

[netlify]

❌ Deploy Preview for actualbudget failed.

Name	Link
🔨 Latest commit	4f667d0
🔍 Latest deploy log	https://app.netlify.com/sites/actualbudget/deploys/67223c928532070008ea5d78

[github-actions]

Bundle Stats — desktop-client

Hey there, this message comes from a GitHub action that helps you and reviewers to understand how these changes affect the size of this project's bundle.

As this PR is updated, I'll keep you updated on how the bundle size is impacted.

Total

Files count	Total bundle size	% Changed
9	5.34 MB → 5.34 MB (+470 B)	+0.01%

Changeset

File	Δ	Size
src/components/manager/BudgetList.tsx	📈 +399 B (+3.44%)	11.32 kB → 11.71 kB
src/browser-preload.browser.js	📈 +71 B (+1.75%)	3.96 kB → 4.03 kB

View detailed bundle breakdown

Added

No assets were added

Removed

No assets were removed

Bigger

Asset	File Size	% Changed
static/js/index.js	3.34 MB → 3.34 MB (+470 B)	+0.01%

Smaller

No assets were smaller

Unchanged

Asset	File Size	% Changed
static/js/usePreviewTransactions.js	1.64 kB	0%
static/js/resize-observer.js	18.37 kB	0%
static/js/indexeddb-main-thread-worker-e59fee74.js	13.5 kB	0%
static/js/AppliedFilters.js	20.96 kB	0%
static/js/narrow.js	82.55 kB	0%
static/js/BackgroundImage.js	122.29 kB	0%
static/js/wide.js	242.95 kB	0%
static/js/ReportRouter.js	1.51 MB	0%

[github-actions]

Bundle Stats — loot-core

Hey there, this message comes from a GitHub action that helps you and reviewers to understand how these changes affect the size of this project's bundle.

As this PR is updated, I'll keep you updated on how the bundle size is impacted.

Total

Files count	Total bundle size	% Changed
1	1.26 MB	0%

Changeset

No files were changed

View detailed bundle breakdown

Added

No assets were added

Removed

No assets were removed

Bigger

No assets were bigger

Smaller

No assets were smaller

Unchanged

Asset	File Size	% Changed
kcab.worker.js	1.26 MB	0%

[Jonathan-Fang]

This seems very cool!

[MikesGlitch]

crdt is a dependency of actual-server. Build it before packaging the sync-server

[Jonathan-Fang]

To clarify your sync server notes, if the server is on a higher server version than the client, the client can't just keep using the previous server version to sync because the server has already updated?

Downtime for the user doesn't seem like a good idea to me.

[MikesGlitch]

To clarify your sync server notes, if the server is on a higher server version than the client, the client can't just keep using the previous server version to sync because the server has already updated?

Downtime for the user doesn't seem like a good idea to me.

It's not really relevant to this code it's just thoughts. I was envisioning more of a warning to the user "Syncing your budget will cause the server web client to be unusable until it has been updated to the latest version."

This POC is just a throwaway code dump btw, don't put any weight on it.

[youngcw]

Just throwing this out there. Could the bundled server have a way to make its own ssl cert? That way we know it setup right and the user just have to know to click past the warning, or have guides on how to install the cert.

[MikesGlitch]

Just throwing this out there. Could the bundled server have a way to make its own ssl cert? That way we know it setup right and the user just have to know to click past the warning, or have guides on how to install the cert.

Still investigating that one. I think it can be done because dev servers do it but still experimenting. Just added an item on the todo for it.