If you are setting up the orchestrator, you will be targeting the local environment.
Clone or fork this repository.
If you are developing this project:
cd to the repository root directory and run:
npm installCopy the config/config.example.json to config/config.local.json.
Use the config/config.js for documentation and further examples.
Long Polling (lp) is supported in both local and cloud environments. Server Sent Events (sse) is only supported in the local environment due to AWS limitations. Both lp and sse are real-time. Both implementations have their pros and cons.
The client-side (PurpleTeam CLI) will use the testerFeedbackComms.medium that the orchestrator instructs it to use.
Using sse is one way communications after the initial subscription from the CLI to the orchestrator. Redis pub/sub is used between the Testers and the orchestrator to publish Tester feedback. If the CLI is stopped (not subscribed) at any point while the back-end is performing a Test Run, events will be lost. If the CLI runs test again before the Test Run is complete, it will receive messages from then to Test Run completion, and will also receive the Outcomes archive on Test Run completion. If the CLI runs test after a Test Run is complete, no messages will be received from the previous Test Run, the previous Test Runs Outcomes archive is destroyed, and the CLI will start receiving messages as Tester feedback is produced.
Using lp is request-response communications. A request is made and only answered when there are Tester feedback messages available, or the application specific (rather than AWS Api Gateway) time-out is exceeded.
As soon as the CLI receives a set (one to many) of Tester feedback messages, it makes another request to the orchestrator (Directly if running in local env. Indirectly via the AWS API Gateway if running in cloud env). Redis pub/sub is used between the Testers and the orchestrator to publish Tester feedback.
A little more detail:
If running in the cloud environment the first set of Tester feedback messages are persisted in the orchestrator's memory, and subsequent Tester feedback messages are persisted to in-memory Redis lists. The reasons for this are at least two-fold. The orchestrator sets up the subscriptions to the Redis channels on behalf of the CLI. The reasons for this is so that:
- The orchestrator knows when the Testers are finished in order to clean-up before the next Test Run
- and to make sure that no messages are missed due to the CLI either being off-line or late to subscribe
This means that if the CLI is stopped momentarily during a Test Run or if Tester messages pile up before the CLI has subscribed (which normally occurs on receipt of the initial set of Tester status messages), as long as the CLI subscribes before the Test Run has completed, it will receive all stored Tester feedback messages and the Outcomes archive when the Test Run is complete.
There are some more details around message flow in this blog post.
Additional background: This may change in the future, WebSockets is also an option we may implement in the future, but implementing WebSockets would mean we would have to change our entire authn approach. Our chosen cloud infrastructure AWS Api Gateway does not support streaming and it does not support the OAuth Client Credentials Flow with Cognito User Pools.
Once you have worked through the above steps, head back to the local setup documentation to continue setting up the other PurpleTeam components.