Modular structure for flux auth modes

[vsoch]

The original design of Flux Restful assumed the server running as a single user (and submitting jobs as such) and this was resigned (commit here) to use an approach of being run by sudo, and signing on behalf of the user. While this is intended for the Flux Operator (where we are running as root and could want to submit jobs on behalf of a user) it's not ideal for the single-user use case - a user running this inside of a flux instance they own from, for example, an HPC cluster. This issue will encompass discussion and design for a refactor that allows this functionality to be more modular. My proposal is the following:

1. Modular Skeleton for Library / Auth functions

Within the library directory, the variances in logic will happen within flux.py (interactions with flux) and auth.py (the authentication setup). My first suggestion is to add a string environment variable to settings here that takes a mode of operation. The default likely would be something akin to single-user. Given this mode, when we import flux_cli or the auth wrappers, instead of just targeting one module, there would be another level of module within the library that is associated with each mode. The design of this (and how/when the import of the correct module) happens can work in multiple ways and I leave up to your design decisions.

The high level goal would be that, regardless of the mode (string) chosen by the user, we are given the same set of functions (same interface) that have slightly different logic to submit jobs, etc.

2. Additional Modules

We can first do the above for the existing functions - likely this would be a multi-tenant-root or similarly named mode (please adjust that appropriately I'm bad at naming things!) When that is done, we can then:

1. Add a single-user mode proper (the old code I linked above before adding the sudo wrapper)
2. Add other auth means such as PAM / LDAP / others?

I'll leave it there for now since that is already a lot! Pinging @khoing0810 this issue is all yours! Feel free to use the thread here for questions and discussion.

[vsoch]

I'll be working on this soon - we need it for the flux operator, for which (ironically) we now run in single user mode!

[vsoch]

I added a simple variable to control mode #65, and it separates the flux auth mode (which was previously tangled). I'll update this issue to be about different kinds of auth, since we just have the basic auth that feeds into token with a signed payload. I'm not sure (for the work we are doing) we need much more than that right now.