Manifest template (#25)

* expose manifest via endpoint/template * prefix env vars with XLWINGS_ * fix static dir * fix .env.template * handle more base_urls * implemented run.py init * templified the manifest further * use default manifest ids in all envs * fix run.py init * fix .env.template * reloed with .env changes * docs * docs and change default env to prod (.env sets it to dev) * docs * show env in tab name * more env related changes * added comment
xlwings · May 21, 2024 · fd0fbec · fd0fbec
1 parent eb66071
commit fd0fbec
Show file tree

Hide file tree

Showing 13 changed files with 179 additions and 65 deletions.
diff --git a/.env.template b/.env.template
@@ -1,8 +1,6 @@
-# Copy this file to .env:
-# cp .env.template .env
-
-ENVIRONMENT=development
-ENTRAID_TENANT_ID=
-ENTRAID_CLIENT_ID=
 # Get a free trial key from https://www.xlwings.org/trial
 XLWINGS_LICENSE_KEY=your-license-key
+XLWINGS_ENVIRONMENT=dev
+
+XLWINGS_ENTRAID_CLIENT_ID=
+XLWINGS_ENTRAID_TENANT_ID=
diff --git a/README.md b/README.md
@@ -17,8 +17,8 @@
 
 ## Instructions:
 
-- Custom functions can be added under `app/custom_functions/examples.py`. There is a sample custom function included that can be run via `=XLWINGS.HELLO("xlwings")`. There's also a streaming function (`=XLWINGS.STREAMING_RANDOM(2, 3)`). The `XLWINGS` prefix ("namespace") can be adjusted in `manifest.xml` and should be different for each environment (DEV, UAT, PROD, etc.)
-- Macros can be added under `app/routers/macros/example.py`. They will need to be bound to a button on either the ribbon (via `manifest.xml`) or task pane (via `app/templates/taskpane.html`). There is a sample button `Hello World` included on both the ribbon and task pane.
+- Custom functions can be added under `app/custom_functions/examples.py`. To add your own Python modules, see the instructions at the top of `examples.py`. There is a sample custom function included that can be run via `=XLWINGS.HELLO("xlwings")`. There's also a streaming function (`=XLWINGS.STREAMING_RANDOM(2, 3)`). The `XLWINGS` prefix ("namespace") can be adjusted in via the settings (`XLWINGS_FUNCTIONS_NAMESPACE` in `.env` file). Except for the prod environment, `-dev` and `-staging` are automatically appended to avoid name clashes. So if you run this under a dev environment, you'll find the custom functions under the `XLWINGS-DEV` prefix.
+- Macros can be added under `app/routers/macros/examples.py`. To add your own Python modules, see the instructions at the top of `examples.py`. They will need to be bound to a button on either the ribbon (via `app/templates/manifest.xml`) or task pane (via `app/templates/taskpane.html`). There is a sample button `Hello World` included on both the ribbon and task pane.
 
 ## Prod deployment
 
@@ -29,7 +29,7 @@
 
 - Set the environment variable: `XLWINGS_LICENSE_KEY=...`
 - Install the dependencies: `pip install -r requirements.txt`
-- Run the app: `gunicorn app.main:sio_app --bind 0.0.0.0:8000 --access-logfile - --workers 1 --worker-class uvicorn.workers.UvicornWorker`
+- Run the app: `gunicorn app.main:main_app --bind 0.0.0.0:8000 --access-logfile - --workers 1 --worker-class uvicorn.workers.UvicornWorker`
 
 **Backend via Docker**:
 
@@ -40,30 +40,31 @@
 
 **Frontend via Office.js add-in:**
 
-- In `manifest.xml`, replace all occurrences of `https://127.0.0.1:8000` with the URL where your server runs. It is recommended to create a copy of `manifest.xml` for each environment (DEV, UAT, PROD, etc.).
-- In `manifest.xml`, set your own ID (see TODO comment in `manifest.xml`). You should use an own ID for each environment (DEV, UAT, PROD, etc.).
-- The `manifest.xml` has to be deployed via the Office admin console, see: https://docs.xlwings.org/en/latest/pro/server/officejs_addins.html#production-deployment
+- You will get the manifest content under your `URL/manifest`. For example, if you run this on localhost, you can go to https://127.0.0.1:8000/manifest. Copy the content into a file called `manifest.xml`.
+- If the manifest doesn't show the correct URL, set the `XLWINGS_HOSTNAME` settings in `.env` to the domain where your backend runs, e.g., `XLWINGS_HOSTNAME=xlwings.mycompany.com`.
+- The `manifest.xml` has to be deployed via the Office admin console, see: https://docs.xlwings.org/en/latest/pro/server/officejs_addins.html#production-deployment. The Office admin console also allows you to point directly to the `/manifest` endpoint.
 
 ## Dev environment
 
 Follow the steps under https://docs.xlwings.org/en/latest/pro/server/officejs_addins.html#quickstart (but using this repo instead of the one mentioned in the quickstart). Mainly, you need to install `mkcert` to create local certificates as Office.js requires the web app to served via https (not http) even on localhost.
 
 **Backend via Python directly:**
 
-- Copy `.env.template` to `.env` and update `XLWINGS_LICENSE_KEY=...`. Make sure that `ENVIRONMENT=development`.
+- Run `python run.py init`: this copies `.env.template` over to `.env` and replaces the default manifest UUIDs under `app/config.py` with your own ones. Make sure to commit `app/config.py` once it has your own UUIDs.
 - Install the dependencies: `pip install -r requirements.txt`
 - Run the app: `python run.py`
 
 **Backend via Docker**:
 
 - Install Docker and Docker Compose
-- Copy `.env.template` to `.env` and update `XLWINGS_LICENSE_KEY=...`. Make sure that `ENVIRONMENT=development`.
+- Run `python run.py init`: this copies `.env.template` over to `.env` and replaces the default manifest UUIDs under `app/config.py` with your own ones. Make sure to commit `app/config.py` once it has your own UUIDs.
 - To run the dev server: `docker compose up`
 - Run `docker compose build` whenever you need to install a new dependency via `requirements.txt`
 
 **Office.js add-in**:
 
 - Has to be sideloaded, see: https://learn.microsoft.com/en-us/office/dev/add-ins/testing/test-debug-office-add-ins#sideload-an-office-add-in-for-testing
+- You'll find the addin by going to the `/manifest` endpoint, e.g., on localhost, go to https://127.0.0.1:8000/manifest. Store the text in a file called `manifest.xml` that you can use to sideload the add-in.
 
 ## Authentication & Authorization with Entra ID (previously called Azure AD)
 
@@ -76,13 +77,11 @@ Follow the steps under https://docs.xlwings.org/en/latest/pro/server/officejs_ad
    ENTRAID_CLIENT_ID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    ```
 
-3. In `manifest.xml`, uncomment the last section and fill in the `CLIENT_ID` (2x). You also need to adjust the domain if you're not running this on localhost. After changing the manifest, it's usually best to restart Excel to make sure everything is loaded properly.
+3. Calling custom functions (via `custom_functions_call` in `routers/xlwings.py`) and any function in `routers/macros.py` use the `get_user` dependency injection to authenticate the user (see application logs).
 
-4. Calling custom functions (via `custom_functions_call` in `routers/xlwings.py`) and any function in `routers/macros.py` use the `get_user` dependency injection to authenticate the user (see application logs).
+4. To only allow specific users to use your application, you can use role-based access control (RBAC): at the bottom of `auth/entraid.py` you can change the definition of `get_user` to require specific roles or create new dependencies (e.g., `get_admin`).
 
-5. To only allow specific users to use your application, you can use role-based access control (RBAC): at the bottom of `auth/entraid.py` you can change the definition of `get_user` to require specific roles or create new dependencies (e.g., `get_admin`).
-
-6. To set up the roles in Entra ID and map them to users, follow these instructions:
+5. To set up the roles in Entra ID and map them to users, follow these instructions:
 
    Go to All Services > Microsoft Entra ID > App registrations > Your app > App roles (left sidebar):
 

diff --git a/app/auth/entraid.py b/app/auth/entraid.py
@@ -92,8 +92,9 @@ def validate_token(token: str):
                     detail="Auth error: Couldn't validate token",
                 )
         logger.debug(claims)
-    except Exception:
+    except Exception as e:
         logger.debug(f"Authentication error for token: {token}")
+        logger.info(repr(e))
         raise HTTPException(
             status_code=status.HTTP_401_UNAUTHORIZED,
             detail="Auth error: Couldn't validate token",

diff --git a/app/config.py b/app/config.py
@@ -2,11 +2,14 @@
 from pathlib import Path
 from typing import List, Literal, Optional
 
+from pydantic import UUID4, computed_field
 from pydantic_settings import BaseSettings, SettingsConfigDict
 
 
 class Settings(BaseSettings):
-    model_config = SettingsConfigDict(env_file=os.getenv("DOTENV_PATH", ".env"))
+    model_config = SettingsConfigDict(
+        env_prefix="XLWINGS_", env_file=os.getenv("DOTENV_PATH", ".env"), extra="ignore"
+    )
     add_security_headers: bool = True
     base_dir: Path = Path(__file__).resolve().parent
     cors_allow_origins: List[str] = ["*"]
@@ -19,14 +22,25 @@ class Settings(BaseSettings):
     entraid_tenant_id: Optional[str] = None
     # Set to False if you have users from external organizations
     entraid_validate_issuer: bool = True
-    environment: Literal["development", "staging", "production"] = "development"
+    environment: Literal["dev", "qa", "uat", "prod"] = "prod"
+    functions_namespace: str = "XLWINGS"
+    hostname: Optional[str] = None
     log_level: str = "INFO"
+    manifest_id_dev: UUID4 = "0a856eb1-91ab-4f38-b757-23fbe1f73130"
+    manifest_id_qa: UUID4 = "9cda34b1-af68-4dc6-b97c-e63ef6284671"
+    manifest_id_uat: UUID4 = "70428e53-8113-421c-8fe2-9b74fcb94ee5"
+    manifest_id_prod: UUID4 = "4f342d85-3a49-41cb-90a5-37b1f2219040"
+    project_name: str = "xlwings Server"
     public_addin_store: bool = False
-    static_dir: Path = base_dir / "static"
-    xlwings_license_key: str
+    license_key: str
+
+    @computed_field
+    @property
+    def static_dir(self) -> Path:
+        return self.base_dir / "static"
 
 
 settings = Settings()
 
 if not os.getenv("XLWINGS_LICENSE_KEY"):
-    os.environ["XLWINGS_LICENSE_KEY"] = settings.xlwings_license_key
+    os.environ["XLWINGS_LICENSE_KEY"] = settings.license_key
diff --git a/app/main.py b/app/main.py
@@ -11,6 +11,7 @@
 from . import settings
 from .routers import socketio as socketio_router
 from .routers.macros.router import router as macros_router
+from .routers.manifest import router as manifest_router
 from .routers.taskpane import router as taskpane_router
 from .routers.xlwings import router as xlwings_router
 
@@ -42,6 +43,7 @@
 app.include_router(xlwings_router)
 app.include_router(macros_router)
 app.include_router(taskpane_router)
+app.include_router(manifest_router)
 
 
 # Security headers
@@ -93,6 +95,7 @@ async def root():
 
 
 # Static files: in prod should be served via a HTTP server like nginx if possible
+# See also pending ASGI branch in https://github.com/evansd/whitenoise
 app.mount("/static", StaticFiles(directory=settings.static_dir), name="static")
 if settings.environment == "development":
     # Don't cache static files

diff --git a/app/routers/manifest.py b/app/routers/manifest.py
@@ -0,0 +1,49 @@
+import logging
+import os
+
+from fastapi import APIRouter, Request
+
+from ..config import settings
+from ..templates import TemplateResponse
+
+router = APIRouter()
+
+logger = logging.getLogger(__name__)
+
+
+@router.get("/manifest")
+async def manifest(request: Request):
+    if settings.hostname:
+        # Settings
+        base_url = f"https://{settings.hostname}"
+    elif os.getenv("RENDER_EXTERNAL_URL"):
+        # Render.com
+        base_url = os.getenv("RENDER_EXTERNAL_URL")
+    elif os.getenv("WEBSITE_HOSTNAME"):
+        # Azure Functions and Azure App Service
+        base_url = f"https://{os.getenv('WEBSITE_HOSTNAME')}"
+    elif os.getenv("CODESPACES"):
+        # GitHub Codespaces
+        base_url = f"https://{os.getenv('CODESPACE_NAME')}-8000.{os.getenv('GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN')}"
+    else:
+        # Mostly localhost
+        base_url = request.base_url
+
+    manifest_ids = {
+        "dev": settings.manifest_id_dev,
+        "qa": settings.manifest_id_qa,
+        "uat": settings.manifest_id_uat,
+        "prod": settings.manifest_id_prod,
+    }
+    manifest_id = manifest_ids[settings.environment]
+
+    return TemplateResponse(
+        request=request,
+        name="/manifest.xml",
+        context={
+            "settings": settings,
+            "base_url": str(base_url).rstrip("/"),
+            "manifest_id": manifest_id,
+        },
+        media_type="text/plain",
+    )
diff --git a/app/static/images/examples/xlwings-16.png → ...tic/images/ribbon/examples/xlwings-16.png b/app/static/images/examples/xlwings-16.png → ...tic/images/ribbon/examples/xlwings-16.png
diff --git a/app/static/images/examples/xlwings-32.png → ...tic/images/ribbon/examples/xlwings-32.png b/app/static/images/examples/xlwings-32.png → ...tic/images/ribbon/examples/xlwings-32.png
diff --git a/app/static/images/examples/xlwings-64.png → ...tic/images/ribbon/examples/xlwings-64.png b/app/static/images/examples/xlwings-64.png → ...tic/images/ribbon/examples/xlwings-64.png
diff --git a/app/static/images/examples/xlwings-80.png → ...tic/images/ribbon/examples/xlwings-80.png b/app/static/images/examples/xlwings-80.png → ...tic/images/ribbon/examples/xlwings-80.png
diff --git a/manifest.xml → app/templates/manifest.xml b/manifest.xml → app/templates/manifest.xml
@@ -3,60 +3,57 @@
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns:bt="http://schemas.microsoft.com/office/officeappbasictypes/1.0"
   xmlns:ov="http://schemas.microsoft.com/office/taskpaneappversionoverrides" xsi:type="TaskPaneApp">
-  <!-- TODO: Create your own ID via https://www.guidgen.com or by running in Python: import uuid;print(uuid.uuid4()) -->
-  <Id>0a856eb1-91ab-4f38-b757-23fbe1f73130</Id>
+  <Id>{{ manifest_id }}</Id>
   <Version>1.0.0</Version>
   <ProviderName>xlwings</ProviderName>
   <DefaultLocale>en-US</DefaultLocale>
-  <DisplayName DefaultValue="xlwings Server" />
-  <Description DefaultValue="xlwings Server" />
-  <IconUrl DefaultValue="https://127.0.0.1:8000/static/images/examples/xlwings-32.png" />
-  <HighResolutionIconUrl DefaultValue="https://127.0.0.1:8000/static/images/examples/xlwings-64.png" />
+  <DisplayName DefaultValue="{{ settings.project_name }}{{ ' [' + settings.environment + ']' if settings.environment != 'prod'}}" />
+  <Description DefaultValue="{{ settings.project_name }}{{ ' [' + settings.environment + ']' if settings.environment != 'prod'}}" />
+  <IconUrl DefaultValue="{{ base_url }}/static/images/ribbon/examples/xlwings-32.png" />
+  <HighResolutionIconUrl DefaultValue="{{ base_url }}/static/images/ribbon/examples/xlwings-64.png" />
   <SupportUrl DefaultValue="https://www.xlwings.org/contact" />
   <AppDomains>
-    <AppDomain>https://127.0.0.1:8000</AppDomain>
+    <AppDomain>{{ base_url }}</AppDomain>
   </AppDomains>
   <Hosts>
     <Host Name="Workbook" />
   </Hosts>
   <!-- SharedRuntime for Custom Functions -->
   <Requirements>
     <Sets DefaultMinVersion="1.1">
-      <Set Name="SharedRuntime" MinVersion="1.1"/>
+      <Set Name="SharedRuntime" MinVersion="1.1" />
     </Sets>
   </Requirements>
   <DefaultSettings>
-    <SourceLocation DefaultValue="https://127.0.0.1:8000/taskpane" />
+    <SourceLocation DefaultValue="{{ base_url }}/taskpane" />
   </DefaultSettings>
   <Permissions>ReadWriteDocument</Permissions>
   <VersionOverrides xmlns="http://schemas.microsoft.com/office/taskpaneappversionoverrides" xsi:type="VersionOverridesV1_0">
     <Hosts>
       <Host xsi:type="Workbook">
-
         <!-- Custom Functions -->
         <Runtimes>
-          <Runtime resid="Taskpane.Url" lifetime="long"/>
+          <Runtime resid="Taskpane.Url" lifetime="long" />
         </Runtimes>
         <AllFormFactors>
           <ExtensionPoint xsi:type="CustomFunctions">
             <Script>
-              <SourceLocation resid="Functions.Script.Url"/>
+              <SourceLocation resid="Functions.Script.Url" />
             </Script>
             <Page>
-              <SourceLocation resid="Taskpane.Url"/>
+              <SourceLocation resid="Taskpane.Url" />
             </Page>
             <Metadata>
-              <SourceLocation resid="Functions.Metadata.Url"/>
+              <SourceLocation resid="Functions.Metadata.Url" />
             </Metadata>
-            <Namespace resid="Functions.Namespace"/>
+            <Namespace resid="Functions.Namespace" />
           </ExtensionPoint>
         </AllFormFactors>
 
         <DesktopFormFactor>
           <!-- FunctionFile is used to bind functions directly to Ribbon buttons -->
           <FunctionFile resid="Taskpane.Url" />
           <ExtensionPoint xsi:type="PrimaryCommandSurface">
-
             <CustomTab id="MyTab">
               <Group id="MyCommandsGroup">
                 <Label resid="MyCommandsGroup.Label" />
@@ -107,32 +104,29 @@
                     <SourceLocation resid="Taskpane.Url" />
                   </Action>
                 </Control>
-
               </Group>
               <Label resid="MyTab.TabLabel" />
             </CustomTab>
-
           </ExtensionPoint>
         </DesktopFormFactor>
       </Host>
     </Hosts>
 
     <Resources>
       <bt:Images>
-        <bt:Image id="Icon.16x16" DefaultValue="https://127.0.0.1:8000/static/images/examples/xlwings-16.png" />
-        <bt:Image id="Icon.32x32" DefaultValue="https://127.0.0.1:8000/static/images/examples/xlwings-32.png" />
-        <bt:Image id="Icon.80x80" DefaultValue="https://127.0.0.1:8000/static/images/examples/xlwings-80.png" />
+        <bt:Image id="Icon.16x16" DefaultValue="{{ base_url }}/static/images/ribbon/examples/xlwings-16.png" />
+        <bt:Image id="Icon.32x32" DefaultValue="{{ base_url }}/static/images/ribbon/examples/xlwings-32.png" />
+        <bt:Image id="Icon.80x80" DefaultValue="{{ base_url }}/static/images/ribbon/examples/xlwings-80.png" />
       </bt:Images>
       <bt:Urls>
         <!-- This must point to the HTML document with the task pane -->
-        <bt:Url id="Taskpane.Url" DefaultValue="https://127.0.0.1:8000/taskpane" />
-        <bt:Url id="Functions.Script.Url" DefaultValue="https://127.0.0.1:8000/xlwings/custom-functions-code"/>
-        <bt:Url id="Functions.Metadata.Url" DefaultValue="https://127.0.0.1:8000/xlwings/custom-functions-meta"/>
+        <bt:Url id="Taskpane.Url" DefaultValue="{{ base_url }}/taskpane" />
+        <bt:Url id="Functions.Script.Url" DefaultValue="{{ base_url }}/xlwings/custom-functions-code" />
+        <bt:Url id="Functions.Metadata.Url" DefaultValue="{{ base_url }}/xlwings/custom-functions-meta" />
       </bt:Urls>
       <bt:ShortStrings>
-        <!-- Uncomment the next line to use a global Namespace. It can also be set per function via xw.func(namespace="...") -->
-        <bt:String id="Functions.Namespace" DefaultValue="XLWINGS"/>
-        <bt:String id="MyTab.TabLabel" DefaultValue="xlwings Server" />
+        <bt:String id="Functions.Namespace" DefaultValue="{{ settings.functions_namespace|upper }}{{ '_' if settings.functions_namespace and settings.environment != 'prod'}}{{ settings.environment | upper if settings.environment != 'prod'}}" />
+        <bt:String id="MyTab.TabLabel" DefaultValue="{{ settings.project_name }}{{ ' [' + settings.environment + ']' if settings.environment != 'prod'}}" />
         <bt:String id="MyCommandsGroup.Label" DefaultValue="MyGroup" />
         <bt:String id="MyFunctionButton.Label" DefaultValue="Hello World" />
         <bt:String id="MyTaskpaneButton.Label" DefaultValue="Show Taskpane" />
@@ -143,14 +137,15 @@
       </bt:LongStrings>
     </Resources>
 
-    <!-- <WebApplicationInfo>
-      <Id>CLIENT_ID</Id>
-      <Resource>api://127.0.0.1:8000/CLIENT_ID</Resource>
+    {% if settings.entraid_tenant_id or settings.entraid_client_id %}
+    <WebApplicationInfo>
+      <Id>{{ settings.entraid_client_id }}</Id>
+      <Resource>api://{{ base_url | replace("https://", "") }}/{{ settings.entraid_client_id }}</Resource>
       <Scopes>
         <Scope>openid</Scope>
         <Scope>profile</Scope>
       </Scopes>
-    </WebApplicationInfo> -->
-
+    </WebApplicationInfo>
+    {% endif %}
   </VersionOverrides>
 </OfficeApp>