All notable changes to this project will be documented in this file.
+The format is based on Keep a Changelog, +and this project adheres to +Semantic Versioning.
+fastapi to 0.100.0sentry_sdk to 1.28.1uvicorn to 0.23.0activity parameter in GET /statementsclickhouse backend
+General improvement for the Helm Chart:downloaded verb)async_lrs backendlrs backendfastapi to 0.97.0sentry_sdk to 1.25.1client_options to a dedicated pydantic modelhttpx to 0.24.1/statements
+GET query agent filteringcachetools to 5.3.1verb.display field no longer mandatory in xAPI models and for convertersentry_sdk to 1.22.2uvicorn to 0.22.0/statements GET method returns a code 400 with certain parameters
+as per the xAPI specificationmore IRL building in LRS /statements GET requestshttpx to 0.24.0fastapi to 0.95.1sentry_sdk to 1.19.1uvicorn to 0.21.1sentry_sdk to 1.16.0fastapi to 0.92.0sentry_sdk to 1.15.0extensions fields3.7+auth subcommand to generate required credentials file for the LRS--version optionfastapi to 0.89.1httpx to 0.23.3sentry_sdk to 1.14.0uvicorn to 0.20.0ca_certs path for the ES backend client option (LRS)https://handbook.openfun.frfastapi to 0.87.0post endpoint for statements implementing the LRS specget
+ method (used in the fetch command)python-dotenv dependencyhost and port options for the runserver cli commandstatus checkinghealth LRS routerpython-legacy handler for mkdocstrings packageclick to 8.1.3elasticsearch to 8.3.3fastapi to 0.79.1ovh to 1.0.0pydantic to 1.9.2pymongo to 4.2.0python-keystoneclient to 5.0.0python-swiftclient to 4.0.1requests to 2.28.1sentry_sdk to 1.9.5uvicorn to 0.18.2websockets to 10.3pyyaml dependency (BC)ws websocket stream backend (compatible with the fetch command)jq, curl and wget in the fundocker/ralph Docker imagerunserver command with basic auth and a Whoami routeget endpoint for statements implementing the LRS specuvicorn to 0.17.4elasticsearch to 7.17.0sentry_sdk to 1.5.5fastapi to 0.73.0pyparsing to 3.0.7pydantic to 1.9.0python-keystoneclient to 4.4.0python-swiftclient to 3.13.0pyyaml to 6.0requests to 2.27.1websockets to 10.1elasticsearch to 7.13.3interacted pydantic modelsterminated pydantic modelscompleted pydantic modelsseeked pydantic modelsinitialized pydantic modelspaused pydantic modelsconvert command to transform edX events to xAPI formatviewed andpage_close eventsplayed pydantic modelsviewed and page terminated pydantic modelsvalidate command to lint edX events using pydantic models--es-op-type optionelasticsearch to 7.13.2python-swiftclient to 3.12.0click to 8.0.1click-option-group to 0.5.3pydantic to 1.8.2sentry_sdk to 1.1.0version immutable field in DC selectorpandas is no longer requiredpush command --ignore-errors option to ignore ES bulk import
+ errorspyyaml to 5.4.1pandas to 1.2.1click_log is no longer required as we are able to configure loggingMIT License
+Copyright (c) 2020-present France Université Numérique
+Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the “Software”), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions:
+The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE.
+ + + + + + +Ralph comes with an API server that aims to implement the Learning Record Store +(LRS) specification (still a work in progress).
+The API server supports the following authentication methods:
+The default method for securing Ralph API server is with HTTP basic authentication.
+The API server can be started up with the following command:
+$ ralph runserver --backend es
+The --backend (or -b) option specifies which database backend to use for
+LRS data storage and retrieval. See Ralph’s backends
+documentation for more details.
However, before you can start your API server and make requests against it, you +need to set up your credentials.
+The credentials file is expected to be a valid JSON file. Its location is
+specified by the RALPH_AUTH_FILE configuration value. By default, ralph
+will look for the auth.json file in the application directory (see click
+documentation for
+details).
The expected format is a list of entries (JSON objects) each containing the
+username, the user’s bcrypt hashed+salted password and scopes they can
+access:
[
+ {
+ "username": "john.doe@example.com",
+ "hash": "$2b$12$yBXrzIuRIk6yaft5KUgVFOIPv0PskCCh9PXmF2t7pno.qUZ5LK0D2",
+ "scopes": ["example_scope"]
+ },
+ {
+ "username": "simon.says@example.com",
+ "hash": "$2b$12$yBXrzIuRIk6yaft5KUgVFOIPv0PskCCh9PXmF2t7pno.qUZ5LK0D2",
+ "scopes": ["second_scope", "third_scope"]
+ }
+]
+To create a new user credentials, Ralph’s CLI provides a dedicated command:
+$ ralph auth \
+ --username janedoe \
+ --password supersecret \
+ --scope janedoe_scope \
+ -w
+This command updates your credentials file with the new janedoe user.
++Note that running this command requires that you installed Ralph with the CLI +optional dependencies, e.g.
+pip install ralph-malph[cli](which we highly +recommend).
(Work In Progress)
+The first request that can be answered by the ralph API server is a whoami request, which checks if the user is authenticated and returns their username and permission scopes.
Use curl to get http://localhost:8100/whoami:
$ curl http://localhost:8100/whoami
+< HTTP/1.1 401 Unauthorized
+< {"error":"Not authenticated"}
+Send your username and password to the API server through HTTP Basic Auth:
+$ curl --user john.doe@example.com:PASSWORD http://localhost:8100/whoami
+< HTTP/1.1 200 OK
+< {"username":"john.doe@example.com","scopes":["authenticated","example_scope"]}
+Ralph LRS API server supports OpenID Connect (OIDC) on top of OAuth 2.0 for authentication and authorization.
+To enable OIDC auth, you should set the RALPH_RUNSERVER_AUTH_BACKEND environment variable as follows:
+
RALPH_RUNSERVER_AUTH_BACKEND=oidc
+RALPH_RUNSERVER_AUTH_OIDC_ISSUER_URI environment variable with your identity provider’s Issuer Identifier URI as follows:
+RALPH_RUNSERVER_AUTH_OIDC_ISSUER_URI=http://{provider_host}:{provider_port}/auth/realms/{realm_name}
+This address must be accessible to the LRS on startup as it will perform OIDC Discovery to retrieve public keys and other information about the OIDC environment.
+It is also strongly recommended that you set the optional RALPH_RUNSERVER_AUTH_OIDC_AUDIENCE environment variable to the origin address of the LRS itself (ex. “http://localhost:8100”) to enable verification that a given token was issued specifically for the LRS.
OIDC support is currently developed and tested against Keycloak but may work with other identity providers that implement the specification.
+The Learning analytics playground repository contains a Docker Compose file and configuration for a demo instance of Keycloak with a ralph client.
(Work In Progress)
+The first request that can be answered by the ralph API server is a whoami request, which checks if the user is authenticated and returns their username and permission scopes.
Use curl to get http://localhost:8100/whoami:
$ curl http://localhost:8100/whoami
+< HTTP/1.1" 401 Unauthorized
+< {"detail":"Could not validate credentials"}
+curl --request POST 'http://localhost:8080/auth/realms/fun-mooc/protocol/openid-connect/token' \
+--header 'Content-Type: application/x-www-form-urlencoded' \
+--data-urlencode 'client_id=ralph' \
+--data-urlencode 'client_secret=super-secret' \
+--data-urlencode 'username=ralph_admin' \
+--data-urlencode 'password=funfunfun' \
+--data-urlencode 'grant_type=password'
+which outputs (tokens truncated for example purpose):
+{
+ "access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJSTWlLM",
+ "expires_in":300,
+ "refresh_expires_in":1800,
+ "refresh_token":"eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICI4MDc5NjExM",
+ "token_type":"Bearer",
+ "not-before-policy":0,
+ "session_state":"22a36735-e35f-496b-a243-152d32ebff45",
+ "scope":"profile email"
+}
+Send the access token to the API server as a Bearer header:
+$ curl http://localhost:8100/whoami --header "Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJSTWlLM"
+< HTTP/1.1 200 OK
+< {"username":"ralph_admin","scopes":["all"]}
+Ralph’s API server can be configured to forward xAPI statements it receives to other +LRSes.
+To configure statement forwarding, you need to create a .env file in the current
+directory and define the RALPH_XAPI_FORWARDINGS variable or define the
+RALPH_XAPI_FORWARDINGS environment variable.
The value of the RALPH_XAPI_FORWARDINGS variable should be a JSON encoded list of
+dictionaries where each dictionary defines a forwarding configuration and consists of
+the following key/value pairs:
| key | +value type | +description | +
|---|---|---|
is_active |
+boolean |
+Specifies whether or not this forwarding configuration should take effect. | +
url |
+URL |
+Specifies the endpoint URL where forwarded statements should be send. | +
basic_username |
+string |
+Specifies the basic auth username. | +
basic_password |
+string |
+Specifies the basic auth password. | +
max_retries |
+number |
+Specifies the number of times a failed forwarding request should be retried. | +
timeout |
+number |
+Specifies the duration in seconds of network inactivity leading to a timeout. | +
++Note that for a forwarding configuration to be valid it is required that all key/value +pairs are defined.
+
Example of a valid forwarding configuration:
+RALPH_XAPI_FORWARDINGS='
+[
+ {
+ "is_active": true,
+ "url": "http://lrs1.example.com/xAPI/statements/",
+ "basic_username": "admin1@example.com",
+ "basic_password": "PASSWORD1",
+ "max_retries": 1,
+ "timeout": 5
+ },
+ {
+ "is_active": true,
+ "url": "http://lrs2.example.com/xAPI/statements/",
+ "basic_username": "admin2@example.com",
+ "basic_password": "PASSWORD2",
+ "max_retries": 5,
+ "timeout": 0.2
+ }
+]
+'
+Ralph provides Sentry integration to monitor its LRS server and its CLI. +To activate Sentry integration, one should define the following environment variables:
+RALPH_SENTRY_DSN={PROTOCOL}://{PUBLIC_KEY}:{SECRET_KEY}@{HOST}{PATH}/{PROJECT_ID}
+RALPH_EXECUTION_ENVIRONMENT=development
+The Sentry DSN (Data Source Name) can be found in your project settings from Sentry application. The execution environment should reflect the environment Ralph has been deployed in (e.g. production).
You may also want to monitor the performance of Ralph by configuring the CLI and LRS traces sample rates: +
RALPH_SENTRY_CLI_TRACES_SAMPLE_RATE=0.1
+RALPH_SENTRY_LRS_TRACES_SAMPLE_RATE=0.3
+++Note that a sampling rate of
+1.0means 100% of transactions are sent to sentry and0.1only 10%.
If you want to lower noisy transactions (e.g. in a Kubernetes cluster), you can disable health checks related ones: +
RALPH_SENTRY_IGNORE_HEALTH_CHECKS=True
+HTTP basic auth implementation uses the secure and standard bcrypt algorithm to hash/salt passwords before storing them. +This implementation comes with a performance cost. +To speed up requests, credentials are stored in a LRU cache with a Time To Live. +To configure this cache, you can define the following environment variables: +- the maximum number of entries in the cache. Select a value greater than the maximum number of individual user credentials, for better performance. Defaults to 100. +
RALPH_AUTH_CACHE_MAX_SIZE=100
+RALPH_AUTH_CACHE_TTL=3600
+