Documentation for developers for the Transfer Digital Records (TDR) project.
The current repositories relating to TDR, and their purpose are:
- tdr-consignment-api - API for managing consignment and file details
- tdr-transfer-frontend - Code for the TDR frontend to allow users to transfer files
- tdr-transfer-service - Code for supporting non-network drive transfers
- tdr-auth-server - Jenkins and docker configuration to build and deploy the Keycloak server
- tdr-auth-utils - A library of useful auth functions
- tdr-keycloak-user-management - Library to create Keycloak users
- tdr-signed-cookies - generates signed cookie for upload
- tdr-file-metadata - Library for extracting file metadata like client-side checksums
- tdr-components - Library of re-usable UI components
- tdr-aws-utils - Scala wrappers for AWS Java SDK methods used in TDR
- tdr-consignment-api-data - Migrations for the main TDR database and Slick helper classes generated from the TDR schema
- tdr-generated-graphql - Generated case classes to use when querying the API
- tdr-graphql-client - A simple scala client for connecting to the API
- tdr-antivirus - Task for scanning uploaded files for malware
- tdr-api-update - Task for sending the results of file checks to the API
- tdr-backend-checks-utils - Reads/Writes step function states to S3 to track file checks during processing
- tdr-backend-checks-results - Aggregates the results of each file check and writes to S3 in a format readable by other file checks
- tdr-checksum - Task for calculating the server-side checksum of uploaded files
- tdr-file-upload-data - Task for retrieving file data to be used by the rest of the file checks
- tdr-file-format - Task for determining the format of uploaded files
- tdr-redacted-files - Task for checking redacted files include original version in upload
- tdr-statuses - Lambda process to generate task statuses and pass them on to API update task
- tdr-consignment-export - Task for exporting a consignment from TDR
- tdr-consignment-export-authoriser - Lambda which authorises a request to export a consignment
- tdr-export-status-update - Lambda which changes the export status of a consignment
- tdr-aws-accounts - AWS account level configuration
- tdr-create-db-users - Task that creates DB users for other processes to use, so master user doesn't have to be used
- tdr-configurations - Private repository for configuration files and parameters
- tdr-ecr-scan - Task which starts an ECR image scan for Docker images which are currently in use
- tdr-ecr-scan-notifications - Task which sends alerts when ECR scans detect a Docker image vulnerability
- tdr-github-actions - shared set of GitHub actions, workflows and other scripts
- tdr-rotate-keycloak-secrets - Script to rotate secrets used on Keycloak
- tdr-service-unavailable - TDR service unavailable page. Deployed through script in
tdr-scriptsrepo - tdr-scripts - set of individual scripts to support automation of regular development operations tasks
- tdr-terraform-backend - Terraform scripts for setting up the Terraform backends in the TDR management account
- tdr-terraform-environments - Terraform scripts for provisioning AWS resources to support the TDR application within the different environments
- tdr-terraform-modules - Terraform modules for use by other Terraform repositories
- tna-custodian - Implementation of Cloud Custodian for National Archives
- tdr-notification - Lambda to send TDR messages.
- tdr-e2e-tests - End to end tests for TDR
- tdr-reporting - TDR reporting
- tdr-xray-logging - Custom open telemetry image for X-Ray logging
- tdr-local-aws - Emulate backend file checks in the development environment
- tdr-dev-documentation - For TDR development documentation
- tdr-dev-documentation-internal - A private repo with development documentation that cannot be added to the public docs
- tdr-design-documentation - Design decisions and wireframes
Prototypes and proofs of concept built during the Beta phase:
- tdr-prototype-cookie-signing - Prototype of a Lambda function which generates signed cookies for S3 uploads
- tdr-prototype-json-logging - Prototype of a Scala application which writes logs in a JSON format
- tdr-interactive-prototype - Interactive prototype using the GOV.UK Prototype Kit
Repos that were part of the final Alpha prototype:
- prototype-terraform - For prototyping with Terraform
- tdr-prototype-sangria - Prototype GraphQL server written in Scala with Sangria
- tdr-prototype-mvc - Prototype Play MVC app
- tdr-prototype-file-export - Prototype of file export for digital archivists
- prototype-state-machine - Prototype of file check tasks, like checksum calculation
- tdr-prototype-wasm-checksum - Prototype of checksum calculations in WebAssembly
- prototype-tdr-jenkins - Infrastructure-as-code for Jenkins
Other ideas that we prototyped, but which didn't form part of the final system:
- prototype-frontend-next - Prototype of the frontend in Next.js
- prototype-front-end - Prototype of the frontend in React
- prototype-cypress - Prototype of Cypress tests for TDR
- prototype-graphql - Prototype GraphQL server written in Node.js with Apollo Server
- prototype-appsync - Prototype AWS AppSync Graphql server
- prototype-server - Prototype REST API
- Developer manual: a guide for developers working on TDR
- Technology considerations: results of prototypes and spikes into different technology choices for TDR
- Architecture decision records: technical decisions that we make while building the production system
Unless stated otherwise, the codebase is released under the MIT License. This covers both the codebase and any sample code in the documentation.
The documentation is © Crown copyright and available under the terms of the Open Government 3.0 licence.