Central Digital Platform

This is a technical documentation for the Cabinet Office’s Central Digital Platform.

Introduction

The Central Digital Platform (CDP) that is being developed to support the new procurement regime.

Overview

Context

The system context diagram below presents the Central Digital Platform system in the context of users and other systems.

Figure 1. System Context for Central Digital Platform

Figure 2. Legend

Software Architecture

Container view

The diagram below focuses on high level responsibilities and technology choices within the Central Digital Platform system.

Figure 3. Container View of the Central Digital Platform system

Figure 4. Legend

Development

This section covers development related topics.

Development environment

Requirements

The minimum set of tools required to work on the project covers:

An IDE of your choice (i.e. Visual Studio, Rider)
Docker

On Windows, it might be handy to set up git-bash or WSL in order to run tools like Make.

Configuration

The application is mostly configured to start with a fresh repository checkout. The only configuration missing are secrets that are not safe to be commited to the repository.

Never commit secrets to the repository.

Secrets are managed with the Secrets Manager and environment variables. IDE depends on the secrets manager while Docker depends on environment variables.

Environment variables for Docker are managed in compose.override.yml file that’s not version controlled. Run the following command to create it with defaults:

make compose.override.yml

Organisation App

The OrganisationApp requires the following secrets / environment variables:

OneLogin:Authority / OneLogin__Authority
OneLogin:ClientId / OneLogin__ClientId
OneLogin:PrivateKey / OneLogin__PrivateKey

These can be set as secrets with the following dotnet commands:

dotnet user-secrets set --project Frontend/CO.CDP.OrganisationApp/CO.CDP.OrganisationApp.csproj OneLogin:Authority "https://oidc.example.com"
dotnet user-secrets set --project Frontend/CO.CDP.OrganisationApp/CO.CDP.OrganisationApp.csproj OneLogin:ClientId "client-id"
dotnet user-secrets set --project Frontend/CO.CDP.OrganisationApp/CO.CDP.OrganisationApp.csproj OneLogin:PrivateKey "-----BEGIN RSA PRIVATE KEY-----SECRET KEY-----END RSA PRIVATE KEY-----"

For Docker, the following environment variables need to be updated in compose.override.yml for the organisation-app service:

  organisation-app:
    environment:
      OneLogin__Authority: "https://oidc.example.com"
      OneLogin__ClientId: "client-id"
      OneLogin__PrivateKey: "-----BEGIN RSA PRIVATE KEY-----SECRET KEY-----END RSA PRIVATE KEY-----"

Organisation WebApi

The Organisation.WebApi requires the following secrets / environment variables:

GOVUKNotify:ApiKey / GOVUKNotify__ApiKey

These can be set as secrets with the following dotnet commands:

dotnet user-secrets set --project Services/CO.CDP.Organisation.WebApi/CO.CDP.Organisation.WebApi.csproj GOVUKNotify:ApiKey "123456"

For Docker, the following environment variables need to be updated in compose.override.yml for the organisation service:

  organisation:
    environment:
      GOVUKNotify__ApiKey: "123456"

Authority API

The Authority API depends on the following secrets / environment variables:

OneLogin:Authority / OneLogin__Authority
PrivateKey / PrivateKey

To set them in the secrets manager, run:

dotnet user-secrets set --project Services/CO.CDP.Organisation.Authority OneLogin:Authority "https://oidc.example.com"
dotnet user-secrets set --project Services/CO.CDP.Organisation.Authority PrivateKey "-----BEGIN RSA PRIVATE KEY----"

Note	The `make generate-authority-keys` command generates a private key that can be used with `PrivateKey`. Make sure to copy the contents of file and not the path.

For Docker, update the authority service in compose.override.yml:

  authority:
    environment:
      PrivateKey: "-----BEGIN RSA PRIVATE KEY-----"
      OneLogin__Authority: "https://oidc.example.com"

Setting up the IDE

Import the project to your favourite IDE to build and run tests from there.

Figure 5. Running tests in an IDE

Alternatively, use the dotnet command or the following make targets to build and run tests:

make build
make test

Any dotnet tools used by the project are installed locally and will be restored by the above commands.

Setting Up Docker

While local development is done within the IDE of our choice, the Docker Compose configuration is provided to conveniently start all (or some) services at once.

First, make sure you configured environment variables in compose.override.yml (see Configuration).

Next, build all the Docker containers with the build-docker Make target:

make build-docker

Finally, we can start all Docker services with:

make up

By default, service and application ports are mapped as follows:

OrganisationApp - - http://localhost:8090/
Authority - - http://localhost:8092/swagger/
Tenant - http://localhost:8080/swagger/
Organisation - http://localhost:8082/swagger/
Person - http://localhost:8084/swagger/
Forms - http://localhost:8086/swagger/
Data Sharing - http://localhost:8088/swagger/
PostgreSQL database - :5432

Figure 6. All services started in Docker

Later, all services can be stopped and destroyed with:

make down

Make targets

There’s a number of Make targets that provide shortcuts during development. Run make help to get an up-to-date list.

Table 1. Make targets
Target	Description
help	Shows available commands
build	Builds the solution
test	Runs all tests
up	Starts all the Docker containers
down	Stops and removes all Docker containers
stop	Stops all Docker containers
ps	Lists all running Docker containers
db	Starts the database Docker container only and runs migrations
localstack	Starts the localstack Docker container for AWS services available locally
generate-authority-keys	Generates the private/public key pair for the authority service

Cookbooks

This section stores recipes useful in development.

Using Docker for development

During development, testing, or debugging it’s often useful to run some services with Docker, while others with an IDE. Others, prefer to run everything in the IDE, but infrastructure dependencies, like a database, with Docker. Both scenarios are supported.

Mixing services started on Docker and IDE

By default, Docker Compose will start all services defined in the Compose configuration.

Figure 7. All services started in Docker

Imagine we’d like to work on the Organisation App. Ideally, we’d work on it within our IDE, but continue running all the other services in Docker to use the application end-to-end.

First, let’s disable the organisation-app service in compose.override.yml by setting replicas to 0.

# ...
  organisation-app:
    deploy:
      replicas: 0
    # ...

This way, when we start Docker services the organisation-app won’t be started as we expect it to be started from the IDE.

Next, point the gateway to where the Organisation App is running (outside of Docker):

# ...
  gateway:
    environment:
      CDP_ORGANISATION_APP_HOST: 'http://host.docker.internal:58090'
#      CDP_AUTHORITY_HOST: 'http://host.docker.internal:5050'
#      CDP_TENANT_HOST: 'http://host.docker.internal:58080'
#      CDP_ORGANISATION_HOST: 'http://host.docker.internal:58082'
#      CDP_PERSON_HOST: 'http://host.docker.internal:58084'
#      CDP_FORMS_HOST: 'http://host.docker.internal:58086'
#      CDP_DATA_SHARING_HOST: 'http://host.docker.internal:58088'
    # ...

We use host.docker.internal to point to the host machine from Docker containers.

Now, we can start all Docker services (but the Organisation App) with make up and run the Organisation App with the IDE (choose http-for-docker launch profile).

Figure 8. Organisation App running in the IDE

We can pick and choose which services are run by the host or by Docker, so if we needed to also run the organisation service in the IDE, we can follow the same pattern. First, edit compose.override.yml:

# ...
  gateway:
    environment:
      CDP_ORGANISATION_APP_HOST: 'http://host.docker.internal:58090'
#      CDP_AUTHORITY_HOST: 'http://host.docker.internal:5050'
#      CDP_TENANT_HOST: 'http://host.docker.internal:58080'
      CDP_ORGANISATION_HOST: 'http://host.docker.internal:58082'
#      CDP_PERSON_HOST: 'http://host.docker.internal:58084'
#      CDP_FORMS_HOST: 'http://host.docker.internal:58086'
#      CDP_DATA_SHARING_HOST: 'http://host.docker.internal:58088'
  organisation:
    deploy:
      replicas: 0
    # ...

Next, start selected services in the IDE, while Docker takes care of the other ones.

Figure 9. Organisation App and Organisation service running in the IDE

Starting everything in the IDE

In case you preferred to run all the dotnet services in the IDE, you can disable them all in Docker by setting replicas to 0.

Alternatively to setting replicas to 0, you can run the db and migrations containers only with:

make db

Use the http launch profile in your IDE to start each service. All services can be run together by using a multi-launch profile (has to be created manually).

Figure 10. Multi-launch profile