fixnotebook

FIX allocation using Jupyter

This document serves as a step-by-step walkthrough of the examples under this directory.

The examples consist of a rudimentary FIX 5.0 allocation workflow. This was chosen as it is similar to the hackathon exercises, yet the standard (FIX vs CDM) and workflows differ substantially.

The target audience is familiar with DAML, knows how to write a basic model with scenarios, load them onto the sandbox, and visualize them with the navigator. This is all covered in the quickstart, up to the Integrate with the ledger section.

To understand the high-level architecture, refer to the reference documentation

The Model

Let's have a look at the root directory:

fixnotebook
├── daml
│   ├── Fix5.daml
│   ├── Main.daml
│   └── Scenarios.daml
├── daml.yaml
└── python
    ├── readyToBookTrigger.py
    └── ui.ipynb

This has been created by the DAML assistant, and has the usual directory structure for a DAML project, with the addition of a python folder to hold our client/UI code.

You will find the DAML model is split across two files, Fix5, which contains only DAML data definitions, and the Main module, which defines the templates. This has been deliberately done to mirror the way the CDM model is structured in the hackathon (except the data model is provided in a pre-built *.dar file).

To recap, in our example, data are pure data structures that could be code-generated from a formal standard such as FIX in our case, or CDM in the hackathon. For instance, the first data type describes an execution:

-- Orders/Executions -------------------------------------------

-- This is a really cut-down version of the actual message
data ExecutionReport =
  ExecutionReport with
    orderID: Text
    parties: [Parties]
    ordStatus: OrdStatus
    tradeDate: Date
    avgPx: Decimal
    cumQty: Int
  deriving (Eq, Show)

These data structures would be mapped from a formal specification, but contain no notion of semantics. Also, they are not contracts - they can't live on the ledger on their own. A DAML contract, on the other hand, must have a signatory, which gives a DAML Party the right to create and archive the contract. Contracts are then composed through choices to delegate rights between users. In the Main.daml file, we wrap this type in a template:

template Execution
  with
    report: ExecutionReport
    broker: Party
  where
    ensure [ broker ] == gatherParties ExecutingFirm [ report ]
    signatory broker
    observer gatherParties ClientID [ report ]

Note the ensure statement, which enforces that the signatory in this contract is consistent with the parties on the execution. It prevents, for example, a client from creating an execution.

You will find two more contracts, ProposeBilateralAllocation and Allocation that follow this same pattern.

Before loading the contracts into the ledger, we can test out that these work through VSCode. Load up VSCode if you haven't done so already with the daml studio command from the ui directory. Make sure you have the official DAML plugin installed.

If you navigate to the file Scenario.daml, click on the link above setup = scenario do, and you should get a table that looks something like this (the second table is only visible when clicking 'show archived'):

Main:BilateralAllocation

broker

client

id

status

initiator

instruction.allocID

instruction.allocTransType

instruction.allocType

instruction.refAllocID

instruction.allocCancReplaceReason

instruction.ordAllocGrp

instruction.side

instruction.instrument.symbol

instruction.instrument.securityID

instruction.instrument.securityIDSource

instruction.avgPx

instruction.quantity

instruction.tradeDate

instruction.settlDate

instruction.allocGrp

responder

affirmation.allocID

affirmation.parties

affirmation.allocStatus

affirmation.matchStatus

affirmation.allocAckGrp

X

X

#1:1

active

'client'

""

Fix5:AllocTransType:New

Fix5:AllocType:Preliminary

none

none

[]

Fix5:Side:Buy

"BARC"

"GB0031348658"

"ISIN"

140.5200000000

100000

2019-09-06T

2019-09-10T

[Fix5:AllocGrp with allocAccount = "Account1"; allocPrice = 140.5200000000; allocQty = 100000; parties = [(Fix5:Parties with partyID = "client"; partyIDSource = "LEI"; partyRole = Fix5:PartyRole:ClientID), (Fix5:Parties with partyID = "broker"; partyIDSource = "LEI"; partyRole = Fix5:PartyRole:ExecutingFirm)]; allocNetMoney = 14052000.0000000000; allocSettlCurrAmt = 140520.0000000000; allocSettlCurr = "GBP"]

'broker'

"<contract-id>"

[Fix5:Parties with partyID = "client"; partyIDSource = "LEI"; partyRole = Fix5:PartyRole:ClientID, Fix5:Parties with partyID = "broker"; partyIDSource = "LEI"; partyRole = Fix5:PartyRole:ExecutingFirm]

Fix5:AllocStatus:Accepted

Fix5:MatchStatus:Compared

[]

Main:ProposeBilateralAllocation

broker

client

id

status

initiator

instruction.allocID

instruction.allocTransType

instruction.allocType

instruction.refAllocID

instruction.allocCancReplaceReason

instruction.ordAllocGrp

instruction.side

instruction.instrument.symbol

instruction.instrument.securityID

instruction.instrument.securityIDSource

instruction.avgPx

instruction.quantity

instruction.tradeDate

instruction.settlDate

instruction.allocGrp

responder

X

X

#0:0

archived

'client'

""

Fix5:AllocTransType:New

Fix5:AllocType:Preliminary

none

none

[]

Fix5:Side:Buy

"BARC"

"GB0031348658"

"ISIN"

140.5200000000

100000

2019-09-06T

2019-09-10T

[Fix5:AllocGrp with allocAccount = "Account1"; allocPrice = 140.5200000000; allocQty = 100000; parties = [(Fix5:Parties with partyID = "client"; partyIDSource = "LEI"; partyRole = Fix5:PartyRole:ClientID), (Fix5:Parties with partyID = "broker"; partyIDSource = "LEI"; partyRole = Fix5:PartyRole:ExecutingFirm)]; allocNetMoney = 14052000.0000000000; allocSettlCurrAmt = 140520.0000000000; allocSettlCurr = "GBP"]

'broker'

The VSCode extension runs a local sandbox and executes your scenario, displaying the results. In this case we created an AllocationProposal and then Affirmed it with a choice.

This type of testing is handy when you are developing your DAML model, and don't want to have to spin up a ledger + clients to visualize the results, as described in the next section.

Interacting with the Ledger

Now we will start up the sandbox ledger ourselves and create some contracts. First, compile your model:

daml build

This creates a *.dar file under .daml/dist, which you must load into the ledger so that it knows about your contracts.

Next, we start the sandbox (in a new shell preferably):

> daml sandbox --port 6865 --ledgerid allocs --wall-clock-time .daml/dist/*.dar

	Sandbox verbosity changed to INFO
DAML LF Engine supports LF versions: 0, 0.dev, 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.dev; Transaction versions: 1, 2, 3, 4, 5, 6, 7, 8; Value versions: 1, 2, 3, 4, 5
Starting plainText server
listening on localhost:6865
   ____             ____
  / __/__ ____  ___/ / /  ___ __ __
 _\ \/ _ `/ _ \/ _  / _ \/ _ \\ \ /
/___/\_,_/_//_/\_,_/_.__/\___/_\_\

Initialized sandbox version 100.13.23 with ledger-id = allocs, port = 6865, dar file = List(.daml/dist/ui-0.0.1.dar), time mode = WallClock, ledger = in-memory, daml-engine = {}

The --ledgerid switch needs to be consistent with the JWT tokens we will use for authentication in our python code. --wall-clock-time is also required; otherwise contract creation will fail.

Now, we start the http REST API (in a new shell preferably):

> daml json-api --ledger-host localhost --ledger-port 6865 --http-port 7575

13:05:36.495 [main] INFO  com.digitalasset.http.Main$ - Config(ledgerHost=localhost, ledgerPort=6865, httpPort=7575, applicationId=HTTP-JSON-API-Gateway, maxInboundMessageSize=4194304)
13:05:37.841 [http-json-ledger-api-akka.actor.default-dispatcher-9] INFO  com.digitalasset.http.HttpService$ - Connected to Ledger: allocs
13:05:39.334 [http-json-ledger-api-akka.actor.default-dispatcher-9] INFO  com.digitalasset.http.HttpService$ - Started server: ServerBinding(/127.0.0.1:7575)

Finally, start your Jupyter notebook, which we will be using as a rudimentary UI:

jupyter notebook python/ui.ipynb

This should open up a web page on localhost:8888. You will also need to have the BeakerX extension installed; otherwise some of the controls will fail to render.

If this is the first time you are using Jupyter, you can get a quick tutorial through the Help > User Interface Tour.

Next, we will create some execution contracts on the ledger. Run all the cells up to and including the "Setup" cell with Shift + Enter (or going to the menu Cell > Run all above). This command within the cell sends the HTTP POST request to the process we started with daml json-api above:

    requests.post(
        "http://{}:{}/command/create".format(host,port),
        headers = brokerHeader,
        json = execution(round(99.0 + random(), 2), int(1000 * random()))
    )

As a side note, you will see that brokerHeader, defined at the top of the notebook, has a JWT token that is used to authenticate the party on the HTTP-JSON API. TODO: link here to the official docs once they're available.

Now, let's check the executions actually exist on the ledger. Run the next cell, "Client", which contain this call to fetch the contracts:

executionsResponse = requests.post(
    "http://{}:{}/contracts/search".format(host,port),
    json = { "%templates" : [{ "moduleName" : "Main", "entityName" : "Execution"}]},
    headers = clientHeader
)

This is similar to the previous call; it provides a URI for the REST endpoint, a token in the header, and some JSON to filter the results. In fact, we could have fetched all contracts from the shell using curl to send a plain HTTP POST request to this process:

curl  http://localhost:7575/contracts/search --header "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJsZWRnZXJJZCI6ImFsbG9jcyIsImFwcGxpY2F0aW9uSWQiOiJhbGxvY3MiLCJwYXJ0eSI6ImNsaWVudCJ9.ayhBmc7qfT1kjF_1AM7RTTQ4ZXsjM9q1sP-CaMYExPg"

To end this section, let's recap what we've done:

• We loaded up our allocations model into a sandbox ledger.
• We started a process that serves HTTP requests on port 7575.
• We POSTed to this process from our python worksheet, creating some Execution contracts as broker
• We POSTed to this process form our python worksheet, to read back the created contracts as client.

To conclude, we suggest you execute the remaining cells in the workbook, reading the comments that explain what is happening. You will create some AllocationProposals and Affirm them, as in the scenario from the previous section.

Automation

We're now going to look at python/readyToBookTrigger.py and run this as a stand-alone process to automate some workflows. You should start up the ledger and REST service as explained above, if you haven't already done so. First however, we need to digress briefly to explain why we can't do this in the UI notebook.

In the formal FIX specification, there is another workflow which we haven't covered, the "Ready to Book" process (FIX 5.0 Rev 2 Spec, Vol 5, p. 50):

Booking instructions can also be communicated Post-Trade (after fills have been received and processed) to signal that a particular order is now ready for booking or to signal that a set of orders for the same security, side, settlement date, etc., are to be aggregated as single booking unit which is now ready for booking.

This workflow sits apart from the allocation/affirmation in the UI notebook; in reality it should be done beforehand allocation takes place, but for demonstration purposes we've not modelled this so that the examples can be run independently.

What is interesting about this process, is that the workflow can't be modelled in DAML directly: we need to trigger an event as a result of a contract creation. To work around this, we can create a bot, as we've done in ../python/readyToBookTrigger.py, using the unofficial DAML python bindings.

In readyToBookTrigger.py you will find three callbacks:

• onInit: creates a global variable execs which we'll use to collect executions.

• onCreate: triggered every time a new Execution contract is created, and appends it to execs.

• aggregateEveryMinute: scheduled to run every 3 seconds, which creates a new AllocationProposal contract, but in the readyToBook state, with the collected executions in execs. It then resets the list.

In reality, the last step would run at the market close, say after 4:30 GMT for the London Stock Exchange, rather than every 3 seconds.

Let's try out this process now. First, install the DAZL client using pip if you haven't done so already:

pip3 install dazl

Next, start the trigger from the bot directory:

python3 readyToBookTrigger.py

You won't see any input until it picks up some Execution contracts. So let's generate some now, using the UI notebook, as we did in the previous section above. Run the "Setup" cell again, and keep an eye on your shell session. You should see something like this (numbers are random):

New executions found, aggregating as:
            quantity          avgPx                                        ordAllocGrp
tradeDate                                                                             
2019-09-10      3839  99.9000000000  [a1b173f4df8d11e98c6548a47202862c, a1ba969cdf8...

This means the process has found some executions, grouped them together into a single block of 3839 shares, and written a new AllocationProposal contract to the ledger. Let's check this is the case. Go back to your Jupyter notebook, and run the last cell. You should see a "Proposed Allocations" record with allocType = ReadyToBook, which was automatically created from the above line.