Earned commission
Keep track precisely on how much you earned on a given timeframe
As a marketplace, you want to know precisely how much commission you're earning, either over a timeframe or at a specific point in time. As you grow and experiment with different commission models and the complexity grows, using a ledger as your single source of truth helps rationalize and build unified, real-time views of your earnings. You can connect Formance Ledger to Elasticsearch and Kibana to start getting insights into your commissions quickly.
Using this pattern
Choosing a Ledger Account Architecture
First, decide on a convention for your account architecture. Most commonly, you'll create a dedicated commission account for accruing commissions separately from other revenue streams. For example, here is a transaction in Numscript that records paying out a sale on order ID 1234 to seller ID 6789, along with a 15% commission on the sale:
send [USD/2 100] (
source = @orders:1234
destination = {
15% to @platform:commission
remaining to @users:6789
}
)By using naming conventions like this, we make it easy for Elasticsearch to index transactions based on what they represent, and not just to whom they are going. Then we can use Kibana to build a dashboard that gives real-time insight into your commissions by showing all money movements in and out of the dedicated commissions account.
Connecting Formance Ledger to Elasticsearch
Formance ships with native Kafka and HTTP publishing integrations, allowing you to sync Formance events with external systems. We'll use a handy tool called Benthos to create a data chain that moves our ledger transactions to Elasticsearch. Benthos facilitates connecting to Elasticsearch by receiving Ledger events over HTTP, and publishing those directly to Elasticsearch:
Ledger Webhook > Benthos > Elasticsearch Prerequisites
An Elasticsearch and Kibana deployment
An up-to-date Benthos binary
Setup
We'll first need to start our Formance Ledger with HTTP publishing enabled. You can enable this with the publisher-http-enabled and publisher-topic-mapping options, here passed as flag:
numary server start \
--publisher-http-enabled \
--publisher-topic-mapping='COMMITTED_TRANSACTIONS:http://localhost:3042/post'Note that we'll post on localhost:3042, which is the endpoint we'll configure Benthos to bind to with the following config:
input:
label: "ledger_hook"
http_server:
address: "localhost:3042"
path: /post
allowed_verbs:
- POST
timeout: 5s
pipeline:
processors:
- bloblang: |
this.payload.transactions.map_each(tx -> tx.postings.map_each(p -> tx.without("postings").merge(p).merge({
"ledger": this.ledger
})))
- unarchive:
format: json_array
- unarchive:
format: json_array
output:
label: ""
elasticsearch:
urls: ['http://localhost:9200']
index: "demo-ledger-dashboards-1"
id: ${!count("elastic_ids")}-${!timestamp_unix()}
max_in_flight: 64The configuration above creates a simple pipeline that deconstructs the ledger payload and creates an object to be indexed in Elastic for each transaction posting — the rationale for this mapping is simply to create something that will be easier to query and visualize with Kibana. You can save this config as config.yaml and start a Benthos process with it:
benthos -c config.yamlWe're all set! Let's commit a basic ledger transaction to verify. This transaction creates a new ledger called demo-ledger-dashboards-1, and moves some money from the outside world into a central bank account, and then from the central bank to a user's account:
curl -X POST \
-H 'Content-Type: application/json' \
-d '{
"postings": [
{
"source": "world",
"destination": "central_bank",
"asset": "GEM",
"amount": 100
},
{
"source": "central_bank",
"destination": "users:001",
"asset": "GEM",
"amount": 100
}
],
"metadata": {
"description": "Wohoo! a first transaction."
}
}' http://localhost:3068/demo-ledger-dashboards-1/transactionsYou can now head to the Kibana index management page, and should now see the demo-ledger-dashboards-1 index available for use.
Monitoring Commission Cash Flow in Kibana
Once our transactions data is in Elastic, we have everything we need at hand to use Kibana to create powerful real-time visualization of cash flow to the dedicated commissions account.
To start, create a new index pattern. Call it demo-ledger-dashboards-1, and use the field labeled timestamp for the timestamps (what else?).
Find your new index in the dashboard. By default, you'll see all the transactions; the histogram shows the volume of transactions over time. This isn't what we want, but we can see everything is hooked up.
Now, let's create the visualization. In the main menu bar, under "Analytics", choose "Visualize Library" to start creating a new visualization.
We're going to use the basic "Lens" visualization, so pick that.
Now, we're at the visualization editor. The key field for our purposes is amount, which is the amount each transaction moves in our ledger. Drag amount into the middle of the visualization builder.
So far so good, but this visualization is showing us the cash flowing through the entire ledger, not just our commissions account. And it's showing us by default the median amount of the transfers each time period. Let's chance that to sum, so we can see totals instead of measures of central tendency. Click on the label under "Vertical axis" on the right-hand panel, and choose "Sum".
Finally, let's make sure we're only looking at commissions being paid. At the top left, click "Add Filter". Select destination.keyword as the field, is as the operator, and platform:commission as the value.
And now, finally, we have what we wanted: A dashboard showing the cashflow into the commissions account, second-by-second. We can save this visualization so we can view and share with the rest of the team!
