humanblnk tutorial

Chapter 11 · Running it

Reconciliation

Scenario: Paystack's daily settlement report lists three deposits. You recorded two of them. Reconciliation finds the one that's missing.

Step 1: say what counts as a match

A rule is a list of criteria that must all hold. Make it once and reuse it. Matching on reference works because a deposit's reference was built from Paystack's own id (chapter 4), and Paystack's report carries that id.

11-reconciliation/main.go
rule, err := hb.Reconciliation.AddRule(ctx, humanblnk.MatchRule{
    Name: "paystack: amount and reference " + demo.Run,
    Criteria: []humanblnk.Criterion{
        {Field: humanblnk.MatchAmount, Op: humanblnk.MatchEquals},
        {Field: humanblnk.MatchReference, Op: humanblnk.MatchEquals},
    },
})
if err != nil {
    log.Fatal(err)
}
Criterion
FieldMatchAmount, MatchDate, MatchReference, MatchDescription, MatchCurrency
OpMatchEquals, MatchGreaterThan, MatchLessThan, MatchContains (with Pattern)
Drifthow far apart they may be: a fraction of the amount for MatchAmount (0.01 is 1%), seconds for MatchDate, and for MatchContains a percentage of the longer text in edits (10 is 10%)

Step 2: run it on the report

Up to 10,000 lines can be sent directly with RunLines:

11-reconciliation/main.go
report := []humanblnk.StatementLine{
    {ID: "stl_1_" + demo.Run, Amount: money.MustParse("10000", demo.NGN), Reference: refs[0], Date: time.Now(), Source: "paystack"},
    {ID: "stl_2_" + demo.Run, Amount: money.MustParse("2500", demo.NGN), Reference: refs[1], Date: time.Now(), Source: "paystack"},
    {ID: "stl_3_" + demo.Run, Amount: money.MustParse("7000", demo.NGN), Reference: "deposit:paystack:ps_missing_" + demo.Run, Date: time.Now(), Source: "paystack"},
}
run, err := hb.Reconciliation.RunLines(ctx, report, humanblnk.Run{Rules: []humanblnk.RuleID{rule.ID}})
if err != nil {
    log.Fatal(err)
}

A line's ID is the provider's id, and it must be unique across every statement: Blnk refuses a line whose id it has stored before.

Run options: Strategy (OneToOne by default, OneToMany or ManyToOne, with GroupBy to group, such as "reference") and DryRun (match without recording the results).

Step 3: wait for the result

It runs in the background, so poll:

11-reconciliation/main.go
var res humanblnk.RunResult
for range 50 {
    if res, err = hb.Reconciliation.Result(ctx, run); err == nil && (res.Status == "completed" || res.Status == "failed") {
        break
    }
    time.Sleep(200 * time.Millisecond)
}
fmt.Printf("run %s: %s, %d matched, %d unmatched\n", res.ID, res.Status, res.Matched, res.Unmatched)

A report as a file

Bigger reports are uploaded: CSV or JSON with the columns id, amount, currency, reference, description and date (RFC 3339), with amounts in major units (10000, not kobo). Then run it with Reconciliation.Run(ctx, statement, humanblnk.Run{…}).

11-reconciliation/main.go
csv := "id,amount,currency,reference,description,date\n" +
    "stl_file_1_" + demo.Run + ",10000,NGN," + refs[0] + ",Bank transfer," + time.Now().UTC().Format(time.RFC3339) + "\n"
statement, lines, err := hb.Reconciliation.Upload(ctx, "paystack", "settlement.csv", strings.NewReader(csv))
if err != nil {
    log.Fatal(err)
}
fmt.Println("uploaded", statement, "with", lines, "line(s); run it with Reconciliation.Run")

The whole example

go run ./docs/examples/11-reconciliation
11-reconciliation/main.go
// Chapter 11: check a provider's settlement report against the books.
package main

import (
    "fmt"
    "log"
    "strings"
    "time"

    "github.com/otyang/humanblnk"
    "github.com/otyang/humanblnk/docs/examples/demo"
    "github.com/otyang/humanblnk/money"
)

func main() {
    ctx, hb := demo.Setup()
    ada := demo.Customer(ctx, hb, "ada", demo.NGN)

    // Two deposits you recorded from Paystack webhooks.
    refs := []string{humanblnk.Reference("deposit", "paystack", "ps_a_"+demo.Run), humanblnk.Reference("deposit", "paystack", "ps_b_"+demo.Run)}
    for i, amount := range []string{"10000", "2500"} {
        m := money.MustParse(amount, demo.NGN)
        if _, err := hb.Transactions.Record(ctx, humanblnk.Transaction{
            Ref: refs[i], From: humanblnk.Single(humanblnk.ClearingAccount("paystack"), m), To: humanblnk.Single(humanblnk.AccountRef(ada.ID), m),
            Description: "Bank transfer",
        }); err != nil {
            log.Fatal(err)
        }
    }

    // 1. What counts as a match. Make the rule once and reuse it.
    rule, err := hb.Reconciliation.AddRule(ctx, humanblnk.MatchRule{
        Name: "paystack: amount and reference " + demo.Run,
        Criteria: []humanblnk.Criterion{
            {Field: humanblnk.MatchAmount, Op: humanblnk.MatchEquals},
            {Field: humanblnk.MatchReference, Op: humanblnk.MatchEquals},
        },
    })
    if err != nil {
        log.Fatal(err)
    }

    // 2. Paystack's report: the two deposits, and one you never recorded.
    // Line ids are the provider's, and must be unique across every statement.
    report := []humanblnk.StatementLine{
        {ID: "stl_1_" + demo.Run, Amount: money.MustParse("10000", demo.NGN), Reference: refs[0], Date: time.Now(), Source: "paystack"},
        {ID: "stl_2_" + demo.Run, Amount: money.MustParse("2500", demo.NGN), Reference: refs[1], Date: time.Now(), Source: "paystack"},
        {ID: "stl_3_" + demo.Run, Amount: money.MustParse("7000", demo.NGN), Reference: "deposit:paystack:ps_missing_" + demo.Run, Date: time.Now(), Source: "paystack"},
    }
    run, err := hb.Reconciliation.RunLines(ctx, report, humanblnk.Run{Rules: []humanblnk.RuleID{rule.ID}})
    if err != nil {
        log.Fatal(err)
    }

    // 3. It runs in the background: poll for the result.
    var res humanblnk.RunResult
    for range 50 {
        if res, err = hb.Reconciliation.Result(ctx, run); err == nil && (res.Status == "completed" || res.Status == "failed") {
            break
        }
        time.Sleep(200 * time.Millisecond)
    }
    fmt.Printf("run %s: %s, %d matched, %d unmatched\n", res.ID, res.Status, res.Matched, res.Unmatched)

    // A bigger report comes as a file: CSV or JSON with the columns id,
    // amount, currency, reference, description and date, in major units.
    csv := "id,amount,currency,reference,description,date\n" +
        "stl_file_1_" + demo.Run + ",10000,NGN," + refs[0] + ",Bank transfer," + time.Now().UTC().Format(time.RFC3339) + "\n"
    statement, lines, err := hb.Reconciliation.Upload(ctx, "paystack", "settlement.csv", strings.NewReader(csv))
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println("uploaded", statement, "with", lines, "line(s); run it with Reconciliation.Run")

    if err := hb.Reconciliation.RemoveRule(ctx, rule.ID); err != nil {
        log.Fatal(err)
    }
}

What it doesn't do (yet)

  • It doesn't list which lines didn't match. Blnk 0.15.6 reports counts only. To find the missing ones, check each line yourself with Transactions.GetByReference.
  • It doesn't fetch reports. Something has to download them and call Upload or RunLines, on a schedule you choose.
  • It doesn't check balances. To confirm the clearing account matches what Paystack owes, read it with Accounts.Get and compare it with the provider's total. Get takes a balance id, not @PaystackClearing: take it from any deposit entry's From posting, which names the clearing account by its balance id.