(Copied from Joseph’s notion)

High Level Observations

This section isn’t necessarily good or bad, just things that I thought we interesting to call out and will be referencing in the next section. I’ve primarily focused on the betting flow, but I think most of this doc applies across the codebase.

  1. Business plans
    1. The business only uses play currency now
    2. The business wants to start real money payouts
  2. Problem domain
    1. The workload in this domain is Embarrassingly Parallelizable™
      1. https://en.wikipedia.org/wiki/Embarrassingly_parallel
    2. Market places lend themselves well to queues and batch processing
    3. There has been a colossal amount of money and thought dumped into this problem
  3. Coding practices
    1. Asynchronous processing is not being used
    2. There is no formally verifiable source of truth regarding balances
      1. More on this below…
    3. There does not appear to be scheduled downtime
  4. Database usage
    1. The team isn’t using an ORM, but has mostly built one from scratch
      1. i.e. updateData()
    2. Not many advanced features being used
      1. I have a feeling this stems from the initial stint in Firebase, which is NoSQL
    3. There are a lot of database functions and triggers
    4. There is very little usage of foreign keys
      1. Also think this stems from Firebase
    5. There doesn’t appear to be a real migration plan for database changes

Priorities

Given what’s written above, I think there are a few “classes” of concerns here.

  1. Correctness
    1. Is every deposit accounted for?
    2. Is every bet accounted for?
    3. Could someone ask for the state of an arbitrary market at an arbitrary point in the past?
    4. Could someone ask for the state of an arbitrary user at an arbitrary point in the past?
  2. Reliability
    1. If the user balances were wiped, could you recover them?
    2. Are there exercised disaster recovery procedures?
    3. What is the failure mode when load goes up?
  3. Performance
    1. How many concurrent users can be on the site?
    2. As load increases, will the markets always reflect reality?

Proposals

Migrate the data model towards an append-only ledger

From a correctness and reliability point of view, this is a worrying gap in the system architecture. While there are potential performance penalties for a naïve implementation, this will enable you to sleep at night when the site burns down during a crazy news day.

A centralized ledger allows you to have a single, rewindable source of truth regarding the entire financial state of your system. Given any user or any contract, you can just start at the beginning of the ledger and sum every relevant entry up to whatever date you’re interested in. There is literally a whole profession based around this (Accounting) as well as text books on their implementations. For an initial proposal, it’s hard to write more without getting into the weeds.

The primary tradeoff here is performance. As you may have surmised, having a centralized ledger effectively turns the transactions table into a linked list with a single head pointer. However you can…

[J] See also this user’s creation of a ledger with Manifold in mind:

https://github.com/syvb/yosoku-core/blob/master/README.md