Inspired by what I read on Twitter and this LW post

Problem

Manifold has a lot of markets with low liquidity and low interest which results in a high variation of accuracy. This is made worse by independent multiple-choice, inefficient conditional markets, and long-term markets.

While Manifold has many super-accurate markets, it is easy to look at all the weak ones and write off the site as a whole. It is also hard for the average person, including journalists, to distinguish which markets they can wholly trust and which to take with a grain of salt.

Solution

Manifold creates a system which automatically assigns markets with a confidence in accuracy score. This lets users know which markets we put our reputation on as being highly accurate, and which should be taken with a grain of salt. While our experienced traders can already do this at a glance, this will make interpreting our data more accessible for the average person. It will also give them confidence to share and cite it.

Implementation

Logic

Create an algorithm that takes in account various factors to automatically determine a confidence level in a market. The following are some factors I look for when determining my confidence in a market:

• Liquidity
• Limit orders: Spread, volume, and # unique traders
• Number of traders who have recently bet on the market
  • Very few recent traders mean their opinion has a larger effect on the odds of the current market lowering its confidence.
  • But lots and lots of traders can often mean the market is still very volatile and there is a lot of disagreement so it needs time to settle which to a lesser extent lowers it’s confidence.
    • Maybe this needs its own status = Highly volatile (rather than sayer it is lower confidence).
• Time until resolution
• Total number of traders (this becomes unimportant after a market has ~15 traders compared to the things above).
• Trading volume? (I don’t look at this, but I think others do).

Frontend