Glossary
The words, in plain language
Every term here is used somewhere on this site and is load-bearing, meaning a sentence containing it does not work if you do not know what it means. Definitions explain the idea rather than the implementation, and none of them is a claim about performance.
- Training gym
- One search program. A gym takes a single bounded question, proposes candidate answers to it, tests them against a standard fixed in advance, and discards the ones that fail. Several run at once, independently. The name is literal: it is where candidates do repetitions until they either hold up or break.
- Referee
- The part of the system that decides whether a candidate passes. It is kept separate from the part that generates candidates, so the thing proposing an answer is never the thing grading it. This is why it has a name of its own.
- Quality-diversity archive, or QD archive
- A store that keeps the best candidate found for each distinct kind of behaviour, rather than just the single best candidate overall. Searching for one winner tends to find something that suits the exact conditions it was discovered in. Searching for a map tells you what works where, and where nothing has been found at all.
- Niche
- One cell in that archive: a specific combination of behaviours, such as a particular reach, regime, direction and cost. A niche is occupied when some candidate currently holds it, and empty when nothing has taken it. Empty means nothing has been found there yet, not that something was tried and failed.
- Generation
- One round of proposing, testing and culling within a gym. Counted separately for each gym, so two gyms at the same generation number have not done the same amount of work.
- Rejection
- A candidate that was tested and did not pass. Rejections are kept rather than deleted, which is the entire point of the null registry below.
- Null registry
- The record of what failed, and why. Most research discards this. Keeping it does two things: the search can avoid re-testing dead ends, and anyone reading a good-looking result can see how many attempts it was drawn from. Without that count, a result cannot be judged.
- Deflated Sharpe ratio
- A Sharpe ratio corrected for how many things were tried. Test four thousand candidates and the best of them will look good by luck alone, so the deflated version asks whether it still looks good beyond what luck would have produced. A raw Sharpe can be high and its deflated Sharpe zero at the same time. Those two numbers do not contradict each other.
- Trials factor, or the look-elsewhere effect
- The reason the correction above is needed. The more places you look, the more likely you are to find something that looks like a signal and is not. This is a property of searching, not a flaw in any particular result.
- Regime
- A stretch of time over which a market behaves in a broadly consistent way. Something that works in one regime and not in others has not been shown to work; it has been shown to fit.
- Holdout
- Data set aside before testing starts and not looked at until the end. Opening it uses it up, because once a result has been seen it can no longer be an independent test of anything. This is why a holdout is only ever opened once.
- Forward observation
- Watching a candidate against data that arrived after its definition was frozen. It is the one test that cannot be adjusted after the fact, which is what makes it worth more than any amount of testing against history.
- Embargo
- A period during which a frozen candidate is watched but neither changed nor acted on. The point is to remove the temptation to intervene while the evidence accumulates.
- Basin
- A region of the search space where many nearby candidates behave similarly. A result sitting in a wide basin is less likely to be an accident than one sitting on a narrow spike, because small changes to it do not destroy it.
- Spanning
- How much of the possible behaviour space the search has actually reached. Low spanning means the map has large unexplored areas, which is a statement about the search rather than about the market.
- Overlap-safe t
- A t-statistic adjusted for the fact that overlapping observation windows are not independent of each other. Without the adjustment, overlapping samples make a result look more statistically significant than it is.
Words from the raw process log
Some exhibits show process events exactly as the research system wrote them, which means
tokens like brood_pairs, learned_deferred_pairs and
sidecar can appear. Those are internal operator output, not vocabulary
anyone is expected to learn, and nothing on this site depends on understanding them. They
are shown unedited so the record cannot be accused of having been tidied, and they are
not part of this glossary because defining them would imply they were meant for you.
A separate vocabulary on the research registry defines the twelve evidence states a finding can hold, which is a different question: not what a word means, but how far a claim has got.