You pull today’s index constituents, fetch history for each one, and backtest a simple rule. The equity curve looks good. You check the arithmetic, reseal the snapshot, confirm the hashes. Everything reproduces.
The arithmetic is right. The experiment still answers the wrong historical question, and nothing in the output says so.
The file you downloaded lists the companies that are in the index today. The ones that were dropped along the way never appear in it. Your strategy was never offered the chance to lose money on them. You didn’t choose to exclude them. The data did, using information from the end of your sample.
You need this article whenever your universe comes from a list that changes over time, such as index constituents. A fixed universe chosen only from information available before the backtest begins avoids this particular retrospective selection problem. A fixed list chosen because its members survived the whole window does not: that is the same selection, made by hand.
Survivorship bias
Selection on what remained observable. When a sample is truncated by survival, the surviving history can show returns the strategy could not have earned. Brown, Goetzmann, Ibbotson, and Ross (1992) measured the effect on performance studies. Shumway (1997) documented the related delisting bias that appears when large negative delisting returns are missing from equity data.
By the end of this article you will be able to declare a universe-aware experiment, explain the assumptions it rests on, and say why its positions, fills, and reported horizon differ from what you asked for.
Here is where the example ends up, so you know what the machinery is for:
| Experiment | Total return | What it rests on |
|---|---|---|
Survivor list (BBB only) |
+14.3% | the company that is still listed today |
Point-in-time (AAA and BBB) |
-14.1% |
AAA carried at its last observed close after it stopped trading |
Point-in-time, AAA worth nothing |
-33.9% | the same run with that holding valued at zero |
The rest of the article builds those three numbers and shows where each assumption is recorded.
Two Companies, Ten Sessions
The example is synthetic and small enough to read in full. DEMO is an illustrative venue, not a real exchange, and none of these rows are redistributed vendor data.
Two instruments are listed on January 6. AAA loses more than half its value over five sessions and stops trading. BBB grinds upward and is still listed at the end of the window.
session_dates <- seq(as.Date("2020-01-06"), as.Date("2020-01-17"), by = "day")
open_day <- as.integer(format(session_dates, "%u")) <= 5L
open_dates <- session_dates[open_day]
prices <- tibble(
date = open_dates,
AAA = c(100, 92, 78, NA, 44, NA, NA, NA, NA, NA),
BBB = c(50, 51, 52, NA, 54, 55, 56, 57, 58, 59)
)
prices
#> # A tibble: 10 x 3
#> date AAA BBB
#> <date> <dbl> <dbl>
#> 1 2020-01-06 100 50
#> 2 2020-01-07 92 51
#> 3 2020-01-08 78 52
#> 4 2020-01-09 NA NA
#> 5 2020-01-10 44 54
#> 6 2020-01-13 NA 55
#> 7 2020-01-14 NA 56
#> 8 2020-01-15 NA 57
#> 9 2020-01-16 NA 58
#> 10 2020-01-17 NA 59
Two holes matter later, and they are not the same kind of hole. The entire feed is absent on January 9 even though the venue was open. Both instruments resume on January 10, but AAA stops printing prices after that session while BBB continues.
A missing price is not a reason
Nothing in this table says why an observation is absent. A delisting, a trading halt, and a vendor feed gap all look identical here. This example declares no lifetime or trading-status facts, so ledgr is never told which one occurred. The prose below supplies that story; the data does not.
The Selection Problem
Before any ledgr call, do the arithmetic by hand.
A researcher who starts from today’s index list sees only BBB, because AAA is no longer a constituent. A researcher who asks what the index actually held on January 6 sees both.
first_price <- function(x) x[is.finite(x)][1]
last_price <- function(x) x[is.finite(x)][sum(is.finite(x))]
simple_return <- function(x) last_price(x) / first_price(x) - 1
returns <- c(AAA = simple_return(prices$AAA), BBB = simple_return(prices$BBB))
round(returns, 3)
#> AAA BBB
#> -0.56 0.18
tibble(
view = c("Survivor list (BBB only)", "Point-in-time (AAA and BBB)"),
equal_weight_return = c(returns[["BBB"]], mean(returns))
)
#> # A tibble: 2 x 2
#> view equal_weight_return
#> <chr> <dbl>
#> 1 Survivor list (BBB only) 0.18
#> 2 Point-in-time (AAA and BBB) -0.19One view reports a gain, the other a loss, from the same file. That is the shape of the problem. It is an illustrative valuation, not a portfolio return anyone could have executed over one horizon: it values AAA at its January 10 close and BBB at its January 17 close.
It is only a sketch, and one assumption is doing quiet work: it treats AAA’s last observed price as its ending value. No settlement was observed. Nobody told us what a holder actually received. Hold onto that, because ledgr will make the same assumption only when you can see it, and will name it in the result.
The Objects You Are About To Build
ledgr splits this problem across a few objects. Each has one job, and they fall into three groups: what the world supplied, what you chose, and what the run recorded.
| Object | Its job in this example |
|---|---|
| Bars | The prices actually observed for AAA and BBB. |
| Facts | Declared expected sessions and dated membership, including when each assertion became knowable. |
| Snapshot | A sealed version of those inputs, so a run can be reproduced and reopened. |
| Strategy | The positions you want at each decision. |
| Experiment | The snapshot and strategy combined with a universe rule, a valuation policy, a terminal-outcome policy, and cost rules. |
| Run | The execution of that configuration, plus the evidence and explanations it retained. |
Bars and facts are source evidence. The universe rule, the valuation and terminal-outcome policies, and the strategy are your choices. Everything a run reports afterwards is a recorded outcome of those two. Most confusion about point-in-time backtesting comes from mixing the three, which a single wide price table encourages you to do.
Declaring The Facts
Everything below is something you assert about the world. The mechanical reshaping is folded away; the claims are not.
Turning the wide price table into bars (mechanical)
bars_from <- function(prices) {
bind_rows(
tibble(instrument_id = "AAA", date = prices$date, close = prices$AAA),
tibble(instrument_id = "BBB", date = prices$date, close = prices$BBB)
) |>
filter(is.finite(close)) |>
transmute(
instrument_id,
ts_utc = date,
open = close,
high = close + 1,
low = close - 1,
close,
volume = 1000
)
}Which sessions were expected?
A session is one trading day at one venue, declared from the venue’s calendar rather than inferred from the rows you received; vignette("missing-data-and-sessions", package = "ledgr") teaches the calendar in full. Here it is an ordinary table that includes the closed days:
calendar <- tibble(
session_date = session_dates,
status = if_else(open_day, "open", "closed"),
session_open = if_else(open_day, "14:30:00", NA_character_),
session_close = if_else(open_day, "21:00:00", NA_character_)
)
calendar
#> # A tibble: 12 x 4
#> session_date status session_open session_close
#> <date> <chr> <chr> <chr>
#> 1 2020-01-06 open 14:30:00 21:00:00
#> 2 2020-01-07 open 14:30:00 21:00:00
#> 3 2020-01-08 open 14:30:00 21:00:00
#> 4 2020-01-09 open 14:30:00 21:00:00
#> 5 2020-01-10 open 14:30:00 21:00:00
#> 6 2020-01-11 closed <NA> <NA>
#> 7 2020-01-12 closed <NA> <NA>
#> 8 2020-01-13 open 14:30:00 21:00:00
#> 9 2020-01-14 open 14:30:00 21:00:00
#> 10 2020-01-15 open 14:30:00 21:00:00
#> 11 2020-01-16 open 14:30:00 21:00:00
#> 12 2020-01-17 open 14:30:00 21:00:00
session_facts <- ledgr_facts_sessions(
calendar |> mutate(knowledge_time = ledgr_utc("2020-01-01")),
venue_id = "DEMO",
timezone = "UTC"
)venue_id names the venue these sessions belong to. knowledge_time says the calendar was published in advance, on January 1, so it was already usable for decisions inside the period it covers. Real calendars can also change inside that period, which is why knowledge time is recorded per assertion rather than inferred from the dates a calendar spans.
January 9 is now a real session with no observations, rather than a date that never happened. The printed prices table has an explicit January 9 row with missing observations, while the printed calendar labels that date open. Saturday January 11 has no observation row and the calendar labels it closed. January 9 is therefore an open-session feed outage, so it stays on the pulse and valuation clocks. January 11 is not a pulse and does not age a stale mark.
bars_input <- bars_from(prices)Which companies belonged, and when was that knowable?
Start from what a provider actually hands you: a constituent list, as of a date. Here there are two. On January 6 the universe was both companies; from January 13, after AAA was removed, it was BBB alone.
membership_lists <- tibble(
effective_from = ledgr_utc(c("2020-01-06", "2020-01-13")),
knowledge_time = ledgr_utc(c("2020-01-01", "2020-01-13")),
members = list(c("AAA", "BBB"), "BBB"),
source = "synthetic_membership"
)
membership_lists
#> # A tibble: 2 x 4
#> effective_from knowledge_time members source
#> <dttm> <dttm> <list> <chr>
#> 1 2020-01-06 00:00:00 2020-01-01 00:00:00 <chr [2]> synthetic_membership
#> 2 2020-01-13 00:00:00 2020-01-13 00:00:00 <chr [1]> synthetic_membershipEach list also needs a second date: when that list became knowable. The January 6 list was published with the calendar on January 1. The removal was announced on January 13, the day it took effect.
The members list-column keeps each provider list intact. The constructor normalizes it to the member rows and complete-set evidence that ledgr hashes and resolves; users do not need to expand the list by hand.
The two dates answer two different questions:
-
effective_from: when does this membership apply? -
knowledge_time: when could a strategy first have known it?
Those clocks are usually different in real data. An index change is decided, announced, and takes effect on three separate days. A downloaded constituent list carries neither timestamp, which is why it leaks.
Which timestamp goes in knowledge_time?
Record when the assertion became available to the research process you are modelling:
- The provider timestamps the announcement: use that.
- The file has a publication or delivery date: use that.
- You only know when you downloaded it: use the download time. It establishes receipt, not historical publication, so disclose it as the assumption it is.
The two timestamps may legitimately coincide when the evidence supports it, as the January 13 removal does here: it was announced the day it took effect. What you cannot do is copy effective_from as a default, because that asserts every change was knowable the instant it took effect. That is a different leak from survivorship: it makes information available too early, while survivorship removes opportunities the strategy really had. The two often travel together, but neither requires the other.
membership_facts <- ledgr_facts_membership_snapshots(
membership_lists,
universe_id = "demo_members",
complete = TRUE
)complete = TRUE is the load-bearing argument. It says each effective_from group is the entire membership set on that date, not a partial report of some members. That is why the January 13 row removes AAA: AAA is absent from a set that claims to be complete. Without complete = TRUE these rows would only add members and AAA would never leave.
instruments <- tibble(instrument_id = c("AAA", "BBB"))
facts <- ledgr_facts(session_facts, membership_facts)
ledgr_facts_validate(facts, bars_input, instruments)
#> ledgr facts validation
#> Can seal: yes
#> # A tibble: 7 x 2
#> category n
#> <chr> <int>
#> 1 facts_accepted 15
#> 2 facts_runtime_conflict 0
#> 3 facts_audit_only 0
#> 4 facts_rejected 0
#> 5 observations_accepted 13
#> 6 observations_quarantine_candidate 0
#> 7 observations_rejected 0Can seal: yes means this bundle and these observations pass the structural checks. It does not certify that your vendor supplied every relevant fact.
What the knowledge clock actually changes
Start with the question a decision actually asks: who was a member at this cutoff, and why?
ledgr_facts_resolve(
membership_facts,
family = "membership",
scope_id = "demo_members",
at = "2020-01-13T21:00:00Z",
instruments = c("AAA", "BBB")
)
#> ledgr facts resolution
#> Family: membership [demo_members]
#> At: 2020-01-13T21:00:00Z
#> Rows: 2
#> # A tibble: 2 x 3
#> instrument_id member reason
#> <chr> <lgl> <chr>
#> 1 AAA FALSE omitted_from_complete_set
#> 2 BBB TRUE member_asserted
#> # i Omitted stored columns: evidence_ids.
#> # i Full rows remain in $rows; supporting evidence remains in $evidence.AAA is not a member at that cutoff, and reason records why: the applicable list is a complete set and AAA is absent from it. BBB is a member on an explicit assertion. This is the resolution ledgr performs for the run’s pulse, and it reports only what was knowable at the cutoff you asked about. A strategy can consume the resolved result when that pulse is a decision.
History is the audit view behind that answer. It is retrospective: it shows all assertions, including ones that were not usable at an earlier decision cutoff.
ledgr_facts_history(
membership_facts,
family = "membership",
scope_id = "demo_members"
)
#> ledgr facts history
#> Family: membership [demo_members]
#> Rows: 5
#> # A tibble: 5 x 6
#> evidence_type instrument_id member effective_from knowledge_time complete
#> <chr> <chr> <lgl> <dttm> <dttm> <lgl>
#> 1 membership_assertion AAA TRUE 2020-01-06 00:00:00 2020-01-01 00:00:00 NA
#> 2 membership_assertion BBB TRUE 2020-01-06 00:00:00 2020-01-01 00:00:00 NA
#> 3 set_header <NA> NA 2020-01-06 00:00:00 2020-01-01 00:00:00 TRUE
#> 4 membership_assertion BBB TRUE 2020-01-13 00:00:00 2020-01-13 00:00:00 NA
#> 5 set_header <NA> NA 2020-01-13 00:00:00 2020-01-13 00:00:00 TRUE
#> # i Omitted stored columns: evidence_id, set_id, effective_to, source.
#> # i Full rows remain in $rows; supporting evidence remains in $evidence.The next block is a what-if for this article only, not a step in an ordinary workflow. It rewrites the example vendor’s knowledge_time to show what that clock changes. Its setup code is folded, because real research resolves facts as supplied rather than rewriting vendor timestamps.
Show the what-if setup
late_membership <- ledgr_facts_membership_snapshots(
membership_lists |>
mutate(
knowledge_time = if_else(
effective_from == ledgr_utc("2020-01-13"),
ledgr_utc("2020-01-14"),
knowledge_time
)
),
universe_id = "demo_members",
complete = TRUE
)
early_membership <- ledgr_facts_membership_snapshots(
membership_lists |>
mutate(
knowledge_time = if_else(
effective_from == ledgr_utc("2020-01-13"),
ledgr_utc("2020-01-10"),
knowledge_time
)
),
universe_id = "demo_members",
complete = TRUE
)
resolve_aaa <- function(member_facts, scenario, at) {
ledgr_facts_resolve(
member_facts,
family = "membership",
scope_id = "demo_members",
at = at,
instruments = "AAA"
)$rows |>
transmute(scenario, cutoff = at, member, reason)
}
bind_rows(
resolve_aaa(
early_membership, "known early, not effective", "2020-01-10T21:00:00Z"
),
resolve_aaa(
membership_facts, "effective and knowable", "2020-01-13T21:00:00Z"
),
resolve_aaa(
late_membership, "effective, not knowable", "2020-01-13T21:00:00Z"
),
resolve_aaa(
late_membership, "effective and now knowable", "2020-01-14T21:00:00Z"
)
)
#> # A tibble: 4 x 4
#> scenario cutoff member reason
#> <chr> <chr> <lgl> <chr>
#> 1 known early, not effective 2020-01-10T21:00:00Z TRUE member_asserted
#> 2 effective and knowable 2020-01-13T21:00:00Z FALSE omitted_from_complete_set
#> 3 effective, not knowable 2020-01-13T21:00:00Z TRUE member_asserted
#> 4 effective and now knowable 2020-01-14T21:00:00Z FALSE omitted_from_complete_setThe first row proves that knowing about a future replacement does not activate it early. The middle rows hold the effective date fixed and move only the knowledge time: AAA remains a member until the removal is both effective and knowable. No strategy, opening position, or run is needed to inspect that decision-time answer.
What A Strategy Returns
Before running anything, be precise about the contract, because the execution behaviour later depends on it.
-
ctx$hold()starts from the positions you currently hold. - The numbers you return are desired position quantities, not orders.
- A target is evaluated against execution rules. It does not guarantee a fill.
- An unfilled target does not persist. It is not a standing order waiting for liquidity.
The strategy below is deliberately plain. Its portfolio rule has no calendar branch. The experiment attaches an explicit two-date schedule: January 6 and the membership change on January 13. The second decision makes the point-in-time availability change part of the executed strategy path rather than merely something inspected after the run.
What happens to a departed holding depends on how the rule is written. ctx$flat() is literal, so a rule starting from it sells a departed holding at the next open. The strategy helpers keep departed holdings, as the rule below does. Increasing a holding outside the universe fails with an error. What ledgr should do by default here is open design work.
rebalance_declared_members <- function(ctx, params) {
ctx |>
ledgr_selection() |>
ledgr_weight_equal() |>
ledgr_target_rebalance(ctx, equity_fraction = params$invested)
}
rebalance_schedule <- ledgr_schedule_dates(
as.Date(c("2020-01-06", "2020-01-13"))
)
scheduled_rebalance <- ledgr_strategy_schedule(
rebalance_declared_members,
rebalance_schedule
)Only those two sessions call the portfolio rule. On the other eight sessions, ledgr records a scheduled skip while it continues to process events, value the holdings, and age stale marks. The scheduling section of vignette("strategy-authoring-tools", package = "ledgr") develops that distinction from a smaller example.
In this availability-aware experiment, equity_fraction is deliberately below 1. Sizing uses the decision close while execution uses the next observed open; the later price and the availability rules together determine whether a target can fill. Leaving headroom is a research choice, and the exercise at the end of this article shows what happens without it. Dense runs share next-open timing, but their refusal reasons need not be the same as this availability-aware case.
The path is explicit. ledgr_universe_members("demo_members") selects the named history. At each decision, its cutoff resolution becomes ctx$members. ctx$universe is the members-plus-holdings axis, so a held former member stays visible. ledgr_selection() without a condition selects every current member that can trade, so a held former member is never selected. The rebalance helper creates one full-axis target; execution and valuation then decide independently what can fill and what can be marked.
Comparing Survivor and Point-in-Time Universes
Now the comparison the opening promised, run properly. Same snapshot, same strategy, same valuation policy, same starting cash. The only difference is which universe is declared.
Seal the evidence first. After this point the inputs are frozen and identified by a hash. The store comes from ledgr_temp_store() because the article closes and reopens it later; a real project passes a persistent path, so the evidence outlives the R session. The snapshot declares price_basis = "split_adjusted": ledgr counts shares in split-adjusted units, and a point-in-time experiment on a snapshot that does not say what its bars are runs on that assumption and labels it as a default.
store_path <- ledgr_temp_store(
file.path(tempdir(), "ledgr_survivorship_bias.duckdb")
)
snapshot <- ledgr_snapshot_from_df(
bars_input,
instruments_df = instruments,
facts = facts,
price_basis = "split_adjusted",
db_path = store_path,
snapshot_id = "survivorship-demo"
)An experiment is the sealed snapshot plus every choice you are making about it. Here is the whole thing, written out once:
point_in_time_experiment <- ledgr_experiment(
snapshot,
scheduled_rebalance,
universe = ledgr_universe_members("demo_members"),
valuation_policy = ledgr_valuation_stale(max_sessions = 2),
cost_model = ledgr_cost_zero(),
opening = ledgr_opening(cash = 10000)
)Read those arguments as four separate decisions. universe selects the membership history you declared, rather than a fixed list. valuation_policy says how many sessions an old close may still value a holding, and what happens after that. By default the holding is frozen: it keeps its last close, the run continues, and the result says so. cost_model and opening fix the trading frictions and the starting account. Only the first of them differs between the two runs below, so the comparison isolates the universe.
ledgr_experiment_plan(point_in_time_experiment)
#> ledgr experiment plan
#> Availability: active
#> Price basis: split_adjusted (declared)
#> Metrics: US equity daily (default) (252 days/year * 1 bars/day = 252 bars/year)
#> Strategy schedule: explicit dates (2)
#> Universe: membership
#> Valuation: stale-mark horizon 2 sessions (declared)
#> History: price carry 5 sessions (default); output carry off (default); segment limit 4 per fact, at least 16 (default)
#> Checks:
#> # A tibble: 4 x 3
#> family status assumption_reasons
#> <chr> <chr> <chr>
#> 1 membership declared ""
#> 2 sessions declared ""
#> 3 trading_status omitted ""
#> 4 lifetime omitted ""
#> Declared assumptions:
#> valuation.max_sessions 2
#> valuation.exhausted freeze (default)
#> history.price_carry 5 sessions (default)
#> history.output_carry off (default)
#> history.segment_limit 4 per fact, at least 16 (default)
#> held_terminal_position.default last_permissible (default)
#> frozen_terminal_position write_off (default)The plan states which checks actually participate. Membership and sessions are declared. Trading-status and lifetime are omitted, and omitted does not mean the source said every instrument was active and listed. It means no such check ran, which is why ledgr never learns that AAA’s silence after January 10 was a delisting rather than a halt or a feed gap. The declared assumptions at the end list the choices that decide what happens if a holding goes unpriced or ends; (default) marks the ones you did not set yourself.
# The survivor experiment repeats every argument above and changes one, so
# wrap the shared choices rather than retyping them.
universe_experiment <- function(universe) {
ledgr_experiment(
snapshot,
scheduled_rebalance,
universe = universe,
valuation_policy = ledgr_valuation_stale(max_sessions = 2),
cost_model = ledgr_cost_zero(),
opening = ledgr_opening(cash = 10000)
)
}
point_in_time <- ledgr_run(
point_in_time_experiment, params = list(invested = 0.9), run_id = "pit-universe"
)
survivor <- ledgr_run(
universe_experiment(c("BBB")), params = list(invested = 0.9), run_id = "survivor-universe"
)The point-in-time run bought both companies. The survivor run only ever saw one.
ledgr_results(point_in_time, "fills") |>
select(ts_utc, recording_pulse_ts_utc, instrument_id, side, qty, price)
#> # A tibble: 3 x 6
#> ts_utc recording_pulse_ts_utc instrument_id side qty price
#> <dttm> <dttm> <chr> <chr> <dbl> <dbl>
#> 1 2020-01-07 14:30:00 2020-01-07 21:00:00 AAA BUY 45 92
#> 2 2020-01-07 14:30:00 2020-01-07 21:00:00 BBB BUY 90 51
#> 3 2020-01-14 14:30:00 2020-01-14 21:00:00 BBB BUY 11 56The decision was made at the January 6 close. The fills are stamped at the January 7 open, when the economic execution happened. The third row is the January 13 rebalance, filled at the January 14 open: AAA has left the declared membership, so the rebalance selects only BBB, keeps the AAA holding it can no longer select, and spends the rest of the budget on 11 more BBB shares. Each fill’s read-only recording_pulse_ts_utc points to the close of its fill session, January 7 for the first two rows and January 14 for the third, where the resulting account state appears on the equity curve.
Two clocks in one fill row
-
ts_utcandpriceidentify the open of the next session. That is where execution happened. -
recording_pulse_ts_utcis that session’s close, the accounting pulse to which the fill belongs.
A target decided at session t is evaluated and priced at session t+1’s open. Strategy code still sees only decision-time evidence. To align fills with equity, aggregate fills by recording pulse first; joining raw economic timestamps to close-stamped equity is not session alignment.
fills_by_pulse <- ledgr_results(point_in_time, "fills") |>
group_by(recording_pulse_ts_utc) |>
summarise(
fill_count = n(),
filled_notional = sum(qty * price),
.groups = "drop"
)
pit_equity_for_join <- ledgr_results(point_in_time, "equity")
pit_equity_for_join |>
left_join(
fills_by_pulse,
by = c("ts_utc" = "recording_pulse_ts_utc")
) |>
select(ts_utc, equity, fill_count, filled_notional)
#> # A tibble: 10 x 4
#> ts_utc equity fill_count filled_notional
#> <dttm> <dbl> <int> <dbl>
#> 1 2020-01-06 21:00:00 10000 NA NA
#> 2 2020-01-07 21:00:00 10000 2 8730
#> 3 2020-01-08 21:00:00 9460 NA NA
#> 4 2020-01-09 21:00:00 9460 NA NA
#> 5 2020-01-10 21:00:00 8110 NA NA
#> 6 2020-01-13 21:00:00 8200 NA NA
#> 7 2020-01-14 21:00:00 8290 1 616
#> 8 2020-01-15 21:00:00 8391 NA NA
#> 9 2020-01-16 21:00:00 8492 NA NA
#> 10 2020-01-17 21:00:00 8593 NA NAThe aggregation preserves one row per equity pulse even when several instruments fill at the same opening.
Check completion before comparing results
run_inventory <- ledgr_run_list(snapshot)
run_inventory
#> # ledgr run list
#> # A tibble: 2 x 11
#> run_id label tags status final_equity total_return complete_performance
#> <chr> <chr> <chr> <chr> <dbl> <chr> <lgl>
#> 1 pit-universe <NA> <NA> DONE 8593 -14.1% TRUE
#> 2 survivor-universe <NA> <NA> DONE 11425 +14.3% TRUE
#> achieved_end_utc valuation_state execution_mode reproducibility_level
#> <dttm> <chr> <chr> <chr>
#> 1 2020-01-17 21:00:00 frozen audit_log tier_1
#> 2 2020-01-17 21:00:00 stale audit_log tier_1
#>
#> # i 2 runs attempted: 2 done, 0 incomplete, 0 failed.
#> # i Of the done runs, 1 froze a holding and 0 used an assumed terminal outcome; see valuation_state and assumed_terminal_outcomes.
#> # i INCOMPLETE metrics describe the achieved prefix only.
#> # i Full identity and telemetry columns remain available on this tibble.
#> # i Inspect one run with ledgr_run_info(snapshot, run_id).The inventory puts completion beside the raw metrics. Both runs reach their requested end, so ledgr_run_compare() can rank them together:
ledgr_run_compare(snapshot)
#> # ledgr comparison
#> # A tibble: 2 x 11
#> run_id label final_equity total_return sharpe_ratio max_drawdown n_trades win_rate
#> <chr> <chr> <dbl> <chr> <dbl> <chr> <int> <chr>
#> 1 pit-universe <NA> 8593 -14.1% -4.6775 -18.9% 0 <NA>
#> 2 survivor-univ~ <NA> 11425 +14.3% 22.602 0.0% 1 100.0%
#> # i 3 more variables: valuation_state <chr>, assumed_terminal_outcomes <int>,
#> # reproducibility_level <chr>
#>
#> # i 2 runs attempted: 2 done, 0 incomplete, 0 failed.
#> # i Only DONE runs are compared; incomplete and failed runs stay listed by ledgr_run_list().
#> # i Fill timing comparable: yes (same_timing_convention).
#> # i Full identity and telemetry columns remain available on this tibble.
#> # i Inspect one run with ledgr_run_info(snapshot, run_id).The valuation_state column is the part a survivor file could never show. The point-in-time run finished frozen: on its last sessions part of its equity rests on a price nobody observed. Read the size of the effect from the figure below, where both paths are drawn against the same clock and the assumed sessions are shaded.
The universe declaration explains the difference between these two runs. The shading marks where the point-in-time number stops being an observation and becomes an assumption, which the next sections work through.
What The Run Recorded
The availability view reports, for every instrument at every pulse, what the configured logic concluded. It is run evidence, not proof that the strategy callback executed on that pulse.
availability <- ledgr_results(point_in_time, "availability")
availability |>
filter(instrument_id == "AAA") |>
select(ts_utc, member, held, admissible, priced, mark_source, mark_age)
#> # A tibble: 10 x 7
#> ts_utc member held admissible priced mark_source mark_age
#> <dttm> <lgl> <lgl> <lgl> <lgl> <chr> <int>
#> 1 2020-01-06 21:00:00 TRUE FALSE TRUE TRUE current_close 0
#> 2 2020-01-07 21:00:00 TRUE TRUE TRUE TRUE current_close 0
#> 3 2020-01-08 21:00:00 TRUE TRUE TRUE TRUE current_close 0
#> 4 2020-01-09 21:00:00 TRUE TRUE TRUE TRUE stale_close 1
#> 5 2020-01-10 21:00:00 TRUE TRUE TRUE TRUE current_close 0
#> 6 2020-01-13 21:00:00 FALSE TRUE FALSE TRUE stale_close 1
#> 7 2020-01-14 21:00:00 FALSE TRUE FALSE TRUE stale_close 2
#> 8 2020-01-15 21:00:00 FALSE TRUE FALSE TRUE frozen_close 3
#> 9 2020-01-16 21:00:00 FALSE TRUE FALSE TRUE frozen_close 4
#> 10 2020-01-17 21:00:00 FALSE TRUE FALSE TRUE frozen_close 5These columns are not stages of one process. They answer four different questions:
| Field | The question it answers |
|---|---|
member |
Does this asset belong to the declared universe at this pulse? |
held |
Do I already own a nonzero quantity of it? |
admissible |
Does the configured availability logic permit new exposure? |
priced |
Can the valuation policy assign it a permissible mark? |
January 13 is the row worth studying. AAA is outside the universe, still owned, and still valued, using a mark that is one session old. Membership, ownership, and the ability to value a position are three separate things, and a survivor-filtered file cannot represent any of them, because the instrument is simply absent.
January 15 is the second row worth studying. The mark is now three sessions old, one more than max_sessions = 2 permits, so mark_source becomes frozen_close. The holding is still owned and still on the equity curve at its January 10 close, but that value is now an assumption rather than an observation, and admissible is FALSE because ledgr will not size new exposure on it.
priced = TRUE is not a tradable price
A permissible mark values a holding on the equity curve. It says nothing about whether an executable price exists. Fills come only from observed bars.
Try it: preserve a held nonmember
At the January 13 pulse, compare ctx$hold() with a target built only from ctx$members. Which one preserves AAA? Then remove one current bar for a still-held instrument and seal the changed bars under a new snapshot_id. A feed gap must not become an accidental liquidation; start from holdings and change only the positions your rule intends to change.
Expected result: ctx$hold() retains the nonzero AAA position. Replacing it with a member-only zero vector requests an exit, but with no observed execution bar that request cannot fill and does not become a standing order.
What The Point-In-Time Run Assumed
AAA is delisted, still held, and from January 15 its last observation is older than the valuation policy permits. ledgr does not drop the position and does not invent a sale. It keeps the quantity and the cash exact, carries the January 10 close, and puts the assumption in front of the metrics:
point_in_time
#> ledgr Backtest Results
#> ======================
#>
#> WARNING: this result relies on declared valuation or terminal assumptions.
#> Frozen valuation: 1 holding carried at the last close beyond max_sessions = 2 for 3 instrument-sessions (exhausted = "freeze", the default).
#> Still frozen at end: 1 holding worth 1980 (23.04% of final equity); total return if worthless: -33.87%.
#>
#> Run ID: pit-universe
#> Period: 2020-01-06T21:00:00Z to 2020-01-17T21:00:00Z
#> Opening Cash: $10000.00
#> Final Equity: $8593.00
#> Total Return: -14.07%
#> Max Drawdown: -18.90%
#> Closed Trades: 0
#> Strategy schedule: explicit dates (2)
#> Decision pulses: 2; scheduled skips: 8
#> Metric Calendar: US equity daily (default) (252 days/year * 1 bars/day = 252 bars/year)
#>
#> Economic events: NOT SUPPLIED - returns may omit distributions
#> Price basis: split_adjusted (declared)
#> Valuation: FROZEN - 1 holding frozen (longest 3 sessions, max 1980 or 23.60% of equity); 1 still frozen at end worth 1980
#> Point in time: membership rule 'demo_members'; valuation stale-mark horizon 2 sessions (declared)
#> History policy: price carry 5 sessions (default); output carry off (default); segment limit 4 per fact, at least 16 (default)
#>
#> Use summary(bt) for metrics and evidenceThe warning comes first and has no threshold: a frozen holding worth one dollar would trigger it too. The valuation line underneath gives the size. ledgr_valuation_summary() returns the same evidence as fields you can test:
pit_valuation <- ledgr_valuation_summary(point_in_time)
pit_valuation
#>
#> Valuation Evidence:
#> Valuation: FROZEN - 1 holding frozen (longest 3 sessions, max 1980 or 23.60% of equity); 1 still frozen at end worth 1980
#> Universe: membership rule 'demo_members'
#> Stale-mark horizon: 2 sessions (declared)
#> Exhausted marks: freeze
#> Stale instrument-sessions: 4
#> Frozen instrument-sessions: 3
#> Longest freeze: 3 sessions
#> Max frozen value: 1980
#> Max frozen weight: 23.60%
#> Frozen at end: 1 holdings, 1980 (23.04% of equity)
#> Return if frozen worthless: -33.87%
#>
#> History Policy:
#> Price carry: 5 sessions (default)
#> Output carry: off (default)
#> Segment limit: 4 per fact, at least 16 (default)
pit_valuation$total_return_if_frozen_worthless
#> [1] -0.3387This is the hand calculation’s quiet assumption coming due, now with a name. The reported total return values AAA at its last observed close. The total_return_if_frozen_worthless field restates the endpoint as if that holding paid nothing. It is a bound for the long-only case, not a corrected return: Sharpe ratio and drawdown still follow the assumed path.
Frozen equity is an estimate
During a freeze, cash, quantity and the ledger stay exact; the equity number does not. It assumes the last close still holds. If the holding later trades again, the whole price move lands on the session it resumes, so the endpoint return converges, provided the carried mark changed no decision in between, but the path statistics stay distorted. A run that ends with a holding still frozen is a complete run built on a stated assumption, which is why every result surface carries valuation_state.
If you would rather have no number than an assumed one, say so: ledgr_valuation_stale(max_sessions = 2, exhausted = "refuse") stops the run INCOMPLETE at the first session a mark expires, and keeps the prefix it could value. Comparison, sweep ranking and walk-forward selection then leave that run out and report it as incomplete, rather than dropping it silently.
Share-changing corporate actions, borrow and short financing, order-management behaviour, and general imputation are outside this availability surface. Cash distributions are modelled, as vignette("corporate-action-cash", package = "ledgr") shows.
Try it: move the valuation boundary
Rebuild point_in_time_experiment with ledgr_valuation_stale(max_sessions = 1), use a new run_id, and rerun it. Predict first: does freezing start earlier, later, or on the same session? Then compare frozen_instrument_sessions in ledgr_valuation_summary(). Finally add exhausted = "refuse" and compare achieved_end_utc with the frozen run’s. One fewer permitted stale session moves the boundary. It does not change when AAA was delisted.
Telling ledgr What Happened To AAA
Freezing is what ledgr does when it knows nothing. Here we do know something: AAA left the index because it was delisted. A lifetime fact says so, dated when the delisting took effect, with the reason as the vendor would label it. knowledge = "assume_effective" makes the delisting knowable the moment it takes effect. That is an assumption for this synthetic example; with real data, record when the delisting was announced.
delisting <- ledgr_facts_lifetime(
tibble(
instrument_id = "AAA",
effective_from = ledgr_utc("2020-01-13 00:00:00"),
assertion = "known_inactive",
terminal_event = "delisted"
),
knowledge = "assume_effective"
)
terminal_snapshot <- ledgr_snapshot_from_df(
bars_input,
instruments_df = instruments,
facts = ledgr_facts(session_facts, membership_facts, delisting),
price_basis = "split_adjusted",
db_path = store_path,
snapshot_id = "survivorship-terminal"
)A delisting is a fact. What a holder received is not in this data, so ledgr still has to assume something, and the corporate-action policy is where you declare it. held_terminal_position maps exact terminal labels to choices. The research default, "last_permissible", disposes the holding at its last permissible mark. Here we also run the conservative alternative, a write-off for the "delisted" label:
terminal_experiment <- function(policy) {
ledgr_experiment(
terminal_snapshot,
scheduled_rebalance,
universe = ledgr_universe_members("demo_members"),
valuation_policy = ledgr_valuation_stale(max_sessions = 2),
corporate_action_policy = policy,
cost_model = ledgr_cost_zero(),
opening = ledgr_opening(cash = 10000)
)
}
at_last_mark <- ledgr_run(
terminal_experiment(ledgr_corporate_actions()),
params = list(invested = 0.9), run_id = "pit-terminal-default"
)
written_off <- ledgr_run(
terminal_experiment(ledgr_corporate_actions(
held_terminal_position = c(delisted = "write_off", default = "last_permissible")
)),
params = list(invested = 0.9), run_id = "pit-terminal-write-off"
)
bind_rows(
ledgr_corporate_action_summary(at_last_mark)$terminal_outcomes,
ledgr_corporate_action_summary(written_off)$terminal_outcomes
) |>
select(instrument_id, terminal_event, choice, assumption_label, proceeds)
#> # A tibble: 2 x 5
#> instrument_id terminal_event choice assumption_label proceeds
#> <chr> <chr> <chr> <chr> <dbl>
#> 1 AAA delisted last_permissible default: last permissible mark 1980
#> 2 AAA delisted write_off declared: assumed worthless 0Each terminal becomes one DISPOSITION on January 13, before the strategy decides, so the default run’s 1,980 in proceeds is already cash when the January 13 rebalance buys more BBB. Neither run freezes anything, because AAA is no longer held when its mark would have expired. Both runs still warn: the outcome is a modeled choice, and assumption_label says whether it was your choice (declared) or ledgr’s (default).
written_off
#> ledgr Backtest Results
#> ======================
#>
#> WARNING: this result relies on declared valuation or terminal assumptions.
#> Assumed terminal outcomes: 1.
#> AAA 2020-01-13: declared: assumed worthless (held_terminal_position["delisted"] = "write_off").
#>
#> Run ID: pit-terminal-write-off
#> Period: 2020-01-06T21:00:00Z to 2020-01-17T21:00:00Z
#> Opening Cash: $10000.00
#> Final Equity: $6613.00
#> Total Return: -33.87%
#> Max Drawdown: -37.80%
#> Closed Trades: 0
#> Strategy schedule: explicit dates (2)
#> Decision pulses: 2; scheduled skips: 8
#> Metric Calendar: US equity daily (default) (252 days/year * 1 bars/day = 252 bars/year)
#>
#> Economic events: MODELED - configured settlement conventions were exercised
#> Price basis: split_adjusted (declared)
#> Valuation: STALE - 2 holdings valued at a permitted stale close (longest age 1 session)
#> Point in time: membership rule 'demo_members'; valuation stale-mark horizon 2 sessions (declared); assumption-backed facts: lifetime (knowledge assumed effective)
#> History policy: price carry 5 sessions (default); output carry off (default); segment limit 4 per fact, at least 16 (default)
#>
#> Use summary(bt) for metrics and evidenceThe write-off lands on -33.87%, the same number the frozen run reported as total_return_if_frozen_worthless: both value the same AAA shares at zero. That equality is specific to this example. AAA paid nothing in either run, so every later decision saw the same cash; a choice that pays proceeds changes the later decisions too, as the default run shows. The three runs differ only in what they assume about how AAA ended, and the spread between them is how much the result depends on that assumption. None of them is evidence of what AAA actually paid. Evidence would be a settlement fact, such as a recorded cash acquisition; it pre-empts every choice above and is reported as evidence rather than as an assumption. vignette("corporate-action-cash", package = "ledgr") shows that path.
When The Bar You Need Is Not There
The January 9 feed outage teaches a different lesson, so give BBB its own small run: ask to halve the position on January 8, and again on January 10.
trim_twice <- function(ctx, params) {
target <- ctx$hold()
day <- format(ledgr_utc(ctx$ts_utc), "%Y-%m-%d", tz = "UTC")
if (day == "2020-01-06") target[["BBB"]] <- 100
if (day %in% c("2020-01-08", "2020-01-10")) target[["BBB"]] <- 50
target
}
trim_run <- ledgr_run(
ledgr_experiment(
snapshot,
trim_twice,
universe = c("BBB"),
valuation_policy = ledgr_valuation_stale(max_sessions = 2),
cost_model = ledgr_cost_zero(),
opening = ledgr_opening(cash = 10000)
),
run_id = "trim-story"
)
bind_rows(lapply(
ledgr_utc(c("2020-01-08 21:00:00", "2020-01-09 21:00:00", "2020-01-10 21:00:00")),
function(ts) ledgr_run_explain(trim_run, "BBB", ts)
)) |>
select(ts_utc, target_before_risk, execution_outcome, execution_reason,
resulting_position)
#> # A tibble: 3 x 5
#> ts_utc target_before_risk execution_outcome execution_reason
#> <dttm> <dbl> <chr> <chr>
#> 1 2020-01-08 21:00:00 50 no_fill "execution_bar_missing"
#> 2 2020-01-09 21:00:00 100 no_action "no_target_change"
#> 3 2020-01-10 21:00:00 50 filled ""
#> # i 1 more variable: resulting_position <dbl>This date branch injects three specific requests to demonstrate a missing-bar execution scenario; it is not a recurring rebalance calendar. Periodic strategy scheduling belongs in ledgr_strategy_schedule() as shown above.
Follow the dates against the contract stated earlier:
-
January 8 decision asks for 50 units. Its execution opportunity is the next session’s open, January 9.
BBBhas no January 9 bar, so the explanation recordsexecution_bar_missingand the position stays at 100. -
January 9 decision holds. The outcome is
no_actionwith reasonno_target_change. The January 8 request did not survive as a standing order. A target states the position you want now. -
January 10 decision asks again. Its execution opportunity is the January 13 open, where a
BBBbar exists, so the trim fills.
That is why a missing fill cannot silently become a later liquidation.
Evidence Survives Reopening
Closing a run handle releases resources. It does not erase what was recorded.
pit_run_id <- point_in_time$run_id
expected <- ledgr_run_explain(point_in_time, "AAA", ledgr_utc("2020-01-13 21:00:00"))
completion_fields <- c(
"status", "requested_start_utc", "requested_end_utc",
"achieved_start_utc", "achieved_end_utc", "stop_reason",
"last_fully_valued_ts_utc", "last_executed_ts_utc",
"complete_performance", "affected_instrument_ids"
)
expected_completion <- ledgr_run_info(snapshot, pit_run_id)[completion_fields]
expected_valuation <- ledgr_valuation_summary(point_in_time)
close(point_in_time)
close(survivor)
close(trim_run)
close(at_last_mark)
close(written_off)
ledgr_snapshot_close(terminal_snapshot)
ledgr_snapshot_close(snapshot)
reopened_snapshot <- ledgr_snapshot_open(
store_path, "survivorship-demo", verify = TRUE
)
reopened_run <- ledgr_run_open(reopened_snapshot, pit_run_id)
identical(
ledgr_run_explain(reopened_run, "AAA", ledgr_utc("2020-01-13 21:00:00")),
expected
)
#> [1] TRUE
identical(
ledgr_run_info(reopened_snapshot, pit_run_id)[completion_fields],
expected_completion
)
#> [1] TRUE
identical(ledgr_valuation_summary(reopened_run), expected_valuation)
#> [1] TRUEThe same recorded decision, completion boundary and valuation assumptions come back from a fresh handle, without running the strategy again.
Invalid Observations Are Refused By Default
Survivorship is selection you did not make. Malformed data is a different problem, and ledgr’s default is to refuse it rather than guess: a bar whose high sits below its open prevents sealing, with ledgr_availability_validation_failed. Explicit quarantine (invalid_observations = "quarantine") is the deliberate alternative, which hashes the rejected row as evidence and excludes it from runtime bars while the expected session stays on the calendar. For that workflow, read vignette("missing-data-and-sessions", package = "ledgr").
Try it: spend the whole account
The snapshot above has been closed, so first point snapshot at the reopened handle with snapshot <- reopened_snapshot, then call universe_experiment(c("BBB")). Rerun the survivor experiment with params = list(invested = 1) and a new run_id. Predict first: does its single target fill? Sizing uses the January 6 close of 50, so the whole account asks for 200 shares, but execution happens at the January 7 open where BBB costs 51 and those shares now cost 10,200. Check ledgr_results(run, "diagnostics") for the execution row and its reason code.
Then try the same on point_in_time. Its two targets cost 9,700 at that same opening and both fill, because AAA fell overnight while BBB rose. Full allocation is not what rejects an order; the overnight move against you is. This is the availability-aware execution outcome in this experiment, not a universal statement about every ledgr path or future broker policy.
Where This Fits
This article keeps a two-company example because it makes the bias visible. For the complete reusable input model – bars, sessions, membership, status, lifetime, and corporate actions – read vignette("point-in-time-inputs", package = "ledgr").
AAA, BBB, and the DEMO venue are local to this article; they are not additions to the shared DEMO_* history. When you want to study a delisting on data that also carries sessions, trading status and corporate actions, load the shared ledgr_demo_pit_inputs bundle. It contains a delisting, and this check finds its lifetime boundary:
data("ledgr_demo_pit_inputs", package = "ledgr")
shared_delisting <- subset(
ledgr_demo_pit_inputs$cases,
type == "delisting"
)
shared_lifetime_boundary <- subset(
ledgr_demo_pit_inputs$lifetime,
instrument_id == shared_delisting$instrument_id[[1L]] &
as.Date(effective_from) == shared_delisting$date[[1L]]
)
shared_lifetime_boundary[
, c("instrument_id", "effective_from", "assertion", "terminal_event")
]
#> instrument_id effective_from assertion terminal_event
#> 6 DEMO_01 2020-01-14 21:00:00 known_inactive delistedWhat This Does And Does Not Establish
The example shows five distinctions worth preserving:
- membership is a dated assertion about opportunity, not a sell order;
- a held former member stays visible until an explicit target changes it;
- valuation marks and execution prices answer different questions;
- expected sessions remain visible when observations are absent;
- a value carried past the valuation horizon is reported as an assumption, and a terminal outcome as a declared or default choice, unless evidence settles it.
It does not show that ledgr detects survivorship bias. A company omitted entirely from the evidence you supply stays invisible, and no amount of downstream machinery recovers it. The narrower claim is the one demonstrated here: a holding that is represented cannot silently disappear because membership ended or observations stopped.
Nor does sealing certify your source. If membership knowledge time is assumed from effective time, ledgr_experiment_plan() labels the family assumption_backed. If trading-status or lifetime facts are omitted, their checks remain omitted. The gap shown here belongs to one synthetic example chosen to make the mechanism legible; it is not an estimate of survivorship bias in any real index.
A carried price is not a missing price
Freezing starts only when a bar is absent. Halts and untraded sessions that a data source encodes as zero-volume bars carrying the previous close, as the reference Sharadar data does, are still read as observed prices: they can fill and they bypass valuation ageing and freezing. If your source fills untraded sessions that way, remove those rows or mark the instrument halted with a trading-status fact before you seal.
For sealing details, read vignette("data-input-and-snapshots", package = "ledgr"). For execution timing and target semantics, read vignette("execution-semantics", package = "ledgr"). For durable run handling, read vignette("experiment-store", package = "ledgr").
Where Next
-
vignette("strategy-authoring-tools", package = "ledgr")tests a strategy on one pulse, sizes shares, and handles missing inputs. -
vignette("point-in-time-inputs", package = "ledgr")covers the complete input model. -
vignette("execution-semantics", package = "ledgr")explains when and at what price a target fills.
References
- Brown, S. J., Goetzmann, W. N., Ibbotson, R. G., and Ross, S. A. (1992). Survivorship Bias in Performance Studies. The Review of Financial Studies, 5(4), 553-580.
- Shumway, T. (1997). The Delisting Bias in CRSP Data. The Journal of Finance, 52(1), 327-340.
- Center for Research in Security Prices. Research Data Products. The permanent-ID discussion is useful context for keeping stable instrument identity separate from mutable symbols and membership.
- Nasdaq Trader. Trading Halt Codes. The status vocabulary illustrates why a valid price row and positive evidence of tradability are different source claims.