A strategy sees one pulse at a time, but most estimators need a window of past values: a covariance matrix, a trailing volatility, a momentum score. ledgr_history_frame() returns that window from inside the strategy, already aligned to the venue’s sessions and cut off at the pulse. Every value in it was observed at or before the pulse, and every fact behind it was knowable then.
This article starts from a snapshot that already exists, runs it once, and then reads history the way a strategy would: defaults first, then the explicit choices, then the evidence that explains each cell. It ends with one portfolio estimate that goes from history to targets without rebuilding any timestamp or instrument name by hand.
Open A Snapshot And Run It
Someone has already sealed a point-in-time snapshot of eight synthetic instruments over 2020. In this article it was built in a setup step that is not shown, from the inputs ledgr_sim_pit_inputs() returns (kept as pit), and written to the DuckDB file whose path is snapshot_path. The next article, on point-in-time inputs, shows that preparation step by step. Opening the file, declaring an experiment and running it takes three calls.
ledgr Backtest Results
======================
Run ID: hello-history
Period: 2020-01-01T16:00:00Z to 2020-12-31T16:00:00Z
Opening Cash: $100000.00
Final Equity: $107536.73
Total Return: 7.54%
Max Drawdown: -6.50%
Closed Trades: 369
Metric Calendar: US equity daily (default) (252 days/year * 1 bars/day = 252 bars/year)
Economic events: NONE - supplied facts did not affect held instruments
Price basis: split_adjusted (declared)
Valuation: CURRENT - every held mark was a current close
Point in time: membership rule 'DEMO_UNIVERSE'; valuation stale-mark horizon 5 sessions (default)
History policy: price carry 5 sessions (default); output carry off (default); segment limit 4 per fact, at least 16 (default)
Use summary(bt) for metrics and evidence
The snapshot already declares a venue calendar, a changing universe, trading halts, a delisting and a dividend, so the experiment reads them without further declarations. The inputs it was built from are in pit; pit$cases lists where each of those teaching cases sits. The rest of this article looks inside single decisions of experiments like this one.
Read History From Inside A Strategy
ledgr_pulse_snapshot() builds the context a run would hand to its strategy at one session, without running anything. That makes it the easiest place to try a history read.
The defaults are the ones most estimators want, but the history width is always explicit:
lookback = Inf returns every expected venue session from the start of the data up to the pulse. A finite lookback returns the last sessions of that same history, so a shorter read is never computed differently.
instruments = "admissible" returns the instruments the strategy may target at this pulse.
min_observations = NULL requires one finite treated value for every requested session. Set a smaller integer when the estimator can work with incomplete columns.
min_instruments = 1L records how many included instruments the strategy needs before it is ready. It does not remove another column.
layout = "wide" returns one row per session, ts_utc first, then one column per instrument.
Read A Matrix Directly
Estimators usually want a numeric matrix, so the context form of ledgr_history_matrix() reads one directly. It uses the same causal plan as the frame reader, keeps the session stamps and instrument IDs as dimension names, and does not build a data frame first. The one-argument form remains an adapter when a frame is useful for inspection.
ledgr_history_info() returns what the result records about itself: its source, cutoff, window, how many sessions it holds and which requested instruments it includes or left out.
Because min_observations performs column admission and the matrix retains the admitted instrument identity, an estimator normally needs only this one read. Its unnamed solution can go straight through ledgr_align_weights(); the complete worked example is vignette("minimum-variance", package = "ledgr").
ledgr_history_info
Source: feature:return_20
Cutoff: 2020-06-30T16:00:00Z
Window start: 2020-06-24T16:00:00Z
Sessions: 5 of requested 5
Selector: admissible
Requirements: 5 finite observations, 1 instruments
Ready: yes
Included: 7 of 7 requested
Used before window: yes
# i Selection table: info$selection
Used before window: yes says that some returned value depends on sessions before the returned window: each 20-session return in these five rows was computed from closes up to 20 sessions earlier.
The ordinary matrix and wide frame are intentionally light. They retain the axis, instrument selection, policy and admission summary, but do not hash the transient values or retain a second cell-evidence payload. That metadata is provenance for the originating read, not a seal over later R transformations. Ask for layout = "long" when you need per-cell diagnostic evidence.
Choose The Instruments And The Required Observations
The defaults can be replaced one at a time. instruments = "decision_axis" adds held nonmembers and members the strategy may not trade now. Here that is DEMO_01, still a member of the universe but delisted on 14 January, so it is not admissible at the cutoff. min_observations leaves an instrument out of the estimate when it has too few finite treated values, and records why. Finite input-carried and output-carried values count because they are values the declared history policy admitted.
structural_exclusion means the window crosses a lifetime or terminal barrier, here DEMO_01’s delisting, rather than a missing observation (insufficient_observations) or a carry that ran out (carry_expired). Leaving an instrument out of an estimate is an estimation choice only. It creates no target, no zero target and no order.
The explicit long diagnostic layout returns one row per session and instrument, with the evidence for each cell next to its value.
The history request records both requirements. min_observations filters each instrument independently; min_instruments asks how many included instruments the strategy needs. ledgr_passed_warmup() reads the resulting readiness bit without rescanning the matrix.
In January there are not yet 60 finite returned observations per instrument, so the answer is FALSE. For history objects the familiar warm-up helper means that the recorded panel requirements passed; it does not rescan cells. Feature-cell readiness remains separate evidence, described next.
Feature History And Indicator Classes
Every indicator belongs to one of two history classes. A bounded indicator, such as a 20-session return or a simple moving average, depends only on its last requires_bars expected sessions. A history-dependent indicator, such as TTR’s recursive exponential moving average, depends on everything before it and starts again after a gap it cannot bridge. Printing an indicator shows its class and where the class comes from.
ledgr indicator
ID: ttr_ema_10
Requires bars: 10
Stable after: 10
Series fn: yes
History class: history_dependent (package catalogue)
Required: close
Input carry: eligible (package catalogue)
Feature cells carry their own evidence. ready turns TRUE once a cell has enough admitted sessions behind it, calculation_valid once it is ready and finite, and missing_reason says why any other cell has no usable value.
ts_utc value ready calculation_valid missing_reason
NA <NA> NA NA NA <NA>
NA.1 <NA> NA NA NA <NA>
NA.2 <NA> NA NA NA <NA>
NA.3 <NA> NA NA NA <NA>
The first 20 cells are not_ready: a 20-session return needs 21 closes.
Carried Prices Are Disclosed
By default the experiment’s history policy carries a missing open, high, low or close forward for at most five expected sessions; volume never carries. A carried cell says so, with its source session and age.
pit$cases
type instrument_id date
1 venue_closure <NA> 2020-01-06
2 missing_observation DEMO_04 2020-01-09
3 delisting DEMO_01 2020-01-14
4 halt DEMO_02 2020-01-03
5 cash_dividend DEMO_03 2020-01-10
# A tibble: 8 x 4
instrument_id required_input_count carried_input_count carried_input_share
<chr> <int> <int> <dbl>
1 DEMO_04 4 1 0.25
2 DEMO_01 NA NA NA
3 DEMO_02 NA NA NA
4 DEMO_03 NA NA NA
5 DEMO_05 NA NA NA
6 DEMO_06 NA NA NA
7 DEMO_07 NA NA NA
8 DEMO_08 NA NA NA
ledgr_history_policy(price_carry = NULL) turns input carry off, so a missing close stays missing.
Read Return Features, Not Differences Of Carried Closes
A carried close repeats the previous close, so a return computed by differencing the close history is exactly zero on the missing session, and the next session carries the whole two-session move as if it were one day. Neither number is an observed daily return. Return and volatility features never consume a carried price: the cell on the missing session is missing with the reason carried_input_excluded, the next one is not_ready while the return restarts, and the first return computed from two observed closes follows.
[1] ts_utc value missing_reason
<0 rows> (or 0-length row.names)
From History To Targets
This is the whole estimation flow in one strategy: read a window of daily returns, turn a covariance estimate into weights with a small stand-in solver, name the solver’s output through the history result, and turn the weights into targets. The solver is defined inside the strategy, because preflight refuses a strategy that depends on a function from the calling session.
The stand-in is not a long-only minimum-variance solver. It solves the unconstrained problem, sets negative weights to zero and rescales the rest, which in general is not the long-only optimum. vignette("minimum-variance", package = "ledgr") builds the constrained strategy with a quadratic-programming solver, screens and decision dates.
ledgr_align_weights() reads the unnamed solver output in the order of the history result’s included instruments and refuses a vector of the wrong length, so no column name is rebuilt by hand.
Two separate choices are at work. Estimation exclusion decides which instruments enter the covariance estimate. Execution intent decides what happens to each position. ledgr_target_rebalance() targets every eligible instrument without a weight to zero, so an instrument the estimate dropped would be sold. Passing the dropped instruments that are held in keep preserves those positions instead. The excluded IDs come from ledgr_history_info(); the held ones from the context. A held instrument that is no longer admissible is not requested under the default selector, so it never appears among the excluded IDs; ledgr_target_rebalance() already preserves such holdings. (The params$keep switch exists only so the next example can show both choices.)
To see keep act, run the estimate with a three-session window, so that it starts trading in early January and DEMO_04’s missing close on 9 January falls inside the window while the position is held. Once with keep and once without:
Both runs buy 357 shares of DEMO_04 on 8 January. On 9 January its close is missing, so the estimates of the next sessions leave it out. With keep the position stays until the estimate itself reduces it on 16 January. Without keep the rebalance targets it to zero and sells all 357 shares at the next open, on 10 January, then buys some back on 16 January once the instrument is in the estimate again.
estimated
ledgr Backtest Results
======================
WARNING: this result relies on declared valuation or terminal assumptions.
Assumed terminal outcomes: 1.
DEMO_01 2020-01-14: default: last permissible mark (held_terminal_position default "last_permissible", the research default).
Run ID: history-estimate
Period: 2020-01-01T16:00:00Z to 2020-12-31T16:00:00Z
Opening Cash: $100000.00
Final Equity: $114198.49
Total Return: 14.20%
Max Drawdown: -3.81%
Closed Trades: 865
Metric Calendar: US equity daily (default) (252 days/year * 1 bars/day = 252 bars/year)
Economic events: MODELED - configured settlement conventions were exercised
Price basis: split_adjusted (declared)
Valuation: STALE - 1 holding valued at a permitted stale close (longest age 1 session)
Point in time: membership rule 'DEMO_UNIVERSE'; valuation stale-mark horizon 5 sessions (default)
History policy: price carry 5 sessions (default); output carry off (default); segment limit 4 per fact, at least 16 (default)
Use summary(bt) for metrics and evidence
When A Fact Arrives Late
Facts can become known after the sessions they describe. A lifetime fact may say an instrument was inactive for two sessions, but arrive only on the third. The two kinds of history treat that differently:
bar history is restated with what is known at the cutoff, so a read after the fact arrives marks those sessions as inactive;
feature history is a vintage view, so each feature cell keeps the value and the evidence it had at its own pulse. The late fact changes feature cells only from the pulse at which it became known.
Read at the sixth session, both sessions are ordinary. Read at the seventh, after the fact is known, the bar history marks them as prints from an inactive interval, while the moving-average cells keep the values they were computed with at the time.
Walk-Forward Windows Read One History
A walk-forward test window does not start its history at the window start. Every run, sweep candidate and window reads the same history from the start of the data, so a feature or a history read at the first pulse of a test window sees everything before the window, as it would inside one long run.
To see what a test window reads, this strategy holds as many shares of one instrument as its history read has sessions. Each test window below starts flat (opening_state_policy = "flat_test_state"), so the first fill of a window is the number of sessions the strategy saw at that window’s first pulse.
Warning: `opening_state_policy = "flat_test_state"` starts every test run from the
experiment opening state. This can distort chained walk-forward evaluation when positions
or cash would carry between test windows.
Each fill happens at the next open after the decision. At the first pulse of the first test window, 1 July, the strategy read 130 sessions: every session since 1 January, not just the window. At the first pulse of the second window, 1 October, it read 196, again from 1 January. Neither window starts its history at its own start.
Outside this demonstration, the default opening_state_policy = "carry_test_state" carries the portfolio across window boundaries, so consecutive test windows form one continuous economic chain.
Read A Finite Lookback
lookback = Inf is convenient for exploring, but a strategy that reads the whole history at every pulse allocates in proportion to everything seen so far, and the total work over a run grows with the square of its length. Read the window your estimator needs.
Both reads end at the same pulse and agree on every session they share; the first grows with every pulse of a run, the second does not.
What ledgr Guarantees
ledgr guarantees that its own history, feature and fact readers return only data known at the decision time. Strategy code outside them is the author’s responsibility. ledgr_strategy_preflight() helps with a best-effort static check: it refuses code that reaches private context internals, non-exported ledgr objects or environment-inspecting functions, which is how a strategy could otherwise reach prepared data from later sessions.
ledgr Strategy Preflight
=========================
Tier: tier_3
Allowed: FALSE
Reason: Strategy reaches private context internals: $.history, .history. Read the context through its public fields and functions and history through `ledgr_history_frame()`.
Preflight is not a sandbox. R gives a strategy no private memory, so code that deliberately works around the check, for example by reading a file of future prices, is outside what ledgr can guarantee.
