Skip to contents

Sweep-level PBO/CSCV diagnostic

Source: R/validation-pbo.R
ledgr_sweep_pbo.Rd

ledgr_sweep_pbo() computes a native Probability of Backtest Overfitting (PBO) diagnostic using Combinatorially Symmetric Cross Validation (CSCV) over retained completed-candidate return panels. It is an evidence surface only: it does not select, promote, filter, or change walk-forward identity.

Usage

ledgr_sweep_pbo(
  sweep,
  candidates = NULL,
  S = 4L,
  metric = NULL,
  metric_name = NULL,
  threshold = 0
)

# S3 method for class 'ledgr_sweep_pbo'
as_tibble(
  x,
  what = c("summary", "cases", "degradation"),
  ...
)

# S3 method for class 'ledgr_sweep_pbo'
print(x, ...)

Arguments

sweep

A ledgr_sweep_results object with retained completed returns.

candidates

Optional character vector of candidate ids to include.

S

Even positive number of contiguous CSCV subsets. S must divide the post-first-row return count.

metric

Optional function that receives a numeric returns matrix and returns one finite numeric score per candidate column. Higher scores are treated as better. When NULL, mean period return is used.

metric_name

Optional character scalar naming the metric in result metadata.

threshold

Numeric logit threshold. PBO is the fraction of CSCV cases with lambda <= threshold.

x

A ledgr_sweep_pbo object.

what

Which table to return: "summary", "cases", or "degradation".

...

Passed to the tibble print method.

Value

A ledgr_sweep_pbo object with summary, cases, degradation, and metadata tables/lists. Use as_tibble(x), as_tibble(x, what = "cases"), or as_tibble(x, what = "degradation") for programmatic access.

Details

The diagnostic consumes ledgr_sweep_returns_panel(..., value = "returns", complete = TRUE). That means the structural first NA_real_ return row is verified and dropped before the CSCV matrix is formed, all selected candidates must share one common timestamp grid, and failed or unretained candidates fail closed through the retained-return panel classes.

S partitions the post-first-row return panel into contiguous subsets. For each symmetric split, ledgr scores all candidates in sample, takes the in-sample winner, ranks that same candidate out of sample, converts the relative rank into lambda = log(omega_bar / (1 - omega_bar)), and reports PBO as the share of cases with lambda <= threshold.

The default metric is mean period return. Custom metrics must return one finite numeric score per candidate column; larger scores are interpreted as better.

See also

vignette("selection-integrity", package = "ledgr") or system.file("doc", "selection-integrity.html", package = "ledgr").

Examples

if (FALSE) { # \dontrun{
pbo <- ledgr_sweep_pbo(sweep, S = 4)
as_tibble(pbo)
as_tibble(pbo, what = "cases")
} # }