Skip to contents

Research Workflow

Source: vignettes/research-workflow.qmd

You have an idea for a trading rule. How do you turn that hunch into evidence you can reopen, inspect, and explain later, while you keep changing the rule?

Most research iterations change the strategy code itself: a new condition, a different exit, a filter you did not think of on the first try. Some iterations ask a narrower question, such as which lookback works best. This article follows both kinds of iteration through one project and shows where each one hands off.

This workflow produces research evidence, not investment advice or a guarantee that a strategy will work in the future. See the project disclaimer.

%%{init: {"theme": "base", "flowchart": {"nodeSpacing": 18, "rankSpacing": 22, "curve": "linear"}, "themeVariables": {"fontFamily": "system-ui, -apple-system, Segoe UI, sans-serif", "fontSize": "15px", "primaryColor": "#f8fafc", "primaryTextColor": "#1f2937", "primaryBorderColor": "#64748b", "lineColor": "#64748b", "tertiaryColor": "#eef2ff", "tertiaryTextColor": "#1f2937", "tertiaryBorderColor": "#64748b", "clusterBkg": "#ffffff", "clusterBorder": "#cbd5e1"}}}%%

flowchart TB
  data["Seal data"]
  rule["Write a rule"]
  run["Run and inspect"]
  compare["Compare committed runs"]
  sweep["Sweep a parameter question"]
  select["Select and validate"]
  note["Record the decision"]

  data --> rule --> run --> compare --> select --> note
  compare -. change the code .-> rule
  compare -. parameter question .-> sweep --> select

Prerequisites 

library(ledgr)
library(dplyr)
data("ledgr_demo_bars", package = "ledgr")

Running this yourself

This article is evaluated when it is rendered, so it writes to a temporary store. In a real project, replace store_path with a durable path such as artifacts/ledgr_store.duckdb.

Project Topology

Start with a small project shape. One script creates the evidence, one store holds the artifacts, and one report explains the research choice.

my-ledgr-project/
  artifacts/
    ledgr_store.duckdb
  reports/
    workflow_review.md
  scripts/
    research_workflow.R

ledgr does not require this exact layout. The point is simpler: keep the script, durable store, and written reasoning close enough that a future review can recover the full research decision.

In a real project, add generated stores such as artifacts/*.duckdb to .gitignore unless you have a deliberate artifact-versioning policy. The script and source data should explain how to recreate the store; Git should not quietly become the database backup.

Seal The Evidence

Every version of the rule below runs against the same sealed snapshot, so the comparison is between rules, not between data sets.

store_path <- ledgr_temp_store()

bars <- ledgr_demo_bars |>
  filter(
    instrument_id %in% c("DEMO_01", "DEMO_02", "DEMO_03"),
    between(ts_utc, ledgr_utc("2019-01-01"), ledgr_utc("2019-12-31"))
  )

snapshot <- ledgr_snapshot_from_df(
  bars,
  db_path = store_path,
  snapshot_id = "research_2019"
)

Version 1: One Rule, One Run

The first idea is plain momentum: hold qty shares of every instrument whose 20-bar return is positive.

momentum_v1 <- function(ctx, params) {
  rising <- ledgr_signal_return(ctx, lookback = 20) > 0
  ctx |>
    ledgr_selection(where = rising, missing = "exclude") |>
    ledgr_target_quantity(ctx, params$qty)
}

exp_v1 <- ledgr_experiment(
  snapshot = snapshot,
  strategy = momentum_v1,
  features = list(ledgr_ind_returns(20)),
  opening = ledgr_opening(cash = 10000),
  cost_model = ledgr_cost_zero()
)

run_v1 <- ledgr_run(exp_v1, params = list(qty = 10), run_id = "momentum_v1")
run_v1
ledgr Backtest Results
======================

Run ID:                            momentum_v1
Period:                            2019-01-01 to 2019-12-31
Opening Cash:                      $10000.00
Final Equity:                      $10432.35
Total Return:                      4.32%
Max Drawdown:                      -1.69%
Closed Trades:                     29
Metric Calendar:                   US equity daily (default) (252 days/year * 1 bars/day = 252 bars/year)

Economic events: NOT SUPPLIED - returns may omit distributions
Price basis: UNDECLARED - distribution double counting cannot be ruled out

Use summary(bt) for metrics and evidence

Before judging the idea, check that the setup behaves at all: did anything trade, and do the numbers look plausible for ten shares of three demo instruments? If a run has no fills or implausible equity, fix the strategy or the feature declarations before you iterate further.

Version 2: Change The Code

Suppose the trades show the rule buying into short rallies inside a falling market. The next iteration is not a new parameter value; it is a new condition. Version 2 also requires the close to sit above its 50-bar moving average. The 50-bar average is missing for longer than the 20-bar return, and missing = "exclude" keeps an instrument out until both inputs exist.

momentum_v2 <- function(ctx, params) {
  rising <- ledgr_signal_return(ctx, lookback = 20) > 0
  above_trend <- ctx$vec$close > ledgr_signal_feature(ctx, "sma_50")
  ctx |>
    ledgr_selection(where = rising & above_trend, missing = "exclude") |>
    ledgr_target_quantity(ctx, params$qty)
}

exp_v2 <- ledgr_experiment(
  snapshot = snapshot,
  strategy = momentum_v2,
  features = list(ledgr_ind_returns(20), ledgr_ind_sma(50)),
  opening = ledgr_opening(cash = 10000),
  cost_model = ledgr_cost_zero()
)

run_v2 <- ledgr_run(exp_v2, params = list(qty = 10), run_id = "momentum_v2")

Each version is its own committed run with its own strategy source and hash. You do not overwrite version 1 to try version 2; both stay in the store.

Compare Committed Runs

Label each version with what changed, then compare them side by side. The store and comparison surfaces are taught in vignette("experiment-store", package = "ledgr").

snapshot <- snapshot |>
  ledgr_run_label("momentum_v1", "20-bar momentum") |>
  ledgr_run_label("momentum_v2", "20-bar momentum above the 50-bar trend")

ledgr_run_compare(snapshot, run_ids = c("momentum_v1", "momentum_v2"))
# ledgr comparison
# A tibble: 2 x 11
  run_id      label  final_equity total_return sharpe_ratio max_drawdown n_trades win_rate
  <chr>       <chr>         <dbl> <chr>               <dbl> <chr>           <int> <chr>
1 momentum_v1 20-ba~       10432. +4.3%               1.55  -1.7%              29 55.2%
2 momentum_v2 20-ba~       10250. +2.5%               0.983 -1.9%              27 48.1%
# i 3 more variables: valuation_state <chr>, assumed_terminal_outcomes <int>,
#   reproducibility_level <chr>

# i 2 runs attempted: 2 done, 0 incomplete, 0 failed.
# i Only DONE runs are compared; incomplete and failed runs stay listed by ledgr_run_list().
# i Fill timing comparable: yes (same_timing_convention).
# i Full identity and telemetry columns remain available on this tibble.
# i Inspect one run with ledgr_run_info(snapshot, run_id).

Here the trend filter did not help: version 2 finished lower with a slightly deeper drawdown. That is still useful evidence, and it stays in the store next to version 1. Read the comparison as evidence about two rules on one window, not as a verdict: both Sharpe ratios are annualized from one year of three demo instruments, so they describe this sample more than the rules. Picking the better of two code versions is already a selection: the more versions you try on the same data, the more room there is for luck to look like an improvement.

When A Parameter Question Appears

Sometimes the next question is narrower: is 20 bars the right lookback, or 50 the right trend length? That is a parameter question, and it has its own tool. Declare the candidates as a grid, sweep them, and promote the one you choose; vignette("sweeps", package = "ledgr") teaches that step, including failure rows and how to reopen a promoted run later.

Keep the two kinds of iteration apart. Code changes are committed runs you compare deliberately. Parameter questions are sweeps over a declared grid. Mixing them, for example by editing the rule between sweep candidates, makes the evidence impossible to reconstruct.

Select, Then Validate

Choosing a version or a promoted candidate records a decision; it does not prove the rule generalizes. Two articles test that decision:

Write The Human Research Note

Do not leave the reasoning only in your head. Write a short report next to the stored artifacts. A compact report should include:

  • hypothesis and data window
  • snapshot hash and data-source assumptions
  • each code version and why it changed
  • feature and strategy declarations
  • candidate grid summary, if you swept
  • the ranking or selection rule
  • issue and failure review
  • equity and drawdown plots
  • promotion note, if you promoted
  • reason for rejecting alternatives
  • selection caveat: the chosen version is not statistically validated by selection itself

Here is the shape of a useful entry:

Hypothesis and data window:
  Short-term momentum may capture persistent moves in DEMO_01 to DEMO_03
  over the 2019 demo window.

Iterations:
  v1 bought every positive 20-bar return; v2 added a 50-bar trend filter after
  v1 bought rallies inside falling markets.

Selection caveat:
  Both versions were compared on the same window. The choice is recorded, not
  validated; walk-forward evaluation is the next step.

The report is where the human reasoning lives. ledgr records what happened, which inputs were used, and which runs you compared or promoted; it does not certify that the research protocol was statistically sound.

Where Next