Point-in-Time Backtest: What Testing a Quality-Value Screen Taught Us

Our Point-in-Time Backtest by the Numbers

What a point-in-time backtest is

A point-in-time backtest reconstructs, for each historical date, only what an investor could actually have known on that date. That sounds like a low bar, but most backtests quietly fail it. In a point-in-time test the stock universe is the set of companies that were members of the index on that date, and the fundamentals are the figures that had been publicly filed by that date, not the figures as they look today. This article explains why that matters, and what we found when we applied it to the quality and value screen we use in our Quality Tilt strategy.

The headline result is a humbling one. Nothing we tested was statistically significant, the more weight we gave to value the worse the screen did, and the best-scored group of stocks had the lowest return of all five. The method is the more useful lesson, so we start there.

Why ordinary backtests overstate results

Two biases do most of the damage, and both are easy to introduce by accident.

Look-ahead bias means using information that was not yet public. Illustrative example: a company’s fiscal year ends December 31, and it files its annual report at the end of February. A backtest that rebalances on December 31 but uses that annual report’s figures is trading on numbers nobody had yet. Databases that store fundamentals by the period they describe, rather than the date they were filed, make this mistake the default. Restated numbers create the same problem: today’s history may differ from what was reported at the time.

Survivorship bias means testing only the companies that exist today. Illustrative example: if you run a 2012 screen on today’s index members, every company that was dropped for shrinking, failing or being acquired in the meantime has been removed in advance, and the survivors look better than the opportunity set did. We cover this effect in more detail in survivorship bias in investing, and the broader craft in backtesting trading strategies.

Both errors push in the same direction: toward making a rule look better than it was. A point-in-time design removes them by construction, which is why we built the test this way even though the data work is considerably harder.

How we built the test

1. Universe. At each rebalance date, the universe is the S&P 500 membership on that date, from Wikipedia’s constituents list plus its change history. Companies later removed are therefore included while they were members.
2. Fundamentals by filing date. Fundamentals come from SEC XBRL company facts, filtered by filing date, so only data publicly filed before the rebalance date is used. Trailing twelve months is the latest 10-K plus the year-to-date 10-Q minus the prior-year year-to-date.
3. Market cap. Market capitalization is the split-unadjusted price times shares.
4. Gates. Financials are excluded, and a company must have positive earnings per share and free cash flow and net debt to EBITDA of at most 3.5x. These are the live screen’s own rules.
5. The real rule. The backtest runs the live scanner’s own ranking code, so it tests the actual rule rather than a re-implementation of it.
6. Schedule. We rebalance quarterly from March 31, 2012 to October 2026: 59 rebalances over about 14.5 years, with 10 basis points of cost, against SPY.

Point-in-time work is never perfectly clean. XBRL company facts are keyed by company identifier, and a holding-company reorganization creates a new one, so some names drop out of early dates (Alphabet’s facts start with 2016 filings). About 35 non-financial S&P companies do not tag operating income, so they have no EBITDA and are excluded. And when a company restates, our trailing figures can mix restated and unrestated numbers. We list these gaps because they tell you where the test is thinnest, not because they change the conclusions.

What the test found

We compared five portfolios. The fair comparison is against the “eligible universe,” meaning every name that passed the gates, because SPY’s 14.3% a year over this window is a high bar for any large-cap screen.

Every variant except the no-floor quality portfolio trailed SPY, and that one roughly matched it at +0.1% a year. Against the eligible universe, which strips out the effect of the large-cap index simply being hard to beat, only the no-floor quality portfolio showed a positive edge, at +1.5% a year.

The lesson: the more value constrained the selection, the worse it did

Line the results up by how much value was allowed to shape the selection and a pattern appears. With no value floor, the quality portfolio beat the eligible universe by 1.5% a year. With a 25% value floor it matched it, at 0.0%. The 50/50 blend returned −1.7%, the quality portfolio with a 50% value floor −2.2%, and value alone −2.5%. The more the selection leaned on value, the lower the result.

The combined-score quintiles tell the same story from another angle. We ranked the universe on the combined quality-plus-value score and split it into five groups, with Q5 the best score and Q1 the worst. Annual returns were 10.9% for Q5, 13.6% for Q4, 14.5% for Q3, 11.4% for Q2 and 12.7% for Q1. The best-scored group had the lowest return and the middle group the highest. There is no monotonic relationship: the combined score did not sort future returns. For readers new to these factors, factor investing explained gives the background, and the quality factor itself is treated in Asness, Frazzini and Pedersen’s “Quality minus junk” (2019).

There is an important qualification. This is S&P 500 large caps over 14.5 years in a market dominated by mega-cap growth, which was value’s worst stretch. Value-only underperformance partly reflects that period and says little about value over other regimes.

Is any of it significant?

No. The best information ratio against the eligible universe was 0.27. Over 14.5 years that gives a t-statistic of roughly 0.27 × √14.5 ≈ 1.0, well short of the conventional 2 and further still from the t above 3.0 that Harvey, Liu and Zhu (2016) argue new factors should clear. Hou, Xue and Zhang (2020) report that 65% of 452 anomalies fail the t above 1.96 bar once microcaps are controlled for, a reminder that even published factors often fail stricter tests.

McLean and Pontiff (2016) add a second warning. Across 97 predictors, anomaly returns were 26% lower out of sample and 58% lower after publication. A screen that shows a small edge in a sample should expect less going forward. Our own result, a small and insignificant difference, fits comfortably in that picture. The related issue of fitting rules to history is discussed in overfitting in backtesting and walk-forward optimization, and the standard way of checking factor exposure is the Fama-French model.

We changed the rule, and that is a risk too

The strategy began as a 50/50 blend of quality and value rank. After the backtest we changed it to a quality-ranked rule with a 25% value floor. We are telling you this because the change is itself a data-mining risk: we looked at the results and then altered the rule. Every such adjustment spends some of the test’s credibility.

Two facts limit the damage. The change is small, and its outcome is still insignificant: the current rule matched the eligible universe at 0.0% a year, with an information ratio of 0.01. It also did not pick the best variant in the table. The portfolio with no value floor scored higher at +1.5%, and we did not adopt it. Still, the honest reading is that the current rule was chosen after seeing this data, so its own results should be treated as in-sample, not as a forecast.

How Alpha Suite uses this

Alpha Suite’s Quality Tilt strategy ranks non-financial large caps on quality (return on assets, return on equity, operating margin) and uses value as a floor rather than half the rank. It gates on positive earnings and free cash flow and on net debt/EBITDA of at most 3.5x, and it publishes this backtest on its strategy page.

Limitations

This is one universe, one period and one rule family. S&P 500 large caps over 14.5 years is a narrow slice, and the period was dominated by mega-cap growth. Value-only underperformance partly reflects that regime, so none of this shows that value or quality will behave the same way in other markets or in smaller companies. SPY’s strong run over 2012–2026 is a demanding yardstick, which is why the eligible universe is the fairer comparison.

The data are also imperfect. Company identifier changes, missing operating-income tags and restatements remove some names, especially in the early years, and the effect on the results is unmeasured. Combined with the in-sample rule change, the right conclusion is modest: the point-in-time test removed two large sources of bias, and what remained was no demonstrable edge. This article is research, not investment advice.

FAQ

What is a point-in-time backtest?

It is a backtest that uses only the information available on each historical date: the index members of that date and fundamentals that had already been filed. This avoids look-ahead and survivorship bias, which otherwise make results look better than they were.

Why filter fundamentals by filing date?

Annual and quarterly figures become public only when they are filed, often weeks after the period ends. Using the period date instead lets a backtest trade on numbers nobody had yet, which overstates the result.

Did the quality and value screen beat the market?

Not in a statistically meaningful way. The current rule returned 13.0% a year against 14.3% for SPY and matched the eligible universe, while the best variant, with no value floor, returned 14.4% with a t-statistic of about 1.0.

Why did combining quality and value not help?

In our sample, the more value shaped the selection, the worse the result, and the best combined-score quintile had the lowest return. The period favored mega-cap growth, so that may not hold in other regimes, and nothing here is significant.

References

1. Alpha Suite. (2026). “Point-in-time quality/value backtest (October 2026).” backtest_quality_value.py, run October 3, 2026; methodology and results summarized in this article.
2. Asness, C. S., Frazzini, A. & Pedersen, L. H. (2019). “Quality minus junk.” Review of Accounting Studies, 24(1), 34–112. doi:10.1007/s11142-018-9470-2
3. Harvey, C. R., Liu, Y. & Zhu, H. (2016). “…and the Cross-Section of Expected Returns.” Review of Financial Studies, 29(1), 5–68. doi:10.1093/rfs/hhv059
4. Hou, K., Xue, C. & Zhang, L. (2020). “Replicating Anomalies.” Review of Financial Studies, 33(5), 2019–2133. doi:10.1093/rfs/hhy131
5. McLean, R. D. & Pontiff, J. (2016). “Does Academic Research Destroy Stock Return Predictability?” The Journal of Finance, 71(1), 5–32. doi:10.1111/jofi.12365