Sharpe, Sortino, Calmar Compared, and How Martingale Fools Them

Risk-adjusted performance means dividing the return by the risk you carried to earn it. Two accounts with the same profit can feel completely different to hold, and carry very different chances of collapsing along the way. Sharpe, Sortino, Calmar, Recovery factor and the Ulcer index all try to capture this, but each one “punishes” something different, and each one can be gamed in a different way. This article computes every one of them on the same example accounts, then simulates how good a martingale system can make them look before the day it blows up.

Two accounts, same profit

Take two 10,000-dollar accounts with these monthly returns (%) over 12 months. The numbers are made up for illustration, not results from a real system.

Month	1	2	3	4	5	6	7	8	9	10	11	12
A smooth	2.0	−1.0	1.6	0.4	−0.8	1.9	1.0	−1.4	2.1	0.6	1.5	1.4
B lumpy	6	−4	5	−7	3	8	−5	2	6	−3	4	−4.3

Compounded, both finish at about +9.6% (A at 10,962 dollars, B at 10,961), but they got there very differently.

Every figure computed (no risk-free rate subtracted, 0% target for Sortino):

Measure	A smooth	B lumpy
Average monthly return	0.775%	0.892%
Monthly standard deviation	1.232%	5.204%
Annual Sharpe	2.18	0.59
Annual Sortino	4.90	0.99
Max drawdown	−1.40% (147 dollars)	−7.00% (748 dollars)
Calmar (this year)	6.87	1.37
Recovery factor	6.53	1.29
Ulcer index	0.55	3.47

Look at the first row. B has the higher average monthly return, yet compounds to the same result. Volatility eats compound growth by itself (volatility drag): a 7% loss needs a 7.5% gain just to get back. Try other numbers in the drawdown recovery calculator.

Sharpe ratio: punishes volatility both ways

Sharpe = (average return − risk-free rate) ÷ standard deviation of returns

William Sharpe proposed it in 1966. The denominator is total volatility, so a big winning month counts as “risk” just as much as a big losing month.

A, step by step:

1. Average monthly return = 9.3 ÷ 12 = 0.775%
2. Standard deviation (sample version, dividing by n − 1) = 1.232%
3. Monthly Sharpe = 0.775 ÷ 1.232 = 0.629
4. Annualised × √12 = 0.629 × 3.464 = 2.18

B gives 0.892 ÷ 5.204 = 0.171 × 3.464 = 0.59.

Annualising multiplies by the square root of the number of periods per year: × √12 = 3.46 for monthly data, × √252 = 15.87 for daily data (about 252 trading days a year). A daily Sharpe of 0.1, for example, is about 1.59 a year. That scaling is only correct when each period’s return is independent of the others, which is exactly where it can be gamed. We come back to this under serial correlation.

As for the risk-free rate: subtract it before comparing with a fund or a deposit. When comparing versions of your own system, 0 is fine, as long as every version uses the same.

Sortino: punishes only the downside

Sortino swaps the denominator for downside deviation: only the months below the target (0% here) count. Square them, average over all months, then take the square root.

A: the losing months are −1.0, −0.8 and −1.4.

1. Squares: 1.00 + 0.64 + 1.96 = 3.60
2. Divide by 12 months = 0.30, square root = 0.548%
3. Monthly Sortino = 0.775 ÷ 0.548 = 1.415 × √12 = 4.90

B: the losing months are −4, −7, −5, −3 and −4.3. The squares add to 117.49, ÷ 12 = 9.79, square root 3.129%, giving 0.892 ÷ 3.129 × √12 = 0.99.

The upside is that big winning months aren’t punished. The downside is that with very few losing months the denominator gets tiny and the ratio shoots up, and with no losing months at all it can’t be computed. A system that “rarely loses, but loses big when it does” gets its best Sortino right up to the day of that loss.

Calmar and MAR: punish the deepest hole

Calmar or MAR = compound annual return ÷ max drawdown (%)

The usual difference is that Calmar is often computed over the latest 36 months, while MAR uses the account’s whole history. With one year of data they’re the same here.

• A: 9.62 ÷ 1.40 = 6.87
• B: 9.61 ÷ 7.00 = 1.37

This asks the question an account holder actually feels: “for this much a year, how deep a hole do I have to sit through?” But it rests on a single data point, the deepest hole. How many other holes were almost as deep, or how many months you were stuck under water, Calmar can’t see. And the deepest hole tends to get deeper the longer the period, so comparing a 2-year test with a 10-year test directly isn’t fair.

Recovery factor: net profit in multiples of the hole

Recovery factor = net profit ÷ max drawdown (money)

• A: 961.62 ÷ 147.23 = 6.53
• B: 961.17 ÷ 747.94 = 1.29

This field is in the MT5 report (every field of the report explained). Unlike Calmar, it uses total profit for the period, not profit per year. For a profitable system, a longer test tends to raise it on its own, because cumulative profit usually grows faster than the depth of the hole. So only compare it over the same period, and know which drawdown it divides by. Divided by balance drawdown, a system that holds on to losing trades looks better than it is. See Equity vs balance drawdown.

Ulcer index: punishes depth and duration

Peter Martin designed the Ulcer index to measure the “pain” of holding an account. For every period, measure how far below the previous peak you are in %, square it, average, then take the square root.

B: the monthly distances below the peak are −4, −7, −4.21, −5, −3.10, −3 and −4.30 (the other months are 0).

1. Squares: 16 + 49 + 17.72 + 25 + 9.61 + 9 + 18.49 = 144.82
2. Divide by 12 = 12.07, square root = 3.47

A gives 0.55 (by coincidence the same as its downside deviation, because every losing month of A was followed by a new high the next month).

A 5% hole that lasts 6 months scores a higher Ulcer index than a 5% hole that recovers in a month, a difference Calmar can’t see. Dividing return by the Ulcer index gives the Martin ratio (A 17.6, B 2.77). The weak spot is that it needs frequent enough data. Measured monthly, a hole that opens and closes within a month vanishes from the formula.

Sharpe in MT5 isn’t the Sharpe in a fund report

The MT5 backtest report does have a Sharpe Ratio field, but we don’t confirm its internal formula. What is certain is that a Sharpe computed from “per-trade results” and one computed from “daily or monthly returns” sit on different scales, for three reasons:

• Trades are held for different lengths of time. A 5-minute trade and a 5-day trade each count as one
• Time with no open trade isn’t counted, even though the capital is still sitting there
• The number of trades per year isn’t fixed, so multiplying by √(trades per year) has no solid footing the way √252 does

Since we don’t know which period and reference rate the MT5 figure uses, only compare it between MT5 tests with identical settings. To compare across software or with a fund, export daily equity and compute every figure yourself with the same formula.

Martingale: a beautiful Sharpe until it blows up

Grid and martingale systems make small, steady profits and hide the risk in rare events. Any ratio built from past data can’t see what hasn’t happened yet. We simulated it with these rules:

• Price takes random steps of 1 pip up or down, 2,500 steps a day. No trend, nobody has an edge
• 10,000-dollar account, first trade 0.01 lot (0.10 dollars per pip) in a random direction, 1 pip of spread paid on every trade
• Every 20 pips against, open a new trade at double the lot. Close the whole basket once its combined profit reaches 1 dollar
• Forced close (stop-out) when equity falls below 2,000 dollars. Equity is measured at each day’s close, including floating profit and loss
• 300 runs of 3 years each (756 trading days)

Results:

1. 175 of 300 runs (58%) blew up within 3 years. 87 blew up in the first year. The median blow-up came on day 256
2. Average final equity across all runs was 7,881 dollars, below the starting balance. The system loses on average, as it should
3. 247 runs were still alive at month 6. Their median annual Sharpe at that point was 2.28, and 79% had a Sharpe above 1
4. Of the runs with a Sharpe above 1 at month 6 (194 runs), 91 blew up within 3 years
5. The runs that survived the full 3 years (125) had a median Sharpe of 1.26 and a +64.5% return. These are the ones that get shown off

The run in the chart shows one more important thing. The numbers change with what you measure, even for the same system over the same 411 days before the blow-up:

Measured on	Annual Sharpe	Max DD
Daily equity (including open trades)	1.47	−6.6%
Equity every 21 trading days (about monthly)	15.57 (not one losing month in 19)	–
Daily balance (closed trades only)	37.6	0%

Balance never fell, because every basket closed in profit while the losing trades were simply held. Sortino and Calmar on balance can’t even be computed (the denominator is zero). That’s why you look at equity, and look often, when evaluating copy-trading signals.

Every ratio in this article is computed from what has already happened. A system that sells tail risk scores its best numbers in the stretch before the tail arrives.

Serial correlation: when √12 overstates

Multiplying by √12 or √252 assumes each period’s return is unrelated to the others. If a good month tends to be followed by another good month (positive serial correlation), real annual volatility is higher than √12 implies, so the simple annual Sharpe comes out too high. Andrew Lo explained this in “The Statistics of Sharpe Ratios” (2002), with a correction:

Correct multiplier = q ÷ √[q + 2 × Σ (q − k) × ρₖ], where q is the number of periods per year and ρₖ is the autocorrelation at lag k

With zero autocorrelation this is exactly √q. Here is the true multiplier relative to √12 when autocorrelation decays step by step (AR(1)):

Lag-1 autocorrelation	−0.2	0	0.1	0.2	0.3	0.5
True multiplier ÷ √12	1.20	1.00	0.91	0.83	0.75	0.61

We simulated how easily this happens. We drew independent “true” monthly returns averaging 1% with a 4% standard deviation for 20,000 years, then built “reported” returns as the average of the latest 3 months, like profit and loss that gets recognised slowly.

• True returns: Sharpe × √12 = 0.87, matching the Sharpe computed from actual annual returns, 0.87
• Smoothed returns: Sharpe × √12 = 1.51, but the Sharpe from actual annual returns is 0.91. Lag-1 autocorrelation 0.66
• Lo’s correction gives 0.905, back close to the truth

In trading, this happens when you measure on balance and losing trades are carried across periods to be closed later, or when month-to-month results run in streaks. The other way round, a system whose periods alternate good and bad (negative autocorrelation) gets a simple figure below the truth. With enough data, the most direct fix is to compute Sharpe from annual returns, or measure the autocorrelation and adjust.

Sharpe itself is uncertain

Lo (2002) also gives the standard error of the Sharpe ratio for independent returns, roughly √[(1 + Sharpe² ÷ 2) ÷ number of periods]. For an annual Sharpe of 1.0:

Data	Standard error	Rough 95% range
1 year	1.22	−1.40 to 3.40
3 years	0.71	−0.39 to 2.39
10 years	0.39	0.24 to 1.76

Using daily instead of monthly data helps little, because the precision of an annual Sharpe is driven mainly by the number of years. A system with a Sharpe of 1 over 3 years still can’t be told apart from one with nothing. And if you pick the best of many variants, the figure inflates further. More in Data snooping and How many trades is enough.

Rough benchmarks

These are rules of thumb people commonly use, not laws, and they only apply to multi-year results after costs.

• Annual Sharpe from daily equity: below about 0.5 is thin, about 1 or more is considered good. If a multi-year backtest shows more than about 3, first look for what’s hidden, such as settings fitted to the past, lot doubling, or measuring on balance
• Recovery factor: as used in our MT5 report article, if a multi-year test comes in below about 2, the profit for the whole period is only about two holes deep
• Calmar / MAR: better used to compare versions over the same period than as a fixed line, because the deepest hole depends on the length of the period
• All of them: an unusually high figure is a signal to check, not a signal to celebrate

Which ratio for which question

Question	Best fit	Punishes	Gamed by
Return per unit of volatility, compared with other accounts	Sharpe	Volatility in every direction	Selling tail risk, smoothing returns, sparse measurement, picking the period
Systems that earn in rare big chunks (trend following)	Sortino	Only periods below target	Systems that almost never lose until the day they lose big
How deep a hole for each year’s return	Calmar / MAR	The single deepest hole	Short test periods, measuring on balance
Does total profit justify the hole, in an MT5 report	Recovery factor	The deepest hole in money	Longer tests, dividing by the wrong drawdown
How long does holding it hurt	Ulcer index	Depth × duration of every hole	Monthly or balance-based measurement

None of them sees risk that hasn’t happened yet. For a system whose position size grows after losses, check the rules, not the numbers. For the risk of an account breaking, read Risk of ruin.

Common mistakes

• Comparing Sharpe ratios measured differently. The Sharpe in an MT5 report and a fund’s monthly Sharpe can’t be compared. Compute both yourself with the same formula
• Using balance instead of equity. A system that holds on to losing trades gets zero drawdown and a sky-high Sharpe, as in the simulation above
• Trusting figures from a short period. A Sharpe of 1 from a single year has an uncertainty range from below zero to above 3
• Looking at one ratio. Each one is blind to something. Use Sharpe together with equity drawdown and the Ulcer index, three angles at least
• Comparing Recovery factor or Calmar across different periods. A longer period has deeper holes and more cumulative profit. Compare over the same period
• Not asking where a beautiful number comes from. If a figure looks better than it should, check whether the system adds lots after losses or holds losing trades before you look at anything else