By Alton Cogert | firstname.lastname@example.org
But, that’s OK, because so is ours.
You see, British statistician George Box put it best when he wrote in 1976, “All models are wrong, some are useful.”
Of course, this has huge implications for your company’s investment strategy, to say nothing about even more encompassing issues such as enterprise risk management.
“If your asset allocation model is wrong, your company is its own worst enemy since you can be producing much better investment results…you just may not know it.”
If your asset allocation model is wrong, your company is its own worst enemy since you can be producing much better investment results…you just may not know it. Please don’t forget that neither GAAP nor statutory accounting has an expense called ‘opportunity cost,’ but it is a real economic cost nonetheless.
Importantly, if all models are wrong, how do we know if the model we are using is useful?
Here are a few important tests you can apply to your model. And, yes, you can apply these to other models being used to aid in decision making at your company.
1. How long has this model been around?
Although a long-used model is not necessarily the most important characteristic of a useful model, the fact that many do find it worthwhile should cause you to think a bit favorably about its usefulness. Of course, just because everyone has used the same model for years does not automatically make it useful.
2. Does it accurately reflect the real world?
Please be careful here, since it is all too easy to assume the model does reflect the real world, while it may be too simplistic to be a good reflection.
For example, this can be found in asset allocation models that are steeply grounded in statistics, yet do not consider the fact that future asset performance has a lot to do with how groups of humans view those assets’ relative return, risk, etc. This is where the confluence of traditional finance and behavioral finance meet and battle for their importance in making an asset allocation decision. In other words, we are not the completely rationale economic beings assumed in many economic models, so why should we automatically accept the results from models that assume that is so.
3. Does it apply well to all assets under consideration?
For example, an asset allocation model that relies upon standard deviation of expected return as a measure of asset risk may be using a good approximation for asset classes where the distribution of expected returns approximates normal or log normal distributions.
But, for asset classes whose returns are not expected to follow anything close to such distributions, using standard deviation would fall under the category of ‘model malpractice’.
You may have heard of the terms ‘skewness’ and ‘kurtosis’, which we will not detail here. However, they can add materially to the depth – and potential usefulness – of your model. Asset classes where this can be an important factor include some structured securities, convertible bonds, hedge funds, private equity and others.
4. Is the model based on first principles?
A first principle is a foundational concept which cannot be deduced from any other concept and must be known. For example, a foundational concept would be a positive relationship between risk and reward (more of one means more of the other). A non-foundational concept might be the assumption that currency risk should always be hedged, as there are academic studies that support both the pro and con of that assumption.
5. Does the model require multiple regress of its principles?
Multiple regress means a model justified by principles, which themselves require justification by other principals, etc. The further model assumptions and structure are away from first principles, the more difficult it will be to assume it is a useful model. Although, careful analysis of this chain of principles may prove it is a useful model.
6. Can you easily explain the assumptions, structure and output of the model in layman’s terms?
This is perhaps more basic than even the other very basic tests of usefulness noted here. Remember that if you are unable to explain the model to a Board of Directors that may not have your knowledge, expertise or point of view, the model will not be accepted no matter how less wrong or more useful it may be. Some good advice here comes from Denzel Washington’s character in the movie “Philadelphia.”
Where does this leave our search for a useful asset allocation model?
It leaves us with a sense that there is more to making a good asset allocation decision than what is derived from models that do not consider at least some of these tests to determine a useful model.
Importantly, it leaves us in a position that doesn’t automatically accept the models provided by so-called experts, but humbly understands that our models are likely wrong, but which one is most useful?