By Steven Davis, CSI Developer
Original 5/17/99
Step 1: Selecting your market
list.
Step 2: Choosing the Data
Series.
Step 3: Choosing the Data
Lags.
Step 4: Choosing the Indicators
Parameters.
Step 5: Correlation of the
Data.
Step 6: Factoring the Data.
Step 7: Setting the parameters
for the Leadership MMA System.
Step 8: Analyzing the Current
Situation.
Step 9: Historical Validation.
Step 10: Developing a Trading
Strategy.
CSI's Unfair Advantage is an unprecedented foundation for Inter-market Analysis by providing virtually every market in the world. CSI's new Multi-Market Analyzer provides a set of new and unique indicators. These indicators describe the market situation, not by staring at the same chart day after day, but by determining the underlying economics and the current market forces.
This installment explains how to use the Multi-Market
Analyzer and how it works in detail. Let us step through a typical session:
selecting the markets to analyze, factoring the data set, evaluating the
multi-market indicators, and determining what these indicators are telling
us.
Step 1: Selecting your market list.
The Multi Market Analyzer can be launched from within UA by choosing the main menu option Database-MMAnalyzer. After dismissing the welcome screen, you are asked which markets to include in the analysis. When selecting markets you should consider the following facts: 1) Analysis will only be done on the inclusive date range. (Date range in common to all selected markets.) 2) Unrelated economically unrelated markets will not make good predictions. 3) Only select as many markets as your computer's memory can hold. Generally each market will consume about 300k of computer memory. If your computer starts using virtual memory, the processing will be very slow.
While every market in the UA database can be included in your analysis, the following list contains just those markets that have history back to 1985 with all highly similar and inactive markets removed.
AG BO BP C CC CD CL CT DJI DJU DM DX ED FC FFI FLG FSS GC HG HO HU IR0 IR1 JDC JRB JY KC KV LB LBA LC LH M09 MNI1 MPB1 O OJ PA PB PL RS S SB SE SF SP TB US W WF YBA YIX YTC
This list is readily available if you open up the selection drop down menu.
When using neural networks, it is common to removed variables as the neural network finds them to be irrelevant. If you run the Multi-Market Analyzer on all of the above markets and apply some advanced techniques, you find that the least significant market, in terms of explaining other markets, is better than 20% of the most explanatory market. In such a circumstance, there is no mathematical reason to remove any further markets.
However, there is a computer performance reason. So for this article, let’s suppose you are wondering what the interest rates/currencies are going to do. An appropriate selection of markets might be: Eurodollar (ed), Dollar Index (dx), S&P 500 Index Futures (sp), US Treasury Bills (tb), Japanese Yen (jy), US Treasury bonds (us), and FTSE 100 Index (ffi). For repeated analysis, it is often easier to drop down the list and choose your previous selection. Since the commodity and stock symbol universes are separately unique, but have many symbols in common, it is sometimes necessary for MMA to guess which market you mean. To assist MMA, select in the market type drop down which symbol universe to search first. If you want to specify, for any given market, which symbol universe to use, explicitly add a "(C)" or "(S)" after the symbol (i.e. sm(C), ibm(S).) ("ED DX SP TB JY US FFI")
One word of caution: do not choose markets that are very
highly correlated. An example might be the same commodity at two different
exchanges that close at the same time. The Multi-Market Analyzer will successfully
identify an arbitrage situation, but if both series are kept in the analysis,
the trade allocation between the exchanges may be inconsistent over time.
Furthermore some of the indicators will not give interesting results for
these markets.
Step 2: Choosing the Data Series.
After selecting the markets to analyze, you are asked which continuous time series you would like to use. We recommend a non-detrended weekly Perpetual Contract® Data Series. You can select whether or not to detrend, the periodicity of the data, and indicate if back adjustment is preferred.
Weekly data is recommended. We are making judgements based on inter-market relationships that take time to evolve. A one-day rise in the corn price relative to the soybean price will neither change supply nor demand. Monthly data tends to be too slow to show trend changes, and has too few bars of data to get trustworthy results.
Notice that the inclusive date range has been calculated for you. If you wish to include data outside of that range, you must remove or exchange the limiting data series to obtain common data for the date range preferred. In our example, the earliest common start date is displayed as "19851120 (DX)". This is because the DX series (dollar index) begins on November 20, 1985. If I want to start earlier, I must discard the DX series contribution or exchange it with a series that goes back further. If you do not see a symbol listed, it is because all of our selected series begin or end, as appropriate, on the same date.
The date range may well have to be reduced so that historical
relationships in the data persist today. For example, the Green Revolution
changed the level and character of the agricultural markets so including
earlier data is likely to worsen your results. We recommend applying our
detrending facility, but the investigator must compromise between having
a consistent economic period and having enough data to get a meaningful
result.
Step 3: Choosing the Data Lags.
Now that you have the series chosen, you are asked how to merge these series. For beginning users, these options are rarely used, but for those economists out there, here is your chance to show off.
Economists tell us that lagging the data is sometimes necessary. A principle example is the Live Hog and Corn markets. Farmers would rather have some return on investment than none. So even if the price of corn goes up, farmers will still, typically, continue forward with their current livestock. In fact, if the corn prices go high enough, hog farmers might even slaughter more hogs than intended pushing the hog prices down. It is only in the new crop of hogs that the old crop of corn prices are directly felt. Thus in studying hog and corn prices, a lag to the corn prices must be used to meaningfully understand the hog prices.
As an aside, the actual perpetual market data does not demonstrate this. It shows that the current corn price has the highest correlation with the current hog price. The second highest correlation is for a lag between 9 and 10 months. Even so, you may find it useful with your markets.
Since perception of prices often matters more than the prices themselves, a series may be replaced with its seasonal index. The seasonal index employed is a 32-bit port of the one available under Unfair Advantage as written by Bob Pelletier.
In case you included an index market to get the index,
a check box is provided so that you can include the cash price series instead
of the Back Adjusted or Perpetual series previously chosen. A word of warning,
however, the date range presented and validated is based on the futures
data. The cash data may have a different range. If the cash series is not
present through out the date range, the gap is treated as a series of holidays
and is filled in with the last price.
Step 4: Choosing the Indicator Parameters.
Before we can proceed with analyzing the data, you get the option to customize the parameters used in determining the Davis MMA indexes described below.
The first option of applying exponential smoothing allow MMA to discount earily history where macroeconomic relationships may have been different. Larger values or unchecking Exponential Smoothing result in larger Davis Stretch Index values and greater disparity between the current price and the Davis Unstretched Index. Small value produce the opposite values. Too large values result in huge stops. Too small values result in whipsawing.
The second option determines how smoothed the Davis Leadership Index is. In general the day-to-day movements towards and against the leadership direction cannot be used to measure Leadership. Rather it is the average tendency of the markets to follow that can be used to measure Leadership. A value too large will make the system unresponsive to changes of leadership. A value too small us likely to result in lots of small losing trades as the system is switching between markets from one bar to the next.
The third option determines how many bars at the begining
need to be discarded so that the correlation matrix has converted to its
correct value. If the value is too small, the early trades will be
unfavorable, and if the value is too large, your historic testing will
be limited by the number of trades from which validation is to be obtained.
Step 5: Correlation of the Data.
When your data selections are complete, the Multi-Market
Analyzer will retrieve the series selected from the Unfair Advantage database
directly. This could take a few seconds or several minutes, depending on
what has been chosen.
Press the "Raw Correlation" button when the data is retrieved to display the correlation coefficient table. The correlation coefficient between two markets is a value between +1 and -1. A large positive value means that the two markets tend to be high at the same times and low at the same times. In our example, Eurodollar and Treasury Bonds have a correlation coefficient of 98.1%; Dollar Index and the Deutschemark have a correlation coefficient of -92.4%; and, US Treasury Bonds has a correlation coefficient of just -2.3% with the Dollar Index. A perfect value of 1 or -1 is an arbitrage situation.
Some consider this table quite interesting, but it is
easy to miss the forest for the trees when doing a large analysis. It is
too easy to misinterpret common effects of a third, unstated factor, and
it is too easy to make convincingly logical falsehoods. Such as, if corn
were to rise, soybeans should rise too; live cattle should rise; consumption
of cattle should fall; production of cattle should fall; and finally, the
price of corn should fall. Even if each step in a five-step argument held
80% of the time, a priori, the conclusion should only hold true (.85=.328)
about 30% of the time.
Statisticians have faced this for quite some time. One statistical technique is to apply eigenvalue analysis to the raw correlation coefficient table.
Suppose that you wished to screen applicants based on personality. You could come up with a questionnaire with lots of questions. Knowing whether applicants prefer coloring to drawing is interesting, but you would like to know if they are jovial, talkative, supportive or combative. You want summary, understandable traits. The situation is analogous to studying the S&P index rather than tracking 500 stocks of your choosing. The index gives us the broad picture where the consistent returns are too be found.
The traits should be independent. Having two different stock indices for the same industry is just confusing unless they really tell you something different. Knowing that the applicant is kind, generous, loving, honest, and sensitive is good, but these traits all go together. For the purposes of qualifying applicants, we might as well lump them into a general Goodness score.
Through eigenvalue analysis, we can take our correlation coefficient table for seven markets and automatically produce seven independent indices. The indices (factors) as a whole completely capture the correlation information inherent in the data. The technique takes 500 stocks and makes 500 unique indices losing nothing. The indices are ranked according to how much variance the index has had over time. Usually, there are a couple of indices that contain, or explain, most of the variance. The remaining low variance indices explain little of the variance, but are very powerful at prediction.
Closing the Correlation Coefficient Table, we see the factors identified in our example.
The variance of each index is measured two equivalent ways: 1) by the percent of the variance contained, 2) by the number of markets which can be replaced, without loss, by including the factor. In our personality test example, the Goodness factor fairly replaces the variables of kindness, generosity, loving, honesty, sensitivity, and (with a negative weight) jealousy.
At the top of the list, we have indices with high variance. The indices tell us, generally, what the market is doing. A dramatic change in one of these indices would require many underlying markets to change drastically. Just as with the S&P Index, the indices behavior is fairly insensitive to the exact weighting used. As a result, using these factors for prediction tends to work well.
At the bottom of the list, we have indices of low variance.
These are like arbitrage opportunities. These are spread trades of low
volatility. In our example, the last index is an example. Where the spread
between the Eurodollar futures and the Treasury Bills widens, it doesn't
widen much and will probably return quickly. These economic facts are essential
to a useful prediction of either, but were that relationship to break,
as the Brazilian Real-US Dollar peg did, the predictions based on this
index's historical variance would be worthless. This is a perpetual quandary
for the investor, when is a trade a deal-of-a-lifetime and when is it a
disaster-in-the-making. This software does not answer such question, but
it does make it clear when a choice must be made.
Step 7: Setting the parameters for the Leadership MMA System.
MMA comes with a system based on the Davis Unstretched Indicator and the Market Leadership Index. These indicators are fully described in the next section, but it is at this point that you are asked what parameter values you wish to use. Normally 0.5, 2, and 4 produce good results for most any market and are the defaults.
You may wish to try other values for the fourth parameter. If you are using daily data, then you most certainly do not want to reenter a trade at the open when you just closed it on the previous close. For weekly or monthly data, it is often better to let the system stop-out during the week and reenter the trade from fresh the next week if the signal persists. This option inhibits reentering a trade after it is stopped out.
To improve upon these settings rerun MMA several times
with different values to see what works the best for you. For now, press
the Next button.
Step 8: Analyzing the Current
Situation.
Once you have determined to your own satisfaction which factors represent real market relationships, we can proceed to analyze the current market situation.
The Multi-Market Analyzer next computes the Davis Stretch Index, the Davis Unstretched Indicator, and the Davis Market Leadership Index to give us a sense of the current market situation.
These indicators assume the data can be modeled by a multivariate
normal distribution with the factors selected. This assumption is usually
false, but often useful. As I discussed in Step 5, be careful about including
distinctly different time periods and selecting low-variance factors. In
either case, you may be requiring MMA to explain current market behavior
from an outmoded perspective.
Davis Stretch Index
If all of the markets fully accounted for all economic factors, commodity trading would be just gambling. The Davis Stretch Index gives us a measure of how much misalignment is in the marketplace. Is the current cattle price too high for the current corn price and the current oil price? Or is it exactly where it should be?
In the assumed normal distribution a value of 3 should only occur about 0.3% of the time. In practice average values of 4 or even 5 are tolerable so long as the results appear to be useful.
Trying to interpret this index can be a little tricky. A value of zero is minimal and means that the markets are in harmony; no spread trades should be profitable. High values of the index mean that major market movement is happening. The other indicators help to sort out which markets are going where.
Davis Unstretched Indicator
The Davis Unstretched Indicator is a bit different. It tells you, for each market, where the price would have to go if all of the other prices remain at their current levels. Suppose you were analyzing corn, soybeans, and live cattle. If the corn prices are very high and the live cattle prices are very high, the "Unstretched" price for soybeans will be very high as it tries to accommodate both the corn price and the live cattle price without regard to its own economics. It measures how far each market would separately have to move to absorb all of the underlying market forces itself.
Don’t be too surprised if the Davis Unstretched Indicator is several large points away from the current price. The group, as a whole, will equilibrate with each member doing some of the accommodating. The problem for a trader is for the trader to predict who is going to compromise. If, however, the spread between Davis Unstretched Indicator and the actual price widened or closed abruptly at an abnormal peak in the Davis Stretch Index, then perhaps the market factors didn't just stretch, but they snapped. That is, that there was an underlying economic change. In this event, try reducing your date range to exclude this event and see if the spread between the Davis Unstretched Indicator and the actual price is more consistent. If the resulting date range is just too small to use, then either wait for the new economic model to mature, or proceed as best you can with a broken model.
A word of warning: These indicators work best with lots of markets. If you use just two markets, the factors are the average of the two markets and the spread of the two markets. The Davis Unstretched Indicator reduces to
where x1
is the price of product 1, m1
is the mean price of product 1, r is the correlation ratio, and
s1
is the volatility of product 1. s is either r, +1, or -1 depending
on what factors were chosen and whether r is positive or negative. If the
markets get highly out of line, the "Unstretched" values of product 1 still
strongly mimic value 2 or a mirror image of value 2. This makes the indicator
appear to be for the wrong variable and possibly in a weird range. This
is a consequence of using too little data. It is like having a couple of
identical twins and trying to predict changes in one by looking at the
other. If one gets a scar on their left cheek, you will predict the other
will get a scar on their left cheek. If, instead, you try to predict what
your teenager is going to do based on what all of the teenagers in your
town are doing, you’ll have better predictive success. There is the propensity
for scarring, but you are not fooled into making unrealistically precise
predictions.
Davis Market Leadership Index
The Davis Market Leadership Index is the piece that completes the puzzle. The Davis Stretch Index tells us that somebody is going to move. The Davis Unstretched Indicator tells us what would happen to a market's price if it followed the other markets. But not every market follows, some lead.
When the OPEC Oil Crisis drove the price of oil up, no one knew whether the American consumers would reduce consumption enough to make the OPEC group up, or whether the American public gave in and paid more. The answer was that the American public gave in, and got used to paying more for everything. The oil prices lead the economic change. The corn and cattle prices followed the economic change.
The value of the Davis Market Leadership Index measures in days, weeks, or months the recent historical tendency for the Davis Unstretched Indicator or lead or follow the market. Large positive values mean that in the recent past the market tended to get ahead of the market changes just as the oil price pulled up the agricultural prices. A negative value means that the market recently has been slow to accommodate economic changes. A value near zero means that the market, in recent history, has promptly fully discounted all factors.
On leading and fully discounted markets, trend following
may be profitable, but spread trades are likely to be risky and unprofitable.
A trailing market gives the trader the advantage of plenty of time to get
in, but it can be a long time to show a profit. (For example, currency
support should eventually collapse, but a national treasury can hold out
along time.)
Step 9: Historical Validation.
If you see anything interesting developing, historical validation is in order. By pressing the "Next" button, the Multi-Market Analyzer displays historic graphs of all the above indicators applied to the first market. By checking and unchecking the items in the legend, you can choose which series and indicators you wish to see. Selecting a market to see the scale to appear on the left side. The commodity price appears as a solid line, the Davis Unstretched Indicator as a dotted line, and the Davis Market Leadership Index as a dashed-dotted line.
The Multi-Market Analyzer also provides a CSV file "mmanalyzer.csv"
in the UA directory. This file contains the price and indicator values
historically. These indicators require a few bars to get started so some
empty cells are expected. Further the Davis Market Leadership Index uses
a centered statistic so it is not available for the last few bars.
Step 10: Developing a Trading Strategy.
Aside from using the built-in Leadership MMA System, you may find other ways to get useful market timing. The best way to decide how, and if, to use these tools is by applying them to markets you find interesting and look to see historically. What happens when these markets get out of line? Does the Davis Stretch Index find the historical major events? Do the interpretations of the Davis Unstretched Indicator and the Davis Market Leadership Index make sense in your markets? If not, you may have to examine which markets are relevant and what time period to analyze. If you do find them applicable, enjoy the power of having the Unfair Advantage database at your fingertips.