An Essay by Bob Pelletier
It is an unending study of an ever-changing subject. It is a quest that takes commodity traders and technicians deep into the history of the markets, brings them rushing back to the present and hurls them pensively into the future. Technical analysis is indeed an exciting, sometimes grueling business; one which leads its practitioners to tackle large quantities of historical data for individual commodities. Speculators demand a workable way to view the markets that simulates the perils, profits and pitfalls of actual trading. Those in the know are finding that the most meaningful results can be found in the study of "computed" contracts, which are derived from, but do not exactly mirror actual market activity. This is a discussion of the various types of computed contracts available to CSI data resource subscribers.
Let's start with the basic fact that futures contracts are relatively short lived. They are created on some date by traders on some exchange floor and eventually die when their delivery dates are reached. This birth-death process for commodity and futures contracts is an inherent characteristic that cannot be ignored. Some commodity contracts have longer lives than others. Grain contracts, for example, trade for a year or two, while financial markets may be traded six to ten years into the future. In all markets, nearby contracts (those about to expire) enjoy much heavier volume and open interest than contracts with later expiration dates. Technical traders are wary of entering illiquid markets, where order-execution slippage can take a significant toll on both actual profits and efficient order execution. Liquidity factors relating to open interest and volume, life span and distance from expiration are all-important considerations.
Nth Nearest Future Contracts
Traders of the 1950s and before were comfortable viewing a concatenation of contracts of the same commodity over time. These were created by manually splicing together the nearest portion of successive delivery months into a series covering ten, 20 or even 50 years. They could then simulate the practice of trading and viewing the more active (and most liquid) period of each successive contract to obtain a feel for trends, volatility and opportunity for profit. Many traders still prefer viewing the markets from a nearest-contract perspective. An advantage to this approach is that the most heavily traded portion of every contract viewed in the concatenated series is a representation of the actual market prices. A major disadvantage is that significant price jumps or drops (discontinuities) occur from one contract to the next which help to discredit, distort and diminish results.
Unfair Advantage accommodates this type of analysis through the Nearest Future Contract choice in its Portfolio Manager. When selecting the Nearest-Future contract option, users may select the calendar delivery months to include. Most users wish to analyze the first nearest future, but the 2nd, 3rd, etc. nearest future may be selected instead, to add distance and time from delivery risk. There is also a prompt allowing users to select the day and relative month of roll-forward. This option also adds distance from delivery risk and clarifies when to expect one contract to roll to the next.
While viewing a chart of this or any other computed series, UA users can display the price, volume and open interest information of the computed contract along with the name of the underlying contract in a movable window. This is done by clicking the Price Window (crosshair) button on the tool bar and positioning the vertical bar on any day.
Gann Contracts
Gann enthusiasts represent another trading group that is interested in simulating markets over a wider spectrum of contract history. This group views the markets similarly to the nearest future proponents, with the exception that like contracts (those with the same delivery month classification) of a commodity are concatenated. For example, a Gann time series might hold the final year of the June 1987 contract, followed by the final year of the June 1988 contract, followed by successive June contracts up to and including the most current June contract that lies within 12 months of its expiration date.
Unfair Advantage accommodates this type of analysis through the Gann Contract choice in its Portfolio Manager. First the single delivery month to be used for this Gann series is chosen. Then comes the roll date, which can be selected as any day relative to month start. This allows for rolling on, say, the first or tenth day of the month, or any date the user selects. The ability to gain more distance from delivery is available for these series by opting to roll in the calendar month prior to expiration. UA users can simply enter a 0 to roll during the delivery month, 1 to roll one month prior to the delivery month, or even higher numbers to roll earlier.
The Gann approach may be better than the nearest-future variety because there are fewer discontinuities. On the other hand, the one-year segments of a "Gann file" may be too long to yield meaningful information. What may have been learned from the distant (early) portion of each one-year segment of the time series may not readily apply to the more volatile later portion of each successive one-year series. As a contract approaches maturity, its characteristics such as volatility and trading volume gradually increase until a maximum level is reached near the end of each delivery month's contribution to the overall series. Unfortunately, the later period of each contract is likely, in a statistical sense, to show no resemblance to the relatively tame earlier period. This phenomenon suggests a lack of stationarity, a statistical property explained in the Perpetual Contract® data discussion below.
Perpetual Contract Data
In 1970, when the computer became more popular for analysis, CSI unveiled its trademarked Perpetual Contract data. This computed contract, very popular among CSI subscribers, represented a time-weighted average price of the two active contracts that lie earlier and later than a fixed number of days and months ahead of the then-current date. This method of calculation remains popular because it provides an accurate view of the market's characteristic waveform over time that is "perpetual" in nature. It is similar to the forward contracts offered by the London Metals Exchange (LME). The major drawback of the Perpetual Contract data approach is that the contracts cannot be traded directly, and can only be used as a guide for overall market direction. They are used to assist in examining long-term analysis alternatives. They should not be heavily relied upon in examining agricultural markets where different supply-and-demand conditions may affect the distinct old and new crops. An alternative to the standard Perpetual Contract data is the open interest-weighted Perpetual Contract which has a near-contract view that results from all contract prices being weighted by their respective open interest.
Advocates of Perpetual Contract data series point out that these series are more likely to exhibit statistical stationarity than, say, a Gann contract. This is particularly true when there is a long enough period from birth to death to change the contract's volatility over time. The concept of "stationarity" is simple to understand. For a serially correlated time series to be stationary (and most time series are serially correlated), the mean and variance of the series must remain statistically constant. Another significant advantage of Perpetual Contract data is that it offers flexibility to focus on near or far contracts as single independent series for analysis purposes. For example, an analyst could pair off far-forward future hogs against nearby corn (the raw material needed to produce the hogs) to study the dependent impact of these two commodities on each other.
Unfair Advantage accommodates this type of analysis through the Perpetual Contract choice in its Portfolio Manager, where many options give the user flexibility to fine-tune the study. Any or all contract months may be included in a Perpetual Contract series, but generally all active trading months (the default response) are represented over time. The Perpetual Contract data user must choose how many months ahead to view the market. Three months is the usual distance, but a two-month forward series may be appropriate for commodities that expire every month or two such as the energy products and perhaps some precious metals. Farther-out studies can also be useful as in the above example of near corn and far-off hogs. Perpetual Contract users have the same roll-forward options as nearest future and Gann traders. The Perpetual Contract choice in the Portfolio Manager is also used to affect open interest weighting on the data, whereby no rolling is involved and weighting is based solely on speculative and trader interest.
Back- and Forward-Adjusted Contracts
More recently, traders have shown an interest in back- and forward-adjusted contracts. Back-adjusted contracts use the actual prices of the most recent contract with a backward correction of price discontinuities for successive earlier active delivery months. In a forward-adjusted contract, the prices of the current contract are changed to eliminate the gap between the current and recently expired contract. An important aspect to remember about forward-adjusted contracts is that current prices do not represent actual values for today's markets. Because of the removal of contract-to-contract price jumps and drops in both back- and forward-adjusted contracts, they appear as smooth, blended, homogeneous price histories representing a sorted and concatenated compilation of successive contracts over time.
Starting with version 1.66 of UA, there are two back adjusters available. One is written with a forward market viewing perspective, and the other views the market forward in time. Both have been thoroughly tested on standard and exotic data as part of our continuing quality assurance commitment. The results may differ in that the roll date may be skewed by a day.
For a more complete comparison of these choices, please go to our Internet site at www.csidata.com. To use the alternate adjuster, click the "Use C++ Back Adjusted" box on the Back Adjusting tab of General Settings (from the Options menu).
The back-adjusted accumulation method of joining contracts in a series over a period of years or decades permits the analyst to focus on the period when one might prefer to trade the markets in actual practice. Traders often wish to communicate their own rolling preferences so that they will not be simulating trading situations when there is either a risk of delivery or an exposure to highly volatile markets. To accommodate these preferences, UA lets users choose their desired delivery months and a roll-forward date. The roll-forward date may be relative to the start or end of the month for rolling. The option of picking a roll date relative to the month end is useful for traders who want to avoid risking delivery of their commodities by rolling out of a contract on or before the first notice day, which is often calibrated relative to the end of the month.
Back- and forward-adjusted files can also roll when heaviest volume or open interest shifts from one contract to the next. The switching of contracts based on volume or open interest is always based on the previous day's data because these values are released one day late by the commodity exchanges. For example, a rollover based on a change in volume or open interest on Monday would not be reflected in the data file until Tuesday. If heavy volume or open interest switches back to an earlier contract, the current delivery month will not change, as it is locked in to avoid confusing oscillations. Although the menu choice of these adjusted files is called "Back-adjusted" the software can forward-adjust the data just as easily.
The selection of representative prices (previously referred to as splicing method and delta) is another user-defined option, which heavily impacts the adjusted files. It refers to the data points used to calculate the back or forward adjustment value. It closes the gap between adjacent contracts by focusing upon the close-to-open, close-to-close or the open-to-open price differential of successive pairs of contracts to be joined. The option of comparing the open price of the new lead contract with the previous day's close price of the former lead contract is an especially desirable feature because it gives the user the opportunity to either ignore or include a real gap in contract-to-contract price movement.
Negative Values in Back- and Forward-Adjusted Series
An advantage of the back-adjusted approach to long-term market synthesis and simulation is that the data observed is precisely the same as the exchange's representation of the final contract in the concatenated series. A flaw in back adjusting is the strong chance that an inflation-sensitive market could produce negative price quantities into the past. The same logic allows forward-adjusted contracts in a deflationary environment to produce negative current prices for today. The suggestion that prices can be negative in actuality is clearly flawed and could discredit the accuracy of such a methodology for longer-term analysis. No one would really pay you to take 50 bars of gold away or pay you to take thousands of pounds of cotton. This flaw demonstrates that a bias is introduced through the removal of contract-to-contract price discontinuities.
When early contract prices in a concatenated set are significantly less than their real contract counterparts, they tend to produce a bias that in simulated trading would heavily favor the act of buying over selling. In addition, even if the early contract prices are not significantly different from their current-contract counterparts, inflation could play a role in influencing buying over selling when such a long series is introduced as representative of current pricing norms. This phenomenon should tell you that your results may be invalid and that applying in the present what you have learned by simulating the past can distort your trading algorithm. Fortunately, there is a way this bias can be removed without compromising the validity of your simulation.
Detrending to Remove Biases
Users of back- or forward-adjusted series can, through a simple time series analysis transformation, remove the upward or downward trend tendencies by detrending the portion of the series that connects the final current contract with the earlier balance of the series. This approach, which is found in all computed contract series and cash prices of the UA data warehousing system, removes any evidence of long-term trend for any length series so that trading can be simulated without the danger of favoring long trades over short trades.
Two alternatives for detrending are offered. One allows detrending up to the very end of the contract that lies before the current contract; the other detrends up to one day short of the period end. This series includes all of the current contract up to, but not including the very last day on file. The latter approach may be most suitable for use with UA's Seasonal Index study.
The idea of detrending is meant to apply only to the longest possible time period. This would be the period of time that incorporates all or virtually all available history for the market to be studied. It wouldn't be practical to detrend the short-to-intermediate oscillatory period. This more fruitful period should be left in the data for the technician to study.
Little or no penalty stems from the detrending process because all data is viewed from today's perspective, with today's prices (when the entire series is detrended). Before detrending, each price in a historical data file is assumed to be measured by today's dollars. Given the effects of inflation over the years, this is clearly a faulty assumption. Consider that a six-cent price move in 1966 may have represented a limit-up or limit-down situation, whereas, the same six-cent move today might be considered insignificant. When data is not detrended, that very significant six-cent move of 1966 is rendered as insignificant as a six-cent move today. Detrending, on the other hand, returns integrity to data from the distant past by putting it back into proper perspective. The importance of detrending is that it increases the chance that analytical results derived from yesteryear will be relevant and comparable to trading conditions in today's market.
Don't be fooled by analysis results which suggest that the simulated performance of a non-detrended series produces greater hindsight profits than the same series in detrended form. Remember that this process removes a bias that may give the false impression that buying is always better than selling. Such results are not achievable in future trading practice. You also cannot easily trade across contract boundaries without paying a heavy slippage and commission tax, even if you have carefully spliced together successive independent series.
Deciding Which Computed Approach to Follow
There are many considerations in choosing computed contracts for analysis and, eventually, for impacting investment decisions. Each category has some unique value. Both the nearest future contract and Perpetual Contract data can view the markets from early and distant delivery perspectives by focusing upon contracts that expire either early or late with respect to any given current date. The Perpetual Contract is the only viable approach that can focus upon a fixed period forward in time and therefore achieve a substantial level of statistical stationarity.
From an astrological perspective, perhaps only the Gann computation is valid. It seems to have the advantage of offering a predictably long period of time to view a market on an annualized basis and may have some longevity benefits not possible with nearest-future contracts. Nearest future contracts have the advantage of focusing upon the most liquid period of a contract's life, but the disadvantage of offering very brief periods of individual contract data.
The overlooked idea of detrending of computed data is especially useful with Unfair Advantage. Without loss of substance, one can get data into a form where profits and losses are not subject to extremes and avoid the problem of compounding profits and losses of the distant past into present value terms. Portions of an entire time series can receive equal weight treatment, and the early portions of a detrended inflationary series are progressively amplified so that they appear in as volatile a form as the most current data.
The back-adjusted contract offers the most flexibility for the user. Current data can be supplied as it was actually traded in exchange-released form and past data can be expressed in adjusted and detrended form. The mechanical effects of back adjusting and price inflation can be removed, making the detrended series an excellent source of information for seasonal analysis. A minor disadvantage to the back- or forward-adjusted contract is the heavy computing requirements necessary to produce the resulting series. Total computing time is measured in seconds rather than microseconds, making it necessary for one to wait for results.
Perpetual Contract data is also a very good choice because of the statistical stationarity issue, but many users dislike using non-tradeable price representations. As an analytical guide or tool for overall market direction and as an auxiliary independent supporting input, it is an excellent choice.
Traders and analysts may be tempted to adopt some form of hindsight market synthesis and simulation to prove a preconceived method of trading. Each of the alternative computed contract methods have certain advantages. In addition to what has been argued above, the Perpetual Contract data has a very distinct advantage over the others where intermarket analysis is involved. Because Perpetual Contract data uses actual prices on a time-weighted forward basis, the data in related markets operating in parallel can be significant to any analytical exercise.
Price ratios of related markets and their correlated or associative properties are directly comparable. Back-adjusted series, on the other hand, cannot benefit from such associative characteristics. This is so because the inherent flaws involving contract splicing and the overwhelming tendency for back-adjusted series to report negative readings all but destroy the relative characteristics of one market with respect to another. Gann and nearest future contracts can be used much like Perpetual Contract data, however, because of inherent price discontinuities, the results are not as consistent for longer-term analysis.
This message is presented to guide you in your study of the commodity markets and to help you understand the ever-changing subject at hand. It is not only for those who are contemplating building trading systems based on computed contract series, but also for those whose trading systems have been derived from such approaches. Each type of computed contract discussed here can add some visibility to market analysis. It is important to consider both the strengths and possible weaknesses inherent in these methods to maximize profits and preserve capital in actual trading.