Update & IPO's
The views and
information expressed in this document reflect the opinions and experience of
the author Robert C. Pelletier. Neither CSI nor the author undertake or
intend to provide tax advice or trading advice in any market or endorse any
outside individual or firm. All recommendations are provided for their
informational value only. Readers should consult competent financial
advisors or outside counsel before making any software purchase or investment
decision. CSI does not stand behind or endorse the products of any
Copyright (c) 1998 Commodity Systems Inc.
(CSI). All rights are reserved.
Topics discussed in this month's
The Truth About Data
The more serious the analyst,
the more careful the consideration of data. This is as true of white-coated
scientists in laboratories as it is of investment professionals on Wall Street.
Another truth: The more careful the consideration of data, the more troublesome
anomalies will be found.
Over the years I've met many CSI customers who
stake their reputations and their livelihoods on the market research they
perform. I have come to know them because they had concerns about data which
they felt only I, as president of the company, would fully appreciate. Whether
or not others could have helped, I was glad for their calls, as they have
helped me stay in touch with the needs of our users. It occurs to me that,
although most CSI customers do not ponder the validity of every data point, all
might benefit from the research we have done and the explanations we have
formulated at the behest of our most ardent researchers.
One of the most common
concerns involves so-called "missing data." Null fields can, indeed, be a
problem when algorithms require daily comparisons or accounting. If a study is
intended to perform a simulation based upon all trading days, ALL trading days
MUST be present. Anything less would be inaccurate. However, missing data is
sometimes simply a part of the factual record of a commodity's trading life.
CME Pork Bellies, for example, were not traded for the brief period between
August 23, 1963 and September 20, 1963. Alberta Barley did not trade between
February 28, 1989 and May 24, 1989. White Sugar, MidAmerica Treasury Notes, the
French Franc, Unleaded Gasoline, the Mexican Peso, London Tin and COMEX Silver
all have trading gaps that are reflected as missing days in data files.
One poignant concern came from
a man who was comparing databases and wanted to know why CSI was the only one
of three data services with a gap in silver prices for three days from August
16th 1971 to August 18th 1971. We frantically researched our files to find
that, low and behold, there was no trading in silver during that time. Back in
1971, our records show, President Nixon imposed an embargo on finished goods
that affected metals and other commodities, forcing the exchanges to close for
a few days until the impact of the President's decision could be absorbed by
the emotional public. Where our competitors got their prices, we'll never know.
We do know you won't find fictitious data points in the CSI database that were
invented for the sole purpose of making the series appear credible. As far as
we know, you won't find any inappropriate gaps in your CSI data.
We have discussed this before,
but continue to get many questions regarding closing prices that are above or
below the day's high and low. Typically only users of Unfair AdvantageŽ have
queries about out-of-range closes. This is because QuickTrieveŽ, our original
downloader, and many products for market analysis do not allow for closes
outside the trading range. Unfair Advantage, however, can report out-of-range
settlement prices, regardless of trading range violations, because they are a
true part of the market data.
The confusion about out-of-range closes stems
from the idea of having a simple "closing" price for commodity contracts.
Although the "closing" price could represent the last trade, at some exchanges
it could be a calculation decided by a committee after the close of trading.
Low volume, far-out contracts are most likely to have adjusted settlement
Unfair Advantage users can present the data in
1) actual exchange released form with highs, lows and settlements left
untouched, 2) with compromised closes that ignore outlier settlements and force
the close to lie within the high-low range, and 3) in compromised high and low
form with accurate settlements and possibly a high or low that has been
modified. Here again, when the analyst adheres to the true market record (by
choosing actual), apparent anomalies are added to the data series. But, like
missing days where no trading occurs, out-of-range settlements are a part of
the record that analysts may benefit from accommodating in their evaluations.
Two Closes for LIFFE
The London LIFFE exchange is
particularly prone to confusing closing prices because of the way they blend
computerized APT data (After Pit Trading) with floor trading data. The APT
session begins about 20 minutes after the pit session is over and lasts about
90 minutes. The data supplied by the exchange is as follows: The open from the
pit session, the high and low statistics compiled from all trades from both the
pit and APT sessions, and the official settlement price, which reflects the
close of the pit session. The close from the APT session is also
available, but it is not "official," and is therefore not quoted on our system.
Neither our price quotation, nor any others that we know of, can accommodate
two closing prices for the same contract. We hope to eventually add a field to
Unfair Advantage to accommodate LIFFE's double closes.
If you were to visit CSI's
facilities, you might be surprised to see source material neatly cataloged and
arranged on shelves that line the walls of the data department. The material is
in the form of microfilm, exchange bulletins, newspapers, old out-of-business
newspapers and bound volumes of trade reports. We have materials that were
published as far back as 100 years ago. We can back up and verify just about
any questionable report and certify relatively ancient data accuracy. We save
just about every possible data record so that our customers can get the most
for their money. The fact that we publish every error we commit is a testament
to our devotion to doing the job right.
If you also use another source and a discrepancy
is discovered, 90% of the time CSI will have the correct reading. None of our
competitors are brave enough to publish their mistakes, and we would be happy
to challenge each and every one of them to a contest on accuracy. We ran an add
several years ago showing a side-by-side comparative record of errors committed
by CSI and one of our competitors, but after a while S&C Magazine refused
to print our reports.
While it is certainly true that our most serious
customers provide us the greatest challenge in providing the highest quality of
data and support, it is also true that careful consideration of the data is
what sets CSI apart from all others in the industry. Anomalies will always be
found in data, simply because we live in a world that is not perfectly
structured. It is important to know that these anomalies reflect our somewhat
chaotic world, not errors on the part of your data provider. The truth is, we
are privileged to serve such a dedicated and thoughtful group of investors and
Soaring Stock Market Calls for CSI
Barring a major downtrend in
tech stocks, the NASDAQ Composite Index will soon eclipse the CSI format's
upper price limit of 262143. That is, if it hasn't already done so by the time
this newsletter is printed. QuickTrieve users and those who use
QuickTrieve-compatible software will note distorted readings for this index on
all days the price exceeds 2621.43. CSI's Unfair Advantage software will not
have such problems in its charts because it can accommodate number sizes beyond
As this letter goes to press, new format
specifications are on our web site for software developers to download and
implement. These will extend the CSI (QuickTrieve) format open/high/low/close
price range from 256K to one million. By the time you receive this newsletter,
revised QuickTrieve software files will also be on the web for users to
download and install. We strongly recommend that everyone upgrade to the
revised format once it is released. The new format will require changes to your
QuickTrieve software and to any compatible software you use to analyze
The CSI format has been updated a few times over
the years and has changed from handling two-byte quantities (16 bits) to the
new convention, which handles 20-bit quantities (2.5 bytes). Periodic revisions
are necessary in any fixed-field format (such as the CSI format) to accommodate
an ever-expanding range of positive number sizes. Developers who stay current
with the CSI format will be able to process data with values of 999999 or less
and should have no trouble reporting data from all exchanges.
Unfortunately, some software developers have
been slow to respond to the expanded precision that CSI offers. Those who
haven't upgraded their software at any time may be limited to expressing and
processing data with a maximum quantity of 65535. If you are not experiencing
problems, then your software may well be compliant. On the other hand, if you
have difficulty charting your data with software other than UA, then your
program is probably operating on the old CSI format. Hopefully, the developer
of that program has already made an upgrade or is preparing one for release.
Let's assume the NASDAQ Composite Index
has already broken through 2621.43. If you haven't upgraded your
QuickTrieve software and/or your analysis software is subject to the original
16-bit number size limitation or the 18-bit version of the CSI format, please
consider these options:
QuickTrieve users can track the NASDAQ Composite
Index with CSI stock number 5740 (COMPZ). The price values in this series are
quoted to one (rounded) decimal place, instead of two decimal places. This
stop-gap measure will produce charts and data which are less precise than the
standard NASDAQ Composite Index data (5799 COMP), but they will not be
Most Unfair Advantage users who store data in
the CSI (QuickTrieve) format will note no distortion because of changes we
transmitted to UA's factsheet and database control files. Those UA users who
have protected the factsheet file from remote revisions should see Tech Talk
for information on revising this file.
The exchanges are neither unaware nor
insensitive to the effects of rapidly rising values on traders. The higher
prices go, the less meaningful incremental changes become. We anticipate that
the NASDAQ will soon change the minimum tick for its index and divide the
values by ten, just as we suggest doing through Unfair Advantage. In either
case, the change will be virtually transparent to the user.
QuickTrieve users and compatible
software vendors are asked to check the Patches page of our web site at
www.csidata.com for emerging information and upgrades. These are provided at no
charge. If you do not have access to the Internet, please call our technical
support staff to request a diskette of the revised QuickTrieve software. A $10
handling fee will apply.