Trader’s roundtable: correllation between ETFs

Image via Wikipedia

In a recent discussion among traders, the idea of how to measure the correlation between various ETFs in the world market model arose.

Here is my method:

I set up a table, updated automatically daily iwant to compare the daily % gains and losses as the data points, then just pick the desired time frames and run =CORREL on it in Excel, that uses the =CORREL function to find the correllations between the broad indices

Iwant to compare the daily % gains and losses as the data points, then just pick your desired time frames and run =CORREL on it

I  use XLQ from qmatix.com to fetch and update daily performance data; a great time saving utility program with a robust  set of functions and a great yahoo group.

see attached snapshot of todays correllations

the left table is a cross-correllation on 200 days of data

the right table is 4 diff times frames of indices vs SPY

green is exceptionally highly correllated,  red is exceptionally uncorrellated for that data set

Related articles by Zemanta

• A reflection on leading and managing a complex Participatory Action Research curriculum project (kansasreflections.wordpress.com)
• A deceptively simple management “game” for appreciating Force Management (usacac.army.mil)
• The Benefits of Equity Commodity ETFs (seekingalpha.com)
• Correlation Between Stocks and Gold: An Explanation (blogs.wsj.com)
• The Shifting Currency-Equity Market Relationship (seekingalpha.com)
• Market Surprise: Stock Market Decouples from Dollar (seekingalpha.com)
• In Search of the Conditionally Correlated Hedge: VXX vs. VXZ (seekingalpha.com)
• ETFs for 2010 (seekingalpha.com)
• Diversification, Schmversification: All Correlations to 1.0(-ish) (paul.kedrosky.com)
• Three Commodity ETFs in ‘Anti-Contango’ (seekingalpha.com)
• Global Equity Markets: So Much for Diversification (seekingalpha.com)
• The U.S. Dollar and Asset Price Correlation (seekingalpha.com)
• culture and climate (kansasreflections.wordpress.com)
• A thought experiment for revolutionary leadership (usacac.army.mil)
• Army Enterprise Task Force: potentially greatest in-house management re-organization ever (usacac.army.mil)
• Reflecting on scenario support to CGSC, TRADOC and the Army (usacac.army.mil)
• A reflection on Hunt’s “Leadership: A new synthesis” (1996) (kansasreflections.wordpress.com)

Traders roundtable: the three phases of new system innovation

At a recent traders roundtable discussion, we started to review the procedures and practices that each of us followed for coming up with new ideas to trade the market. While each trader have their own specific routines and follow, there were some aspects of their practice that were in common with each other. Here are the three phases that each trader went through in one form or another when developing new systems ideas:

Phase 1: Saturation. in this phase the trader is hungry for knowledge and reads everything he can get his hands on. Searching YouTube for different trading ideas is now feasible and reasonable as people will give you their best ideas online in a way that is very accessible and time efficient. In this phase you’re trying to cram your mindful of facts, ideas, concepts, themes without regard for trying to make sense of it yet. Think of it as filling your hard drive with data.

Phase 2: Incubation. In this phase, the trader allows the subconscious mind to stew on facts and assumptions below the level of consciousness. You can think of this as churning butter or slow cooking a stew. It’s not unusual to have trouble sleeping or to feel distracted from day to day events in this phase, because you can sense something is happening at a deep level that you can’t put a name to get. It is often helpful to distract the conscious mind with strenuous physical activity or listen to classical music, especially Baroque music like Bach which seems to have a generative effect.this phase continues until your moment of breakthrough.

Phase 3: Illumination: this phase begins with the shock of an “A Ha!” moment. the nervous energy released by the shock of recognition of a new insight can be almost overwhelming. I can remember times having to pull my car over on the highway in order to let the new thought fully develop. the energy will run its course while you visualize and or here are the first full manifestation of the idea and its implications. It’s a good idea to have pencil and paper with you at all times or a digital voice recorder so that you can capture the full stream of consciousness insights while it is still fresh.

I strongly recommend that at least once a month you set aside some time to go through these three steps and make them part of your trading research ritual.

Profitable ETF trading strategies: cooperating with the voice of the future

Trading is by its very nature a crapshoot, filled with uncertainty.

People find trading so difficult because it rewards the ability to maintain an open mind with respect to performance and uncertainty, whereas our pattern matching brain desires to create order out of chaos no matter what the cost.

We like to believe that there is order in the universe and that things can be known. Based on this belief we seek to improve our pattern matching capabilities in order to better understand the world around us.

This leads us to believe that our performance is attributable to our own skills as the primary cause of our results.

Trading can be frustrating for people seeking certainty and deep explanations that are robust and reliable, precisely because it is a complex adaptive system that seems to have an irreducible complexity that resists complete explanation.

Even very reliable systems go through periods of time when they underperform random entries. This is because no single system can be as complex as the market in the act of simplification creates conditions in which the system will under perform. Humans being what we are, we interpret these periods of underperformance as the system being broken or ourselves to blame when all that is happening is that the market conditions no longer favor a particular rule set.

It is a normal response after a losing streak of some length, to doubt yourself and begin to anticipate that every decision you make will lead to failure.

It’s normal to start hearing a judgmental voice in your head that tells you in the moment of decision just how wrong are going to be in the future. The voice reminds you that it will be there then in order to say “I told you so”.

Typical human bias will cause us to remember only those times when the voice is right and not all the times when the voice was wrong and we will begin to reinforce our emotion-based belief in ourselves in a harmful way.

I call this voice, the voice of the future.

I recommend that you listen carefully to that voice and find out the name of the character who is doing the talking.

Conduct some deep reflection and inner theater to find out the source of this character and the emotional events in your past that give rise to this judgment of character.

This character can be very helpful to you in many life circumstances.

It may not be necessarily helpful however when you’re trying to trade the markets under conditions of uncertainty.

Psychological work can help you reconcile yourself with that voice in the function it performs and help you come to an agreement on how this naysayer can actually be helpful to you.

For instance, if you set this naysayer onto observing you following your rules and alerting you to those times when you take shortcuts, this critical observer can be immensely powerful.

When it crosses the boundaries into judging you for being wrong than it has overstepped its boundaries and is no longer being helpful.

Understanding and appreciating this deep part of yourself can have a powerful impact on your trading at times when it gets difficult.

Profitable ETF trading strategies: a thought experiment with relative strength

Typically, traders will apply various technical analysis indicators to the behavior of price of an instrument in order to find moments of advantage. Many times these are combined in systems of ever-increasing complexity and apparent sophistication. All of these concepts however are based off of direct behavior of price. That’s not the only way that technical analysis can be used however.

Here is an example of using technical analysis in a nontraditional way to quickly and consistently assess the relationship between a broad index and one of its underlying components.

Begin by calculating the relative strength of an instrument against its index. Perform this calculation over a look back. That makes sense based on the timeframe you intend to hold the instrument. As an example, you may have the belief that swing trading large-cap stocks in the Dow 30 industrials offers an advantage to the agile trader who cannot afford to watch the market throughout the day.

You might take the Dow 30 industrials is a reasonable set of ultra-large-cap stocks which are not likely to go bankrupt overnight and which behave in a conservative fashion compared to some small-cap and overseas stocks based on the wealth of analysis and broad ownership by institutional money. It is rare for these stocks to be wildly mispriced and their stock performance is characterized by relatively smooth changes in price from day to day.

At any given time, one or more of these stocks will exhibit leadership qualities in that it will be outperforming the broad index and its peers for one to four weeks. At the same time there will be a set of these stocks that are underperforming for the same time.

There comes a moment when the individual leaders and laggards can no longer maintain their extreme behavior and they begin to revert to the mean of the index itself. By calculating the relative strength of each of the stocks on a daily basis and plotting that performance on a standardized scale of 0 to 100 for each stock, you may be able to find moments when the extreme condition begins to stabilize and reverse.

In the case of the laggards, this may alert you to a possibility to buy value just as it is beginning to revert to the mean.

In the case of the leaders, this may alert you to the possibility of the outperformance period coming to an end.

By applying these relative strength performance curves on a channel-based technical indicator like Williams %R, you might treat the upper 20 and lower 20 percentiles as the regions of extreme behavior and thus ready yourself for a reversion of the previous trend.

I have not subjected to this idea to rigorous back testing yet, but the concept is intuitively appealing. The insight comes from inductive reasoning based on years of observation and may be worthy of closer study.

Profitable ETF trading strategies: the power of focus

I have a good friend who has been struggling for years to become a professional trader in the stock market. He is well-capitalized, smart, a quick learner, humble, energetic, self disciplined and passionate about his desire to be a full-time trader. And yet, after years and years of trial and error he has not made enough progress towards his goal to feel confident that his goal is realistic.

I told him that I was willing to be his coach provided he put in the effort and was willing to go beyond his beliefs which may have been the source of his limitations.

The good news is that in a matter of weeks through rigorous self-examination and brutal honesty and a great deal of personal work, he has made major strides after all these years in realizing his goals.

After several weeks of observing his trading patterns in real time, together we concluded that his curiosity and eagerness were not really strengths at all in the way he was  applying them.

He was more than willing to pull the trigger on ideas that seemed to make sense in the moment, but what he lacked was disciplined focus to master and implement a strategy that was most suited for his personality and strengths.

Once we had enough trades to establish a statistically significant population to examine, we focused on the patterns and targets where he had performed to the highest levels with the least amount of effort.

In his case, it turned out that the best thing that he was doing was trading Coca-Cola on an intraday basis, because he found himself able to tune into its normal price patterns and find price levels of support and resistance that gave him favorable reward to risk ratios when traded at his level of capitalization.

By sticking to a couple patterns that were simple and robust, he was able to develop new levels of mastery in a remarkably short period of time on his favorite target. His performance statistics were so outstanding that there is every reason to believe that he could trade just that symbol for living forever.

Coca-Cola is a marvelous company and its stock behaves in a way that makes short-term trading and swing trading very accessible to the beginning trader.

It is liquid, smooth and orderly, and  sufficiently volatile to allow outsized gains for measured risks. Because it is a large-cap US stock, it must be owned by large-cap mutual funds so there is always a population of people who are ready to buy Coca-Cola, which as a certain degree of fundamental safety to the stock.

Human nature being what it is though, it is a daily task of management to keep my friend focused on what’s working for him because the very success he experiences in Coca-Cola encourages him to go back to his old ways and try on other new ideas. If those don’t work, the negative emotions he feels find their way back into the way he trades Coca-Cola.

This is the insidious nature of human emotions in the trading arena. It must be monitored and guarded against daily basis. Each new day brings its own new challenges and you cannot let down your guard.

The results though speak for themselves. If my friend has the will to do it, he already has the skill to be a full-time professional trader Coca-Cola.

Maybe the most important quality in a coaches the ability to remind us to remember and apply the things that we already know.

Profitable ETF trading strategies: reflection theory and its implications on back testing

A simple explanation of reflection theory comes from the mathematical field of set theory and statistics. The theory can be made as complex as you want it to be, based on your interest in statistics and math, but the essence of it is intuitive and has some interesting insights for back testing your systems in the stock market.

To begin with, the reason we conduct back testing is to ensure that our intuitive insights, which are triggered by enthusiasm and emotion in our pattern matching brain, may actually have some objective truth to them when subjected to rigorous analytical confirmation.

The manner in which we conduct back testing and the assumptions that we used to drive the analytical engine are all important when it comes to having reliability and confidence in the results. If you start with a bad set of assumptions, then even the most perfectly applied back testing technique will come up with invalid and unsupportable conclusions.

Back testing is like logic and deduction in that sense. If you start with bad information, you can be perfectly logical but come up with conclusions that are logically sound and don’t work in the real world.

Reflection theory suggests that in normally distributed populations, that for a sufficiently large population there will be subsets which have the same characteristics as the larger population from which they are drawn. This leads you to conclude that the inferences you draw on samples can be applied with great degrees of confidence to the behavior of the population at large.

At the simplest level, if you believe that the market returns are normally distributed, then you can take samples at will and data mine them for relationships between variables of your choosing and then apply any edges that you find to the broader population with great confidence.

Everything we know about the distribution of stock market returns, though, is that they are not normally distributed and that therefore great care must be taken in your sampling technique and in your out of sample testing for validation for any rule set that you propose to put real money on.

There are proponents of the idea that the market is so abnormally distributed that there is ian irreducible chaotic nature about the market which should put an upper limit on any risk you propose to take, no matter what the statistical evidence from back testing suggests.

The short answer to this is: place your confidence wisely and always have an out. Never be the first mouse, unless you have to be, and remember that there are no old, bold mountain climbers.

Profitable ETF trading strategies: the importance of the initial capital preservation stop

Chat rooms and discussion boards are filled with the talk of what to do about stops for active traders? This discussion can seems take on an almost religious fervor and I have never actually seen a single discussion thread where anyone’s mind was ever changed or consensus was ever reached.

That probably relieves me of the burden of trying to convince you one way or the other and simply let the described my thoughts on stops and how I apply them to my own trading. It’s not impossible that something I say may trigger a similar idea that you can apply in your own trading. That’s my hope. my strongest belief about stops actually does take on an almost religious belief in its importance and purpose. That is my belief in the importance of placing your initial stop in the market immediately after entering your trade and leaving it alone, never to be lowered.

I feel so strongly about this belief, that on the day that I’m no longer able to do this I will hang up my trading career and walk away. I believe it’s that important.

Your initial entry into the market is up function of your belief in the edge that you perceive you have and how it applies to the current conditions of the market.

This insight comes with it your analysis of what price action would convince you that your positive outlook is wrong. In passing, if there’s no price that would cause you to change your mind and I would advise you not to quit your day job. once you have anchored your initial capital preservation stop in the order book, you can monitor your reward to risk ratio was a function of the profit targets that emerge as reasonable based on subsequent price action. You don’t know what prices going to do after you enter, but you can be sure about the price that will tell you your initial idea was wrong.

The market will tell you if you’re wrong, and if you cannot put your initial stop in the order book and deal with it, then you’re not prepared to admit you’re wrong and you’re not prepared to be a trader.

By placing your initial capital preservation stop in the order book, you have established the lower limits of uncertainty based on your best knowledge and skills at the time.

To adjust this stop lower at any time after you have entered the trade is to mistrust your judgment and would be an indicator that you’re not prepared to trade that position to begin with.

By putting this stop physically in the order book you have established your belief in black-and-white. This will be the source of great psychological strength for you to record trading career.

And when you can take a professional loss when the price moves immediately against you after your entry, you can consider yourself to be a trader.

In my years of trading, I have never seen anyone trade who could adjust their capital preservation stops lower on the fly for some psychological and analytical reasons.

It has always led to the disaster on their part.

I’m not saying that these people don’t exist, it’s just that I have never met any of them in 20 years of active trading.

You will do what you’re going to do, but I recommend you put that order in the book.

Profitable ETF Trading Strategies: appreciating the power of rehearsals

Rehearals are considered to be one of the highest payoff practices in the military planning process. It’s where units develop and reinforce the patterns of action and decision-making that make all the difference in combat.  Rehearsals  will improve your trading practice as well, if you understand how to do them well.

There are 4 main reasons why rehearsals can improve your trading practice:

1. Practice essential tasks.  By identifying the critical tasks in your systems, you can focus on the ones that contribute the most  to success or failure

 2. Identify weaknesses or problems in the plan. You often will not discover  gaps in your logic or problems with the concept until you have “driven the route ” from start to finish from the perspective of the operator. This is especially true if you have built your plan out of component pieces, each which are individually sound, but have not yet been linked together before.

 3. Coordinate subordinate element actions.  When you use a building block approach to trading system development, sometimes  you will discover that the sum of the parts is different than the whole. This means that there are unknown or unintended consequences of piecing things together which are not revealed until you put the plan into operation, or better yet, have a decent rehearsal to test the seams.

 4. Improve understanding of the concept of operations. Once you have driven the entire plan, you will develop a sense of completeness and appreciation for its qualities or problems areas from the top down. You will be able to see the seams, where pieces come together in utual support or in sequence.

By paying close attention to your rehearsals, and making them as realistic as possible, you will be able to dramatically improve your trading practice.

Taken together with effective After Action Reviews, rehersals are an important part of your preparation phase. Time spent here will add directly to your bottom line.

Profitable ETF Trading Strategies: understanding Average True Range (ATR)

One of the most powerful measurements in the technical trader’s toolbox is Average True Range, invented by Welles Wilder, and found prominently near the top of every major charting package that can be found on the web.  Its power is what makes it so popular. 

I always recommend to traders that they use ATR for their systems if they need a decent measure of short term volatility that is sensitive enough to detect important changes in near term volatility, smooth enough to prevent whipsaws of opinion and action, and reliable enough to be useful in all types of market conditions. 

The way the indicator is calculated and displayed though makes it very easy to misinterpret by a trader or investor who is not familiar with how it works. In my opinion, the problem is so profound that it should not be displayed in the raw form on your charts because the visual display of the information is so misleading.  Even experienced traders, who look at a lot of indicators, can overlook the problem. In fact the more indicators you use, the easier it is to forget this insight about ATR. 

The problem is simply that ATR is measured in dollars and cents and so when you see the ATR line rising on a chart you are naturally led to believe volatility is increasing.  The problem is that if price is also rising, volatility may be steady or even declining, which you cannot easily tell by inspection. 

For example: an ETF priced at 20 with an ATR of 2 is much more volatile than an ETF priced at 100 with an ATR of 5. If it were the same ETF, and you watched its ATR climbing from 2 to 5 while price went from 20 to 100 you might assume that it was more  volatile as price was climbing. The opposite would be the truth of course, but your eyes could lead you to believe otherwise unless you were constantly alert. 

Here is the refinement that I recommend. Instead of using ATR in its basic form, simply divide ATR by the current price. This will give you a number which I call “ATR%”. This is interpreted to describe the percent fluctuation of the value of the asset that can be considered normal given recent price action. By recent price action I mean the last 14 trading days, in the default construction of the indicator. 

Use of the ATR% will allow you to do the following things very easily, things which cannot be done using straight ATR numbers, which represent a dollar denominated “normal” fluctuation. 

You can use a time series of a single asset to see how its volatility is truly changing over time. 

You can compare the ATR% of an asset against a set of other assets, and rank them into a scheme of relative volatility at that moment. 

You can use statistics to further classify and describe a set of assets into logical, statistical sets 

You can compare sets of volatilities against other time periods to help you with market classification. 

Average True Range is an important tool of technical analysis, but one which has serious  problems in interpretation for novice or very busy traders. Know your tools!

Profitable ETF Trading Strategies: Respect Volatility, use its power for good

At the most fundamental level, volatility is the fluctuation in the price of an asset. The greter the price swings in the shorter periods of time, the greater the volatility. Periods of great volatility are like thunderstorms. They get your attention.

Volatility is an absolute requirement for a trader to make money. Most investors look at volatility with fear and trepidation, and properly so, because wild swings in price are an indication of uncertainty about the fair value of the asset in question. This uncertainty will play havoc with your bottom line as an investor. It is a normal tradeoff consideration for investors to give up the potential of outsize gains in exchange for protection against downside volatility.

 

It is clear from scholarly studies that increases in volatility correlate strongly with declines in equity value. If you look at bear markets you see volatility everywhere, leading to tremendous gains on up days and tremendous losses on down days. This volatility  is what drives longer term investors to the sidelines and creates the window of opportunity for longer term value players to establish excellent entry points for long term holdings in beaten down companies and sectors. This eagerness to buy value at a discount is why we see buying pressure even in the midst of the worst bear markets.

 

Traders who are looking to make their loving off the buying selling of inventory need the volatility of longer term position traders and short term scalpers to move price in swings that last long enough for them to realize their gains while offering the buyers and sellers from other time frames reasons to get in and out of these positions as well.

 

Because swing traders do not need fundamental beliefs in their positions they are able to operate successfully in swing trades during bear markets, and providing that they can manage their risk in the periods of higher volatility, should be eager to trade on the most violent of das in the market.  For a swing trader the greater the intraday volatility, the easier it is to see opportunities and frame favorable trades in terms of reward to risk. 

Like electricity, volatility can be your best friend or your worst nightmare. As a trader you must learn to use the power of volatility responsibly and effectively.  Stay grounded and respect the power for your own good.