Torture the Data long enough and it will confess to anything...

The core motivation for this Blog was to promote some rigor in business and economic analysis and research. In our posts we had talked about how research is becoming less and less exploratory and more and more confirmatory.

I was just reading an article on Yahoo! Real Estate from CNNMoney.com titled "Americans Tame Their Wanderlust". The article is pretty cool as it focuses on domestic migration patterns, which obviously has a strong bearing on regional shifts in Real Estate demand and pricing. What was not so cool was the lack of connectivity between the data presented and the corresponding insights developed. For instance the article's title is based on the following statement "Only about 2.4% of Americans moved from state to state in 2008, down from 2.5% the previous year". The author relies on U.S. Census Bureau population data, which is publicly available at the Census Bureau website. Now this is a 0.1% change- 307,000 absolute decline assuming a total U.S. population of 307 Million. This seems like a large number if the numbers aren't estimates but actuals, but it is impossible to measure this data with 100% accuracy, so the Census Bureau like most other folks in Business and Social research settles for 90% confidence (with a 10% probability that the estimates are wrong). A key metric in gauging how much importance to put into this change is the "standard deviation" of how this percentage varies year over year. So if this percentage varies +/-0.05% every year, there is only a weak possibility that there was actually a decline and for all purposes the change would be statistically "insignificant".

I am not saying that the article is inaccurate, all I am saying is that in my opinion enough data may not have been presented to justify the title. It is especially risky to base real estate investment decisions using just two data points as typically the gestation period for these investments to bear fruit is significantly longer than 2 years (if more folks had been mindful of this we probably wouldn't have had a real estate bubble).
A little rigor around analysis of Census Bureau would have yielded the author a few interesting nuggets around domestic migration trends in place. For example the chart below shows net migration trends by 4 U.S. regions.
 

The chart indicates that with the exception of 2002-2003, the South seems to significantly lead all other regions in migration. The West seems to have some cyclical pattern going on but seems to generally mean-reverting around an equilibrium balance. Over the past 10 years the exodus out of the Midwest has progressively abated. The exodus out of the Northeast seems to have peaked out in 2004-2005 and is progressively becoming less negative in net domestic migration.

Looking at this from a State level may have generated too much noise as there are several factors to be considered like the fact that Washington D.C. is at the confluence of three states and has a significant number of people that at one point in time or another lived in both D.C. and one of the neighboring states. So either analysis and insights should be limited to the granularity permitted by raw data or an appropriate statistical model should be leveraged that controls the influence of potentially confounding factors.

Eye on The U.S. Economy: September 2009 Employment Report

The Employment Situation Report comes out on Friday September 4th, 2009 at 8:30 A.M. Based on other reports that came out earlier this week, it may go either way vs. expectations. Consensus estimates an average decline of 200,000 in Nonfarm Payrolls.


Although the ISM Manufacturing Index came out on September 1st at a very strong at 52.9 vs. a Consensus of 50.5, the Inventories Index was weak, suggesting Manufacturers are drawing down existing Inventories, which doesn't help Employment. but production was up.
From a historical perspective, last month ISM Manufacturing Index had an upside surprise, ADP came in at -371K, Consensus estimates on Nonfarm Payroll were -300K and Actual came in at -247K. So if we see a repeat trend this month, we could actually come better than -200K.

So far I see a couple of positive factors supporting better than expected Employment and one negative factor (ADP).

Here's the reason I am honing into Employment this month- Personal Income usually lags Employment and leads Consumption, therefore it is logical that Employment trends in September and October could be a major influence on Holiday sales. If you look at the last ten years Retail Sales estimates from the Census Bureau (excluding Auto), the monthly SAAR (Seasonally Adjusted Annual Rate) correlation vs. the Month-over-Month Employment percent change peaks out at a 3-month lag. The correlation with monthly change in Seasonally Adjusted Retail Sales (ex-Auto) peaked at a 2-month lag. Unless we have significant improvement in Employment and Personal Income over the next two months, we are looking at another year of bleak Holiday sales.