Japan External Trade Organization(JETRO) released a report of the Japanese content business in North America. The report showed a steep shrinkage of the anime market in US between 2003 and 2007 (See the chart in the spoiler).
The anime market in US mostly consists of the sales of the character goods and it is strongly affected by the popularity of Pocket Monster series. The drop between 2003 and 2004 and the slight increase in 2007 reflected the business status of Pokemon. JETRO concluded that US anime market won't expand without the appearance of another big hit title comparable to Pokemon.
Ref1. US Anime DVD Sales Ranking For 2006 (The relative indexes are calculated assuming the sales of the top title as 100)