The short answer is that this practice is marketing orthodoxy, and supported by empirical evidence; companies do this because it makes them money. The exact cause is complex and perhaps unknowable, but the effect is real, and reasonably well documented.
Some common "folk" psychological explanations of the phenomena of the "9-fixation" are outlined in the paper "Why 99 Cents?" (pdf) written by Economist Oz Shy:
Rounding illusions: Consumers tend to approximate the prices they pay by a lower integer or a lower decimal number rather than a higher price. Thus, depending on the level of rounding numbers, consumers may state or report a price of $999.99 as nine-hundred-ninety-nine, as nine-hundred-ninety, or simply nine-hundred dollars. Whence, people tend to under quote a price according to the leading digits of the price. Stores, then, maximize profit by setting the last digits as large as possible, in which case they simply add the relevant sequence of 9s.
Consumers like to receive change: Therefore, if stores take into their pricing consideration the assumption that consumers' utility is enhanced by receiving change, they will maximize profit by handing out the minimum-possible change, which is 1¢. Hence, all prices end in 99¢.
Attractive digits: People find the combinations of 99, 999, 9999, and so on to be "nice." Thus, consumers are attracted to looking at prices involving many digits of nine. Hence, this "elegant" statement of the price serves as a store-advertising mechanism since after looking at the price itself consumers are more likely to pay attention to other details and features of the store.
Image of a discount retailer: Discount stores tend to utilize black & white ads in order to create an image that the store is engaged in a significant cost cutting. Similarly, a price of 99¢ may indicate that a store is concerned with all levels of cost cutting and that even a 1¢-cost reduction is being transferred to the consumers in the form of a 1¢ price reduction.
There are various proposed psychological and economic "causes" to this problem.
In "Why 99 Cents?", Shy proposes a game-theoretic solution, which has retailers reaching pricing equilibrium with 99 cents after the dollar amount.
The book The Strategy and Tactics of Pricing: A Guide to Profitable Decision Making by Tom Nagle and Reed Holden suggests that people observe the "dollar" and "cent" parts of prices independently, and often ignore the cent portion as insignificant. Capitalizing on this phenomena, retailers should maximize profit by setting the "cent" part of the price to the highest possible value.
In his paper "Why are so many goods priced to end in nine? And why this practice hurts the producers" Kaushik Basu additionally offers that this phenomena is due to pricing equilibrium caused by consumers always assuming that the "cent" part of a price will be 99, and so when an item is "four-dollars", for instance, a consumer will assume that it's $3.99 and retailers will "lose" a penny.
Whatever the actual cause and effect of this phenomena, it is an empirical fact observed by André Gabor in his 1988 book Pricing: Concepts and Methods for Effective Marketing, who observed that the decimal portion of a price has no effect on the consumers of an item.
This same phenomena was observed by Phillip Gendall in his paper Estimating the Effect of Odd Pricing, who noted that "odd pricing generates greater-than-expected demand, at least at the individual brand level, and for the common practice of setting retail prices that end in 95 cents or 99 cents."
