It was two years ago, almost to the day, that Warren Buffett said on CNBC he would buy up, "a couple hundred thousand" single family homes if it were practical to do so. Some claim that opened the floodgates to investors in the single-family rental trade.
Buffett, however, was just reacting to a trend already underway, and a new asset class that had formed. Firms like Colony Capital, Waypoint and Silver Bay Realty Trust were just a few big names buying thousands of properties and turning them into rentals.
Between 2011 and 2013, institutional investors, defined as those who purchased at least 10 properties in a calendar year, accounted for 3.87 percent of all home sales across 1,264 counties, according to a new study by RealtyTrac. Those counties were selected because rental data was available and allowed RealtyTrac to create a so-called "heat map" to show institutional investors' impact in those counties. The average home price appreciation in those counties was 14 percent and the increase in rents on three-bedroom homes was 7 percent.
RealtyTrac Vice President Daren Blomquist elaborated on the overall market for institutional investor home buyers:
"In 2011 institutional investors—purchasing at least 10 residential properties in a calendar year—purchased 219,000 residential properties nationwide, representing 5.12 percent of all residential sales. That increased to 259,000, representing 5.82 percent of all residential sales in 2012, and 354,000, representing 7.40 percent of all sales in 2013. That's a 44 percent increase in the institutional investor share of the residential sales market from 2011 to 2013."
Certain markets that were particularly hard hit by the housing crash, like Phoenix and Las Vegas, saw a far higher share of investors and far higher home price appreciation as a result. These investors, at times, accounted for nearly half of all home sales, and home prices soared well above 20 percent.
(Read more: Google to invest $50 million in real estate site Auction.com)
Now, as investors slow their purchases, home prices are following suit. In Phoenix, for example, active listings are now up 47 percent from a year ago, even though the supply of distressed homes (foreclosures and short sales) are down 13 percent, according to the Center for Real Estate Theory and Practice at Arizona State University. Prices are still up 21 percent from a year ago, but down 4 percent from the previous month, the first monthly drop since last summer.
"Demand has been weakening since July, especially demand from investors," noted Michael Orr, the center's director. "The situation has changed dramatically from January 2013 when the luxury market was 10 percent smaller than the low end market in terms of dollars spent. In contrast, 99 percent more dollars were spent on homes over $500,000 in January 2014 than on homes under $150,000."
Nationwide, home price gains are slowing, despite monthly fluctuations. Trulia's chief economist Jed Kolko points to quarterly comparisons: "The quarter-over-quarter change in asking prices topped out at 3.5 percent in April 2013 and now, at 1.9 percent, the increase is just over half of that peak," Kolko wrote in a report. However, he pointed out, home prices are still rising much faster than the historical norm.
(Read more: 4 million more homeowners now 'above water')
—By CNBC's Diana Olick. Follow her on Twitter @Diana_Olick.
Questions?Comments? facebook.com/DianaOlickCNBC