"It appears increasingly uneconomic for the customer to drive 20 to 30 miles round trip to a supercenter to save a marginal amount on consumable goods—particularly when consumables, including food, are becoming more broadly distributed," Exstein said.
Exstein, who upgraded his rating on the retailer to "outperform" from "neutral" this week, said he expects that newly appointed CEO Doug McMillon will tackle the issue and accelerate its plan to add small-store formats. Exstein pointed to an announcement made at Wal-Mart's October analyst day, when it disclosed plans to quicken the expansion of its neighborhood markets, which are about a quarter the size of its supercenters. In January, Wal-Mart opened 12 new neighborhood markets, which is the same number as they averaged for an entire quarter last year, Fishman said.
This is a change from the mid-2000s, when the retailer would open an average of five stores a week, and 80 percent of them were supercenters, Fishman said.
But the problem, Sozzi said, is that Wal-Mart isn't closing its megastores when it opens a smaller format. It's simply adding to its store count.
"There's just too many Wal-Marts," he said.
In response to a request for comment, Wal-Mart pointed CNBC toward its press release, and said it would have additional details on results in its earnings release on Feb. 20.
—By CNBC'sKrystina Gustafson.Follow her on Twitter@KrystinaGustafs.