Emanuel Lutchman, 25, was charged with attempting to provide material support to ISIS, which carries a sentence of up to 20 years in prison and a $250,000 fine.
"This New Year's Eve prosecution underscores the threat of ISIL even in upstate New York but demonstrates our determination to immediately stop any who would cause harm in its name," said William Hochul Jr., the U.S. attorney for the Western District of New York.
Lutchman allegedly received direction from an ISIS member overseas and planned to attack revelers in order to join ISIS.
According to a criminal complaint, Lutchman had been in contact with a man overseas who claimed to be in ISIS. Lutchman told him he wanted to come to Syria but was informed he couldn't come now and that he should kill Americans to prove himself, according to the document.
On Monday, the document alleges, Lutchman met with a confidential source for the Justice Department. The source said Lutchman wanted to sneak a bomb into the restaurant and kidnap some people to kill.
Authorities allege he and the source went to a Walmart on Tuesday and bought two ski masks, two knives, a machete, zip ties, duct tape, ammonia and rubber gloves.
"The operation is a go. We just gotta do it, man," the complaint says Lutchman told the source. He also talked about making a video before the attack, FBI Special Agent Timothy J. Klapec says in the document.
The FBI said the confidential source has been paid for cooperation on another case, and has a felony and misdemeanor on his criminal record. The source paid for the items at Walmart, it said.
John Paige said business owners are familiar with Lutchman.
"He's been around the neighborhood for quite some time. He's a panhandler. A lot of businesses have asked him to leave, including myself," he told WHAM.
Lutchman appeared before U.S. Magistrate Judge Marian Payson on Thursday morning.
The city announced that it was canceling its fireworks show and adding police officers to patrols.
The announcement comes as President Barack Obama has been seeking to calm fears in the wake of the San Bernardino, California, terrorist attack, while reassuring Americans of his strategy for handling ISIS.
Obama was briefed on a possible terror threat targeting three major U.S. cities happening between Christmas and the New Year's Eve holiday.
CNN's Steve Almasy and Jennifer Moore contributed to this report.