Immigration is set to be a key election issue in 2008 |
This followed a series of amendments to the proposed bill, which undermined a fragile bipartisan compromise.
The bill would have tightened border controls, but given 12 million illegal immigrants already in the US a way to legal status and citizenship.
Illegal immigration is among voters' top concerns and is set to be a key issue in the 2008 presidential poll.
President George W Bush has made immigration reform one of his priorities and has previously said he wants to see new legislation in place by the end of the year.
Party leaders have promised they will work to try to bring a bill back to the Senate for a new attempt to make progress within the next few weeks.
But the vote is a serious blow to the president's chances of achieving comprehensive immigration reform before he leaves office, correspondents say.
And unless the measure is reconsidered soon, it may get caught up in the upcoming election cycle as politicians turn their attention to the 2008 presidential vote.
The House of Representatives, where views on immigration reform are even more polarised, has still to consider its version of the legislation.
Guest workers
The bipartisan bill had elements which were hard for both Democrats and Republicans to accept.
Some critics called it an "amnesty" for those who had illegally entered the US, while other opponents argued its guest worker programme threatened American workers.
After two weeks of debate, the bill finally stalled late on Thursday, when only 45 senators voted in favour of bringing debate to an end and moving the bill to its final passage.
Sixty votes were required for the motion to pass.
Senate Majority Leader Harry Reid removed the bill from the business schedule and proceeded to other matters.
""I, even though disappointed, look forward to passing this bill," he said.
However, Republicans - many of whom believe the bill should be tougher on illegal immigrants - had sought more time for debate.
"I simply do not understand why some of my colleagues want to jam this legislation through Congress without a serious and thorough examination of its consequences," said Republican Senator John Cornyn of Texas.
Border security
Under the proposed Senate bill, illegal immigrants could seek a renewable "Z visa" after paying a $5,000 (£2,530) fine.
They could ultimately be placed on the path to permanent residency - a process that could take several years.
The bill also set out a "points system" that emphasises immigrants' education, language and job skills over family connections in awarding green cards.
But these measures would not come into force until 18,000 new border guards are deployed, the fence with Mexico reinforced and hi-tech surveillance in place.
Some Democrats say the bill would destroy families, because people's employment prospects, rather than links to relatives already in the United States, would be a key criterion in allocating visas.