The Israeli military said the two brothers were members of the militant group Hamas and were planning attacks on Israel.
Hamas said they were sons of a member but were not involved in Hamas. It called the abduction a crime.
Israel pulled out of Gaza last summer but there remains a regular exchange of missile fire.
However, such incursions by Israel are rare.
Computer
The raid started at 0330 (0030 GMT) in the village of Umm al-Nasser, near the Rafah refugee camp and only one kilometre from the Israeli border, Palestinian sources say.
The two captured men were brothers Osama and Mustafa Abu Muamar, sons of Hamas activist Ali Muamar.
An Israeli army spokesman said the two men taken were Hamas militants intending to carry out imminent attacks on Israel.
"These Palestinians were transferred to Israel where they will be questioned," the spokesman said.
Hamas denied the men were its members.
Ali Muamar told Reuters news agency he awoke to see Israeli troops scaling down his wall.
"They blindfolded and handcuffed me and started beating me up with the butts of their rifles and kicking me," said Mr Muamar, who was treated in hospital for wounds.
Mr Muamar said the soldiers seized his computer and left after about an hour.
The Israeli army occupied Gaza for nearly 40 years before it withdrew last summer.
It has launched a number of air strikes which it says are targeting militants who fire rockets into Israel.
Hamas, which refuses to recognise Israel, won control of the Palestinian government in elections in January.