Syed Sharif Khan was stripped naked by the mob, beaten and dragged through the streets before being hanged in Dimapur, the main city in Nagaland state.
Hundreds of policemen have been patrolling the streets of Dimapur since the incident amid increasing tension.
A curfew was also imposed in the city on Sunday to maintain law and order.
In earlier reports, Mr Khan was named as Farid Khan, but police have now identified him as Syed Sharif Khan.
Police said Mr Khan was a Bengali-speaking Muslim trader from the neighbouring state of Assam. He was arrested in February on charges of raping a 19-year-old tribal woman three times.
Mr Khan’s brother said he was picked on because of ethnic reasons. According to reports, the woman was related to Mr Khan by marriage.
After Mr Khan was taken from prison on Thursday, he was dragged through the streets by thousands of people. Police opened fire to try to stop the mob, wounding several people.
On Monday, police said the arrested men had been charged with rioting, arson and unlawful assembly.
“Some of these 42 arrested men were also directly involved in the lynching, and will face additional charges,” news agency AFP quoted Inspector General of Police Wabang Jamir as saying.
There are conflicting versions of how a large mob entered the prison and then removed – and killed – one selected inmate.
One version is that initially a small group of men went to the prison, and demanded to take out Mr Khan, and that some prison officials complied and led them to him.
Only later, several thousand people collected outside the jail, demanding that the prisoner be handed over to them.
The lynching happened in the midst of a controversy in India over the government’s decision to ban India’s Daughter, a documentary about the 2012 gang rape and murder of a student in Delhi.
But correspondents say the killing is also linked to rising ethnic tensions in Nagaland, whose indigenous tribespeople have blamed migrants from Assam state and Bangladesh for settling on their land.
Vigilante justice is not unheard of in India but it is rarely seen on this scale.
Analysis: Soutik Biswas, BBC News, Delhi
The lynching points to rising anxiety among locals in Nagaland about Muslim Bangladeshi migrants who, they say, have illegally settled on their land and are encroaching on their resources.
Many say it also proves how easily mass hysteria can be whipped up among local people to brand any Muslim migrant as Bangladeshi.
Syed Sharif Khan, the suspected rapist, was not a Bangladeshi but was a Bengali-speaker from the neighbouring state of Assam. His father served in the Indian air force, and two of his brothers are in the army.
The incident points to a complete breakdown of law and order in Nagaland, which is wracked by a tribal insurgency.
Chances of a settlement to the 58-year-old conflict continue to look dim after 17 years of negotiations because rebel factions who cannot agree on the territorial limits of a future Naga homeland or state.