The government of Uttar Pradesh state, run by India's ruling Bharatiya Janata Party (BJP), suspended the head of the state-run BRD Medical College, Rajeev Misra, late on Saturday and ordered an investigation.
Indian media have said the deaths of 60 children, 34 infants among them, were caused in part by oxygen shortages after a private supplier cut the supply over unpaid bills.
Hospital officials deny lack of oxygen caused the deaths, saying alternative supplies were found, and blamed many of the deaths instead on encephalitis and unspecified issues related to delivery of the infants.
On Sunday, J.P. Nadda, health minister in Prime Minister Narendra Modi's cabinet, visited the hospital in the town of Gorakhpur, over 500 miles east of New Delhi, accompanied by the state's chief minister, Yogi Adityanath.
After the visit, the chief minister urged patience until the investigation was complete.
"We will know - whether it was because of an oxygen shortage or due to a lack of proper treatment," Adityanath told reporters. "Those found guilty will not be spared."
Nadda said a team of doctors from New Delhi was working with the local authorities and the federal government was ready to send more assistance. He said Modi was also monitoring the developments.
The issue of the unpaid bills for oxygen supply has become a flashpoint in relations between the hospital and the state government, after the suspended hospital chief on Saturday accused state officials of not answering his requests for money.
"I wrote at least three letters," Misra told television reporters on Saturday, adding that he had flagged the issue in video conference discussions.
Reuters was unable to immediately contact Misra for comment.
Adityanath, who had visited the hospital on Aug. 9, said no issue of unpaid bills was brought to his attention and all requests for funds were met promptly.
Opposition parties have stepped up the pressure on the state government, demanding the resignations of Adityanath and the state health minister.
"This government is a murderer," said Raj Babbar, head of the opposition Congress party in Uttar Pradesh.
Uttar Pradesh is India's most populous and politically-prized state, where the BJP's thumping victory has strengthened Modi's claim to a second term in 2019.
Gorakhpur, a down-at-heel town near the border with Nepal, is Adityanath's political base, which elected him to parliament five times before Modi asked him to lead Uttar Pradesh, after a landslide BJP election victory in March.
A study of government data by nonprofit body Brookings India showing the district has a 26 percent shortage of primary health centres.
Encephalitis outbreaks kill hundreds in India every year, especially during the monsoon season.
India's expenditure on public health is about one percent of GDP, among the world's lowest. In recent years, Modi's government has increased health spending and vowed to make healthcare more affordable.