Not directly related to your source(s), but see the article pit of despair on Wikipedia. Warning, the article and the experiment itself can be rather disturbing.
Harlow's first experiments involved isolating a monkey in a cage surrounded by steel walls with a small one-way mirror, so the experimenters could look in, but the monkey could not look out. The only connection the monkey had with the world was when the experimenters' hands changed his bedding or delivered fresh water and food. Baby monkeys were placed in these boxes soon after birth; four were left for 30 days, four for six months, and four for a year.
After 30 days, the "total isolates", as they were called, were found to be "enormously disturbed". After being isolated for a year, they barely moved, did not explore or play, and were incapable of having sexual relations. When placed with other monkeys for a daily play session, they were badly bullied. Two of them refused to eat and starved themselves to death.
Harlow also wanted to test how isolation would affect parenting skills, but the isolates were unable to mate. Artificial insemination had not then been developed; instead, Harlow devised what he called a "rape rack", to which the female isolates were tied in normal monkey mating posture. He found that, just as they were incapable of having sexual relations, they were also unable to parent their offspring, either abusing or neglecting them. "Not even in our most devious dreams could we have designed a surrogate as evil as these real monkey mothers were", he wrote. Having no social experience themselves, they were incapable of appropriate social interaction. One mother held her baby's face to the floor and chewed off his feet and fingers. Another crushed her baby's head. Most of them simply ignored their offspring.
So if the results for monkeys apply also to humans, the answer is no: social isolation will not directly kill babies, but it will severely damage them, and some babies will never recover.