This is often a concern especially of parents that have lost a child. They realize that to go to heaven you must make a conscious decision accepting Jesus Christ. So what about the child that was not old enough to make a decision such as this?
We can find the answer to this in a story found in the Bible, 2 Samuel 11:2 - 2 Samuel 12:23. This is the story of where David lusted after Bathsheba who was the wife of Uriah. David had Uriah sent off to war in the front lines so he would be killed. In this way he could take Bathsheba for himself. David's plan succeeded, he married Bathsheba and had a child with her. After this had taken place, David was confronted by the Prophet Nathan. He was told because of this awful thing he had done the child born to him by Bathsheba would die. The child then got very sick and David began to fast and pray hoping the Lord would let the child live. When the child died he quit fasting and the servants asked him why? David responded, " why should I fast? Can I bring him back again? I shall go to him, but he will not return to me."
David knew that the child was not old enough to consciously acknowledge God. Still David expected to be with him after he himself died. David was told his sin was forgiven even though he would still suffer consequences because of it.So it is reasonable to also say David still believed he himself would go to heaven. In order for David to go to be with the child the child would have to be in heaven. So yes babies do go to heaven. We know not everybody goes to heaven, so it would seem, there is some kind of age in which we become accountable. The Bible however does not tell us what that age might be.
Now for the grieving Mom or Dad they often want to know why God allowed such a horrible thing to happen. Many people will argue that God is not fair or even say he is cruel. I believe that the Bible teaches that even if we do not understand; all things will indeed be reconciled in the end. That being said I absolutely do not think that the grieving parent should be given a lesson in theology while they are grieving! Rather they need a hug! They need support prayer or anything you can offer.