Ok, I'll try.
I think what makes it tend to be scarier is two things.
1. The fact that whatever the evil that is being dealt with is SO evil that it harmed a child somewhere down the line to a point that the child is now involved. And the knowledge (concious or sub-concious) that if the evil that's there is so cold and dead that it would hurt a child, then it has no boundaries and has no mercy. So it tells you that you have no way of reasoning with it and it will act with reckless abandone without any conscience what so ever. Quite simply, there will be no mercy.
2. We all have the responsibility to protect our innocent children. If the child (voice or whatever) is being used for evil, it gives you the feeling that a child has a sinister power beyond our own and whatever is behind the child would obviously be even more powerful than the child. Plus, given that we are all responsible to protect our children, the child could be exacting revenge for not being protected in the first place.
All rolled into one.
That's the best way to describe my thoughts on it.