Unfortunately, this is not the case in reality. In fact, a child may become more difficult to understand for some time after they have amassed a rather sizable vocabulary. For example, when your son says his first word: dog (which sounds like "da") you know when he excitedly squeals "DA" that he saw the dog walk by his line of sight. Now imagine that his next words are Dada ("dada"), that ("da"), ball ("bah"), Mama ("mamama"), and duck ("da da da"). All of a sudden "da" could mean just about any of those things are seen. Better yet, as development continues, so does the ability to talk about things that aren't present. Maybe your darling baby boy is remembering the fun he had with his toy ducks in the bath last night, or trying to tell you he'd like to go off in search of the dog. Of course, as your child grows and gains more experience with speaking and the world around him, his language will become easier and easier to understand - first to those who spend time with him daily and then to friends, acquaintances, and even strangers.
If you aren't already living it on a daily basis, I'm sure you can imagine how frustrating this process can be for both adults and children. We want, so desperately, to convey our wants, needs, interests, excitement, fears, etc. We want, just as desperately, to understand our children when they try to convey these things to us. Unfortunately, it takes a few years for the muscles and experience necessary for speech to develop to the point of real fluency.
Luckily, we have the ability to provide our children a means of early communication while those muscles develop. Much like crawling provides an early means of transportation while the muscles and experience required for walking develop, using American Sign Language provides support while spoken language is still emerging.