I for one, was baptized as an infant. Growing as a child in the Church I can tell you it did and continues to have an effect on me. I was enermously curious about the day I was baptized, I would look at the photos and secretly pry into the drawer where my special baptismal candel was kept. Even though I don't remember the event of being baptized, as I grew I could always grasp something of its significance.
I left the Church in my early teenager years, but recently I have been moving to closer communion with it and have started re-attending Mass. I like walking into the Church, looking around and knowing this is my home, this is my tradition, this is what I was baptized into and it is now my privlege to be part of the Church that I was initiated so long ago.
I have a very strong pull towards traditional religions. In the past year I have been drawn, not only to Eastern Orthodoxy but as well as Orthodox Judaism to the point that I was considering converting to Judaism. In the cases of both of these I was always discouraged to an extent, because I was very aware that these were not the traditions of my family and that I could not partake authentically in them without actually converting and going through all kinds of "hoops" so to speak.
On a recent trip to Europe, seeing some of the most gorgeous Catholic cathedrals and pilgrim sites, I realized that this was indeed my root. In my baptism I was already a part of the Church, part of the continuity with my family and community and that, in my baptism, I was heir to a 2000 year Christian tradition.
For some of us, the feeling of finding our place in history is a very powerful feeling, and something that the Church, being so old, can offer. Its traditions also make membership a strong bond. I hope you can see from this post that, at least for Catholics, there is high value in infant baptism, especially for those of us that have a tendency to wander.