Here is a quote from an article posted on the Tenth Presbyterian Church of Philidelphia website explaining why we baptize infants. The entire article can be found here.
The article is written by Richard Phillips.
We must address the question, “Who should be baptized?” Our church baptizes adults upon profession of faith, and baptizes children presented for baptism by one or more believing members of the church.
Infant baptism is based upon the continuity between New Covenant baptism and the Old Covenant rite of circumcision. A considerable amount of Scripture upholds this continuity. In Romans 4:11, the apostle Paul notes that circumcision was the seal of “the righteousness of faith.” In Colossians 2:11, 12, Paul points to the same spiritual reality (separation from the sinful nature) by means of both circumcision and baptism. Furthermore, the privileges of the New Covenant are hardly less than those of the Old Covenant. The inclusion of children in the covenant is not rescinded under the New Covenant, but is directly affirmed in Jeremiah 31:33 and in Acts 2:38-39. Finally, Gal. 3:27- 29, one of the key texts on baptism, relates baptism to circumcision by rejoicing that not only men but women may receive the covenant sign, not merely Jews but Greeks, etc.
By baptizing infant children of believers, we take seriously the promise at the core of God’s covenant: “I will be your God and the God of your children after you” (Genesis 17:7; Exodus 19:5, 6; Deuteronomy 7:6, 14:2; Jeremiah 31:33). We take seriously Jesus’ words regarding the little children, as found in Matthew 19:13, 14: “Then little children were brought to Jesus for him to place his hands on them and pray for them. But the disciples rebuked those who brought them. Jesus said, “Let the little children come to me, and do not hinder them, for the kingdom of heaven belongs to such as these.”When he had placed his hands on them, he went on from there.”
This is not just a story about how Jesus was nice to little children. These are the covenant, circumcised children of Israel (at least the boys were circumcised). And here is the Messiah, the Lord of the Covenant, laying His hands on them for blessing and praying for them to the Father. What an encouragement this is for us to bring our children to receive the blessing of the covenant sign of our Lord.
Along similar lines, we note that children of believers are accountable as members of the covenant. In Ephesians 6:1, 4 and Colossians 3:20, 21, the apostle Paul commands children to obey their parents “in the Lord”. Parents, he says, are to bring their children up “in the training and instruction of the Lord”. Here we find that the children of believers are treated as Christians, to receive the benefits and obligations thereof. Furthermore, in 1 Corinthians 7;14, in a passage discussing marriage and divorce, Paul remarks that children of a believing parent are “holy”. Paul is not saying that these children are automatically saved or that they automatically come to faith in Christ; however, he is saying that such children are set apart in God’s sight.
Finally, we take note of the household baptisms seen in the New Testament (Acts 16:15, 33, 34; 1 Corinthians 1:16). Of only twelve actual baptisms recorded in the New Testament, three of them are household baptisms. Scripture does not tell us that infants were baptized in these occasions, nor does it say that all who were baptized believed. Indeed, the clear inference from these household baptisms is that those under the headship of the head of the house received a benefit from his/her belief. That benefit was baptism, and a public identification with the church.
It is because of this biblical data that our church, along with the whole of the Reformed tradition, baptizes infants of believing adults. In so doing we gratefully lay hold of the promise of our God, when he says:
“I will establish my covenant between me and you and your seed after you throughout their generations for an everlasting covenant, to be a god to you and to your seed after you.” Deuteronomy 29:10-13.