Circumcision in the Old Testament was a sign of the Mosaic Covenant. Paul tells us in Romans 4:11 that circumcision was "a seal of the righteousness that he had by faith" when speaking of Abraham. What were Abraham's instructions for future circumcision? One might assume with an individualistic culture such as our own that Abraham would only then circumcise those who subsequently made a profession of faith. However, Abraham was to circumcise all of his descendants. Isaac was commanded to circumcise both Jacob and Esau, though we read in Romans 9 that God already had decreed the salvation of Jacob, and knew that Esau would fall away. The sign was to be applied to those of faith and their children.
Paul writes in Colossians 2:11,12 "In him you were also circumcised, in the putting off of the sinful nature, not with a circumcision done by the hands of men but with the circumcision done by Christ, having been buried with him in baptism and raised with him through your faith in the power of God, who raised him from the dead." Baptism is now the sign of initiation into the people of God. It has replaced circumcision. Now the question that must be asked is, has this household principle changed? Who receives this sign? It must be assumed that if this had changed the authors of the New Testament should have made it clear to their readers or they would inevitably give the sign to their children.
There are many times when baptism occurs when the baptism is for a believer and his household. Some examples of this are: 1 Corinthians 1:16, Acts 16:15, Acts 16:33 and Acts 11:14. Paul and Luke here are extending the same practice which already happened in circumcision. The phrase used here for household is "Oikos." This phrase denotes in Greek, an entire family including children, and may indeed be pointing specifically to children and infants. This same phraseology is used in the Old Testament when discussing whole households which include children: Gen 7:1, 45:11, 1 Sam 25:6, 2 Kings 8:1 and several other places. Early church uses of the word denote a similar meaning such as in Hermas and Ignatius, both early 2nd century writers.
Proselyte baptism was practiced within Judaism at the time of the New Testament. This was baptism given to gentiles who converted to the Jewish faith. It is clear that it existed before the New Testament through the discussions of Shammai and Hillel. They were Rabbis who lived shortly before the life of Christ. Thus when John began baptizing, he was using a right already instituted but gave it new meaning. In proselyte baptisms, if a parent converted to the Jewish faith, their children would also receive baptism. There is no reason to believe that this changed.
Jesus himself says, "do not permit the children from coming to me" In Luke 15:16-17. The language here is similar to early baptismal language as Jesus says "do not hinder them". In early baptisms, one who had faith and was baptized was asked if anything hindered them from being baptized. Luke knows this and is using this phraseology to get the point across.
This idea that children of believers are separate from heathen children and therefore should be baptized is explained by Paul in 1 Corinthians 7. "For the unbelieving husband has been sanctified through his wife, and the unbelieving wife has been sanctified through her believing husband. Otherwise your children would be unclean, but as it is, they are holy." (7:14) The argument Paul is making here is not about the children. The issue he is dealing with is whether a Christian should stay married to a non Christian. He argues that since the child is holy with one Christian parent, the marriage is ok. The principle that the child is Holy Paul simply assumes that the Corinthians understand as he uses that principle to defend himself. He does not offer any defense for the principle of the holiness of children itself.
The evidence in the early church shows that this has been practiced since the beginning of the church. Origen (185-254) Hippolytus (170-236) Irenaeus (115-202) and Tertullian (160-220) all mention the practice. None of them try to defend infant baptism but simply assume it. Hippolytus whose family was in the Christian faith for a few generations testifies that it has been a tradition since the beginning of the church. He must have known of his grandfather or possibly great grandfather being baptized as an infant. This takes one to the very beginnings of the church.