This is funny... I almost wrote a blog post about the issue of "psalms only" vs. "inspired songs only" just yesterday. I still might.
Mr. Foster was rather disingenuous throughout that debate. It is a big difference between inspired songs only, or the Psalter with a few select hymns; and his church's practice of having a full hymnal. (I'd also like to examine the question of Thomas Manton's views on the subject sometime, which he likewise skewered and misrepresented; but that would have to occupy a separate discussion.)
Three different Dutch Reformed denominations of which I am aware follow this practice, or something similar to it.
1. The Netherlands Reformed Churches still use the Church Order of Dordt, and follow Article 69: "In the Churches only the 150 Psalms of David, the Ten Commandments, the Lord's Prayer, the Twelve Articles of the Faith, the Song of Mary, that of Zacharias, and that of Simeon shall be sung. It is left to the individual Churches whether or not to use the hymn, 'Oh God! who art our Father.' All other hymns are to be excluded from the Churches, and in those places where some have already been introduced, they are to be removed by the most suitable means."
2. The Protestant Reformed Churches have changed it to, "In the churches only the 150 Psalms of David, the Ten Commandments, the Lord's Prayer, the Twelve Articles of Faith, the Songs of Mary, Zacharias, and Simeon, the Morning and Evening Hymns, and the Hymn of Prayer before the sermon shall be sung."
3. The Free Reformed Churches have, "In the worship services only the metrical version of the 150 Psalms (contained in the Psalter published through Wm. B. Eerdmans Publishing Company, 1965 edition and the Dutch Psalms, 1773 edition) and the 9 hymns composed of portions of Scripture, which have been approved by Synod, shall be sung. The approval of Synod shall be required before any other hymns composed of portions of Scripture shall be used in the worship services."
But it is noteworthy that individuals from the Protestant Reformed Churches have recently arisen as defenders of exclusive psalmody (Mr. Stewart among them); and that a young lady at my church back in Colorado Springs was raised in a Free Reformed Church, and they only sang from the Psalms, never from the few Scripture songs in the back of the Psalter; and that Joel Beeke of the Netherlands Reformed said in Semper Reformanda several years ago that they do not sing the Apostle's Creed or that other Dutch hymn (and that his Reformation Heritage Publications carries works specifically arguing for exclusive psalmody). There is a great deal of overlap between the two positions; and although I disagree with having any songs other than the Psalms sung in public worship, I have more essential agreement with them than with a church that uses Trinity Hymnal or something worse. It is a world of difference between a church that uses a hymnal primarily composed of man-made hymns, and a church whose songbook is "The Psalter," which happens to have a few non-Psalter material in the back of the book, which may or may not get sung.
That having been said, I would argue against the position, because,
1. We have neither precept nor example from Holy Scripture for the singing of any of those pieces in public worship.
2. Several of them are recognized to have not been songs, and there is question as to whether the "Songs" of Mary, Simeon, and Zacharias were actually songs; which would overthrow their appointment for singing God's praise in public worship.
3. For what purpose were the Psalms put together into a single book, but to constitute the full, complete, perfect hymnbook of God's people? The fact that certain songs appear outside of the Psalter which were also included in it, and that other songs appear outside the Psalter which were not included in it, demonstrates that these songs were selected for the purpose of singing God's praise in corporate worship to the end of time, while the others were not.