This is a question I addressed several years ago on this blog when I was reading J.C. Ryle's book, Knots Untied, where he discussed the matter specifically in the context of the early 20th-century Church of England, with those who were wrestling with whether or not to remain in that denomination. I was interested to see the question also come up in John Frame's book, The Doctrine of the Christian Life, where he makes these observations:
Remarkably, Scripture never says that believers should leave a church organization and form a new one because of false teaching. Israel in the Old Testament was often guilty of idolatry. Revivals of true worship occurred from time to time, but the nation, including the religious establishment, relapsed. After the exile, the scribes and Pharisees represented movement toward religious purity, but Jesus said they 'shut the kingdom of heaven in people's faces' (Matt. 23:13) and made each proselyte 'twice as much a child of hell as yourselves' (Matt. 23:15). They were 'full of hypocrisy and lawlessness' (Matt. 23:18). Jesus says that God will judge these religious leaders (Matt. 23:32-36), a threat fulfilled in the destruction of the temple in AD 70.
But nowhere in the Old Testament, nor in Jesus' teaching, does God command believers to abandon Israel and to form a new nation, church, or denomination. God himself brings a separation between the followers of Christ and Judaism when the synagogues expel Christians from their fellowship, and when the temple is destroyed. But there is no exhortation in the New Testament for Jewish Christians voluntarily to leave the synagogues. Rather, it is assumed that believers, like the apostles, will bear witness within the synagogues to God's grace in Christ, as long as they are able to do so. This was the practice of the apostle Paul, who preached the gospel in the synagogues wherever he traveled.
...There is doctrinal and practical corruption in the New Testament church as well. But again, the apostles do not call on believers to leave their churches and form new ones because of corruption. Rather, the churches themselves are to take action against it (as in 1 Cor. 5). Even the church at Laodicea, which Jesus threatens to spit out of his mouth (Rev. 3:16), is still a church (Rev. 3:14), and Jesus does not counsel true believers to leave it. Rather, he tells the whole church to repent (431).
That all being said, Frame then points to the history of the church and laments the breakdown of the church in the post-apostolic period as Christians tragically abandoned the call of God to pursue unity and things gradually unravelled. Emerging from the disunity is what we know today as denominations, something Frame suggests is a poor substitute for the middle-level structures or networks of local churches that we see in New Testament church, that sit between the local and universal church. Our modern denominations are especially problematic when it comes to dealing with false teaching. 'Denominationalism,' Frame writes, 'cripples the church in its battle with false religion in its midst. The church cannot speak with one voice against error, as it could in the first century' (433). Anglicans and Lutherans do not deal with apostasy in Presbyterian or Baptist churches, for example, and anyone who is excommunicated from one denomination can easily go join another. The division in the church means then that, in many ways, false teaching is never really dealt with.
As much as I don't like to be cynical, there is little hope of recovering the type of unity we see in the New Testament church on this side of Christ's return (and efforts to do so have usually sought a type of unity that does not reflect the biblical idea of unity), and the reality we know of different traditions and denominations is the reality we are likely to continue living in. So what does this mean for us then as we think about the question of when it is appropriate and right to leave our denominations to join others or to form new ones? Frame suggests the Reformers were on to something when they spoke of the 'marks of the true church' – the pure preaching of the Word, the right administration of the sacraments, and effective church discipline – and that these can still function as a helpful guide in this matter today (although we must humbly acknowledge that no church on earth is able to uphold these marks perfectly). Where a denomination as a whole abandons these things, we can begin to consider moving in a different direction.
Of course, all of this raises many more questions than I can deal with in a simple blog post, but there is much here to think about. Still, in the end, there are no straightforward answers to the question. Frame concludes,
So far as I know, nobody has ever made a serious theological study of the level of error that renders a church apostate, or, to put it differently, the kinds of theological differences that are tolerable or intolerable within a church body. I commend that study to theologians younger and more astute than I. I see this as a daunting task. Perhaps it is not possible to determine such levels objectively (435).
In most cases, then, it seems that it is best for us to remain where we are and to continually work towards reforming our churches and networks and denominations in line with Scripture. Surely any further splintering and division of the church is something we must reject outright. And as all of us work towards reforming our churches and denominations and traditions, we must be earnestly praying that these efforts will coalesce and draw us together to ultimately re-establish the real and tangible unity of the body of Christ here on earth.