The Speaker of the House of Commons, John Bercow, is in trouble. Rumours of a plot to unseat him have been swirling around Westminster this week, with Deputy Speaker, Nigel Evans, being touted as a possible replacement – and preferably sooner rather than later – by some Tory MPs. If Evans is making any attempt to distance himself from these manoeuvrings, I have yet to notice. Bercow’s most recent critics, including MPs Alan Duncan and Mark Pritchard who bellowed after a passing Bercow entourage;  “You are not ****ing royalty, Mr. Speaker” after being told to stand aside for him, were joined this week by a much respected former Speaker, former Tiller Girl, Betty Boothroyd. She witheringly chastised Bercow for his “scruffy appearance”; a sartorial inelegance which in turn, she believed led to a growing lack of respect towards the Speaker from public and politicians alike. For on being elected Speaker, John Bercow had dropped the traditional Speaker’s costume of collars, cuffs and buckled shoes, instead opting for a black gown over a lounge suit. So, is all of this simply another week of over excitable intrigue in the Palace of Varieties otherwise known as Westminster?  Up to a point, Lord Copper. But Speaker Bercow would be foolish to ignore the ratcheting up of pressure on him that has been taking place.

Despite running against him as a Labour candidate in the 2002 General Election, I have no personal animus towards John Bercow. He remains an effective and assiduous constituency MP and my late Uncle was one of his close political supporters. However, through the manner of his election as Speaker, (largely through the votes of Labour MPs determined to enrage the Tories opposite), to his house flipping to avoid paying capital gains tax, to his extraordinary ability to alter his opinions on almost everything to suit his political career, Bercow today remains a symbol of a still unreformed House of Commons and is held by some to be partially responsible for the crisis of legitimacy that still grips Parliament. Junking the Speaker’s traditional garb of cuffs, collars and tights, as John Bercow did, was never going to be enough to persuade many that it was otherwise. For during the 2009 Parliamentary expenses scandal, it was revealed that Bercow changed the designation of his second home on more than one occasion – meaning that he avoided paying capital gains tax on the sale of two properties. He also claimed just under £1,000 to hire an accountant to fill in his tax returns. Bercow denied any wrongdoing, but agreed to pay £6,508 to cover any tax which he may have had to pay to HM Revenue and Customs.

For all of these reasons it is now time for Speaker Bercow to pack away his school master’s cloak, and return to the back benches as an Independent, perhaps as a Labour leaning, MP. He wouldn’t be the first Labour MP for Buckingham; that somewhat dubious honour goes to the late Robert Maxwell. Bercow would, after all, be entitled to an extraordinary lifelong pension of £85,000 a year, alongside an MPs pay.

For now Parliament desperately needs a statesmanlike figure, a new Speaker, who not only commands the respect of both sides of the House, but more importantly, the country as a whole. The candidate John Bercow originally defeated to become Speaker, with all of those Labour MPs votes, Sir George Young might be persuaded to run again. Or there is former Liberal Democrat leader and former Foreign Affairs Spokesman, Sir Menzies Campbell, who would command the respect of all parties; and not to be overlooked there is always Ken Clarke who would look rather splendid in the wig, tights and breeches of high office.

Many Tory MPs have never forgiven Bercow for his political trajectory, which took him from far Right Monday Clubber to touchy feely liberal leftie within a couple of decades. Once upon a time Bercow argued for voluntary repatriation of immigrants, for hanging Nelson Mandela and was a hot favourite at Unionist gatherings in Northern Ireland. Today his is a more mellowed voice, one that advocates gay rights and human rights in countries like Burma.  Some, especially in the Labour Party, applaud Speaker Bercow’s political journey, and the role of Labour supporting wife and ‘Twitter’ fiend, Sally in edging him along the way. But, argue his Tory critics, Bercow has cut his cloth to suit his interests, and those interests for a long time were directed at being elected as Speaker.

The gift for which Bercow pined lay largely in the hands of Labour MPs, and who subsequently put him there, doing so in the sure fire knowledge that this would infuriate his erstwhile colleagues on the Tory benches (it is estimated only three Conservative MPs actually voted for him). So even without all of the controversies and vexations, Bercow’s election, when as tradition demanded the ‘Buggins turn’ principle of a Tory choice of Speaker, was always a time bomb waiting to go off.

Yet history may one day record that it a was simple letter to a constituent on headed notepaper that proved the final un-doing of Mr Speaker, the Rt Honourable John Bercow, Member of Parliament for Buckingham. For this flimsy piece of paper has helped light the blue touch paper of revolt touted by Bercow’s growing band of critics in the House of Commons who want him to go and to elect a new Speaker, who they believe will be less biased against them.

Speaker John Bercow, recently wrote to a constituent saying that fox hunting had “no place in the 21st century” and that he remained in favour of the hunting ban. Few Bercow expostulations are complete without added supportive ‘twitterings’ from wife Sally Bercow, and as the double act blundered on, not just a few a few wiser heads were left wondering what on earth the supposedly impartial Speaker thought he was doing?

But the truth is that Speaker Bercow usually does know exactly what he is doing. It is just this time he has been rather caught out. If it is possible to change one’s political convictions on almost everything, it is really quite easy to say one thing to one person and something quite different to another. The point, for old style politicians at least, is to avoid being caught out doing so. In 2004, John Bercow voted strongly against Tony Blair’s hunting ban. Today, Guy Portwin, master of the Kimblewick Hunt in Bercow’s constituency says rather diplomatically: “We are all sorry that John Bercow has changed his previous strongly held convictions on this subject.” In fact as late as Spring last year, Bercow was still sticking to his pro-hunting convictions and told an election meeting in Brill, Buckinghamshire that he remained strongly in favour of hunting. “Bercow’s support for hunting last year was every bit as strong as Flurry Knox and Friedrich Engels!”, recalls Independent candidate John Stevens who also spoke at the meeting and in favour of lifting the hunting ban.

The next Speaker, whoever it is and whenever the time dictates, will be dragged by colleagues protesting from his or her Commons seat to that of the Speakers high dais, as is the tradition. This next Speaker will preferably be genuinely reluctant to be dragged from his or her seat, but once there don the uniform that goes with it with grace and be seen to be utterly without reproach. And to help whoever it may prove to be, the Speaker should immediately cease representing a parliamentary constituency, and simply be accountable to Parliament. For if a majority of MPs still think that the status quo, and one in which John Bercow as Speaker is still an integral part, is good enough, the public feels quite differently. For there are few places anywhere where the disconnect between the rulers and the ruled is as profound as it is in Britain in 2011.