A nightmare scenario is unfolding for the Californian parents of 12-year-old Jane Smith. Their child has been in a car accident and lies unconscious in A&E. The doctors say that Jane is bleeding internally — only an immediate operation will save her life.

Unfortunately it’s a Wednesday. That’s the day the surgeon on call is Dr Jones, aka Dr Death. He has killed every patient under his care for the past 10 years. The hospital would give anything to be rid of him. But Jones has tenure and that means he’s untouchable. In the past 10 years only 0.0007% of Californian surgeons have been sacked for incompetence. Bad luck to the Smiths; little Jane picked the wrong day to need surgery.

As far as I know, this scenario has never happened. American doctors simply aren’t that powerful. But until three months ago its teachers were. The dismissal rate of 0.0007% is a genuine statistic. That is to say, over the past decade just 19 incompetent teachers in California have been sacked out of a workforce of almost 300,000.

It’s no secret as to why: teachers in the state receive tenure after a mere 18 months. From then on, union regulations ensure that it takes years of hearings and can cost more than $1m (£610,000) to remove a single teacher.

Just to be clear, I’m not only talking only about schools being lumped with teachers who have no ability to teach whatsoever or who hate what they do. (Although it is impossible to get rid of them.) I mean schools with teachers who have caused lasting and irreparable damage to countless children. Teachers such as Mark Berndt who coerced his Los Angeles elementary school pupils into degrading sexual acts.

The school district did try to sack Berndt eventually. He appealed, so it offered him the usual escape clause: no mark against his record, $40,000 in cash, plus almost $4,000 a month for life to go quietly. The deal unravelled only after he was jailed in 2013 for 25 years for lewd acts against children.

The response of the two main US teaching unions, the National Education Association (NEA) and the American Federation of Teachers (AFT), to the Berndt case was to fight a 2012 California state legislature bill that would have made it easier to dismiss teachers accused of criminal misconduct against children. The unions’ success in mobilising money and foot soldiers against the bill turned out to be a Pyrrhic victory.

The ease with which they were able to strong-arm state legislators into ignoring public opinion confirmed to reformers that a political solution was impossible.

In May 2012, a few weeks before the bill died in committee, a little known pressure group called Students Matter filed a lawsuit on behalf of nine schoolchildren, claiming the teacher tenure system violated their right to a fair education. On June 10 this year Judge Rolf Treu of the Los Angeles Superior Court ruled in the group’s favour, stating that it was poor, minority children whose education suffered the most from union intransigence. “The evidence is compelling. Indeed, it shocks the conscience,” observed Treu.

The following month the former CNN news presenter Campbell Brown filed a similar suit in New York on behalf of seven families. More are expected to follow.

The big question is how on earth did the teaching unions go from being one of the great forces for good in America to becoming a vast protection racket? Back when the NEA was founded in 1857 by Pennsylvanian teachers its chief focus was remedying social wrongs. Raising teacher salaries was a concern but equally so was the education of emancipated slaves. ending child labour and fighting the forced assimilation of Native Americans.

The founding of the AFT in 1916 added a new dimension to the education fight with its rigorous defence of teacher independence. Starting with the famous 1925 “Scopes monkey trial” in Dayton, Tennessee — in which John Scopes was found guilty of teaching evolution — the AFT could be counted on to support the right to educate without religious or political interference.

Long before the civil rights movement, both unions fought to remove the petty rules and pay discriminations that kept down women and African-American teachers. If there was ever a group that deserved and needed a social champion it was the nation’s overworked and underpaid teachers.

In the 1960s, however, the two bodies began to compete for members. As part of that struggle they turned from being merely professional organisations into fully fledged labour unions.

Within a short space of time the NEA and the AFT, plus their countless local affiliates, had found the holy grail of labour politics: the ability to combine collective bargaining power with huge political influence. The former has allowed the teaching unions to dictate every aspect of school organisation; the latter has enabled them to block any attempt by government to impose reforms or new regulations.

Somewhere along the line the needs of children, their rights and futures became irrelevant. During the NEA’s 2009 convention the union’s general counsel spelt it out in case anyone was confused about the issue, saying: “It is not because we care about children and it is not because we have a vision of a great public school for every child.

“The NEA and its affiliates are effective advocates because we have power . . . there are more than 3.2m people who are willing to pay us hundreds of millions of dollars in dues each year because they believe we are the unions who can most effectively represent them.”

Ironically the NEA and AFT are so effective at winning battles over money that they don’t realise how badly they are losing the war for hearts and minds. It was while watching the Netflix version of House of Cards in the episode when Kevin Spacey’s Machiavellian congressman Frank Underwood achieves a political and moral triumph by cleverly forcing the self-serving teachers’ unions to accept reform that I realised they had entered the zeitgeist in the worst possible way.

The teachers’ unions are out of step with America and now, with the California court ruling, they are running out of time.