WHY DID Great Britain stay neutral during the American Civil War? Back when I was researching this question, one answer that seemed particularly intriguing was the claim — made at the time in America and by subsequent historians — that it was due to a severe wheat shortage.

Repeated crop failures in the early 1860s had led to a massive reliance on imports from America and Russia. Ergo, Britain intervening in the war between the states would have been an unaffordable risk.

I combed through four years of cabinet reports, memoranda, letters and diaries, looking for proof. Cotton, slavery, Canada, blockade running, the balance of power: these were frequent subjects of fretful debate, but never wheat. The paper record showed the theory to be an utter dud, thereby freeing me to find the true causes of British neutrality.

I tell this story because I don’t see any point in hiding the fact that I am entirely partisan in the debate about government transparency. I believe that everything should be maintained in its proper place. What is classified should remain so, what is public should be open, and all must be preserved for future scrutiny.

Yet leaving aside my particular interest in history, I think what is happening right now in Washington beggars belief — starting with the revelation last week by The New York Times that former secretary of state Hillary Clinton used a personal email account for all official business during her four years in office.

By itself the revelation is bad for a number of reasons: nobody knows what security lapses may have taken place, or how much official State Department history is now the private property of the Clintons, or whether other departments are running blind because of information that was withheld either inadvertently or on purpose.

It would be a huge mistake to concentrate on Clinton alone, however, or to see her misuse of government information as just another example of the way the Clintons operate. In this case she was acting in accordance with established practice. For a country with such a short history, its leaders seem to have had almost no regard for respecting what little there is.

When the republic was in its infancy, James Madison, the father of the country’s constitution and its fourth president, warned: “A popular government, without popular information, or the means of acquiring it, is but a prologue to a farce or a tragedy; or, perhaps both.” He was one to talk, having cut up documents to give his signature to autograph-hunters.

From George Washington onwards it was a matter of luck whether presidential papers ended up on the bonfire or in the hands of a reputable institution. It was only in 1978 that Congress declared such archives to be government property. This is why the papers of at least 14 presidents are either gone or incomplete, and those of a further 10 have perpetual restrictions on their access.

You may think it doesn’t really matter if the record is skewed for presidential nonentities such as Franklin Pierce, but I defy anyone to say the same for Richard Nixon, who fought tooth and nail to keep his papers out of public hands. Moreover, there remain high-ranking officials who refuse to accept that the rules apply to them too.

In 1972 somebody swiped almost the entire archive of former Supreme Court justice Felix Frankfurter from the Library of Congress. In 1996 then CIA director John Deutch was discovered to have transferred highly classified documents to his home computer. More recently, in 2003, former national security adviser Sandy Berger visited the National Archives in order to smuggle out confidential documents prior to appearing before the 9/11 Commission.

Finally there’s former CIA head General David Petraeus, who pleaded guilty last week to giving classified information, ranging from his Afghanistan notebooks to the password of his CIA account, to his former mistress and biographer, Paula Broadwell.

With the advent of email, politicians can hide their papers in plain sight by using personal email accounts such as Clinton did. During the administration of George W Bush, at least 88 officials, including Bush’s adviser Karl Rove, conducted official business using their Republican National Committee email address.

Then there’s Richard Windsor — the name and government email account created by Lisa Jackson, the former head of the Environmental Protection Agency under President Barack Obama, for use when writing to certain officials and activists. Windsor was so active the agency awarded him an ethics certificate three years in a row. When found out, Jackson’s excuse was that it was common practice for officials to use more than one government account in order to control the flood of emails.

The reasonableness of that explanation was exploded by the Associated Press (AP) in 2013. The news agency discovered that many of Obama’s top appointees were using secret email addresses for official business.

Which raises the question of whether any congressional or internal investigations, or public information requests, would ever uncover these shadow email accounts. The labour department did everything possible to frustrate AP’s request to hand over all government emails, including demanding $1m (£660,000) or more for “research” costs.

Frankly, Washington is running amok. In addition to the usual chicanery that plagues all governments, information has become its own form of black market currency. There’s power in controlling, leaking, hiding, stealing and twisting it, and this permeates Washington culture from the bottom to the very top. Bush was guilty of it — in 2007, for example, leaking news about the proposed troop surge in Iraq — as is the White House now; the latest being the accusation by Senator John McCain last month that officials leaked details of the plan to retake Mosul from Isis, also known as Islamic State.

The consequences are profound. Why should anyone follow the rules, let alone a traitorous weasel such as Edward Snowden, when the government doesn’t? It isn’t just the destruction of public trust that’s at stake. It’s the truth of our past, our present and our future.