After a slow start, Washington’s spymasters used mathematics, chemistry, and sneakiness to turn the tide against the British
AS THE REVOLUTIONARY WAR was beginning, the rebels were staring into a critical void. When it came to passing secrets, Americans had few methods and few experts. Europe abounded with codemakers and codebreakers, but as an outpost of empire colonial America had no history of spycraft, and anyone there with such knowledge served the crown. The Continental Army and Congress were sieves. Something had to change, and George Washington, as in many areas, led the way in use of codes, ciphers, and other tools to conceal meaning and messages from all but intended recipients.
Recorded use of encryption dates to ancient Egypt. Cryptography emerged in Europe in the 1300s, beginning to resemble its modern form after 1499, when Johannes Trithemius published books on the topic. The best-designed and best-implemented forms did keep secrets secret, but the craft relied largely on human skills, such as poring over lengthy lists or tables and noting the results a letter or word at a time, which placed practical limits on enciphering and deciphering.
Queen Elizabeth I’s spymaster, Sir Francis Walsingham, used codebreakers to decipher messages being sent by Mary Queen of Scots, who had recourse to 100 ciphers and codes. By the 1700s, Europeans could count centuries of experience obscuring and uncovering messages. In government offices known as “black chambers,” analysts intercepted, opened, transcribed, decoded, resealed, and resent mail, often with no sign of tampering, and functioned like today’s cryptanalysts, identifying and breaking codes.
Cut off from these sources, Americans did as always: capitalize on scraps from the mother country combined with new ideas. Revolutionary mathematicians, chemists, and spies developed ways to help turn the tide of war.
George Washington and the Culper Ring
Gentlemen of the 18th century regarded spies as regrettable necessities, affronts to legitimate warfare. Officers who spied escaped
censure, but along with their fees hired agents earned customers’ contempt. To counter Britain’s battalions of spies and informants, on February 4, 1777, George Washington authorized Nathaniel Sackett to use his counter-intelligence skills to start a network that would gather and pass military intelligence in plain sight of the British. John Jay had recommended Sackett based on Jay’s experience working with him on counterespionage in New York. When Sackett washed out, Washington recruited Long Island-born Major Benjamin Tallmadge. In summer 1778 Tallmadge founded an intelligence service. A favorite of the general and a novice at the job—he briefly had worked for Sackett—Tallmadge took strongly to the spy game. He had a gift for organization and a knack for keeping his crew safe and undiscovered through the Revolution, instilling a code of silence that extended to writing memoirs or speaking of the spying campaign; historians are still trying to uncover what the Ring did. Tallmadge’s interest may have come from friendship: he was close to patriot Nathan Hale, executed as a spy by the British in September 1776.
Tallmadge recruited among friends and acquaintances from his hometown, Setauket, a British-controlled port on Long Island’s north shore, across the Sound from Connecticut. His hires included Lieutenant Caleb Brewster, referred by Washington, and farmer Abraham Woodhull, dubbed “Samuel Culper” after Culpeper County, Virginia, where Washington had done surveying. Woodhull refused wages but insisted on having his expenses covered. The enterprise came to be called the “Culper Ring,” after Woodhull’s alias.
At first, Brewster functioned as a courier, sailing between Connecticut and Long Island to fetch from Woodhull information he passed to Tallmadge. Riders ran material to Washington. As the war progressed, complexities multiplied. A “Culper Jr.”—Robert Townsend—was recruited in New York City, where he constantly risked discovery by the enemy. Some accounts say a Setauket woman, Anna Strong, arranged black and white petticoats and handkerchiefs on a wash line using a code Brewster was able to discern from his boat. The ring also relied on dead drops: unassuming locations at which operatives would leave documents, money, even disguises for comrades to retrieve. These could be as simples as a hollow in a trees or burial in an agreed-upon location in Woodhull’s fields.
Weaknesses and interception threatened constantly. By 1779, the Culper Ring badly needed better methods. “All it would take would be one overly curious trooper on patrol to find the incriminating documents and disaster would befall the Ring,” wrote Alexander Rose in his book Washington’s Spies, basis for the AMC series Turn.
Washington thought the answer lay in a new invisible ink, called in those days a “sympathetic” stain, a meaning rooted in the contemporary belief that chemical bonds resembled those, such as sympathy, among humans, Peter H. Reill explains in Vitalizing Nature in the Enlightenment. This advancement in concealment was the work of James Jay, brother of patriot John. James, knighted by George III in 1763 for reasons lost to history, was an insufferably vain bore and amateur chemist. He returned to America in 1778 a zealous patriot who toward the Revolution’s end grew disenchanted. Ostensibly undertaking a peace mission in 1782, he arranged his own capture by the British. He left for England after the Treaty of Paris. His more famous brother never spoke to him again. In 1807, James Jay lobbied Congress for repayment of a loan and in recognition of his wartime service and invention of invisible ink. Congress ultimately paid him a sum in 1813.
British agents had long known of sympathetic inks. Black chamber personnel knew to use heat and moisture to try to bring up messages. James Jay developed formulations resistant to those techniques. His notes on his solutions have been lost, but according to Rose’s reconstruction, Jay mixed gallic acid, made from tree galls, and gum Arabic, from acacia plants, into an invisible ink. To expose a message, a recipient applied a solution incorporating the common chemical ferrous sulphate. Ferrous sulphate was common in Britain, where it was used in ink, certain acids, and colorings, but scarce in America.
Jay’s complementary solutions functioned straightforwardly. A sender carefully wrote a message with the ink on relatively high-quality white paper, itself a rare commodity. Washington advised writing on the blank leaves in books and pamphlets, or between the lines of innocent correspondence. To expose the message, a recipient brushed on the ferrous solution, taking care not to smear. James Jay made these sympathetic fluids at a secret lab in Fishkill, New York, and supplied Washington with limited amounts for divvying up among American operatives.
Keen to put the Jay fluids to work, Washington described the system in an uncoded letter to Tallmadge. British forces intercepted his communique. The letter also referred obliquely to Tallmadge’s espionage work. Upon reading it, the British raided Tallmadge’s camp. He was not present, but the enemy force seized his saddlebags. In one they found a second unencoded letter from Washington, who later took his spymaster to task. “The loss of your papers was certainly a most unlucky accident and shows how dangerous it is to keep papers of any consequence at an advanced post,” Washington wrote to Tallmadge, ignoring the recklessness he had displayed.
Washington and others continued to use invisible ink, but Tallmadge, intent on better protecting his agents, shifted to encryption. His system combined a code and a cipher, a typical means of handling words not in the codebook. He first compiled 710 words appearing in Entick’s Spelling Dictionary. To indicate transposed digits, verb tenses, and plurals, he designated certain marks. His men often were spotty in use of code words but their technique only had to pass casual inspection, and it sufficed.
Along with successfully reporting on enemy troops and movements, the Culper Ring had particular triumphs. In July 1779, “Culper Sr.”—Woodhull—and “Culper Jr.”—Townsend—warned Tallmadge that British general William Tryon was planning a raid in Connecticut. The aim was to cause the American leader to divide his forces, setting up Sir Henry Clinton to crush him. Alerted, Washington sent only a portion of his Connecticut forces to the target. They arrived late, unintentionally allowing a British rampage so severe even Loyalists criticized Clinton’s forces for overdoing it. However, Washington eluded Clinton, who having failed to crush the mid-Atlantic rebels, shifted his attention to the war in the South.
In July 1780, the Culper Ring foiled Clinton again. Washington, whose Continental Army needed reinforcements, was expecting French ships to deliver a ground force in Rhode Island to accomplish that. Clinton planned to ambush the Frenchmen, a scheme Townsend learned of in Manhattan. Townsend used a trusted rider to pass a letter in sympathetic stain to an extremely ill Woodhull. He got the message to Brewster, who had the dispatch sped to Washington, who was away. Alexander Hamilton processed the message and alerted the French.
Forearmed, Washington feinted toward Manhattan, where the Englishman had his headquarters, causing Clinton to cancel the surprise attack and chase the Americans south. Also during 1780, Ring operatives learned that the British, who had stocked up on the paper used to print Continental currency, were about to flood the war zone with counterfeit bills. Forewarned, Congress foiled the scheme by recalling all paper money.
The Benedict Arnold affair involved code. Arnold and British accomplice John André successfully used ciphers to communicate on the sly. Not knowing this, Tallmadge nonetheless divined that André, caught carrying a plan of West Point given him by Arnold, was in cahoots with the American general. Aware Tallmadge was spying for Washington, Arnold had attempted and failed to lure the agent to his doom by telling him to rendezvous with “James Anderson”—André’s alias—and accompany him to a meeting with Arnold at West Point. Learning inadvertently of André’s capture, Arnold narrowly escaped. Tallmadge and even Washington admired André as a man of character, but the Americans hanged the British officer as a spy in counterbalance to the British execution of Nathan Hale, Tallmadge’s friend and fellow spy.
By forming a secret unit that employed code, invisible ink, dead drops, and rudimentary signal by laundry, with spies and relays traveling under cover of legitimate mercantile trips and illegal smuggling, Washington compensated for Continental forces’ severe deficits in funding, soldiers, training, supplies, and materiel.
Benjamin Franklin had little interest in cryptography, but as a master of statecraft he used such tools as he had. At the outset of the Revolution, working from Philadelphia, Franklin engaged Charles Guillaume Frédéric Dumas, a German supporter of American independence living in the Netherlands, to foster alliances for the Americans and nose around The Hague’s crowd of diplomats.
Dumas provided Franklin with an advanced cipher. Both were careless in stream ciphering (see sidebar), but apparently never were discovered. Between 1776 to 1781, Dumas helped the Americans negotiate a loan from Holland through the French and got the Dutch to recognize America’s independence, receive its representative, and sign a treaty.
No American in the 18th century worked harder than James Lovell to advance use of cryptography. Lovell, elected to the Continental Congress, served five years on its Committee of Foreign Affairs, sometimes as that body’s only member. Among his jobs was deciphering Dumas’s dispatches from Holland.
Lovell, a talented mathematician and linguist, adapted encryption techniques from schemes then in circulation, but could not teach anyone his methods, often because his own encoding errors garbled messages. Complaining by letter in 1780 to Lovell about her husband’s frustrations, Abigail Adams said, “I have seen my friend sometimes rub his forehead upon the receipt of a Letter, walk the room—What does this Man mean? who can find out his meaning.” She thought the better of that line, crossed it out, and wrote, “my Friend is no adept in investigating ciphers and hates to be puzzeld for a meaning.”
But Lovell did make great contributions to the science and art of decoding messages and examining systems of encryption, two of which paid off for his side of the war. In August 1781, American troops in the southern theater of war intercepted two British letters written in code. Lovell decoded the messages for Washington, noting that the British had made only minor changes to a key the Americans knew. With this information, Washington’s staff was able to decipher at least one dispatch that confirmed reports about conditions among British forces besieged with Cornwallis at Yorktown. On October 14, 1781, Lovell deciphered messages from General Clinton intended for Cornwallis but captured by rebels and revealing that Clinton intended to break a French blockade. Washington got the news October 20, three days after Cornwallis had offered terms of surrender. Nevertheless, Washington was able to alert the French naval commander to the possibility of a last-stab British attack.
John Adams was ambassador to Britain, between 1785 and 1788. Assuming his letters were being read, Adams used encryption—a chore, because of a dizzying profusion of codes until 1789, when a standardized nomenclator (see sidebar, below)
came into use. Between 1776 and 1789, however, American diplomats were using more than 17 ciphers, 10 mixed cipher/code sets, and 23 codes, Ralph Weber notes in United States Diplomatic Codes and Ciphers: 1775-1938.
In London, Adams sometimes assigned ciphering to daughter Abigail II, pet name “Nabby.” Writing to her brother of deciphering correspondence to Adams from ambassador to France Thomas Jefferson, Nabby told him, “When there is nobody elce to desyper I have the agreable task.”
Having no alternative, Adams dutifully persisted with codes. “He does understand it’s a necessary part of the job, but I think he’s very concerned with the delays it might cause,” said Sara Georgini, series editor for The Papers of John Adams at the Massachusetts Historical Society. “The use of cryptography becomes part and parcel of the story of the professionalization of the early American diplomatic corps.”
Like James Lovell’s, Thomas Jefferson’s analytical mind revealed solutions to him long before contemporaries grasped them. This was a problem, because Jefferson came up with cryptographic tacks beyond almost anyone, such as a cipher wheel for which he made undated notes sometime before 1802. The device, often pictured online, almost certainly was never made or used, says James McClure, General Editor of the Papers of Thomas Jefferson at Princeton University Library. Jefferson did envision his cipher wheel as a series of wooden cylinders, each with the alphabet written around its circumference, “jumbled & without order,” he wrote. He suggested stringing 36 cylinders side-by-side on a rod. A sender and recipient first would agree on the order in which to arrange the cylinders. To encrypt a message, a sender would rotate the wheels to spell out 36 characters of clear text, then copy any one of the other 25 rows formed by the aligned wheels. A recipient would dial in that scrambled mess and look along the other positions to find a line of text that made sense.
Rather than put his wheel into production, however, Jefferson seems to have set it aside in favor of a non-mechanical solution devised by Robert Patterson. A professor of mathematics at the University of Pennsylvania following the Revolution, Patterson as of 1805 was director of the United States Mint. His was a complex but seemingly easy-to-deploy cipher. Upon receiving details on the concept from Patterson, Jefferson wrote to him in March 1802, “I have thoroughly considered your cypher, and find it so much more convenient in practice than my wheel cypher, that I am proposing it to the Secretary of state for use in his office.”
Patterson had a sender write a message as a series of lines of text, transcribe them in the perpendicular—that is, if written left to right, transcribed top to bottom—and rearrange those lines. At the beginning of each line the send would insert “padding”—a specified number of meaningless letters. To decipher a Patterson message, a recipient needed a key giving the correct order of each line and the number of meaningless letters to remove from that line. The system was error-prone, even for Jefferson and Patterson, who could not get anyone else to use the method. One of Patterson’s messages remained unbroken until 2009.
Jefferson had no enthusiasm for encryption, only a profound aptitude he exercised if he beheld a need. For example, in advance of Meriwether Lewis’s western expedition he created a code which never saw duty. As soon as an impetus faded, Jefferson moved on—a shame, because he had made such progress with his wheel cipher, a concept that resurfaced in 1922, when the U.S. Army developed a new encryption tool: an arrangement of alphabetical wheels reflecting principles similar to those Jefferson posited. His notes about his wheel coincidentally resurfaced that same year. Put into production, this device was employed for decades by government cryptologists. Between the wars, pioneering modern codebreakers Elizebeth and William Friedman (“Hidden Heroine,” April 2018) significantly advanced the science, bringing paper-and-pencil methodology into the computer age.
Tools of Tradecraft
At the time of the Revolution, three methods of encryption saw wide use, along with a fourth, more difficult and less commonplace method.
Codes. A code substitutes a number or word for the name of a thing or person, or even an entire phrase. In a code John Adams employed, he was “Steady.” All parties using a code must work from the same list, or nomenclator. “One-part” nomenclators list words alphabetically and number them sequentially, usually starting with 1. Culper Ring leader Benjamin Tallmadge developed a code that looked in part like this:
As is common with codes, Tallmadge assigned proper names higher numbers, like 711 for General Washington and 728 for Long Island. Once a codebreaker had decoded one such word, the job of unmasking alphabetically adjacent words became easier because each code word had a corresponding number one higher than the one preceding, as shown above.
To improve security, cryptographers developed two-part nomenclators. The sending code was an alphabetical list of words assigned arbitrary numbers; the receiving code listed those numbers in ascending order with the corresponding code word. This deterred alphabetic solutions.
A book or dictionary cipher. In this gambit, parties agreed on a common work, perhaps a specific edition of Entick’s Spelling Dictionary, as Benedict Arnold sometimes did when corresponding with General Sir Henry Clinton and aide John André. To encode a word, a sender would find the page upon which it appeared in the dictionary and record the page number, the column, and the word’s position numerically, typically from the top of the page.
John Jay used dictionary ciphers. In a 1789 letter of his, “43d465” apparently referenced the word “politeness” for the 43rd word in the fourth (“d”) column on page 465. Users could layer on subterfuges, like the page number counted backwards from the end, or a number added to the second digit of the word count. Book ciphers had a flaw: other people had the same books. The Americans and British each used the 1777 edition of Entick’s for messages.
Substitution cipher. Many people know substitution ciphers from childhood, when they first learn how to compose secret messages. Readers of a certain age and fans of the cult film A Christmas Story will recall that radio advertisements made use of them. A substitution cipher substitutes one letter of the alphabet for another character. That can be a random letter, a unique symbol, or simply an alphabetical offset, such as A for E, B for F, C for G, and so on. Ciphers typically omit spaces, punctuation, and other telltales. Germany’s WWII Enigma encoding system, a staple of recent films and documentaries, was a sophisticated substitution cipher.
Even without a computer, a would-be codebreaker can unravel simple ciphers by analyzing word and letter frequency. Pro tip: E is the letter occurring most frequently in English, whose most frequent word is “the.” To plug this vulnerability, cryptographers created polyalphabetic ciphers, or multiple sets of offset substitution alphabets, which rotated through a different cipher in sequence, complicating the task of decoding.
Stream cipher. Benjamin Franklin in America and his operative Charles Dumas in the Netherlands relied on a stream cipher, an advanced and effective form of substitution cipher whose complexity made it difficult to use manually. Instead of rotating through a few alphabetic ciphers, stream cipherers agree on a common passage of text. Dumas once suggested Franklin use the first 682 letters of a book Dumas had had printed especially for Franklin and for which Franklin had written the introduction. This passage began, “Voulez-vous sentir la différence…” The stream cipher assigns each letter in the passage a number, starting with 1—1 is V, 2 is O, 3 is U, and so forth. Common letters have many numbers assigned, allowing the sender to decree any value. In the Dumas/Franklin passage, E had 128 different possibilities, a daunting task for a cryptanalyst. Franklin and even Dumas spurned subterfuge, often relying on a letter’s first few appearances, making messages easier to crack. It’s unclear whether foes broke any of their messages. —Glenn Fleishman