Colossus and the breaking of the wartime "fish" codes

Michie, Donald

COLOSSUS AND THE BREAKING OF THE WARTIME "FISH" CODES*

ADDRESS: Medina Apartments, 63-65 St. Marks Road, Randwick, Sydney NSW 2031 AUSTRALIA.

ABSTRACT: One of the authors of the recently released "General Report on Tunny," here describes his three-year experience as a founder member of the "Testery" and "Newmanry" teams. Their combined use of innovative methods and machines led from the breaking of the German Lorenz military traffic to its large-scale daily decipherment.

KEYWORDS: Bletchley, Lorenz, Tunny, Fish, Heath Robinson, Colossus, Tiltman, Tutte, Turing, Tester, Newman, Flowers, high-speed electronic computing.

PERSONAL PREHISTORY

In late 1941, following my 18 th birthday, a normal next phase would have been two further terms at boarding school, with an option for scholarship holders to proceed to a shortened University degree course before joining up. But over that Christmas my teenage imagination was fired by a tale from my father concerning a mysterious establishment at Bedford. He had it on the authority of the then War Minister, Sir James Grigg, that as preparation for doing something unspecified but romantic behind enemy lines there were opportunities to sign up for a Japanese course starting in a couple of months' time. I duly journeyed to Bedford and presented myself at the address given.

Sorry, wrong info

My request to enroll elicited from the Intelligence Corps officer who saw me a somewhat puzzled reply: "Who told you that we have a Japanese course now? That particular exercise is planned for the Autumn." Noting my confusion he added: "But we have courses on code-breaking. There's a new intake just starting. Would that interest you instead? I'll have someone find you a billet nearby. Make sure to be back here at 9 a.m. Monday."

In World War II one did not mess about. Returning to the London suburbs just long enough to pack a suitcase, I was back and signed in to the School of Codes and Ciphers, Official Secrets Act and all, on the Monday morning. With the rest of the new class I was soon held in thrall by our instructor, a certain Captain Cheadle, and by the black arts of codes and ciphers.

With nothing to occupy my evenings, I arranged to have my own key to the building and classroom. My habit became to return after hours to the texts and exercises. The resulting accelerated learning curve made my selection inevitable when a Colonel Pritchard arrived from Bletchley. He was on a mission to recruit for the new section that was being formed by Ralph Tester to follow up John Tiltman's and William Tutte's successive coups. The hope was that breaking and reading Fish traffic could be placed on a regular basis. The Pritchard interview lasted no more than a few minutes. I was to present myself within 48 hours at the entrance to Bletchley Park with a sealed letter.

After admission and a visit to the billeting office, I was parked in the Mansion House. My first task was to memorise teleprinter code until I could fluently sight-read punched paper tape. Pending completion of the Hut assigned to Major Tester's new section I sat as an ugly duckling in a large room filled to capacity by members of the Women's Auxiliary Air Force. What were they doing? Who knows? New arrivals were imprinted with a draconian DON'T ASK DON'T TELL principle in regard to anyone's immediate business but their own. I did, however, discover that those whose boy friends were on active service felt only contempt for an apparently fit young male in civilian attire. Some of them had lost boyfriends in the RAF, and many had boyfriends still alive but in daily peril.

Charm of a Second Lieutenant

The experience did nothing to ease my sense of disorientation in the new surroundings. Relief appeared in the person of a uniformed and exquisitely charming Intelligence Corps officer, Second Lieutenant Roy Jenkins. My task was to bring him up to my own recently acquired sight-reading skills. Roy's post-war career was to include Cabinet Minister and Chancellor of Oxford University. In my isolation, his company was rescue and balm. We departed to swell the ranks of Tester's new section, in my case via a most curious diversion.

Forty men and a teenager

On reporting to Ralph Tester I was immediately dispatched to take charge of a room like a small aircraft hangar. It was located at some distance from his new Hut. Within it there sat at tables several dozen uniformed men who remain in my memory as being all of the rank of Lance Corporal. What I can attest beyond error is that I quickly became convinced of the infeasibility of the operation which it was now my job to supervise.

As later explained, once the offset had been determined of two intercepts known to constitute a "depth" (the same message retransmitted with the same key, but with the plain-language message at a different offset with respect to that key) they were added so as to cancel out their common key. The resulting "depth-sum" text must logically then consist of the addition to itself at that offset, or "stagger", of a German plain language message. Given the text of such a depth, if one guessed that some character-sequence, say "GESELLSCHAFT" was likely to appear somewhere in the plaintext, then the experiment could be tried of adding that 12-character sequence (a "crib") to the depth's first 12 characters, inspecting the result, then to characters 2-13, 3-14, ... etc. in tedious progression through the text, a procedure known as "dragging". To keep track of what follows, the reader needs only to keep two things in mind: (1) that in international teleprint code "9" stands for "space" (as between words), and (2) that "add" refers to an operation known as "modulo-2 addition" which makes addition and subtraction indistinguishable. In consequence, if, say, HONES + OBDZE = NEST9 then NEST9 + OBDZE = HONES.

Let us return to the Lance Corporal dragging the crib GESELLSCHAFT step by step through the text of a depth-sum, pausing at each step to see what resulted from each successive trial addition. He stopped only if the result of his addition at any stage yielded, say, "SELLSCHAFT9U" (the symbol "9" denotes characterspace), at once concluding from such a local break that the offset was 2 and that the plaintext contained the sequence "GESELLSCHAFT9U". In the hands of a cryptanalyst the immediate next step would be to extend by two characters the "crib" that had been dragged, yielding, perhaps, "SELLSCHAFT9UNT", which would strongly suggest some further extension, say "GESELLSCHAFT9 UNTER", which might possibly be rewarded by "SELLSCHAFT9UNTER9A" and so forth.

Misplaced task-decomposition

So what was wrong with the reasonable-seeming thought that the task could be decomposed into a brute force (crib-dragging only) component and a skilled (extending the breaks) component? Why not first throw brute force at it and then pass the text on to the cryptanalysts with candidate breaks already found and flagged? Take a few dozen Intelligence Corps clerks each equipped with a list of cribs to be dragged, together with rule-sheets for the boolean addition of teleprint characters and for the recognition of common fragments of military German. Let them do the dragging, marking all local breaks found or suspected. Marked-up texts could then be sent on to the Testery proper, to receive the attention of cryptanalysts whose time would thus be conserved by prior delegation of the drag-work.

It sounded good. Experience soon convinced me otherwise. But my conviction had to be validated in the eyes of others. My only course was to drive the project along until its futility became evident, not to the band of massed Lance Corporals, but to the authors of the original proposal, whoever they were (this I never knew).

The flaw lay in the non-decomposability of a task once talent and much practice has melded it into a fluent unity. The cognitive psychologists speak of "automatization". To the eye of an observer from Mars, delivery of the serve at tennis might appear to be a sufficiently separate and stereotyped task to suggest a change of rules. The expert player might be allowed to employ a brute-force server (crib-dragging posse of Lance Corporals) who would on delivery of his service instantly quit the court, leaving the tactically highly skilled tennis professional (cryptanalyst) to continue the rally.

Trade-offs can be debated for each separate athletic or intellectual skill, but can only be quantified empirically, case by case. In tennis, as in depth-breaking, each opening move (the serve) flows smoothly and subliminally into the movesequence (the rally) that follows. The gains from continuity of the single-agent scheme probably outweigh in tennis the sacrifice of sheer serving speed. The same principles were eventually shown to dominate the depth-breaking case.

To the Testery

The dogged endeavours of my well-drilled force of crib-draggers in due course generated sufficient documentation for me to report that the "human wave" assault was unlikely to contribute effectively and was best disbanded. After this interlude, depressing for all concerned, I gained the long-sought shore of the Testery proper. I was turned over to a young graduate, now the internationally distinguished mathematician P. J. Hilton, for instruction in the earlier mentioned method known as "Turingery".

Peter knew all the Testery hand-procedures backward and forward, and played a massive part in perfecting them. My first and vivid memory was that, although only a year or two older than me, he smoked a pipe. My second was of his didactic strictures on my fetish of tidiness and aesthetics in paper-and-pencil work. I should say "my then fetish". With efficiency and speed at an unimaginable premium, not to mention justified awe of my new mentor, I was cured of this kind of perfectionism for life!

Other vivid images of my first encounter with the Testery are first and foremost of Major (later Colonel) Ralph Tester himself. I recall his mesmeric impact on female spectators in the lunch break as he leapt, daemonic and glowing about the tennis court with an animality that I had only ever envisaged as radiating from the great god Pan. Yet a year later when I was already in the Newmanry, engaged in a machine-based attack on the same Fish ciphers, the same man was ashen under his tan. He had had to summon me (presumably at Newman's request) to reprove my conduct. Why had I been canvassing the cryptographic staff of both Newmanry and Testery for signatures to a petition for the administrative merging of the two sections? With the naivety of a nineteen-year-old I was oblivious of such facts as that, even if a Foreign Office section and a War Office section could have been merged, one or other of Tester and Newman would have had to be dumped, and that it would not have been Newman. An ingenious administrative compromise resulted. A fictional "Mr. X" appeared on Newman's books whose fake identity four selected Testery staff assumed for periods in rotation, acting as a species of internal consultant. This gave good technical liaison, previously absent.

Tester had the sense of purpose and personal humility of an outstanding leader. At the time of Rommel's retreat to Tunisia, we suddenly found that some mysterious change in the system had locked us out of the Berlin-Tunis channel. A group of us offered to go flat out round the clock. Ralph's cryptographic skills were really too unpractised to be of material help, as he and we knew. But he sat among us, bolt upright as was normal for him, unflagging as the hours raced by. In the end the hours were not racing, and we young Turks were drooping and nodding. Ralph, focussed and refulgent as ever, saw this: "You know," he said tactfully, "it's easy for me. Most things go downhill with age. Stamina for some reason goes the other way. So you're no good at this sort of thing until you're at least forty. Another coffee, anyone?"

During the glory days of the American space programme, when the mean age of space vehicle commanders seemed to be getting more and more venerable, I recalled Tester's words.

Strange incident, best forgotten

The Testery's machine operators were ATS girls ("Auxiliary Territorial Service" I think). One of them, Helen Pollard (now Currie, see References) in her reminiscence of the Testery speaks not only of the thrill of it all but briefly hints at a romantic attachment. That attachment outlasted the war. If there is to be a dedication of this memoir then let it be to her.

For all the attractions of the new life, or perhaps because of them, I could not drop from my mind the initial "white feather" impact of that roomful of WAAF girls. While on leave visiting my home in Weybridge, I learned from my father of questions from his peers at the St. Georges Hill golf club about what his son was doing for the war effort. Apart from knowing that I was not after all learning Japanese, his mind was unavoidably blank on what I was now up to. It was out of the question to give information of any kind to any person outside the wire beyond "sort of clerical work" or the like. He asked me whether I had ever considered active service.

Back in BP I asked for an interview with Colonel Pritchard, and requested a transfer to the North African desert. Pritchard let me finish. Then he said: "Who's been getting at you?" Taken off guard, I waffled. "No-one?" he enquired politely, and let his question hang in the air.

Eventually I blurted out that my father had mentioned such a possibility, but had applied no pressure. Anyway, I maintained, it had nothing to do with my decision. There was another uncomfortable pause. Then: "I have to instruct you to return to duty. You see, Mr Michie, we have a war on our hands. Inconvenient, but unfortunately true. Unless you have further questions, you are free to return at once to your Section." Pause. "And by the way, I do not expect you to raise such matters again." Pause. "Either with me or with anyone else." Longer pause. "As for your father, I do not anticipate that he will raise them either."