Inside the NSA: The secret world of electronic spying
Eavesdropping on the world
CNN (19-23 mars 2001)
From its headquarters outside Washington, D.C., the supersecret National Security Agency eavesdrops on literally billions of communications worldwide. But for some, the awesome power of the NSA is a source of concern. CNN National Security Correspondent David Ensor takes a behind-the-scenes look at the NSA in a special series airing the week of March 19-23.
Agency's challenges more complex in post-Cold War era
It was simpler during the Cold War when the National Security Agency had one major target -- the Soviet Union. Now there are many new targets and problems.
"NSA is definitely an agency in crisis right now because the world has shifted under its foundation," said James Bamford, author of "Body of Secrets," a forthcoming book about the NSA. "This is not the Cold War anymore. We're facing brand new realities, and NSA has been very slow to move into this new world."
The agency is at the forefront of technological research and has long used highly technological methods in its mission. It listens to a vast amount of communications traffic worldwide, using listening posts around the world like Menwith Hill, a location in England with more than 20 satellite dishes.
"Its technological capabilities are really into science fiction," said Bamford, who also wrote "The Puzzle Palace: A Report on America's Most Secret Agency." "I mean they can really listen to communication thousands of miles away from anybody being there. Somebody in NSA could be listening in real time to somebody making a phone call in China, for example."
One of the NSA's technological headaches is the increasing use of fiber-optic cables. Signals transmitted through the air can be listened to, but tapping into traffic transmitted via fiber-optic cables is much harder.
A fiber-optic cable is a thin glass cable a little thicker than a human hair surrounded by a plastic coating. Data is transmitted through the glass cable in light waves instead of electrical pulses through copper cables. Since fiber-optic cable passes electrically nonconducting photons through a glass medium, it is immune to electromagnetic interference and much harder to wiretap than traditional phone lines.
Another problem is the increasing use and sophistication of encryption software available to the general public. Public key encryption allows the sender of a message to scramble its text, and only the recipient who has an electronic key can unscramble the message and read it. If a third party such as the NSA intercepts the message, the encryption can theoretically be cracked. But depending on the level of encryption, decrypting the message could take a massive amount of computing power.
Hayden declined to say if there are publicly available encryption tools that the NSA cannot overcome.
"I can't talk about what we can and can't do, but I can say that the global availability to encryption products and services is one of the challenges that we have to face," he said.
"Where we are today is that there is too much of it, and it is too hard to understand. So it is a volume, velocity and variety problem for us," said Maureen Baginski, director of the NSA's Signals Intelligence (SIGNIT) program.
In early 2000, the flood of data overwhelmed the NSA's vast complex of supercomputers. For 3 1/2 days, the overloaded system simply shut down.
"The NSA is drowning in material. I mean, it has been since its beginning and even more so today because it's able to intercept so much more," Bamford said.
Hayden said the cause of the computer shutdown was the inability to create a system that could meet the agency's operational needs. He declined to characterize the agency as drowning in data but said it's a "far better metaphor than going deaf."
"It's a challenge for us to keep technological pace with the adversaries of the United States," he said.
The NSA said its computer problems are now fixed, but it is asking Congress for billions in new funding to create what it calls "Trailblazer" -- a computer system designed to better process and gain useful intelligence from the vast quantities of information the NSA collects around the world.
The agency says Trailblazer is one of its major modernization efforts to the Signals Intelligence (SIGINT) programs. The agency is working on studies to research and create the system concepts and architectures for the Trailblazer initiative.
The NSA: Spying on you?
"What's happening, of course, is that the NSA says, 'Trust us, we're the government. We won't abuse the law,'" said Barry Steinhardt, associate director of the American Civil Liberties Union. "Of course, what they're really saying is: 'Trust us, we're the government spies and we won't abuse the law,' but since there is no real check on them, there is no way to know that."
The 1998 movie "Enemy of the State," starring Will Smith and Gene Hackman, portrayed the NSA as an evil Big Brother spying on Americans. The plot involves the NSA's deputy director, who oversees the murder of a congressman due to his opposition to a bill that the NSA official wants passed.
A Washington lawyer unwittingly comes into possession of evidence of the murder, and the NSA official turns the full force of the agency's eavesdropping capability on the lawyer's life. In reality, the NSA is prohibited from spying on U.S. citizens unless it can prove to a special court that there is probable cause to believe that a citizen is committing espionage or other crimes as an agent of another government.
Lt. Gen. Michael Hayden, the NSA's director, saw the movie in a South Korean theater filled with other military personnel while he was under consideration for his current job. The movie did not paint a positive picture of the NSA, he said.
"I made the judgment that we couldn't survive with the popular impression of this agency being formed by the last Will Smith movie," Hayden said.
Hayden sensed an image problem. That recognition in part explains why the NSA decided to let CNN inside the agency to see where code breakers work to crack the secrets of other nations and where code makers protect U.S. secrets.
Hayden said he wants to ensure that the NSA isn't viewed as trampling on the privacy rights of U.S. citizens. He said the nature of the NSA's mission requires it to be a secretive agency. But he added that the agency is trying to make certain that Americans know it follows the law while enhancing U.S. liberty and security.
"Could there be abuses? Of course, there could, but I am looking you and the American people in the eye and saying there are not," Hayden said.
Oversight reforms passed in the 1970s
"After Church and Pike, on this question, the ball and strike count on the agency is no balls and two strikes," Hayden said. "We don't take any pitches that are close to the strike zone. We are very, very careful. We can't go back to the American people with, 'Oh, well, we're sorry for this one, too.' We don't get close to the Fourth Amendment."
But the ACLU's Steinhardt said the laws dealing with these issues need to be revised and congressional intelligence committees don't exercise much oversight.
"Unfortunately, the Congress -- which is the only entity that can really ask these questions, has not so far asked them ... in an aggressively satisfactory way," he said.
The technological advances that have increased the ability for people to communicate also increase the possibility that the NSA may run across more U.S. citizens while performing its mission. When eavesdropping on a drug ring in Colombia, separating other countries' nationals -- who can legally be bugged -- from U.S. citizens or residents is not always easy.
Domestic law enforcement agencies also pressure the NSA to help out with issues such as drug trafficking.
"Increasingly, intelligence agencies are becoming domestic law enforcement agencies," Steinhardt said. "The excuse for that is that some law enforcement issues are international, like the drug trade, for example."
Staying on the safe side of the law
Hayden said the burden of proof is on the NSA when seeking such authorization. He declined to say whether the agency had ever been turned down.
"It's hard for me to get into operation matters," he said. "Let me just say that we are very careful about what we ask for, that we have to be very confident that we have a very strong case, and therefore, when we go forward, we think we have very powerful arguments, and our track record reflects that."
In Europe, the debate about the NSA and privacy centers on the surveillance facilities in Menwith Hill, England. A 1999 European Parliament report suggested the United States might have conducted economic espionage to help American companies against European competitors. Hayden dismissed the allegation.
"That is absolutely not true," he said. "Everything we do is for a foreign intelligence purpose, and everything we produce is handed to an official agency of the U.S. government that is authorized to receive that information. We do nothing on behalf of American enterprise; we do nothing to gain advantage for American industry."
However, Hayden said if the NSA detects criminal activity, it is required by law to pass the information to other U.S. agencies such as the State Department. If the NSA learns that an overseas company is offering bribes to get a contract, that information does not remain a secret.
No matter what the scenario is, however, the issue of balancing privacy with the need to garner intelligence is unlikely to go away for the agency.
"It's a battle that goes on behind the lines in a great deal of secrecy, and how close they get to the line or whether they slip over sometimes is a matter that has to be watched closely," said James Bamford, author of "Body of Secrets," a forthcoming book on the NSA.
High-tech advances keep crack code team in security vanguard
"We see hundreds
of probes -- perhaps thousands on a daily basis. So it's a constant game,
if you will, of moving as quickly as your potential adversary, staying
current on the technologies that are necessary to maintain the security
and integrity of the networks," said Michael Jacobs, the NSA's director
of information assurance.
Passwords of the future
"I have four fingers enrolled so if I actually have a bandage on one finger, you can go to an alternate finger to gain access or you could require two fingers to gain access for the added security, " said Bob Rahikka, the NSA's lead researcher on fingerprint scanning.
Iris recognition is also a possibility as the human iris is a unique identifier for each person. NSA engineer Bob Kirchner said the agency's systems scan an image of the iris, convert it into a template and compare it against a stored template in a database.
"Iris has some
very unique advantages," he said. "One, we believe it is more
accurate than fingerprints. Two, it has the flexibility of not having
to contact the sensor or surface like fingerprints."
The researchers are testing another version that captures a three-dimensional image of a person's face and coloring.
The NSA has long been one of the most highly technological agencies in the federal government, and a building at NSA headquarters contains the world's largest collection of supercomputers in one place. A Cray Triton supercomputer used by the agency can handle 64 billion instructions per second. Machines such as the Cray are used for making as well as breaking codes.
Though spies have given some U.S. secret codes to adversaries like the former Soviet Union, NSA officials said they know of no critical American code system that has ever been broken.
Linux is a version of the Unix operating system, which is used to run many of the server computers that operate the Internet. Linux, created by Finnish programmer Linux Torvalds in the early 1990s, is an open-source operating system, meaning its source code is open to all, unlike a proprietary operating system such as Microsoft's Windows.
"Open-source software plays an increasingly important role in federal information technology systems. I'm delighted the NSA's security experts are making this valuable contribution to the open-source community," said Jeffery Hunker, senior director for critical infrastructure at the National Security Council, when the announcement was made in January.
The NSA's version is aimed at strengthening the operating system's security and preventing unauthorized users from gaining access to government computers. The NSA released its version under the same general public license as most other versions of Linux in hopes of attracting more development from computer programmers who work on open-source software.
Linguist up to his ears tracking security threats
It could be any office building, but Everette Jordan's workplace is one of the most unusual in the country. He cannot take his office keys home as he must punch in a code to get them each morning.
Jordan is a spy -- but not in the way you probably imagine. He is a "listener."
Jordan, 41, is a longtime linguist for the National Security Agency, which is in charge of eavesdropping on behalf of the U.S. government. He is fluent in Russian, Spanish, French, German and Arabic and spends his days listening to tapes of conversations the NSA has acquired via its worldwide array of powerful surveillance technology.
As he listens, Jordan must not only translate but pay attention to the subtleties of the spoken word, no matter what language.
"You have to listen for irony, for sarcasm, for tension," he said. "You have to listen for rhetorical statements being made. You also have to listen for humor."
But the focus of his eavesdropping is to listen for any threats made against the United States. And in his NSA career, he's heard items that were of critical importance to U.S. national security.
"There have been many cases, and that's one of the fun things about being a linguist, knowing that the work you have done has gone right downtown to the president of the United States or the United States Congress, or any of the decision- and policymakers that would provide them with the information they need to make a very critical decision," he said.
As a child, Jordan enjoyed languages and ended up at the NSA after joining the U.S. Army. He heard about the agency needing linguists, applied and got a job.
The agency has a variety of linguists, some who are listeners such as Jordan, while others focus on reading and translating written material. Given that Jordan is unlikely to travel to Russia because of his job as an intelligence officer, he keeps his language skills sharp in a variety of ways.
"We take as many courses as we possibly can," he said. "We have other avenues of trying to keep our language skills up. We have a lot of foreign films we listen to or watch at the agency."
Nevertheless, he admits to wanting to travel to Russia, having spent so much time learning about the language and listening to native speakers. "There will always be a soft spot in my heart," he said.
In all the conversations and other material that Jordan has heard, however, he said he's never listened to a tape of a U.S. citizen. The NSA is banned by law from eavesdropping on conversations of U.S. citizens.
"We have very strict protocols toward handling that, those sorts of situations and really we erase the thing," he said. "But we also report that such and such a thing has happened. It's an incident report, but we are enjoined by law to, against, you know, listening to Americans."
The possibility also exists that the NSA might record the conversations of U.S. citizens speaking other languages while living and working abroad.
"But there are places you don't expect Americans to be, and in the pursuit of our business, we try to focus on places Americans are not expected to be," he said.
While Jordan works in a serious business, he also comes across the humorous in his job, such as trying to discern what a drunk is saying or a person who is missing their false teeth. Jordan said he must gauge whether a drunk's conversation is more or less true because of his or her inebriated condition.
"It depends on what kind of drunk they are," he said, laughing. "But we have to know the language to a degree that we can understand when this person is drunk and you're ... really trying to pay attention to get through it all without laughing."
Because of its trafficking in secrets, the NSA remains a very closed agency. Its budget is classified and an old joke is that NSA really stands for "No Such Agency." For many years, its headquarters at Fort Meade did not have a sign.
Jordan is limited to what he can say to his family and anyone else outside the NSA. He said that NSA employees accept these restrictions when they take the oath to protect and defend the U.S. Constitution.
"It's enough satisfaction to know that you made a difference," he said. "It doesn't matter that you're never going to be able to tell anybody."
To say that NSA employees are security-conscious is putting it mildly. Jordan is the first NSA listener ever to give a television interview. As CNN toured NSA headquarters to film its special report, employees were warned ahead of time so they would not discuss anything classified.
Asked if most of his colleagues would be unwilling to give an interview, Jordan quickly replied, "You got that right." The NSA's secretive culture doesnÕt encourage employees to talk to the media.
"One of the ways we're very successful is the work we do is very quiet, and in some cases, in many cases, our work force has been indoctrinated not to draw attention to themselves because in some cases they would be traveling on official government business, and to sit here in front of the camera as an NSA employee is something like killing one's career," he said.
But Jordan is not living in the secret world anymore, appearing at job fairs and recruitment drives for the NSA. The agency is looking to hire 600 workers with experience or education in computer science, mathematics, engineering, signal analysis, data collection, cryptanalysis and intelligence analysis.
"My name is open; my face is out there," he said. "And I have no career aspirations in the areas that are more, you could say, quiet. So I can basically do this with the understanding that my face may go all around the world, but my body will have to stay here at Fort Meade, Maryland."
Jordan, however, is starting to pay a physical price for listening to pops and screeches of surveillance tapes. The frequent tests that Jordan undergoes have started to show a high-frequency hearing loss in high decibels.
Though he hopes soon to move into management, Jordan said he wouldn't have wanted any other career up to this point -- protecting the nation with his ears and his gift for languages.
U.S. cryptography timeline
1924: The U.S. Navy creates its first cryptanalytic group within the Code and Signal Section of the Office of Naval Communications.
1940s: The U.S. military creates the Army-Navy Communications Intelligence Board (ANCIB) to facilitate cooperation in intelligence gathering.
1946: STANCIB becomes the U.S. Communications Intelligence Board (USCIB) and adds the FBI to its membership.
1947: Congress passes the National Security Act, which moves toward centralizing U.S. intelligence gathering and forms the framework for the modern national security structure. The act establishes the National Security Council (NSC) and the CIA.
1949: Secretary of Defense Louis A. Johnson issues a directive creating the Armed Forces Security Agency with the mission of conducting communications intelligence and security activities for the military.
1952: In June, the special committee, chaired by New York lawyer George Brownell, recommends the creation of a unified communications intelligence agency with greater powers commensurate with clearly defined responsibilities. In October, Truman and the NSC adopt most of the Brownell committee's recommendations, issuing a top-secret directive creating the National Security Agency. In November, Maj. Gen. Ralph J. Canine is named the first NSA director.
1959: Congress establishes a civilian career service at the NSA, like similar services at the State Department and FBI. To maintain this career service, the agency conducts its own recruiting and employment programs.
1960: The NSA begins intercepting messages and communications traffic revealing the Soviet military buildup in Cuba, including the installation of air defense systems and missile capabilities.
1962: In October, the Cuban missile crisis explodes into the public eye, leading to a tense standoff between the United States and Soviet Union. The crisis marks the first time the NSA creates an around-the-clock command center.
1971: The NSA receives operation control over the cryptologic agencies of the Air Force, Army and Navy. The three agencies are reorganized into the newly created Central Security Service (CSS) headed by the NSA director. The move centralizes the government's signals intelligence (SIGINT) and communications security (COMSEC) programs under the NSA.
1975-1976: Investigations by congressional committees headed by U.S. Sen. Frank Church of Idaho and U.S. Rep. Otis Pike of New York reveal that government agencies, including the NSA, performed clandestine surveillance on U.S. citizens who participated in the civil rights and anti-Vietnam War movements. The NSA's activities included passing intercepted information to other agencies and maintaining files on key figures. In response, the House and Senate create permanent committees to oversee the actions of the U.S. intelligence community.
1978: Congress passes the Foreign Intelligence Surveillance Act to regulate electronic intelligence gathering. The act includes the creation of a special court to handle requests by the NSA to perform electronic surveillance on targeted U.S. persons. Other classified regulations created later cover the handling of foreign intelligence electronic surveillance.
1984: President Ronald Reagan issues a directive giving the NSA responsibility of maintaining security of government computers.
1987: Congress passes the Computer Security Act, which makes clear that in the area of unclassified computing systems, the National Institute of Standards and Technology (NIST) and not the NSA, is responsible for the development of technical standards for civilian communication systems.
1993: The federal government proposes the Clipper Chip initiative, which would have placed a chip inside communication devices to allow the encryption of private communications. But the proposal also calls for a third party to maintain the "keys" to unlock encrypted communications, which government agents could access with court permission. The NSA invented the chip's underlying cryptographic algorithm, known as Skipjack, which the agency classified on national security grounds. But the proposal comes under severe criticism from privacy advocates and the technology industry, and eventually the Clinton administration abandons it. In 1998, the NSA releases the Skipjack algorithm to computer security companies interested in developing off-the-shelf encryption products that will work with existing federal government communications systems.
1998: Controversy breaks out over the alleged NSA "Echelon" project, which privacy groups describe as a worldwide surveillance network that eavesdrops on all communications traffic. The NSA allegedly maintains the network, which privacy groups say involves the sharing of intelligence gathered by the United States, Great Britain, Canada, Australia and New Zealand.
A European Parliament report states that the project targets civilian communications, including political advocacy groups like Amnesty International and Greenpeace. The NSA declines to comment on the project but reiterates that it follows U.S. law.
1999: The director of an Australian intelligence agency publicly acknowledges a long-rumored relationship between U.S. and British intelligence agencies known as UKUSA that allows them to share data.
The U.S. House Select Committee on Intelligence Oversight requests that the NSA provide it with "legal memoranda, opinions rendered and other documents" to see how the agency's legal counsel governs its activities. The agency refuses a portion of the request on the grounds it violates the attorney-client privilege. The agency's response comes under criticism from members of Congress, and legislation is passed requiring the NSA to submit a report detailing the legal standards it uses in carrying out its activities. The Electronic Privacy Information Center files suit in federal court, seeking the release of NSA documents concerning potential surveillance of American citizens by the Echelon project.
2000: NSA director Lt. Gen. Michael Hayden testifies before the House intelligence committee. He says media reports that the NSA collects all electronic communications, spies on U.S. citizens and provides intelligence information to U.S. companies are "false or misleading." However, Hayden and CIA director George Tenet, who also testifies, neither confirm nor deny the existence of the Echelon system. U.S. Rep. Bob Barr, R-Georgia, who pushed for the hearings and also testified, says the testimony of Hayden and Tenet leaves the committee with "more questions than answers."
Sources: Federation of American Scientists, Computer Professionals for Social Responsibility, American Civil Liberties Union and National Security Agency .
Anechoic chamber: The National Security Agency's headquarters in Maryland includes a massive cathedral containing fire retardant plastic foam spikes, creating an almost perfect acoustic environment for testing transmitters and antennae.
Cipher: A system for concealing plain text by transposing the letters or numbers or substituting other letters or numbers according to a key. It is also called a "cryptosystem."
Code: A system for replacing words, phrases, letters or numbers by other words or groups of letters or numbers for concealment or brevity.
COMINT (Communications Intelligence): The NSA program for information obtained for intelligence purposes from the intercept of foreign communications (by other than the intended recipient).
COMPUSEC (Computer Security): The NSA's program for protecting U.S. government computers from exploitation and intrusion by foreign intelligence services and computer cracking and hacking groups.
COMSEC (Communications Security): The protection of communications from exploitation by foreign intelligence services. This includes ensuring the security of U.S. cryptosystems, the prevention of electronic emissions from various communications equipment and the physical protection of communications security equipment.
Cryptology: The science and art of making and breaking codes and ciphers.
Cryptography: The science and art of making codes and ciphers.
Cryptanalysis: The conversion of encrypted messages into plain text without having the initial knowledge of the key used in encryption.
CSS (Central Security Service): The CSS comprises all U.S. military branches --Army, Navy, Air Force and Marines -- and works to ensure a full partnership between the NSA and the military.
Decode: To convert an encoded message into the equivalent plain text.
Encode: To replace plain text words with code groups.
Encrypt: To conceal plain text by use of a code or cipher.
ELINT (Electronic Intelligence): The NSA acronym for information obtained for intelligence purposes from the intercept of foreign electromagnetic noncommunications transmissions (by other than the intended recipient). The most common sources of this type of information are foreign radar signals.
HPSCI: The House Permanent Select Committee on Intelligence, which provides oversight on U.S. intelligence activities.
INFOSEC (Information Systems Security): The protection of information systems against unauthorized access to or modification of information, whether in storage, processing or transit, and against the denial of service to authorized users or the provision of service to unauthorized users, including those measures necessary to detect, document and counter such threats.
Intercept: The acquisition of electromagnetic signals such as radio or radar by using electronic equipment for the purpose of gathering intelligence information on foreign entities. The material collected is itself sometimes referred to as "intercept."
IOB: The President's Intelligence Oversight Board, which is charged with oversight of all U.S. foreign intelligence activities.
OPSEC (Operations Security): The process of denying potential adversaries any information about capabilities and/or intentions by identifying, controlling and protecting generally unclassified evidence of the planning and execution of sensitive activities.
National Security Council: The NSC is the executive branch agency that advises and assists the U.S. president on national security and foreign policies. Established in 1947, the council also coordinates these policies among various government agencies on behalf of the president. The primary council members are the president, vice president and secretaries of state and defense. The military adviser to the council is the chairman of the Joint Chiefs of Staff, while the CIA director is the council's intelligence adviser. Invited to all NSC meetings are the treasury secretary, the U.S. ambassador to the United Nations, the national security adviser and the president's chief of staff and economic policy assistant. Other officials, such as the attorney general and the director of the Office of National Drug Control Policy, also attend meetings pertaining to their jurisdiction.
NSOC (National Security Operations Center): The National Security Operations Center is the NSA's nerve center, often the first place that word of a crisis reaches the United States. It was the first to learn of the bombing of two U.S. embassies in Africa and of the bombing of the USS Cole in Yemen. It was also the first to be notified in the United States that a pilot had been downed over Serbia during the Kosovo war. The NSA played a key role in giving his rescuers information they needed to get him out safely. A blue flashing light goes on in the center at the time of a crisis. When people without security clearances visit, such as a CNN camera crew and correspondent, a red flashing light is lit to warn staff not to discuss classified matters.
STICS (Scalable Transportable Intelligence Communications Systems): These systems are based at the NSA ready for rapid deployment to hot spots when needed. The units set up a downlink to receive encrypted, classified intelligence data for ground commanders. Units were deployed in the Balkans, Panama and Somalia. The units set up a tent, with secure telephones and other communications. The STICS are temporary field units. Once deployed, a decision is made as to whether U.S. forces will be staying more than a few months. If so, a more permanent arrangement is set up.
SIGINT (Signals Intelligence): Information that contains (either individually or in combination) communications intelligence (COMINT), electronics intelligence (ELINT) and foreign instrumentation signals intelligence (FISINT), however transmitted.
SSCI: The Senate Select Committee on Intelligence, which provides oversight on U.S. intelligence activities.
Supercomputer: The fastest type of computer used for specialized applications that require a massive number of mathematical calculations. The NSA supercomputer center contains the largest accumulation of computer power in any one building on Earth. One Cray Triton supercomputer at the facility can handle 64 billion instructions per second, and there are many of them at the NSA. The computers are used for breaking adversaries' codes, creating U.S. top-secret codes and sifting through billions of information intercepts made by the NSA searching for useful intelligence. The government also bans the sale of supercomputers to countries such as Iraq for fear they may be used to create nuclear weapons.
Sources: National Security Agency, White House
Cryptography in the modern age
Ancient civilizations were the first to practice cryptography, but modern efforts date back to World War I when the United States dedicated military units to capturing and deciphering communications intelligence and the British decoded a key telegram.
The famous Zimmermann telegram from Germany to Mexico was intercepted by the British, who decoded the message in which Germany promised Mexico a chunk of the United States if it entered World War I. The British leaked the message to the United States, which helped end U.S. neutrality.
But it was during World War II that the United States and Great Britain achieved a great deal of success in cryptography.
German researchers created the cipher machine known as Enigma, which the German military used during World War II. However, a British team of code breakers, including the brilliant mathematician Alan Turing, managed to crack the Enigma code against staggering odds.
The British efforts, code-named Ultra, were based at Bletchley Park, 65 miles north of London. The researchers there also built Colossus, one of the world's first computers, to crack the Lorenz cipher, another encryption scheme used by the Nazis.
The 'Magic' effort to crack the Japanese code
While the British focused on Germany, U.S. intelligence worked on Japanese communications in the Pacific. The U.S. naval intelligence effort, code-named "Magic," cracked the Japanese diplomatic cipher that was produced by an encryption machine nicknamed "Purple." U.S. naval intelligence also cracked other Japanese codes, including the naval and merchant ship codes.
The ability to read the Japanese code helped the Navy in several important battles, most importantly at the Battle of Midway in 1942. The Japanese had six aircraft carriers to the three that the United States operated in the Pacific. Japanese naval commander Admiral Isoroku Yamamoto devised a plan to attack Midway Island in hopes of luring the entire U.S. fleet into battle and use the carrier advantage to crush the U.S. forces.
Yamamoto placed his main carrier force, consisting of four carriers, into a single formation, making all four vulnerable to attack if one was discovered. When naval code breakers pieced enough Japanese messages together, they discovered the plans and objectives of the Japanese fleet.
In the ensuing battle on June 4, 1942, U.S. pilots sank all four Japanese carriers from the main strike force while only one U.S. carrier was lost. The battle helped turn the tide of the Pacific war against the Japanese.
An intercepted Japanese message also led to Yamamoto's death. U.S. code breakers discovered plans for Yamamoto to inspect the naval base at Bougainville in the Solomon Islands. On April 18, 1943, Yamamoto's plane was ambushed as it approached Bougainville and was shot down by U.S. fighter planes.
U.S. code breakers also helped in the invasions of the islands of Guadalcanal and Tarawa. Decoded Japanese messages led to the U.S. Marines invading Guadalcanal before the Japanese could fortify the beaches. During the battle for Tarawa, naval intelligence discovered key details of the Japanese order of battle and logistical status.
Navajo code-talkers baffled the Japanese
While the Japanese were able to decipher the codes used by the U.S. Army and Army Air Corps, they never cracked the code of the U.S. Marines, which used Native Americans speaking in Navajo to communicate between units.
The idea for using Navajos belonged to Philip Johnston, who grew up on a Navajo reservation as the son of a missionary. He also was one of the few non-Navajos fluent in that language. A World War I veteran, Johnston knew that the military used Native American languages -- notably Choctaw -- in the earlier war to encode messages.
Johnston believed Navajo was suited for encoding messages because it was an unwritten language with no alphabet or symbols and was spoken only by Navajos in the American Southwest. Once the military agreed to the idea, 200 Navajos were recruited, and the first 29 recruits created the Navajo code, including a written dictionary and the creation of words for military terms.
Roughly 400 Navajos became what were called "code talkers," serving in various Marine units in the Pacific. Their primary job was to transmit battlefield communications via telephones and radios. The Navajo language stumped the Japanese, and they never cracked the code.
Venona captures Soviet communications
Also during the war, the U.S. Army's Signal Intelligence Service, a forerunner of the National Security Agency, began the top-secret Venona program, which captured and exploited encrypted Soviet diplomatic communications.
According to a declassified NSA history of the project, a Signal Corps reserve officer, Lt. Richard Hallock, discovered weaknesses in the cryptographic system used to encrypt Soviet trade communications. The discovery helped break four other Soviet cryptographic systems used by Soviet diplomats, the KGB and the Soviet army and naval intelligence services.
Eventually, more than 2,200 Soviet messages were translated in the Venona program, but not all the messages collected were successfully decrypted. However, enough information was garnered to help expose activities of some high-profile spy cases, including Kim Philby, the high-ranking British intelligence officer who defected to Moscow, and Ethel and Julius Rosenberg, U.S. citizens who were convicted of selling atomic secrets to the Soviets and executed in 1953.
The trial and execution of the Rosenbergs was protested worldwide. When the NSA released its Venona archive, the children of the Rosenbergs said it contained nothing that proved the guilt of their parents.
The Venona program ended in 1980. In 1995, the NSA ended its 50-year silence on the project and publicly released some of the declassified messages.
The development of public-key encryption
Cryptography and encryption were primarily the domain of intelligence agencies until the rise of the information age and the Internet. A huge step forward in encryption was the simultaneous development of public-key encryption in the 1970s.
A person who encrypts a document uses what's called a "key" to encode the message. Until public-key encryption, the same key had to be used to decrypt, or "unlock" the document. If a person sent an encoded document to someone else, the key also had to be sent to that person via a secure method, like a face-to-face meeting.
Public-key encryption eliminated the problem of securely transmitting the key by using two keys, a public and private one. The sender of a message encrypts the document using the recipient's public key and the recipient of the message decrypts the message using the private key, which the recipient keeps secret. The public and private keys are the inverse of each, so a person's private key -- which can be kept secret -- can unlock a message that was encrypted by the public key.
The idea of public-key encryption was first developed in 1976 by Martin Hellman, a Stanford University professor, and two graduate students who worked with him at the time, Ralph Merkle and Whitfield Diffie.
A similar idea was developed by three Massachusetts Institute of Technology professors, Ron Rivest, Adi Shamir and Len Adleman. Their approach was labeled RSA after their initials, and the three went on to form a business that is now RSA Data Security. RSA is one of primary encryption ideas used on the Internet.
However, in 1997, the British government released a document showing that three employees of its eavesdropping organization known as the Government Communications Headquarters discovered a similar approach earlier in the 1970s. British officials said the discovery by researchers James Ellis, Clifford Cocks and Malcolm Williamson was kept secret for national security reasons.
In 1991, programmer
Phil Zimmermann released the first version of his encryption program,
Pretty Good Privacy (PGP), as a free program. Zimmerman released the program
after the FBI pushed for access to encrypted communications. The program
has now become a worldwide standard.