Security in network communications

Doc File 838.50 KByte,

Security in Network Communications

Deepti Reddy

Suphannee Sae chai

Sarah Summers

Fall 2006


Abstract 3

Introduction 3

Network Communication Security Issues 4

Network Communications Attacks 4

Access Attacks 4

Modification Attacks 5

Repudiation Attacks 5

Network Security Tools 5

Symmetric Key Algorithms (Private Key Algorithms) 6

Data Encryption Standard (DES) 6

Advanced Encryption Standard (AES) 7

International Data Encryption Algorithm (IDEA) 9

Asymmetric Key Algorithms (Public Key Algorithms) 13

Diffie-Hellman 13

RSA (Rivest-Shamir-Adleman) 14

Digital Signatures 14

Digital Signature Algorithm (DSA) 15

Message Digests 17

MD5 17

SHA 19

Achieving Secure Network Communications 20

General Communications Security 21

IPSec 21

Remote Authentication Dial In User Service (RADIUS) 24

Keberos 27

Email Security 27

PGP (Pretty Good Privacy) 28

S/MIME (Secure / Multipurpose Internet Mail Extensions) 29

Web Security 30

Secure Sockets Layer (SSL) 30

Secure Shell (SSH) 33

Future Work 35

Conclusions 35

References 37



Over recent years, there has a dramatic rise in the diversity of the types of communications carried out over networks. Current network communications include email, file transfer, e-commerce, messaging and VOIP. Many of these communications techniques require some security. This paper reviews the types of attacks that we need to secure network communications against, a variety of algorithms used for protecting confidentiality and data integrity amongst other things and some of the protocols that are used to provide security.


In the past computer networks were used primarily by universities, government institutions and large corporations. The networks were predominantly used for sharing resources such as printers and the sending of emails. However, over recent years, there has been a rapid increase in the uses to which computer networks are applied. Today, it is common place for the general public to use computer networks, albeit unknowingly in many cases, not only to send emails but also to carry out such things as online banking and e-commerce.

Many of the applications for which networks are currently used involve the transfer of personal data which if compromised could lead to criminal activity amongst other things. Obviously, protecting this information is paramount and has led to increased awareness of network security, or lack of it.

Network Security is a very broad and complex subject, covering a multitude of issues such as protecting resources from attack, unauthorized access etc. As a result, we have chosen to focus on one particular aspect of network security, specifically network communications security in terms of protecting information which is transferred over a computer network.

Network Communication Security Issues

We have stated the network communication security is of increasing importance, but what exactly are the issues that we need to consider? As stated in the introduction, network communications can involve the transfer of sensitive information; it should therefore be obvious that this information should be concealed from unauthorized parties, which identifies confidentiality as one of the issues that any security mechanisms must address. Secondly, when carrying out a transaction we want to be sure of the identity of the person we are dealing with which is an authentication issue. Thirdly, we want to be assured that the data we receive has not be been modified by an unauthorized party which gives us a data integrity issue. Finally, we need proof that the communication has taken place, so that at some time in the future the originator of the message cannot deny that it took place – this is a non-repudiation issue. [1]

Network Communications Attacks

In order to be able to address the four security issues mentioned in the previous section, it is necessary to understand how an attacker can carry out attacks against them.

Access Attacks

Access attacks are attacks against confidentiality. A variety of these types of attack exist, these include snooping/sniffing, eavesdropping and interception.


Eavesdropping is when an attacker monitors the communications on the network, it is frequently known as snooping or sniffing. Eavesdropping can occur on both local area networks (LAN’s) and wide area networks (WAN’s). On LAN’s eavesdropping is achieved by installing a sniffer on a computer system which is connected to the network. The sniffer works by capturing all of the network traffic. However, most frequently they are used to capture user ID’s and passwords. Sniffing on WAN’s requires greater knowledge and equipment. [2]


In order to intercept network traffic, an attacker must place their system in the communication path between the sender and receiver of the information. This can be achieved on the internet by causing a name resolution change thus allowing them to redirect the network traffic to their system as well as its intended destination. Interception can be achieved by taking over or capturing a session already in progress. [2]

Modification Attacks

Modification attacks are attacks against data integrity. The attacker may alter the information being transmitted by inserting or deleting information. In order to carry out a modification attack on data that is being transmitted, the attacker must first intercept the data by carrying out an access attack. An example is the man-in-the-middle attack. This allows the attacker to read, insert and modify transmission between sender and receiver without either knowing that the link and transmission has been compromised.

Repudiation Attacks

Repudiation attacks are carried out in an attempt to deny that a transmission or transaction took place or to provide false information. Masquerading is one type of repudiation attack. When an attacker launches this type of attack they are invariably attempting to impersonate another person or system.

Network Security Tools

Nearly all of the security mechanisms for enforcing network communications security are based on cryptographic principles. Therefore, in this section we will discuss the various methods by which data can be encrypted.

Symmetric Key Algorithms (Private Key Algorithms)

Symmetric key algorithms utilize a single key for both the encryption and decryption process. There are a number of private key algorithms, these include the Data Encryption Standard (DES), the Advanced Encryption Standard (AES), the International Data Encryption Algorithm (IDEA), RC5, Skipjack, Blowfish, CAST-128 and GOST. The most widely spread used of these algorithms are DES and AES. In this paper, we focus on these two algorithms and also IDEA which is used for email security in the Pretty Good Privacy (PGP) protocol.

Data Encryption Standard (DES)

Figure 1 shows a general depiction of the DES encryption algorithm. The encryption process is comprised of three phases. The plaintext to be encrypted initially passes through the initial permutation, the output of this permutation is then passed through 16 rounds during which both permutation and substitution functions are carried out. The output resulting from the 16 rounds becomes the input for the 32-bit swap. The output from this swap is then passed through a reverse initial permutation which results in the final ciphertext.


Figure 1: General Depiction of DES Encryption Algorithm

One of the main problems with DES is that it utilizes a short key length (56 bits). As a result, in order to attempt to improve security, DES with triple encryption was developed. Triple DES differs from standard DES in that two keys and three stages are used. During the first stage, the plaintext is encrypted using standard DES with the first key as described previously. During the second stage, the ciphertext produced in the first stage is decrypted using the second key with DES in decryption mode. In the third and final stage, the output from the second stage is encrypted using DES in encryption mode with the first key. [3]

Advanced Encryption Standard (AES)

The Advanced Encryption Standard (AES) algorithm is a symmetric block cipher capable of both encryption and decryption. There are three variations of the algorithm AES-128, AES-192 and AES-256. The differences lie in the length of the cryptographic keys that are utilized. The algorithm encrypts and decrypts data in blocks of 128 bits. [25]

Unlike DES, AES is fast in both hardware and software implementations. It is relatively easy to implement and requires little memory.

The input and output cipher bit sequences are processed as 4 x 4 arrays of bytes, known as the state. These arrays of bytes are formed by dividing the blocks into groups of 8 contiguous bits to form a range of bytes.


Encryption using AES consists of rounds with each round except the last consisting of four stages. During the first stage, shown in Figure 2, each byte in the state is combined with the round sub-key using the XOR operation. The round sub-key is derived from the cipher key using a key schedule which we discuss later.


Figure 2: Stage 1 of AES Encryption – XOR addition of round sub-key

During the second stage of the round, a non-linear substitution step is used. Each byte is replaced by another one. The replacement is determined from a lookup table. This stage is shown in Figure 3.


Figure 3: Stage 2 of AES Encryption – Byte Substitution

The third stage of the round, a transposition state takes place, Figure 4. Each row of the state is cyclically shifted to the left. The amount of shift differs for each row. The first row remains unchanged, the second row is shifted one position, the third row is shifted two positions and the fourth row is shifted three positions.


Figure 4: Stage 3 of AES Encryption – Transposition

During the final stage of the round, the four bytes in each column of the array are combined using an invertible linear transformation. The input and output is four bytes and each input byte influences all four of the output bytes. This stage of the round is shown in Figure 5.


Figure 5: Stage 4 of AES Encryption

As stated previously, the final round is different from the preceding rounds. During the final round the first three stages are carried out as described above but the fourth stage is replaced by another instance of the first stage.

International Data Encryption Algorithm (IDEA)

The International Data Encryption Algorithm (IDEA) is a block cipher algorithm. It is used for email security in the PGP (Pretty Good Privacy) program. [4] The algorithm operates with 64-bit plaintext and ciphertext blocks and utilizes a 128-bit key.

IDEA Encryption

Figure 6 shows the functional representation of the encryption process. It is comprised of 8 identical encryption processes which are known as “rounds”. These processes are followed by a final output transformation.

Prior to or during the first round, the plaintext is partitioned into four 16-bit sub-blocks. During the 8 encryption processes, three operations are used to combine two 16-bit values to produce a 16-bit result. These operations are addition, XOR and multiplication.

During each of the 8 rounds, the 16-bit key sub-blocks (which are different in each round) are combined with two of the 16-bit plaintext blocks using addition modulo 216, and with the other two plaintext blocks using multiplication modulo 216 + 1. The outputs are the further processed, as shown in Figure 6, where two more 16-bit key sub-blocks are processed using bit-by-bit exclusive OR. Each of the encryption rounds produces four 16-bit values which are used as input into the subsequent rounds. The four 16-bit values produced by round 8 are combined with the last four of the 52 key sub-blocks using addition modulo 216 and multiplication modulo 216 + 1 to form the resulting four 16-bit ciphertext blocks. Table 1 shows the encryption of the key sub-blocks and an example of the IDEA encryption algorithm is shown in Appendix A. [4, 5 ]


Figure 6: Structure of IDEA [5]

|Round 1 |Z1(1) Z2(1) Z3(1) Z4(1) Z5(1) Z6(1) |

|Round 2 |Z1(2) Z2(2) Z3(2) Z4(2) Z5(2) Z6(2) |

|Round 3 |Z1(3) Z2(3) Z3(3) Z4(3) Z5(3) Z6(3) |

|Round 4 |Z1(4) Z2(4) Z3(4) Z4(4) Z5(4) Z6(4) |

|Round 5 |Z1(5) Z2(5) Z3(5) Z4(5) Z5(5) Z6(5) |

|Round 6 |Z1(6) Z2(6) Z3(6) Z4(6) Z5(6) Z6(6) |

|Round 7 |Z1(7) Z2(7) Z3(7) Z4(7) Z5(7) Z6(7) |

|Round 8 |Z1(8) Z2(8) Z3(8) Z4(8) Z5(8) Z6(8) |

|Output Transformation |Z1(9) Z2(9) Z3(9) Z4(9) |

Table 1: Encryption of the Key Sub-blocks

IDEA Decryption

Table 2 shows the encryption of the key sub-blocks. The decryption process is essentially the same as the encryption process. The only differences are that different 16-bit key sub-blocks are generated with each of the 52 16-bit key sub-blocks being the inverse of the key sub-blocks used during encryption, and the key sub-blocks are used in reverse order. [4, 5]

|Round 1 |Z1(9) - 1 -Z2(9) -Z3(9) Z4(9) - 1 Z5(8) Z6(8) |

|Round 2 |Z1(8) - 1 -Z2(8) -Z3(8) Z4(8) - 1 Z5(7) Z6(7) |

|Round 3 |Z1(7) - 1 -Z2(7) -Z3(7) Z4(7) - 1 Z5(6) Z6(6) |

|Round 4 |Z1(6) - 1 -Z2(6) -Z3(6) Z4(6) - 1 Z5(5) Z6(5) |

|Round 5 |Z1(5) - 1 -Z2(5) -Z3(5) Z4(5) - 1 Z5(4) Z6(4) |

|Round 6 |Z1(4) - 1 -Z2(4) -Z3(4) Z4(4) - 1 Z5(3) Z6(3) |

|Round 7 |Z1(3) - 1 -Z2(3) -Z3(3) Z4(3) - 1 Z5(2) Z6(2) |

|Round 8 |Z1(2) - 1 -Z2(2) -Z3(2) Z4(2) - 1 Z5(1) Z6(1) |

|Output Transformation |Z1(1) - 1 -Z2(1) -Z3(1) Z4(1) - 1 |

Table 2: Decryption of the Key Sub-blocks

Asymmetric Key Algorithms (Public Key Algorithms)

Asymmetric key algorithms are a more recent introduction than symmetric key algorithms. The main difference between private and public key algorithms is that public key algorithms utilize two keys for encryption. One key is used to encrypt the information and the second key is used to decrypt the information.


Diffie-Hellman is a protocol that allows two users to generate/exchange a shared private key. This shared private key then allows them to exchange information securely over an insecure channel. [6]

The implementation requires both users to agree on two prime numbers p and g; p must be large, a minimum of 512 bits and g is a primitive root modulo p. The numbers p and g can be transferred between the users without security. Each user then chooses a large random number and calculates a new number using the equation shown below. [6]

A = ga(mod p) user 1 Equation 1

B= gb(mod p) user 2 Equation 2

a and b are the numbers chosen by each user.

Each user sends the value they have calculated to the other user. The two users then calculate their shared key which is obtained from the equations 3 and 4 below. [6]

K = Ba(mod p) = (gb)a(mod p) Equation 3

K = Ab(mod p) = (ga)b(mod p) Equation 4

The share key, K, is then used to exchange information securely.

The Diffie-Hellman algorithm is used by a number of protocols such as Secure Socket Layer (SSL), Secure Shell (SSH) and Internet Protocol Security (IPSec) which are discussed later in this paper.

RSA (Rivest-Shamir-Adleman)

RSA is a public key algorithm which can be used for both encryption and decryption. The security provided by RSA is based on the fact the factoring large numbers is considered a hard problem when the numbers are 512 bits or greater.[2]

Generating RSA Public and Private Keys

The public and private keys are generated using the formulae below, where p and q are larger prime numbers. An example is shown in Appendix RSA.

n = pq

Ф(n) = (p -1)(q -1)

Determine e such that it is relatively prime to Ф(n)

Determine d such that (d)(e) = 1mod Ф(n)

RSA Algorithm for confidentiality

Ciphertext = (Plaintext)e mod n

Plaintext = (Ciphertext)d mod n

Private Key = {d, n}

Public Key = {e, n}

RSA Algorithm for authentication

Ciphertext = (Plaintext)d mod n

Plaintext = (Ciphertext)e mod n

Private Key = {d, n}

Public Key = {e, n}

Digital Signatures

A digital signature is a protocol producing the same affect as a real signature. It verifies that the message received was in fact sent by the person claiming to have sent it. As such, it must not be forgeable, it must be authentic, unalterable and not re-useable. [7]

Digital Signature Algorithm (DSA)

For creating a digital signature the Digital Signature Algorithm (DSA) can be used. The DSA digital signature is a pair of large numbers which are represented in a computer by a string of binary digits. DSA allows both generation and verification of signatures. Figure 7 shows an overview of the signature generation and verification processes when using SHA with DSA. [8]


Figure 7: Overview of digital signature generation and verification using SHA with DSA

In order to use DSA both the sender and receiver must have a public/private key pair. The private key is used by the DSA in the signature generation process while the public key is used in the signature verification process. In both processes the data (message) to be transmitted is reduced by means of the Secure Hash Algorithm (SHA).

As stated above, the signature of the message to be transmitted is a pair of large numbers. The numbers are calculated using the two equations shown below.

r = (gk mod p) mod q Equation 1

s = (k-1(SHA(M) + xr)) mod q Equation 2

In the above equations, p, q and g can be public and common to a group of users; p is a prime modulus while q is a prime divisor of p – 1 and g has order q mod p, k is a randomly or pseudo-randomly generated integer and x and y are the user’s private and public keys respectively. It should be noted that x and k are only used for signature generation and must be kept secret. Full details can be found in the associated reference. [8]

The value of SHA(M) is a 160-bit string output produced by SHA. The two numbers generated by DSA are transmitted along with the message to the receiver.

Once a message with a digital signature created using DSA has been received, the receiver can verify the signature. In order to verify the signature, the receiver must have access to the values of p, q and g and the sender’s public key. During verification, the receiver first checks to see if the value of the received value of r lies between 0 and q and that the received value of s lies between 0 and q. If either value lies outside the specified ranges the signature is rejected. However, it the values are within the specified ranges the receiver computes the following values.

W = (s′)-1 mod q Equation 3

u1 = ((SHA(M′)w) mod q Equation 4

u2 = ((r′)w) mod q Equation 5

v = (((g)u1 (y)u2) mod p) mod q Equation 6

s′, M′, and r′ are the received versions of the message and the two calculated numbers.

If v is found to be the same as r′, then the signature can said to be verified and the receiver can have a high confidence that the received message was indeed sent by the claimed sender. If v does not equal r′, then the message should be considered invalid. [8]

Message Digests

Message digests are numbers that are created algorithmically from a file. They represent the file uniquely and can therefore be used to determine whether or not it has been altered during transit. Message digests are hash functions.


MD5 is a commonly used message digest. The first version of the message digest was MD2 which was developed in 1989. Since then there have been various modifications to the algorithm and the current version in use is MD5. It is somewhat slower than the previous version, MD4, it is more secure. Therefore, we will only consider this version.

MD5 algorithm takes as its input a message of random length and produces a 128-bit “message digest” of the input message. Computationally, it is not feasible to produce two messages having the same “message digest”. The MD5 algorithm is designed to run fast on 32-bit machines and is fairly easy to code.[9, 10]

Let us assume we have a b-bit message as an input where b is a random integer greater than or equal to zero and it may not be a multiple of 8. Let the bits be as follows,

m0 m1 …. m(b-1)

We then do the following steps to compute the “message digest”.

Step 1: Append Padding bits

The message is padded such that its length in bits is congruent to 448 modulo 512. Then the resulting message is extended so that it is 64 bits less of being a multiple of 512 bits in length. In MD5, padding is always performed on the message even if the message is already congruent to 448, module 512. A single bit of ‘1’ is appended to the message and following that ‘0’ bits are appended till the length of the padded message is congruent to 448, modulo 512. This means that a minimum of 1 bit and a maximum of 512 are appended to the message.

Step 2: Appending length

A 64-bit representation of b is appended to the result of step 1. If b happens to be greater than 2^64, then only the low-order 64 bits of b are appended. The 64 bits that are appended to the result from step 1 are appended as two 32-bit words with the low order word followed by the high order word.

The message now will be an exact multiple of 512 bits. Equivalently, the length of this message is a multiple of 16 words. Let M[0….N-1] denote the words of the resulting message where N is a multiple of 16.

Step 3: Initialize Buffer

Four 32-bit registers are used to compute the message digest.

Let us assume A, B, C, and D are the 32-bit registers. The registers are initialized to the following values in hexadecimal with the low-order bytes first.

A: 01 23 45 67

B: 89 ab c def

C: fe dc ba 98

D: 76 54 32 10

Step 4. Processing the Message in 16-Word Blocks


MD5 uses a table T that contains 64 elements in it. To speed up computations, this table is always calculated beforehand. Element “i” is indicated using symbol “Ti”. The sin function below is used to compute the elements of the table:

Ti = 232 * abs(sin(i + 1))

Four auxiliary functions:

Md5 uses four auxiliary functions. These functions take three 32-bit words as inputs and produce an output containing one 32-bit word by using a combination of logical operators “and”, “or”, “not” and xor to the input.

The four auxiliary functions are represented as F, G, H, I and are described below.

F(X,Y,Z) = (X and Y) or (not(X) and Z)

G(X,Y,Z) = (X and Z) or (Y and not(Z))

H(X,Y,Z) = X xor Y xor Z

I(X,Y,Z) = Y xor (X or not(Z))

The four buffers A, B, C and D described in Step 3 are mixed with the words used as inputsThe contents of the four buffers (A, B, C and D) are now mixed with the words of the input, using the four auxiliary functions (F, G, H and I). There are four rounds, each involves 16 basic operations. One operation is illustrated in Figure 8 below. [9, 10]


Figure 8: One operation in MD5


SHA (Secure Hash Algorithm) is another commonly used message digest. Like MD5, SHA is a cryptographic message digest algorithm. However, it differs from MD5 in that it utilizes an additional expansion operation. The algorithm works by taking a message of length less than 264 bits and produces a 160-bit message digest.

The SHA1 algorithm sequentially processes blocks of 512 bits when computing the message digest. As a result, the first stage of SHA1 is to pad the message to make its total length a multiple of 512 bits. The message is padded by adding a 1 followed by m “0”s which is followed by a 64 bit integer. The 64 bit integer is the length of the original message. The final padded message contains 16 * n words where n > 0 and is considered as a sequence of n blocks M1 through Mn with each M containing 16 words.

To convert the padded message to the message digest, two buffers consisting of five 32-bit words and a sequence of eighty 32-bit words are used. Words in the first five word buffer are labeled A through E, while words in the second five word buffer are labeled H1 through H4. The words of the 80-word sequence are labeled W0 through W79.

In order to generate the message digest each of the 16 word blocks are processed in order. Each processing operation involves 80 steps. Prior to processing, the word buffers are initialized in hex. Once this has been done, each M is divided into 16 words, W0 through W15 and the following steps are carried out:

1. For t = 16 to 79, let Wt = S1(Wt-3 XOR Wt-8 XOR Wt-14 XOR Wt-16)

2. Let A = H0, B = H1, C = H2, D = H3, E = H4

3. For t = 0 to 79 do

Temp = S5(A) +ft(B, C, D) + E + Wt + Kt

E = D; D = C; C =S30(B); B = A; A + Temp;

4. Let H0 = H0 + A, H1 = H1 + B, H2 = H2 + C, H3 = H3 + D, H4

= H4 + E

Once processing has been completed, the resultant message digest is the160 bit string represented by the five words H0H1H2H3H4. [SHA1]

It should be noted that SHA1 is no longer considered to be secure since attacks against it have been found. The current version is SHA2.

Achieving Secure Network Communications

As stated previously, this paper focuses on security during transmission of data. In this section we will consider the various ways in which secure network communications can be achieved. As we have mentioned several times already security is a very broad topic and even by narrowing our focus to network communications we cannot hope to cover all aspects.

In the previous section we discussed some of the commonly used algorithms, digital signatures, and, message digests that can be used to obtain confidentiality, authentication, data integrity and non repudiation. Generally these techniques are used within communications protocols. In this section, we discuss some the commonly used protocols that make use of these tools to provide secure communications.

General Communications Security


The protocol stack does not provide any security mechanisms as a result IPSec was developed as an extension of the IP protocol in order to provide authentication, data integrity and confidentiality.

IPSec provides security at the IP layer, it enables a system to select the required security protocols, the algorithms for use and put in place any required cryptographic keys. It can be used to protect one or more "paths" between a pair of hosts, between a pair of security gateways, or between a security gateway and a host. It has certain advantages over other security methods in that it operates in the network layer of the OSI model. However, it only protects IP-based traffic.

IPSec is comprised of two protocols Authentication Header (AH) and Encapsulating Security Payload (ESP) that can be used separately or in combination [11, 12] It can be used in one of two modes transport mode or tunnel mode. Transport mode is used to protect transmission of data end-to-end between hosts. In this mode only the payload of the packet is encrypted. While in tunnel mode the entire IP packets are encapsulated inside a new IP packet with a completely new IP header, providing effectively VPN functionality.

Authentication and Data Integrity Using IPSec

Authentication and Data Integrity using IPSec is achieved using the Authentication Header (AH). However, it should be noted that AH does not encrypt IP traffic so does not provide confidentiality. Authentication in the AH is achieved by computing a cryptographic hash-based message authentication code over all the fields of the IP packet excluding TTL or header checksum which are modified during transit. The computed code is added AH header which is transmitted with the original message. On receipt of the message, the hash is recomputed. If the transmitted value and recalculated value are not the same, this indicates a problem with the authenticity and/or data integrity of the transmission [12]. Figure 9 shows an AH packet diagram.

|0 |1 |2 |3 |

|0 1 2 3 4 5 6 7 |0 1 2 3 4 5 6 7 |0 1 2 3 4 5 6 7 |0 1 2 3 4 5 6 7 |

|Next Header |Payload Length |RESERVED |

|Security Parameters Index (SPI) |

|Sequence Number |

|Authentication Data (variable) |

Figure 9: AH packet diagram

It should be noted that AH is incompatible with NAT (network address translation). If NAT is in operation then ESP should be used.

Confidentiality Using IPSec

Confidentiality using IPSec is achieved using Encapsulating Security Payload (ESP). ESP encrypts the data. ESP also provides authentication and data integrity. Figure 10 shows an ESP packet diagram.

In ESP, an encryption algorithm combines the data in the datagram with a key to transform it into an encrypted form.

ESP packages fields in different ways to AH. Instead of just having a header, the fields in ESP are divided into three components – ESP header, ESP trailer and ESP authentication data.

The ESP header contains two fields, the SPI and sequence number. This header comes before the encrypted data which is encrypted using symmetric encryption algorithms. It’s actual placement depending upon whether ESP is to be used in transport or tunnel mode. The ESP trailer is placed after the encrypted data, it contains padding which is used to align the encrypted data, through a padding and pad length field. The trailer also contains the next header field. The third component is the ESP authentication field. This contains an Integrity check value. This is used when ESP with authentication is used.[11, 12]

|0 |1 |2 |3 |

|0 |

|Sequence Number |

|Payload * (variable) |

|  |Padding (0-255 bytes) |

|  |  |Pad Length |Next Header |

|Authentication Data (variable) |

Figure 10: ESP packet diagram

IPSec operates at the network layer of the OSI model and as such it has an advantage over other protocols such as SSL and other methods that operate at higher layers since applications do not need to be written to be aware of and use it. However, IPSec also has disadvantages in that it is complex suite of protocols and could therefore be easily be poorly implemented thus resulting in poor security. [11, 12]

Remote Authentication Dial In User Service (RADIUS)

RADIUS is an authentication, authorization and accounting protocol. It is used in network environments that utilize MD5. It is most commonly used in embedded devices such as routers, switches etc.

Due to the limited storage available in embedded systems, they cannot deal with large number of users and authentication. ISPs have millions of users where users are deleted and added frequently and authentication for user’s changes frequently. RADIUS provides a centralized way for user administration.

Below is a summary of the RADIUS packet [13]:

0 1 2 3

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1


| Code | Identifier | Length |


| |

| Authenticator |

| |

| |


| Attributes ...


The type of the RADIUS packet is established by the code where the codes are the following:

|Value |Description |

|1 |Access-Request |

|2 |Access-Accept |

|3 |Access-Reject |

|4 |Accounting-Request |

|5 |Accounting-Response |

|11 |Access-Challenge |

|12 |Status-Server (experimental) |

|13 |Status-Client (experimental) |

|255 |Reserved |

A one octet value called an identifier enables the RADIUS client to match a RADIUS response with the corresponding outstanding request.

A random number of attribute fields are stored in the attributes section of the packet.

The attributes section is where an arbitrary number of attribute fields are stored. The only pertinent attributes for this discussion are the User-Name and User-Password attributes. Here, we only look at a RADIUS exchange involving a username-password pair. This involves an Access-Request involving the username and password followed by an Access-Accept, Accept-Reject or access failure. The client has the information required that the client needs to validate while the server has access to the authentication database that contains the information to be used to validate the client’s authentication request.[14]

Initial Client Processing

An Access-Request RADIUS packet is generated by the client that includes the username and password attributes. The Access-request packet identifier is generated by the client and is usually a simple counter value that is incremented for each Access-Request.

The Access-Request packet's identifier field is generated by the client. The generation process for the identifier field is not specified by the RADIUS protocol specification, but it is usually implemented as a simple counter that is incremented for each request.

The 16-octet Authenticator field is a randomly chosen 16 octet string. The username and password fields are protected. A shared secret message between the client and server followed by the Request Authenticator are put through a MD5 hash algorithm to create a 16-octet value which is then XORed with the password. If the password is larger than 16 octets then additional MD5 hash calculations are performed. Below is an example:

Let the shared secret between the client and the server be S and the 128 bit Request Authenticator be AU. Let the password in 16-octet blocks be p1, p2, ….., pn. If pn is not on a 16-octet boundary it is padded with 0’s to a 16-octet boundary.

c1 = p1 XOR MD5(S+AU)

c2 = p2 XOR MD5(S+c1)

cn = pn XOR MD5(S+ c1)

The username-password attribute will contain c1 cat c2 cat ….. cn where cat stands for concatenation.

The Access-Request packet contains a 16 octet Request Authenticator in the authenticator field. This Request authenticator is a randomly chosen 16 octet string. [14]

Server Processing

When the server receives the Access-Request packet it verifies that it contains a shared secret for that client. If it does not contain a shared secret for the client, the server silently discards the request.

If the server possesses the shared secret, it obtains the original password by going through a process similar to the client’s protection process on the user-password attributes. Once the server authenticates the username-password pair by accessing its authentication database, it sends to the client an Access-Accept packet if the username-password is valid or an Accept-Reject if the username-password pair is invalid.

The server uses the same identifier received from the client’s request packet. The server adds a Response Authenticator in the authentication field. This is the MD5 hash value of the response with the request packet’s Request Authenticator and the shared secret.

Therefore, the ResponseAuthenticator would be MD5(Code+ID cat Length cat Request Authenticator cat Attributes cat Secret) where cat implies concatenation. [14]


Kerberos is a distributed authentication service for networks. It allows a process (client) running on behalf of a principal (user) to prove its identity to a verifier (application server or server) without sending data across the network. In addition there is the option to provide integrity and confidentiality.[15, 16]

Kerberos is based on the Needham and Schroeder authentication protocol. It utilizes a series of encryption messages with the encryption using the DES algorithm. Initially, the client and server do not share an encryption key, sharing is achieved via a Kerberos ticket.

Kerberos has a number of limitations; it is not effective against password guessing attacks and requires a trusted path through which to enter the passwords. In addition, both client and server applications must be modified to use Kerberos for authentication. These limitations can be addressed by combining Kerberos with other techniques. As a result of the limitations and need to combine Kerberos with other techniques to address these limitations, we will not discuss this protocol further. [15, 16]

Email Security

There are a number of protocols that can be used for secure email such as PGP (Pretty Good Privacy), S/MIME (Secure/Multipurpose Internet Mail Extensions) and PEM (Privacy Enhanced Mail). We will only discuss the first two since PEM is no longer used as it was not widely accepted and used.

PGP (Pretty Good Privacy)

PGP is effectively a hybrid cryptosystem since it combines features of both symmetric and asymmetric cryptography. It is effectively a complete email security package since it provides confidentiality, authentication, digital signatures and compression. That being said, it does not have email facilities, rather it is more of a preprocessor that takes plaintext as input and produces an output of signed ciphertext in base 64. This can then be emailed.[3]

PGP Encryption

When PGP is invoked by a user, it first compresses (hashes) the plaintext using MD5. The resulting hash is then encrypted using RSA with the sender’s private key. PGP then concatenates the encrypted hash and the original message into a single message and compresses it using a ZIP program. PGP then generates a session key with IDEA. The key is generated using random input from the mouse movements and keystrokes. The session key is then encrypted using the receiver’s public key. The encrypted session key and zipped message are concatenated and converted to base 64. [PGP1, 3]

PGP Decryption

Once the message is received, the receiver reverses the base 64 encoding and decrypts the IDEA key using their private RSA key. The unencrypted message is then decompressed and the receiver separates the plaintext from the encrypted hash and decrypts the hash using the sender’s public key. Assuming that the plaintext hash agrees with the receiver’s MD5 computation then the receiver knows that the message has not been altered and indeed has been sent by the sender. [17]

The operation for encrypting and decrypting a message with PGP is shown in Figure 11.


Figure 11: PGP in operation for sending a message [compNets]

S/MIME (Secure / Multipurpose Internet Mail Extensions)

S/MIME (Secure/Multipurpose Internet Email Extensions) is a version of the MIME protocol that provides security services, confidentiality, authentication, message integrity and non-repudiation for messaging applications. For confidentiality S/MIME uses encryption and for non-repudiation it uses digital signatures. [18, 19]

In order to use S/MIME it is necessary to establish a Public Key Infrastructure (PKI) to support the sender and receiver of the S/MIME messages. The PKI is used to authenticate the identity of the S/MIME users and also to provide confidentiality, non-repudiation and digital signatures. The use of X.509 Cetrificates is also required to bind an entity’s identity and public key for secure operations of S/MIME.

S/MIME implementations must support the following suites of cryptographic algorithms:


RSA for digital signatures with SHA1

RSA for key transport

Triple DES for content encryption


DSA for digital signature with SHA1

RSA for key transport

Triple DES for content encryption

Other algorithm suites are also supported [18, 19]

Web Security

Secure Sockets Layer (SSL)

The Secure Sockets Layer (SSL) protocol was originally developed by Netscape Communications. It was designed to provide secure access of a web browser to a web server. The Transport Layer Security (TLS) standard developed by the Internet Engineering Task Force (IETF) is based on SSL. SSL allows authentication and data integrity through the use of digital signatures and encryption for confidentiality. [20, 21]

“The SSL protocol runs above the TCP/IP and below the higher level protocols such as HTTP or IMAP. It uses TCP/IP on behalf of the higher-level protocols, and in the process allows an SSL-enabled server to authenticate itself to an SSL-enabled client, allows the client to authenticate itself to the server, and allows both machines to establish an encrypted connection” [22]

The SSL protocol utilizes a combination of symmetric and asymmetric key encryption. The reason that this combination is used is that symmetric key encryption is faster than asymmetric key encryption, but asymmetric key encryption provides superior authentication. SSL is comprised of three protocols, the Handshake Protocol, the Record Protocol, and the Alert Protocol. Descriptions of these protocols are given below. [22]

SSL Handshake

When carrying out transactions using SSL, a session must first be established between the client and server. This is achieved via a handshake sequence (Figure 12). During the handshake, the server authenticates itself to the client using asymmetric techniques; the client and server negotiate a cipher suite (key exchange method, cipher for data transfer and message digest for creating the message authentication code (MAC)); establish and share a session key. The handshake also allows the client to optionally authenticate itself to the server.[20]


Figure 12: Simplified SSL Handshake Sequence [20]

Key Exchange Method

The purpose of the Key Exchange Method is to define how the shared secret symmetric cryptographic key that will be used for the transfer of the application data will be agreed upon by the client and server. In the case of SSL 2.0 only RSA key exchange is used. However, SSL 3.0 supports a choice of key exchange algorithms if certificates are used. If no certificates are to be used, Diffie-Hellman key exchange is used.

Cipher for Data Transfer

Messages in an SSL session are encrypted using symmetric key cryptography. They can be encrypted using any of the Stream ciphers or CBC Block ciphers shown in Table 3.

|Stream Ciphers |CBC Block Ciphers |

|RC4 with 40-bit keys |RC2 with 50 bit key |

|RC4 with 128-bit keys |DES with 40 bit key |

| |DES with 56 bit key |

| |Triple-DES with 168 bit key |

| |IDEA(128 bit key) |

| |Fortezza (96 bit key) |

Table 3: Ciphers used for data transfer

Digest Function

The message digest is used to create the Message Authentication Code. It is encrypted with the message to be sent to provide both integrity and to prevent replay attacks. The message digests supported by SSL are MD5 (128 bit hash) and SHA1 (160 bit hash). It is possible for no digest to be used.

Record Protocol

The record protocol handles the encryption for all messaging in SSL. It provides the common format to frame all Alert, ChangeCipherSpec, Handshake and application protocol messages. [23] The SSL records consist of the encapsulated data, digital signature, message type, version and length. An example of an SSL record is shown in Figure 13 (fig 6 of Cisco systems paper)

Alert Protocol

The purpose of the alert protocol is to handle questionable packets. If an error is detected by the server or client, then they send an alert which contains information about the error. Three types if alert messages exist – warning, critical and fatal. If a warning or critical alert is received then the session can be restricted and if a fatal alert is received the session is terminated.

Secure Shell (SSH)

Secure Shell (SSH) was first designed and implemented in 1995. It is a network protocol which enables the establishment of a secure channel between a local and remote computer. It allows execution of commands on the remote computer. It is used to provide strong authentication and secure communications over insecure channels.

SSH provides both confidentiality and data integrity of transmissions between two computers using encryption and message authentication codes (MACs). The original version of SSH, known as SSH1, is no longer in wide spread use since it has inherent design flaws which make it vulnerable to attacks such as man-in-the-middle attacks.

The current version SSH2 provides better security through the use of the Diffie-Hellman key exchange. Stronger integrity checking is provided via MAC’s.

The SSH architecture operates over three separate layers – transport layer, authentication layer and connection layer.

Transport Layer

The transport layer typically runs on top of TCP/IP, it handles the initial key exchange and server authentication. It also sets up encryption, compression and integrity verification. [24]

Once a connection has been established, both computers must send an identification string. This string identifies the protocol and software versions to be used. It also provides information for the Diffie-Hellman key exchange. The key exchange commences immediately after the string identifier has been sent. Information regarding the public key algorithm, symmetric encryption algorithm, message authentication algorithm and hash algorithm are also negotiated. During the key exchange, two values are produced, a shared secret key and an exchange has. From these values, the encryption and authentication keys are derived. [24]

When encryption is being used, the packet length, padding length, payload and padding fields of each packet must be encrypted using the negotiated algorithm.

A variety of ciphers are currently acceptable for SSH2, these include, but are not limited to, Triple DES, Blowfish in CBC mode, AES with 128, 192 and 256 bit keys and IDEA in CBC mode.

Data integrity in SSH is protected by including a MAC with each packet. The MAC is calculated from the shared secret key, packet sequence number and packet contents.

As with the cipher, a variety of MAC algorithms are currently in use. These include SHA1 and MD5.

The SSH2 protocol was designed to operate most public key formats, encoding and algorithms for signmature and/or encryption. Public keys/certificate formats currently defined include ssh-dss, ssh-rsa, pgp-sign-rsa and pgp-dss.

Signing and verifying using this key format is done according to the Digital Signature Standard using the SHA-1 hash.

Since the majority of the security is provided by the Transport Layer protocol, only a minimal description is provided for the Authentication and Connection Layers.

Authentication Layer

The authentication layer handles client authentication, it runs over the SSH transport layer protocol. The authentication layer makes the assumption that the underlying protocols provide integrity and confidentiality protection.

Connection Layer

The connection layer protocol is designed to run on top of the transport and authentication layers. This layer provides interactive login sessions, remote execution of commands, forwarded TCP/IP connections, and forwarded X11 connections.

Future Work

It should be apparent from in this paper that cryptography, in one form or another, forms the basis of all of the security methods/protocols discussed. In recent years, a number of the supposedly secure algorithms have been shown to be insecure. Developments in the processing capabilities if computers may in the future lead to the current secure algorithms becoming insecure. Therefore to stay ahead of the attackers, new algorithms/methods must continually be developed. When considering the future of secure communications close attention must be made to current security protocols since decisions will have to be made as to whether current implementations can be modified to accept new cryptographic techniques or whether totally new protocols will have to be developed. If the latter route is chosen, it will be necessary to ensure that the protocols and cryptographic techniques are widely accepted.


Network communications security is achieved in the main through the application of some form cryptography. A cryptographic algorithm may be used alone or in combination with other means.

As stated elsewhere in this report network security and network communications security are very broad topics. In this report we have focused on what we consider to be some of the most important aspects. A complete review of the area is beyond the scope of this report.

On noteworthy point is that in order to achieve network communications security and network security in general, it is not enough to employ security mechanisms it is essential that a good security policy be in place and that it is continuously implemented.


[1] Communication Security – available techniques, AXIS Communications White Paper,

[2] Network Security: A Beginner’s Guide, Eric Maiwald, McGraw Hill, 2001, ISBN

[3] Computer Networks, Andrew S. Tanenbaum, 4th Edition, Prentice Hall, 2003, ISBN 0-13-066102-3

[4] IDEA (International Data Encryption Algorithm)

[5] International Data Encryption Algorithm – Technical Description,

[6] Diffie-Hellman Protocol, David Terr

[7] Security in Computing, Charles P. Pfleeger and Shari Lawrence Pfleeger, Third Edition, Prentice Hall, ISBN 0-13-035548-8

[8] Digital Signature Standard, Federal Information Processing Standards Publication 186, May 1994,



[11] Securing Data in Transit with IPSec, Deb Shinder, July 2004,

[12] An illustrated guide to IPSec, Steve Friedl,



[15] Keberos: An authentication service for Computer Networks, B. Clifford Neuman and Theodore Ts’o,

[16] The History of Kerberos Authentication, Daniel Calloway,

[17] How PGP Works,

[18] S/MIME Secure Email - A Beginners Guide, Mark Noble,


[20] SSL/TLS Strong Encryption: An Introduction,

[21] Introducing SSL and Certificates using SSLeay, Frederick J. Hirsch, Web Security: A Matter of Trust. World Wide Web Journal, Volume 2, Issue 3, Summer 1997 0-07-213324-4


[23] SSL and TLS Essentials: Securing the Web, Thomas




Appendix A - Example of IDEA

Appendix B – Example of RSA

Appendix A

Example of IDEA Algorithm


Let the four quarters of the plaintext be called A, B, C, and D.

Let the 52 subkeys called K1 through K52

Before or during round 1:

1. Multiply A by K1

2. Add K2 to B

3. Add K3 to C

4. Multiply D by K4

Round 1:

Calculate A xor C and let the result be E

Calculate B xor D and let the result be F

Multiply E by K5

Add the new value of E to F.

Multiply the new value of F by K6

Add the new value of F to E

Change both A and C by xoring the current value of F with each of them.

Swap B and C

Round 2 through 8 repeat the operations in round 1 with the exception that the following keys are used:

Round 2 – keys K7 through K12

Round 3 – keys K13 through K18

Round 4 – keys K19 through K24

Round 5 – keys K25 through K30

Round 6 – keys K31 through K36

Round 7 – keys K37 through K42

Round 8 - keys K43 through K48

After round 8 the swap of B and C is not performed.


Multiply A by K49

Add K50 toB

Add K51 to C

Multiply D by K52

Appendix B

Example of RSA

Generating RSA Keys

The following example uses numbers which are significantly smaller than those which would be used in a real application.

1. Choose two prime numbers.

Let p = 11 and q = 13

2. Calculate n = pq

n = 11*13 = 143

3. Calculate Ф(n) = (p -1)(q -1)

Ф(n) = (11 – 1)(13 -1) = 120

4. Select a number e such that e is relatively prime to Ф(n)

Let e = 7

5. Determine d such that (d)(e) = 1mod Ф(n)

(d)(7) = 1 mod120 and d must be less than 120

d = 103

6. The private key is {103, 143}

7. The public key is {7, 143}

Algorithm for Confidentiality

Ciphertext = (Plaintext)e mod n

Plaintext = (Ciphertext)d mod n

Private Key = {d, n}

Public Key = {e, n}

For the example above, lets consider that we wish to send the message 9.

Using the encryption algorithm, we get:

Ciphertext = (9)7 mod 143 = 48

To decrypt the information on receipt, we use the decryption algorithm.

Plaintext = (48)103 mod 143 = 9

Download Doc File