T325: Technologies for digital media

1. T325:Technologies for digital media Second semester – 2011/2012Tutorial 3 –Error Control Coding Arab Open University – Spring 2012

2. Introduction • Overview of error control coding • Block codes: Rectangular and Hamming codes • Interleaving • More block codes: LDPC and RS codes Outline Arab Open University – Spring 2012

3. Possibility of fault in all memory technologies means one or more bits being in error • Bit error : 1  0 or 0  1 • The probability of bit error increases with the increase of number of bits stored on devices • The same problem occurs in digital communication while transmitting data • In this part we will be looking in more detail at how with appropriate coding the presence of errors can be detected and sometimes corrected. Introduction Arab Open University – Spring 2012

4. Overview of Error Control Coding Arab Open University – Spring 2012

5. Bit error rate (BER), or bit error ratio, is one measure of how badly a signal is affected by errors • Questions : • Which is better: a high BER or a low BER? • If there are 10 errors in 10000 bits, what is the BER? Bit Error Ratio (BER) Arab Open University – Spring 2012

6. Messagesto be conveyed from the data source to the data sink • The transmitter converts each message to a codeword (a process called encoding or just coding), which is sent through the channel • The receiver converts the codeword back to the message(decoding). • The rules which determine how you get from any given message to a codeword are called the code. Coding/Decoding Arab Open University – Spring 2012

7. Redundancy is a measure of the proportion of a code that is ‘extra’ beyond what is strictly needed to convey the message. • Redundancy can be quantified • If the message has k bits and the codeword n bits, the redundancy of the code is: R = (n – k) / n • Example: if a message has 4 bits(it is, say, 1101) and twoextra bitsare added for error control coding so that there are 6 bitsin the codeword, the redundancy of the codeword is 2/6, which is 33%. • It is usual to describe a code that converts messages of k bits to codewords of n bits as an (n, k) code. Redundancy Arab Open University – Spring 2012

8. The code rate of an (n, k) code is given by k/n, so the rate of a (6, 4) code is 2/3. • Code rate is normally written as a fraction, reduced to the smallest integers. • The extra bits added by the error control codes are described as ‘redundant’ bits • these bits do not add any further information to the message • ‘Raw’ signals contain redundancy and, because of that, can be compressed to reduce the bit rate. Code rate Arab Open University – Spring 2012

9. What do you think is better : Compress then Code or Code then Compress? • Code then compress: • Redundancy bits added by the code will be removed by the compression and thus the result will be vulnerable to errors • Compress then Code: • Redundancy is minimized in the message signal by efficient source coding and/or advanced data compression, and then redundancy is added back through error-control coding. • Compressing then adding redundancy allows the use of efficient error control codes Compress then Code Arab Open University – Spring 2012

10. Code Distance(Sometimes called Hamming distance after Richard W. Hamming (1915 – 1998)): is the minimum number of places in which any two codewords differ  This is a fundamental parameter of error control codes! Code Distance Arab Open University – Spring 2012

11. What is the Hamming distance of the code specified by this Table? • To determine the Hamming distance of the code  check all possible pairs of codewords, counting in how many places they differ. • Table above: minimum value is 3, so the Hamming distance for this code is 3. Error control coding Arab Open University – Spring 2012

12. Error control coding - classification Arab Open University – Spring 2012

13. Parity Check: a simple method of detecting single errors • Underlies more advanced codes • Parity checking can be done either on even parity or on odd parity. • Even parity is more common • As the transmitting device sends data, it counts the number of set bits (ones) in each group of seven bits. • If the number of set bits is even, it sets the parity bit to 0 • if the number of set bits is odd, it sets the parity bit to 1 Parity Check Arab Open University – Spring 2012

14. Suppose a system uses an even parity check code, creating 8-bit codewords by adding a check bit to 7-bit messages. • What would be the codewords if the messages were: • 1011010 • 1100001 • An extra bit is added to the end of the message, chosen to be a 1 or 0 in such a way as to ensure that the codeword contains an even number of ones. • 10110100 • 11000011 Parity check - Activity Arab Open University – Spring 2012

15. b) Assuming that the probability of there being more than one error in any one codeword is negligible, which of the following codewords contain errors? • 00111011 • 00000001 • 01001000 • 01001100 • If the codeword contains an even number of ones it is error free. If it contains an odd number of ones it must contain an error. On this basis, (i), (ii) and (iv) contain an error, while (iii) does not. Parity check – Activity cont’d Arab Open University – Spring 2012

16. A more powerful type of error detection code is a Cyclic Redundancy Check (CRC) code. • CRC codes add one or more bytes (a collection of bits) to a message. • This added redundancy makes these codes more flexible than Parity Check codes because they can detect many different combinations of errors. Error detection codes – Cyclic Redundancy Check Arab Open University – Spring 2012

17. How to correct errors once detected? • Automatic Repeat Request (ARQ): get a replacement copy of the data. • ARQ introduces delays while you wait for the repeat, which might not be acceptable, also the correct data will generally not be available. • Forward Error Correction (FEC) : used to overcome the limitations of ARQ How to correct errors once detected? Arab Open University – Spring 2012

18. Error correction codes Arab Open University – Spring 2012

19. Error Correction codes are more useful than Error Detection codes in that they can actually correct errors • As would be expected, the ability to correct errors requires the use of more redundancy compared to error detection codes • Error correction codes can be broken down into two basic types: • Convolutional codes • Block codes Error correction codes Arab Open University – Spring 2012

20. Encode blocks of input bits as blocks of output bits (like block codes) but, the encoder has memory. • Convolutional codes get their name because the encoding process can be viewed as convolving (combining) the message stream with the impulse response of the code. • A code tree is often used to help visualize the encoding process. Convolutional codes - Encoding Arab Open University – Spring 2012

21. The most popular technique for decoding convolutional codes in practice is the Viterbi algorithm • One way of thinking about how (Viterbi) decoding of convolutional codes works is that at any point the decodertakes account of what has gone before andwhat comes after in deciding how to decode the current data. • With Viterbi decoding of a convolutional code, the output from the decoder is delayed so that the final decoding of the data depends on what comes later, as well as what came before. Convolutional codes - Decoding Arab Open University – Spring 2012

22. ⊕ Storage element • What will be the output sequence of this encoder if the input sequence is 1101 and the storage element is initialized to zero? Convolutional codes - Activity Arab Open University – Spring 2012

23. ⊕ 1 1 1 1 0 1 Arab Open University – Spring 2012

24. ⊕ 0 1 1 1 1 1 Arab Open University – Spring 2012

25. ⊕ 1 0 0 0 1 0 Arab Open University – Spring 2012

26. ⊕ 1 1 1 1 0 1 Arab Open University – Spring 2012

27. What will be the output sequence of this encoder if the input sequence is 101 and the storage element is initialized to zero? Convolutional codes –Activity Arab Open University – Spring 2012

28. With just four input bits we already have a tree with 16 possible paths through it, and the number of paths grows exponentially with the increasing numbers of input bits. • The points at which branches split two ways, up and down, we call nodes, and there are only two types of node: • From one type of node the upper branch is labeled ‘00’ and the lower branch is labeled ‘11’. • From the other type of node the upper branch is labeled ‘10’ and the lower branch is a labeled ‘01’. Convolutional codes: Tree representation Arab Open University – Spring 2012

29. Tracing through the tree forms the encoded bit stream. • When a branch is encountered: • upper path is taken if the input bit is a 0 • lower path is taken if the input is a 1 • The corresponding outputs at each step are the bits above the path. • Example: the message is 01, and the resulting output stream is 0011. Convolutional codes: Tree representation Arab Open University – Spring 2012

30. The coding of the input data 1100 is shown by the red dashed path, resulting in the code 11 01 10 00. (The spaces between the pairs of bits are for ease of reading.) • What would the output stream be if the input data were: • 1010 • 0110 Error control coding - Activity Arab Open University – Spring 2012

31. Arab Open University – Spring 2012

32. There is a lot of repetition in the tree representation • It is possible to collapse equivalent nodes at a given level on the tree onto each other, which leads to a trellis rather than a tree. • When it comes to decoding, the trellis makes life much simpler. • Whereas possible paths diverge indefinitely in the tree, they diverge and converge in the trellis. • Viterbi decoding is based upon making appropriate decisions at the point at which paths converge. • The trellis can have more levels in general! Convolutional codes : Trellis representation Arab Open University – Spring 2012

33. Turbo codes have found a very wide range of applications, mainly in wireless communications, ranging from the third generation mobile systems to deep-space exploration. • Turbo-codes promise to achieve the ultimate limits of capacity of a communication channel • Announced in 1993 • Wide range of applications mainly in wireless communications : 3G mobile systems to deep-space exploration. Turbo codes Arab Open University – Spring 2012

34. Claude Shannon showed that the data rate that can theoretically be transmitted over a certain type of ‘standard’ communication channel depends only on the channel bandwidth and the ratio between the signal power and the noise power (the ‘signal-to-noise ratio’, SNR) • C is the data rate in bits per second (‘channel capacity’), B is the channel bandwidth in hertz, S/N is the signal power divided by the noise power and the logarithm is to the base 2. Turbo Codes - Capacity of a communication channel Arab Open University – Spring 2012

35. Error correction codes Arab Open University – Spring 2012

36. Block codes are the most popular type of error correction codes • Why they got their name? • Encoder takes in a message block of finite length • adds redundancy • sends out a code word that is also a block of longer length than the message. • The most widely used type of block codes is Linear Block codes. Block codes Arab Open University – Spring 2012

37. Rectangular code (also known as a product code) detect the presence of a single error, and identify which bit is in error and, therefore, correct it. • Sequences of message digits are grouped as ‘rows’, making up a fixed-size block. • A ‘horizontal’ parity check digit is inserted at the end of each row. • Then a final row of ‘vertical’ parity checks is added, one for each ‘column’. Block codes: Rectangular and Hamming codes Arab Open University – Spring 2012

38. Figure below represents a codeword from a rectangular code. • How many parity digits are used per codeword to check for errors? • Describe this code using the (n, k) notation. • Calculate the code rate and the redundancy of this code. • Assuming that no more than one digit is in error, how many different errors can be corrected using this code? Rectangular codes – Activity Arab Open University – Spring 2012

39. The diagram shows that eight parity digits are used to check for errors, out of 20 digits in total. • n is the total number of bits in the code, 20, and k is the number of bits in the message, which is 12. The code is therefore a (20,12) code. • The code rate is k/n, which is 12/20, or 3/5. The redundancy is which is 40%. • The rectangular code can correct a single error at any digit position, including the parity digits, so it can correct 20 different errors. Rectangular codes –Activity – Solution Arab Open University – Spring 2012

40. Although rectangular codes are very simple, they are not very efficient because they use a relatively large number of parity digits per data digit. • It is possible to design a code which uses eight parity digits to correct any single error in a block of exactly 255 digits. The code is one of a family of codes called Hamming codes. • For any integer m, it is possible to construct a Hamming code which uses m parity digits to correct any single error in a codeword of size n digits, where n = 2m-1 Rectangular code Arab Open University – Spring 2012

41. As with rectangular codes, Hamming codes can only correct one error per block. • The parity check digits in a Hamming code are applied to groups of message digits in such a way that a single error can be located and then corrected. • The parity digits can be positioned in the coded word so that a binary number representing the result of the checks, the syndrome, points directly to the error, if there is one. Hamming codes Arab Open University – Spring 2012

42. Parity check scheme for a seven-digit Hamming code • This binary number is 0 if there is no error, and gives the position of an error if there is one Hamming codes Arab Open University – Spring 2012

43. Hamming codes Arab Open University – Spring 2012

44. The way in which the parity digits are generated leads to a code with a Hamming distance of three. • If a single error occurs in the transmission of a Hamming codeword, the sequence received will differ by just one digit from the transmitted codeword. • This means that a sequence received with a single error cannot possibly be a valid codeword, and it will therefore be possible to detect the error. • The sequence will differ by just one digit from the transmitted codeword, but by at least two digits from all the other codewords. Hamming codes Arab Open University – Spring 2012

45. What should the Hamming distance be if: • Detections is needed? • Correction is needed? • Suppose that we wish to send messages, each of which consists of two binary digits. Fill in the table with the codewords for the other messages. • What is the Hamming distance of this code?  2 • How many other three-digit sequences are there? Hamming codes - Activity Arab Open University – Spring 2012

46. Suppose you received the three-digit sequence 010. You know that this is not a valid codeword and that only one error can have occurred. Which of the possible three-digit sequences could have been transmitted? • The transmitted sequence must differ from 010 by just one digit. The possibilities are 011, 000 and 110. • These are all valid codewords, but there is no way of telling which one of them was transmitted. So this code, with Hamming distance 2, can detect one error, but cannot correct it. Hamming codes Arab Open University – Spring 2012

47. Therefore, that a parity check code has a Hamming distance of 2 and allows single errors to be detected but not corrected. • Hamming codes, on the other hand, have a Hamming distance of 3 and can correct a single error. Hamming codes Arab Open University – Spring 2012

48. In general, codes with a minimum Hamming distance of ncan correct • up to (n - 1)/2 errors if n is odd, • or up to (n/2) - 1 errors if n is even; • or can detect up to n - 1 errors without correcting them. The relationship between Hamming distance and error detection or correction Hamming codes Arab Open University – Spring 2012

49. If the following codes are received, state whether there have been any errors, and give the decoded output. (Assume that the probability of there being more than one error in a received codeword is negligible.) • 1110000 • 1101011 • 0010010 • The decoding can either be done by looking for the closest match among the codewords in Table 3.4, or by calculating the syndrome. Error control coding - Activity Arab Open University – Spring 2012

50. 1110000. • The three parity checks are as follows: • Parity1 tests parity on digits 1, 3, 5 and 7: 1110000. This has an even number of ones (two), so passes. • Parity2 tests parity on digits 2, 3, 6 and 7: 1110000. This has an even number of ones (two), so passes. • Parity 3 tests parity on digits 4, 5, 6 and 7: 1110000. This has an even number of ones (none), so passes. • All three parity checks pass, so the syndrome is 000, and there are no errors. The decoded data is just extracted from the codeword (digits 3, 5, 6 and 7): 1000. • Alternatively, scanning through the codewords in Table 3.4, the received codeword 1110000 is found to be a valid codeword corresponding to the original data word 1000. Error control coding Arab Open University – Spring 2012