Exploiting 2-Dimensional Source Correlation in Channel Decoding with Parameter Estimation

Received Mar 9, 2018 Revised Jun 25, 2018 Accepted Jul 2, 2018 Traditionally, it is assumed that source coding is perfect and therefore, the redundancy of the source encoded bit-stream is zero. However, in reality, this is not the case as the existing source encoders are imperfect and yield residual redundancy at the output. The residual redundancy can be exploited by using Joint Source Channel Coding (JSCC) with Markov chain as the source. In several studies, the statistical knowledge of the sources has been assumed to be perfectly available at the receiver. Although the result was better in terms of the BER performance, practically, the source correlation knowledge were not always available at the receiver and thus, this could affect the reliability of the outcome. The source correlation on all rows and columns of the 2D sources were well exploited by using a modified BahlCocke-Jelinek-Raviv (BCJR) algorithm in the decoder. A parameter estimation technique was used jointly with the decoder to estimate the source correlation knowledge. Hence, this research aims to investigate the parameter estimation for 2D JSCC system which reflects a practical scenario where the source correlation knowledge are not always available. We compare the performance of the proposed joint decoding and estimation technique with the ideal 2D JSCC system with perfect knowledge of the source correlation knowledge. Simulation results reveal that our proposed coding scheme performs very close to the ideal 2D JSCC system. Keyword:


INTRODUCTION
Many improvements were implemented to channel coding including turbo coding which was introduced in 1993 [1]. The advantages of the turbo codes include high reliability and near to Shannon's capacity performance and hence, make turbo codes to be well adopted in various applications [2]- [4]. Shannon's separation theorem states that an optimal performance in separate source and channel coding system can be achieved if channel coding and source coding are optimal with zero delay [5]. However, it is impossible to get the finest performance with zero delay as the restriction on latency definitely occurs in practical applications. Joint Source and Channel Coding (JSCC) has been introduced by making use of the residual redundancies left in source encoders to improve the error correction capability of a channel code [6]- [8].
Recent research of [9], [10] have introduced 2-Dimensional (2D) source correlation for a JSCC system was exploited and utilized in the system design to improve the Bit Error Rate (BER) performance. However, these studies assumed that the source correlation knowledge is perfectly known at the receiver [9]- [11]. This assumption improves the performance but it affects the reliability of the information of source correlation to be exploited at the receiver. Practically, source correlation knowledge is unknown and needs to be estimated. Thus, many researchers have taken up this issue and developed various parameter estimation techniques to estimate the source correlation at the receiver to ensure the performance is improved and applicable in real practices [12], [13].
Several methods of parameter estimation have been presented in various studies based on a Hidden Markov Model (HMM) such as the Baum-Welch Algorithm (BWA) [14]- [20]. This enables the decoder to utilize the estimated statistics of the source correlation knowledge in channel decoding to improve the system performance. In this paper, we propose 2D JSCC system with BWA-based estimation technique to estimate the source correlation knowledge at the channel decoder on the non-HMM model to get the performance as close as possible to the ideal system. The proposed design is then compared with the existing 2D JSCC system with the perfect assumption on source correlation knowledge or known as the ideal 2D JSCC system [9] and another technique of parameter estimation exploiting the extrinsic information gleaned from the decoders adopted from [11]. This paper has been organised in the following way. The related works of parameter estimation techniques on channel decoding and the mechanism of BWA are discussed in Section 2. The proposed model on the coding scheme is presented in Section 3. Section 4 deliberates on the difference between BER performance from the proposed coding scheme and existing work. Finally, Section 5 draws upon the entire thesis, gives a conclusion and identifies areas for further research.

RELATED WORKS ON PARAMETER ESTIMATION TECHNIQUES 2.1. Parameter estimation on turbo decoder
Several attempts have been made previously to describe the parameter estimation of Markov model at the Turbo decoder in [15], [16]. These approaches have known to be complex due to the essence of building the supertrellises combining both constituent encoders and the HMM models. The modification on the decoder [17] has been done to improve the performance while the simplified method for modifying a decoder has been described in [18] to reduce its complexity while maintaining the same performance as in [16]. The parameter estimation has been refined in [19] to enhance the connection between BWA as the parameter estimation technique and Turbo decoder. BWA is implemented in the first iteration in order to have an initial estimation of Hidden Markov source parameters. The correlation pattern is considered and characterized by HMM [19]. The expected number of transitions from state to state that generate correlation pattern v are represented by the product of the branch probability and the number of trellis transitions. Hence, making it easy to estimate the expected state transition. Anwar, in [20], suggested that the stability and efficient exploitation of BWA can be improved by using a forward-backward algorithm or the uni-directional algorithm.

The simplified Baum-Welsh algorithm
The potential algorithm associated with the learning problem of an HMM is the BWA which has been developed in 1970 [21] and also known as Forward-Backward algorithm which is considered as the unique case of the Expectation-Maximization or EM algorithm [22]. In the learning problem, the parameters of an HMM denoted as are developed and adjusted depending on the applications given an observation sequence O and the set of possible states Q. Meanwhile, for the non-HMM considered in this work, we only used one parameter to be estimated to reduce the complexity of the system while maintaining its peak performance. In this method, only transition probabilities are used to estimate the source statistics in the decoder. The equation of transition probability is adopted from the previous work done by Garcia-Frias and Villasenor in [19]. The BWA is generally performed by calculated the initial estimation for the probabilities, and iteratively uses the information from the estimated value to perform better estimation upon the probabilities on the next iteration. The forward variables can be given as: where is forward variable, and is the partial observation sequence while the backward variable can be stated as: where () t i  is the probability of the partial sequence 12  The forward and backward variables in Equation (1) and Equation (2), respectively can be implemented to initialize the transition probability. The transition probability can be estimated as expected number of transitions from state to state ˆe xpected number of transitions from state ij ij a i  In order to calculate the numerator of, the joint probability of being in state at time and state at time denoted as is defined as: where the observation sequence and are given. The equation also can be written as: By using the forward and backward variables, the final equation of can be modeled as where is the final state.
Thus, the numerator of the transition probability can be calculated as a summation of for all t while the denominator of which is the summation of all the expected number of transitions in state can be denoted The final formula for can be stated as In summary, the estimation of the transition probability can be re-estimated using Equation (7) again to exploit the previous estimated value. The re-estimation process is considered as the basis of the iterative forward-backward algorithm. Figure 1 illustrates the top-level design of the proposed communication system. The source natural redundancy was considered in this study for simplicity and generalization purposes as there is no source encoder employed in the proposed system. The source is a 2D binary source having a 2D source correlation. The 2D source is converted into a 1D sequence represented by a vector u before being encoded by a channel encoder to output the encoded sequence c. Then, the encoded sequence c is modulated by a BPSK modulator to generate modulated output sequence g before being transmitted over an AWGN channel. During the modulation process, the binary encoded sequence is transformed into a signal form that can be physically transmitted, i.e., bit 0 maps to a signal amplitude -1 and bit 1 maps to a signal amplitude 1.

SYSTEM DESIGN
In a real environment, distortion and noise always be the factors that disturb the received signal at the receiver. Most commonly used and simple model to denote the channel behaviors of a communication system is the Additive White Gaussian Noise (AWGN), which is due to the thermal noise produced by the electronic hardwares and devices. In this model, the independent Gaussian noise signals with zero mean n and variance are added to the transmitted signal g. The energy of the transmitted signal is normalized to 1 and the examined energy per bit to noise power spectral density ratio, is used to measure the channel quality. The signal will be distorted and sent to the receiver as z = g + n. At the receiver, the received signal sequence z is demodulated by a BPSK demodulator to generate an output sequence r. The a posteriori information of the source correlation knowledge is estimated during channel decoding using the proposed parameter estimation technique. The new estimated value of source correlation knowledge is exploited at the decoder before outputting the decoded binary sequence. The detail of the proposed transmitter and receiver model will be presented in the following sections.

Transmitter design
The proposed encoder employs two channel codes that are serially concatenated to each otherwhich are Turbo Single Parity Check Code (TSPCC) [23], [24] and a rate-1 Recursive Systematic Convolutional Code (RSCCs) [25], [26]. The TSPCC acts as an outer code and comprises a concatenation of multiple SPCCs arranged in parallel with interleavers separating the SPCCs. A rate-1 RSCC is employed as the inner code, to improve the performance of the proposed system and to remove the high error-floor problem encountered by TSPCC [23], [24]. Figure 2 illustrates the block diagram of the transmitter model for the proposed JSCC system exploiting 2D source correlation. The 2D sources of is generated by S. The research target is to come out with an estimation technique for the 2D correlation parameters that can work well with the decoder of [9], thus the same design is used for the transmitter model. Two component encoders for TSPCC, and are considered. The interleaver, is a block interleaver that is used to arrange the source sequence to a sequence corresponding to the different direction of the source correlation denoted by. A TSPCC uses SPCCs as its component codes and it is known that SPCCs are simple codes that only generate a single parity check bit for any length of information bits. The advantage of this code is the flexibility in working with any length of information bits and it can operate at a very high coding rate. The parity bit sequences, and and source sequences are multiplexed after the source sequences have been encoded by the component codes. This joined sequence w is then rearranged by a random interleaver before passed through another encoding process by. The final encoded sequence c is modulated using a BPSK modulator before being transmitted over an AWGN channel.

Receiver design
The detailed receiver model of the proposed JSCC system is shown in Figure 3. The signal from the channel is received and demodulated to generate an output sequence, r. By using Bahl-Cocke-Jelinek-Raviv (BCJR) algorithm in, the sequence r is decoded via Maximum A Posteriori (MAP) decoding process [27]. Soft information in Log-Likelihood Ratio (LLR) form is used while performing the decoding process. The a priori LLR of coded bit, at a time index t, can be computed as: (8) where is the probability that has the value of 1, while is the probability of holds the value 0. The extrinsic LLR, that is generated from is de-interleaved using before fed into and. The two component decoders correspond to the decoding of the horizontal and vertical direction source sequence. Modified BCJR algorithm is employed to exploit the source correlation throughout the decoding process [9]. Each component decoder yields a posteriori and extrinsic LLRs outputs. The a posteriori are then sent to the parameter estimation technique block which employs the BWA given in Equation (7)

RESULT AND ANALYSIS
In this work, the main goal is to develop an accurate parameter estimation technique to estimate the unknown source statistics at the receiver for the 2D JSCC system performing as close as possible to the ideal 2D JSCC system as proposed in [9]. The BER performance is evaluated, by finding the number of errors of the decoded binary sequence from the original binary sequence from the source and this number of errors is divided by the frame length of the source sequence [28]. The Markov source was initialized with 0.5 to both dimensions. To illustrate the result, a simulation of BER against SNR simulation was performed using MATLAB software.The configuration used in the simulations to evaluate the performance of the proposed 2D JSCC system are given in Table 1. Turbo-Single Parity-Check-Code (TSPCC) was used as the outer code. The Single-Parity-Check (SPC) code consisting of 8 codeword length and 7 information bits is used for the component code of the outer code. The rate-1 Recursive-Systematic-Convolutional (RSC) code which had generator polynomial, ( , ) = (given in octal numeral system) is used in the inner code where is the feedback polynomial and is the feed-forward polynomial. Figure 4 illustrates the comparison in BER performance of the proposed 2D JSCC system and the ideal 2D JSCC system with source correlation in horizontal and vertical direction varying from = until =. The performance of 2D JSCC system assisting with the BWA-based estimation technique or also known as 2D JSCC system with Parameter Estimation 1 (2D-JSCC-PET1 system) is very close to the ideal 2D JSCC system and improves when the source correlation becomes stronger. For example, at BER level of, an improvement of 1.18 dB is achieved is achieved with the proposed 2D-JSCC-PET1 system over the non-JSCC system for p = 0.7 and the improvement increases to 4.48 dB for p = 0.9. The performance gain between the ideal 2D JSCC system and the proposed 2D-JSCC-PET1 system is about the same for each value of p. As can be seen in the Figure 4, the ideal 2D JSCC system is only slightly better than the proposed 2D-JSCC-PET1 system. However, the ideal 2D JSCC system relies on the assumption of perfect source knowledge at the receiver, which may not be suitable practically. Usually, the parameter estimation technique requires complex computations concerning its accuracy that might lead to performance degradation but despite the degradation that may occur, the gain for our proposed 2D-JSCC-PET1 system is not too far from the ideal system and hence, ensuring the system performing better in terms of the accuracy and the reliability of the data transmission at the receiver.
In another simulation, the performance difference between the proposed 2D-JSCC-PET1 system and the similar system without any parameter estimation denoted as the 2D JSCC system reveals that without the estimation technique, the performanceof the system could not do any better for unknown source statistics. The gain is measured as a difference of Eb/N0 at the BER level of between the 2D JSCC systems and the non-JSCC system. The performance gains of different systems are tabulated in Table 2. Figure 5 compares the BER performance between the proposed 2D-JSCC-PET1 system and 2D-JSCC-PET2 system when the correlation parameters are unknown for sources, exhibiting correlation varying from = until =. The performance gain between the proposed 2D-JSCC-PET1 system and the similar 2D JSCC system using different parameter estimation or the Parameter Estimation 2 (PET2) is summarized in Table 3. Figure 5 indicates that the PET1 module is much better than the PET2 module which simply comparing the reliable extrinsic LLRs gathered from each decoder [11].
It can be seen from Table 3 that there is a significant difference in BER performance between the proposed 2D-JSCC-PET1 system and the 2D-JSCC-PET2 system which indicates that the proposed 2D-JSCC-PET1 system is performing better than the 2D-JSCC-PET2 system. Based on the performance in Figure 5, it can be seen that the gain difference over the non-JSCC system between the proposed 2D-JSCC-PET1 system and the 2D-JSCC-PET2 system becomes larger for stronger source correlation, where at p =0.7, the gain difference is 0.84 dB and it increases to 3.47 dB at p =0.9. The performance achieved by the 2D JSCC systems with the parameter estimation technique show a significant difference when compared to the 2D JSCC system without employing any estimation method (the 2D JSCC system) and the non-JSCC system.  Figure 4. Comparison in BER performance of the proposed 2D-JSCC-PET1 system and the ideal 2D JSCC system [9] as the value of p increases    Non-JSCC