Compressed sensing with continuous parametric reconstruction

Received Apr 12, 2020 Revised Jul 27, 2020 Accepted Aug 7, 2020 This work presents a novel unconventional method of signal reconstruction after compressive sensing. Instead of usual matrices, continuous models are used to describe both the sampling process and acquired signal. Reconstruction is performed by finding suitable values of model parameters in order to obtain the most probable fit. A continuous approach allows more precise modelling of physical sampling circuitry and signal reconstruction at arbitrary sampling rate. Application of this method is demonstrated using a wireless sensor network used for freshwater quality monitoring. Results show that the proposed method is more robust and offers stable performance when the samples are noisy or otherwise distorted.


INTRODUCTION
Compressed sensing (CS) is a relatively new method of signal acquisition and compression applicable to sparse signals. The defining feature of CS is that the compression is performed during the sampling process, by taking fewer samples than with conventional sampling. The original signal is then reconstructed from this reduced record. The main goal of compression remains the same-lower data transfer rate and reduced demands on communication channels. Furthermore, CS reduces the computational load required for compression to almost none by taking just enough samples to observe the needed information without redundancy. As a compression method CS can be implemented on devices with limited power and computation resources, such as wireless sensor nodes [1] or wearable electronics [2,3].CS is also naturally encrypting, as there is no way to reconstruct the original signal without detailed knowledge of the sampling process [4]. The reason why CS is not yet widely implemented is that it is not versatile. It can only be applied on certain signals and the entire framework-sampling and reconstruction-has to be tailored to each individual application. Despite this disadvantage CS has found its way into applications such as medical imaging [5][6][7][8], audio [9] and video [10][11][12] processing, vibration sensing [13,14] data gathering [15] etc.
The basic idea of CS was originally proposed by [16] with a number of sampling methods for CS proposed afterwards. Most often used are the earliest random demodulation (RD) [17], random modulation pre-integration (RMPI) [18] and random sampling (RS) [19], and the more recent non-uniform wavelet sampling [20] The goal is to capture as much information as possible with as few samples as possible without introducing aliasing. Reconstruction also presents a specific challenge-solving an underdetermined system of linear equations. Reconstruction is conventionally based on a basis matrix defining signal features,  [21] or using methods such as principal component analysis [22][23][24][25], feature extraction [26][27][28] or methods of machine learning [29][30][31]. A number of reconstruction methods have been established, (orthogonal) matching pursuit [32], CoSaMP [33], iterative hard thresholding [33] or matrix pseudoinverse [22] are commonly used. None of these methods is truly versatile, some are more advantageous in a certain application than others. Research in this area continues, general approaches [6,14,35] as well as specific use cases [4,12] are being proposed.
The main problem with CS is its weak robustness. There are a lot of conditions and rules that have to be followed, which is not easy to guarantee in a practical application. The reconstruction process is sensitive to noisy samples, unexpected signal deviations, jitter [36] and other variables associated with input circuitry [37]. Any of these easily results in distorted output or a failed reconstruction. Some reconstruction methods are more robust than others, with the robustness inversely proportional to the computation load. However, CS in general is not a robust framework.
This paper proposes a continuous reconstruction method that offers stable performance and is more robust than conventional matrix-based methods, the continuous parametric reconstruction (CPR). Conventional CS reconstruction is reliant on matrices to describe both the signal features and the sampling process. Such reconstruction will be referred to as direct linear reconstruction (DLR). Since a matrix description is discrete, there are limits in what precision can be achieved. The proposed method is not utilizing matrices to describe the sampling process and possible input signal, instead using continuous models. Such approach has several advantages that will be discussed. Performance of the proposed method will be demonstrated on a specific freshwater monitoring sensor network and directly compared to DLR. The article is organized as follows: section 2.1 provides theoretical background on conventional CS; section 2.2 shows the limitations of a discrete matrix-based CS framework; the proposed reconstruction method is described in section 2.3. Section 3 describes the sensor network and signals that were used as a benchmark in 3.1, and provides an example of how the proposed method can be implemented in section 3.2. Section 4 shows the results in direct comparison to conventional matrix-based reconstruction and discusses the differences. The paper is concluded in section 5.

THE PROPOSED METHOD 2.1. Compressed sensing background
A conventional approach to CS will now be described in order to put the authors' proposed method in context. CS can be applied if the input signal can be expressed as vector ∈ ×1 , which is compressible and can be classified as sparse. Let there exist a set ∈ × of basis functions ∈ ×1 , 1 ≤ ≤ , such that any possible input signal can be described as

=
(1) Vector ∈ ×1 performs linear combination of the basis functions. If has only s entries of appreciable value and << , the signal is denoted as s-sparse. This signal can be sampled by correlating it with with measurement signals, < , defined by the measurement matrix ∈ × . Resulting is the signal If displays certain properties [38] and was chosen correctly [39], has fewer samples (lower sampling frequency) than but contains all the information needed for reconstruction. is hence referred to as the information signal. The measurement matrix usually has random entries so aliasing is avoided and matrices and must be mutually incoherent. If the matrix is subject to implementation on a physical circuitry, entry values of 0 and ±1 are preferred. The mostly studied sampling methods (see introduction) each have their own structure of measurement matrix.
In the reconstruction phase only bases , and the information signal are known. The original signal can be recovered if the conditions above are met and < << . Let = 0,1,2, … Depending on the solver used. In (1) to (5) with solving a system of linear (5) represent the DLR, a conventional approach to CS. (5) can only be solved if the sparsity condition is satisfied and enough samples have been taken [38]: CS framework with DLR is summarized in Figure 1.

Elaboration of shortcomings
As was mentioned in the introduction, the DLR CS framework is sensitive to noise and other disturbances and it is not inherently robust. That is because there are a lot of strict conditions (see previous chapter) that need to be satisfied. The input signal Nyquist limit, associated to , has to be met. Input signal has to comply to the dictionary , with given upper limit of sparsity related to . These properties all meet in condition (6), which has to remain satisfied for successful reconstruction. The designer can in no way influence the measured signal, only anticipate its properties and set up the CS framework accordingly.
Complying with the Nyquist limit can be ensured by analog input filter. But, any dictionary is just a prediction and any measured signal will not comply to it with mathematical precision. This translates to change of sparsity, a small signal difference can cause a great increase of sparsity on a given dictionary . As a result (6) may no longer be met because would have to be increased. Noise that is inadvertently present in any measured signal causes that these ideal conditions are rarely met. The reconstruction methods intended for CS are designed to be noise tolerant, however, the reconstruction tends to be near-perfect up to a certain point and fails beyond that. The behavior under adverse conditions is strongly dependent on application and used basis.
Another source of imperfection is the sampler circuitry. Investigations on this subject have been performed e. g. by [36,37], including clock jitter and analog multiplier transitions. The jitter has to be tolerated by the reconstruction algorithm, but the multiplier characteristics could be moddelled in . What prevents this modelling is the codependence of M and in DLR CS framework. Let us illustrate on an example in Figure 2. Sampler architectures such as RD and RMPI use analog multipliers usually implemented as switching circuits. describing such sampler typically contains only values of ±1, typically a few entries per period, assuming ideal instant transition of the multiplication factor between the two values. A real transition however is continuous, with the discrepancy resulting in increased reconstruction error. The real transition could be modelled in , but that would require upscaling it and increasing N. As a result, (6) might no longer hold and the reconstruction would be impossible.
A sampler that is close to ideal is the RS, since it comprizes of just ADC, timing circuit and no other sources of imperfection. Neither this type of sampler is immune to the M and codependence. By tying M and together by means of the measurement matrix, sampling can not be as close to random as the circuitry potentially allows. This has an adverse effect on mutual coherence [38]. Let us describe the RS measurement matrix as suggested by [19]. Ξ contains indices of randomly selected samples within the processing frame : Entries of Ξ are random numbers, following Elements of measurement matrix representing RS can then be obtained as and the information signal elements become = Since is limited by range in (8), it is apparent that RS has a limited variance of inter-sample interval. The issue is illustrated in Figure 3 where two sampling series with different elementary time increments are shown. The smaller the increment, the closer sampling gets to truly random, but has to be increased possibly to the point where (6) no longer holds.

The proposed solution
The authors believe that a lot of the CS framework shortcomings can be circumvented by abandoning the discrete matrix descriptions used in DLR and transitioning to a continuous reconstruction method based on modelsthe continuous parametric reconstruction (CPR). The authors propose to emulate the behavior of (5) by Where (̂,̃) is a parametric signal model with parameters ̂. This model substitutes a discrete dictionary by a suitable multi-parametric function. Signal model is sampled according to ̃, a descriptor of the sampling process. ̃ may be a continuous operator, for example ideal RS may be described using a series of modified delta functions so that Time instants , = 1,2, … , at which the samples are taken are not limited to indexes within available positions. CPR hence does not bound the sampling process to the number of reconstructed samples. Besides reconstruction this has also implications on the sampling circuitry. In the case of RS, the elementary time increment is only limited by the capabilities of circuitry and the size of quantized timestamp which has to be transferred along with . With DLR the reconstruction is performed by finding a linear combination of up to basis functions. A large number of constant basis functions within the dictionary is available to choose from. By contrast, CPR could be seen as having a dictionary of just one or a few basis functions that are variable along their parameters. Reconstruction is performed by finding the right parameter values. The authors believe that this approach is potentially more resistant to adverse conditions if a suitable signal model is used. By forcing the reconstruction to adhere to a specific signal model the CPR is a form of curve fitting, returning the best fit. DLR is more rigid and forces quantization in both time and shape of the signal. The proposed method is continuous in both time and parameters, potentially allowing for greater resolution at reduced computation cost. The CPR is summarized in Figure 4. An issue with CPR is ensuring the signal model's orthogonality and convergence to a single correct solution. A solution space needs to be defined, over which the model meets this criterion and still simulates any input signal. Furthermore, (12) in general is a nonlinear, nonconvex problem. The authors propose to use differential evolution [40] for solving (12), as it can search large solution spaces with no regards to the model's linearity. In this regard the conventional DLR is advantageous, as it involves solving a convex problem (5) for which a number of efficient solvers is available [33].

RESEARCH METHOD
The performance of the proposed CPR will be evaluated using a wireless sensor network (WSN) used for automated monitoring of river water quality [41]. This network was deployed near the Slovak-Hungarian border in the Ipel River. The authors investigated CS application in this network because of tight energy budget, which can potentially be improved by lowering the sampling frequency. Results of previous investigation [22] of conventional DLR CS in the same network will be provided as reference.

Freshwater monitoring network
The WSN was organized as two sub-networks, each containing 5 buoys carrying multiparameter probes and sending data to onshore gateways as shown in Figure 5 [41]. The sensor nodes measured multiple water parameters such as temperature, salinity, dissolved oxygen, pH, total dissolved solids etc. [42]. Measured signals were acquired by on-board sensors, transferred to a central database and stored. The developed system was tested in pilot operation spanning over a year, covering periods with extreme weather and flash floods. The sensor nodes used solar power for charging the integrated battery. This proved to be of concern during winter months, when only a limited amount of solar power is available and the battery performance is reduced due to low temperatures. The WSN node power budget comprises of three main parts: the transmitter that draws power when transmitting data (~0.8%), MCU and other control circuits (~0.9%), and the probes that draw power upon triggering of the measurement process (~98%). Since probes consume most of the power, this WSN is a textbook case of saving power by utilizing CS and lowering the sampling rate.

Sparse signal model
This subchapter will demonstrate how to use the proposed CPR (12) and reconstruct a certain type of signal. Data acquired by the WSN described above show that signals of all the measured parameters correlate and display the same basic pattern. A typical signal contains a large slowly changing DC component, a faster AC component with relatively small amplitude and a small amount of noise [22]. Examples are shown in Figure 6. The AC component is a distorted sine wave with fundamental frequency D=1.157×10 -5 Hz (1 per day). Frequency analysis shows that there are only up to 4 higher harmonics of appreciable magnitude. The signal thus carries only limited information content and can be classified as sparse, meeting the criteria for CS application.

Figure 6. Examples of acquired signals
A suitable parametric signal model is needed for reconstruction via CPR. The following signal model is proposed for the discussed application: ( ) is a polynomial of order Λ with coefficients

Sampling and reconstruction
RS was chosen as the sampling method because it can be implemented in the existing WSN with no hardware changes. The authors chose to implement RS according to (11). It does not utilize the whole potential of CPR as (13) (see section 2.2), but it can be tested using existing data. By inserting (13) and (14) into (12) (20) where ̂ are samples of the modeled signal at random sampling instances Parameters ̂,̂,̂ can be used with model (14) for original signal reconstruction at arbitrary time instances. The authors propose to use polynomial fit prior to CPR in order to get a rough estimate of (16), which can be fed into (20) as a starting point. Reconstructed samples are not bound to integer multiple s , = 1,2, . . . , , as was previously discussed. It should be noted that (14) is not a hydrological model and it cannot be used for forecasting, which would be done by setting t outside the processing frame.

RESULTS AND DISCUSSION
The performance of proposed sensing and reconstruction method was tested on data obtained during WSN pilot operation. Testing was performed via simulations that sufficiently represent implementation on the physical system. 200 signal frames with length N=120 (5 days) were chosen randomly from the available database for testing. Signal-to-deviation ratio (SDR) was used for evaluation, with ̂ being samples of the reconstructed signal. 10 2 test runs were performed on each of the test signals with a new sampling sequence generated every time. For every scenario, SDRs of all the runs of all the test signals were averaged. CPR needs to have the solution space bounds defined prior to reconstruction. The algorithm is not particularly sensitive to correctly set bounds but doing so aids the computational cost. The maximum number of harmonic components in (17) was set to K=4 (see sect. 3.2). The polynomial order of (15) was set to = 3. This number was found by investigation of inflection points of acquired signals after filtering out the AC component. Both K and could also be found by an educated guess or by trial and error, since they are small numbers by the nature of model (14). The study [22] utilized CS with DLR based on extensive trained dictionary. Methods such as orthogonal matching pursuit, iterative hard thresholding, CoSaMP etc. [33] were tested for solving (5), with pseudoinverse matrix performing the best. These results will be provided as reference.

Compression ratio
The entire purpose of employing CS is the reduction of power consumption, which is in direct relation to the average sampling frequency. This reduction can be expressed as the compression ratio = (23) or the ratio of number of hidden Nyquist samples over the number of samples taken nonuniformly. Figure 7 shows the SDR vs. CR along with projected power consumption (PPC) in % of the original-PPC=100% for CR=1. The PPC was computed based on the power consumption of individual sensor node components. Implementing CS in the physical WQM system will require choosing a suitable CR. CR=6 is used in further experiments, as it displays a reasonable trade-off between high SDR and low PPC with both methods.

Resistance to noise
Noise can represent any uncorrelated interference that can alter the acquired samples. Noise is one of the inherent signal features that make CS application more complicated. Some CS reconstruction methods can be quite noise sensitive, the investigation of noise resistance of a particular CS application is therefore prudent.
The experiment was conducted by adding an artificial noise of varied standard deviation to acquired samples after RS. SDR was computed by comparing reconstructed signal to the original signal without noise. The test signals were normalized to 1 so that the introduced noise see in Figure 8 presents a consistent degradation level.

Resistance to quantization
The data that are read out of the probes inside buoys are of high resolution (float, ASCII) data types. This high-resolution data are featured out of opportunity rather than necessity-high resolution but low speed ADCs are cheap and common. The range and resolution of featured data types are not utilized and not practically needed, yet they are being transmitted. A considerable reduction of payload could be achieved by simple rescaling and quantizing of the measured data. Provided that the reconstruction algorithm can handle it, quantization is a viable means of reducing the data payload and increasing the reliability of a WSN.
An experiment was conducted where the acquired signals after RS were quantized by a simulated ideal unipolar ADC with a range of <0;1>. Testing signals were normalized to 1 prior to RS and quantization in order to ensure a comparable level of degradation. SDR was computed by comparison of reconstructed and original non-quantized signals. The results are shown in Figure 9.  Figure 7 shows that under ideal conditions the proposed CPR reconstruction method underperforms in comparison to dictionary based DLR. A trained dictionary obtained from an extensive signal database is capable of near-perfect-reconstruction. An intentionally simple model like (14) cannot do this as can be seen in Figur 10, leading to the 5dB lower achievable SDR (45dB vs. 40dB with CR=6).

Discussion
For the end user-hydrologist, recovering all the intricate details is not practically needed and reliability is more important. Near-perfect-reconstruction capability may be counterproductive if the conditions are not ideal. Figures 8 and 9 show that the DLR performs worse than the proposed CPR under adverse conditions. This is most prominent in Figure 8 where even a small amount of noise causes the two methods to become equal in achievable SDR. Further increase of noise level causes the proposed CPR method to perform up to 8dB better. The reason for this turn can be seen in Figure 11.
Conventional DLR has too many options when trying to solve (5), which causes it to "invent" its own noise and artifacts to fit the distorted samples. The proposed CPR by adhering to the semi-analytical signal model yields the most likely fit, resulting in both higher SDR and subjectively more realistic output. Model based reconstruction is also naturally denoising, similar to methods used in biomedicine [43,44]. Situation is similar when quantization noise is degrading the samples. Quantization noise is correlated and does not lower SDR as much as random interference. Here DLR also starts with higher SDR until it is surpassed by the proposed CPR at lower resolutions. Minimal effective resolution needed is 10 bits for DLR and 8 bits for CPR.
The authors propose that with the proposed CPR the compression ratio can be set to CR=6 which would reduce the buoys' power consumption to app. 17% of the original. Implementation of random sampling in the analyzed WSN would require no hardware changes at all and no buoy firmware changes. Further reduction of data traffic could be achieved by rescaling and quantizing the high-resolution data, although this step would require a revision of the buoys' firmware. Energy cost of data transmission is insignificant in this application, but reduced traffic aids the reliability of networks in general. Presented reconstruction method can handle 8-bit full scale resolution without degradation of the reconstructed signal. 12-bit quantization would provide the necessary dynamic range excess while reducing the data payload by over 60%, resulting in overall compression ratio over 10. It can be concluded that DLR trades robustness off for near-perfect reconstruction, while the proposed CPR offers modest but stable performance in challenging conditions. A considerable disadvantage of DLR may be the need for Nyquist signal database. WSN described in sect. 3.1 already made a pilot run during which the database has been acquired, so in this case both reconstruction methods are directly comparable and feasible. However, if the signal database was not available, the proposed CPR would be much easier to implement than DLR. The limits of model (14) required to be known for implementation of the proposed CPR can be observed in just a few days, possibly by sample grabbing. Obtaining a training database for conventional DLR would require deploying the WSN itself, since hundreds of days of Nyquist record are needed [22].
A great advantage of the proposed CPR is that it is not necessarily bound to any underlying Nyquist sampling. When designing a CS system with DLR the underlying Nyquist sampling must be kept in mind. Sampling that takes place on a physical circuitry is described by the matrix ∈ × . If proper discrete representation of the sampling process requires high sampling rate (tied to N), the limit (6) can be easily exceeded, making the reconstruction impossible. For RS this means that sample positions must adhere to a sufficiently long elementary time increment . The proposed CPR allows to model the sampling continuously, in case of RS by timestamps, and samples can be theoretically taken at truly random time. Results show that (6) still has to be met, but with N corresponding to input signal Nyquist rate only, not the Nyquist rate required for discrete sampler description. On the side of recovery, the proposed CPR allows to sample the output signal at arbitrary sampling rate or at arbitrary time instances. With conventional DLR the output sampling is tied to the sampling of bases and .

CONCLUSION
A novel CS reconstruction method based on continuous models and parametric estimation has been presented. Performance of this method was tested on an example of WSN and directly compared to conventional linear CS reconstruction. Results show that the proposed method is more robust and offers stable performance with varied number of samples or samples that are degraded by noise and quantization. The proposed method is compatible with but not limited to random sampling, which is simple to implement in general and particularly on the WSN analyzed in this work.