Optimal Signals and Correlators for Detection and Communication in non-Gaussian Noise

In this work, we explore a Neymann-Pearson hypothesis testing scenario where, under the null hypothesis (H0), the received signal is a white noise process Nt, which is not Gaussian in general, and under the alternative hypothesis (H1), the received signal comprises a deterministic transmitted signal st corrupted by additive white noise, the sum of Nt and another noise process originating from the transmitter, denoted as Zt, which is not necessarily Gaussian either. Our approach focuses on detectors that are based on the correlation and energy of the received signal, which are motivated by implementation simplicity. We optimize the detector parameters to achieve the best trade-off between missed-detection and false-alarm error exponents. First, we optimize the detectors for a given signal, resulting in a non-linear relation between the signal and correlator weights to be optimized. Subsequently, we optimize the transmitted signal and the detector parameters jointly, revealing that the optimal signal is a balanced ternary signal and the correlator has at most three different coefficients, thus facilitating a computationally feasible solution. In the second part of this work, we derive results along the same line of thought for the problem of communication across a channel that is not necessarily Gaussian using correlators. Due to the high degree of complexity of the problem, most of our results pertain to the case of M=2 equal-energy signals, but we also outline a path towards a possible extension to larger signal sets. Here too, we first derive the optimal correlators for given signals and then jointly optimize both the signals and the correlators. Numerical results will be presented as well.

M.Sc. student under the supervision of Prof. Neri Merhav.