|
|
|
“Information Rates of Autoregressive Sources”
by Robert M. Gray
August 1969
The rate distortion function, R(D), of a source can be interpreted as being the average amount of information that must be transmitted about a source for the receiver to be able to approximate the source within an average distortion D. It is demonstrated that for the class of time discrete autoregressive sources the rate distortion function for any difference distortion measure is lower bounded by the rate distortion function of the independent letter source that generates the autoregressive source. Autoregressive sources are constructed by passing such an independent letter source through a time discrete linear filter whose z-transform has only poles. This behavior holds even if the autoregressive source is nonstationary. The lower bound is shown to hold with equality for a non-zero range of small average distortion for two important special cases: the class of possibly nonstationary Gaussian autoregressive processes with a mean square error fidelity criterion and the binary symmetric first order Markov source with an average error per bit fidelity criterion. The positive coding theorem is proven for the constraint on its parameters.
Similar results are presented for the case of an independent letter Gaussian source passed through a more general linear filter and for the time continuous Gaussian autoregressive process. Some original results on the asymptotic behavior of the eigenvalues of Toeplitz and approximately Toeplitz matrices are derived.