Closed book, closed notes test. Honor Code applies.

Problem 1. Draw general diagrams of two adaptive systems:

(i) equalization (de-convolution ) system

(ii) predictor system

and describe in 3 - 4 sentences per case what each of them does and how (approximately).

Problem 2. Given is the correlation matrix  and the cross-correlation vector. The variance of the signal is 1.

The non-normalized eigenvectors of R are  and .
 

a) write down the expression for the Mean Square Error corresponding to these data.

b) find the optimal weight vector

c) find the minimum of MSE

c) express the function MSE as the sum of minimum MSE plus a term dependent on weight deviations  from the optimal weights.
 

Problem 3. Using the same data as in Problem 2:

a) write down the iterative formula for the method of steepest descent

b) represent the steepest descent algorithm in the principal coordinate system, using weight vector and exhibit the diagonal matrix involved in this iterative process.

c) from the above formula find the upper bound for the step size of the steepest descent method (note that the eigenvalues of R can be calculated very easily).
 

Problem 4. Using the data from Problem 2
 

a) write down the formula for the Newton method, showing in detail the matrices involved

b) write or derive the formulas for the learning curve in both the Newton and the steepest descent method.

c) interpret the latter in terms of the two geometric ratios , n=1,2 for this particular problem

Problem 5. Solve either problem 5A or 5B: (solve one of your choice, not both).

5A: Given are two periodic signals composed of +1, -1, and possibly 0, whose sequences look as follows:

signal A: 1,1,1,1,-1,-1, -1, -1 and repeat the same sequence indefinitely

signal B: 1,1,1,1,0,-1,-1, -1, -1, 0 and repeat the same sequence indefinitely

For each of these signals construct a 3 x 3 Toeplitz correlation matrix, by using the 'average-over-a-period' instead of the true mathematical expectation E.

5B: A discrete-time zero-mean gaussian white noise with standard deviation is passed

through a FIR filter with impulse response h(k)= ½, 1/4, 1/8. Find the 3 x 3 correlation matrix of the signal at the output of the filter. Then find an analogous matrix for the signal that results from adding a unity white noise to the preceding (output) signal.

Problem 6. Use the matrix R from problem 2 above. Assume we know R, but we do not know the gradient and we measure it by using the finite difference method with at each iteration of the Newton method. The total number of measurements of at each iteration is 120, that is we have four combinations of weights where we measure the error. (You need to find number N). Let .
 

a) Calculate the covariance of the estimated gradient 
 

b) Calculate the weight vector covariance 
 

c) Calculate the time constant of the weight adjustment process from the equation 

where is the geometric ratio of the Newton method, that you are supposed to be able to calculate from the data of this problem.

Time allocated for this test: 140 minutes.