# Homeworks

• ### 4th Homework:

• Type: Manual+Simulation
• Manual Part: The problems given during the course, that are:
1. Prove that the L0-norm (more precisely L0 "pseudo-norm") satisfies the triangluar inequality, that is, ||x1+x2||_0 <= ||x1||_0 + ||x2||_0.
2. D is an overcomplete dictionary resulted by joining two n by n orthonormal dictionaries (two Bases). Show that M>(1/sqrt(n)), where M is the mutual coherence of D.
• Computer program: Implement Basis Pursuit method for finding the sparse solution of un underdetermined linear system. You can use this guide.
• Due date: Manual: Tuesday 97/3/7, and Simulation: Thursday 97/3/9.
• ### 3rd Homework:

• Type: Manual+Simulation
• Manual Part: Prove equations (11) and (15) of Cardoso's paper ("BSS: Statistical Principles"). Solve the following problem, too:
• Till now in the course, we look at the samples of a signal s[n] as realizations of a random variable S. For a sinusoidal signal (s[n]=cos(Omega_0 n)), calculate the probability density function (pdf) of this random variable (S). Assume that Omega_0 is not a rational number.

• Simulation Part: Implement the BSS method based on mutual information minimization. For estimating score functions, use two methods: polynomial and kernel estimators. For the iterations, use both equivariant and non-equivariant (usual steepest descent) and compare their sensitivity versus the conditioning of the mixing matrix. Your program has to be in the form of a MATLAB function:
• [y1,y2]= separate(x1,x2);

x1 and x2, each one of is mixture of two signals. The function takes them, and gives us y1 and y2, which have to be separated signals. To test the function, you create in MATLAB two signals (s1 and s2), mix them (for example using x1=0.8*s1+0.2*s2; x2=0.2*s1+ 0.8*s2) and then give x1 and x2 to the above function to get y1 and y2. Then, measure the difference between separated and original images using SNR in dB, defined as (assuming that you have no permutation, that is, y1 is an estimation of s1 not an estimation of s2):

SNR = 10 * log10 (mean(s1.^2) / mean( (s1-y1).^2 ) )

and include these SNR's in your report.
• Due date: Manual: Tuesday 98/2/24, and Simulation: Thursday 98/2/26.
• ### 2nd Homework:

• Type: Mostly Manual + 1 simulation
• Manaul part:Problems 3.3, 3.4, 4.1, 4.2, 4.7, 4.9, 4.11, 4.13, 4.18, 5.5 of Hyvarinen's book.
• Computer program: Write a MATLAB program to compute the differential entropy of a random variable from its PDF via numerical integration. This program takes a "Delta_x" to devide the integration interval to intervals of length "Delta_x". If you have written the program correctly, the estimated entropies for different "Delta_x"'s should converge something, when you decrease "Delta_x" to zero. Check this point. Calculate the entropies of a uniform and a Gausian random variable (both of zero-mean and unit variance) using your program and compare the result with true values.
• Due date: Tuesday 97/1/19.
• ### 1st Homework:

• Type: 6 manual problems PLUS one computer program.
• Manual part: Problems 2.2, 2.7, 2.9, 2.12, 2.15, 2.24 of Hyvarinen's book.
• Computer program: Write a MATLAB program to compute the kurtosis of a random variable from its samples. Test you routine on U(0,1) and N(0,1) random variables, and compare the results with true values.
• Due date: Sunday 97/12/19.