Barack Obama to Joe "The Plumber" Wurzelbacher: I think when you spread the wealth around, it’s good for everybody.
I was out of the country last week and as always, was glad to hear the words "welcome home" from the immigration officer upon my return. I came back to America, but apparently not to "real" America. I participated in the IEEE International Workshop on Machine Learning for Signal Processing (MLSP), which was held in Mexico this year. At the conference banquet, I sat between J. J. Remus of Duke University and Sergios Theodoridis of the University of Athens. Remus, who is originally from Wasilla, Alaska (for certain a part of "real" America), had a poster on distance-weighted nearest neighbor classification. Theodoridis gave a plenary talk and a regular oral presentation on using a sequence of projections onto convex sets in reproducing kernel Hilbert spaces to find feasible, but not necessarily optimized, solutions to problems such as finding classifiers.
A paper on sparsity measures, which relates to the quotation at the top of this post, was presented by Niall Hurley and Scott Rickard. Rickard had proposed six desiderata for sparsity measures in 2004, of which four were originally proposed by economists in the early part of the last century in the context of wealth inequity. In the MLSP work, it is shown that among fifteen sparsity measures, including the popular ℓ0 and ℓ1, the Gini index alone satisfies the six desiderata. Considering a vector of coefficient absolute values, where a large valued coefficient is analogous to a rich person, the six desiderata are:
- Robin Hood. Stealing from the rich and giving to the poor decreases sparsity.
- Scaling. Multiplying the wealth distribution by a constant does not change sparsity.
- Rising Tide. Adding a constant to each coefficient decreases sparsity.
- Cloning. If there is a twin population with identical wealth distribution, the sparsity in one population is the same as in the combination of the two.
- Bill Gates. As one individual becomes infinitely wealthy, the wealth distribution becomes as sparse as possible.
- Babies. Adding individuals with zero wealth increases sparsity.
According to the analogy, a sparse signal representation is clearly not an Obama signal representation. I think I'll put in a request to change the title of a journal paper of mine to Palin Representation in Structured Dictionaries.