My goal is to find the most important features that differentiate two classes. It makes sense to use one of the many approaches for feature selection out there to do that.
But here's my problem: I have a lot of correlated features.
Usually the goal of feature selection would be to eliminate those redundant features. But the features have a semantic meaning and I want to avoid loosing that information.
So if a group of correlated features has strong predictive power for the class variable, I want them all to be identified as important. (Bonus problem: If I include ten correlated features in my model, their resulting weights will end up being only a tenth of their "actual" importance.)
Can you think of a feature selection approach which finds important features even if they show up in groups of correlated festures?