Microsoft's anti-malware engineering team is proposing the use of distance measure and machine learning techniques to come up with automated way classify virus families.

MS Researchers Tackle Automated Malware Classification