Improving Speaker Diversity