Even Dwarfs Started Small