Labels Lead Us Astray and Back Again