More than half a million foreign workers leave UK in past year