This is the statute of the Torah that the Lord has commanded, saying: Speak to the children of Yisrael, and they shall take to you a red heifer, perfect, in which there is no blemish, and upon which a yoke has not been laid.
Torah Temimah: Sotah 46a — I have only [the disqualification of] a yoke; from where [do we know] other labors? The verse says “asher lo alah aleha ol” – in any case. If so, why does it say “ol”? A yoke disqualifies whether during work or not during work; other labors disqualify only during work.
English: It was taught: I have only a yoke; from where other labors? The verse says “upon which a yoke has not been laid” – in any case. If so, why does the verse say “yoke”? A yoke disqualifies whether at the time of work or not at the time of work; other labors disqualify only at the time of work.
Ribui from the broader language and limitation of the specific term “ol.”
Any labor performed with the animal can disqualify it for the para aduma, but only a yoke (or equivalent) disqualifies even when not actively working; other forms of labor disqualify only while being performed.