TT #701 · Parashas Ki Seitzei · Seventh Aliyah · Passuk 25:07
And if the man does not desire to take his yevamah, then his yevamah shall go up to the gate to the elders and say: “My husband’s brother refuses to establish a name for his brother in Israel; he will not perform the duty of a husband’s brother for me.”
Torah Temimah points to Yevamot 39b.
TT lemma: וְאִם לֹא יַחְפֹּץ הָאִישׁ.
לֹא יַחְפֹּץ – מִשְׁמַע שֶׁיֵּשׁ לוֹ בְּרֵרָה; אֲבָל אִם אֵינוֹ רָאוּי לְיַבֵּם – אֵין כָּאן חֲלִיצָה.
English: “Does not desire” — implies that he has a choice; but if he is not fit to perform yibbum, there is no chalitzah either.
On the daf / in context: The verse presupposes a man who could have chosen yibbum but declines it.
Implication from the language of desire/choice.
Chalitzah is available only to a man who was eligible to perform yibbum.