438.7.a. "Swapping Back" is an extension of the Replace action.
438.7.b. To Swap Back, the token stops existing and the original card is returned to the space that the token just occupied, inheriting all current effects and statuses.
438.7.c. If there is nothing in Banishment to swap back to, usually because a token was replaced, then this object can never swap back.