Yosuke leaps off The Luggage before it can get any ideas that involve causing him bodily harm or killing him. He makes sure to put some distance between himself and it, as he answers Rincewind.

"Yeah, I'm not done in here. There's something in here we gotta find. Though this place seriously gives me the creeps..."

And he feels a little claustrophobic too, being used to the open plains around Citadel. Possibly one of the worst places for him to be. Plus, he does not care what happens to the dumb box either. If Rincewind leaves, then hopefully The Luggage goes with him. One less stress factor.