Uncle left Bailuo, saying he was heading to Yizhou World. This statement was not wrong, but he was not going immediately.
The Flood had opened the barrier, but even without the Flood, with Uncle's current Ability, he could come and go freely. Otherwise, he wouldn't have told Bailuo that if summoned, he could return immediately.
Being able to return means being able to leave; the two are no different.
"Did you wait here specially to tell me something?"
A mist shrouded the area, and a figure appeared beside Uncle, shockingly it was The Silencer who had previously made a deal with Bailuo.
"You shouldn't have let that child bring me back."
"I have done that before," The Silencer spoke, "but the results weren't good."
"This time I'm here."
"However, this has nothing to do with me."
The Silencer discerned the invitation hidden in Uncle's words but had no intent to be deeply involved, as this simply wasn't her war.
