Julia Gillard's right-hand man Wayne Swan has refused to talk about the next leader for the Labor party. Former treasurer Wayne Swan isn't overly confident of holding onto his north Brisbane seat.