No one's opinion on this matter is any more valid than another. It's just that...opinion. You are not a moron nor a conspiracy theorist to think that once a guy's stock rises, he might leave. If anything, you should be able to learn by now that it is always a possibility. If Toronto was known as an NBA destination instead of a nice place to visit then fine, but it's not.
And yes, time will tell because we don't need to learn the same lesson a million times. If it gets close to re-up time and the humming and haw-ing starts, trade that ass.