"What else can I do? They know where I am now, they'll know where I'll be, no matter where I choose to go."

At least if it's SHIELD who knows, he has just a little more trust that they won't do him harm, or try to use him for some obscure purpose. Doesn't mean he won't be cautious - knowing Fury, he'll probably want him to join the Avengers again, and he can't do that. Not right now, at least.

"At least I won't be alone."