Remove Remote Desktop history

HKEY_CURRENT_USER\Software\Microsoft\Terminal Server Client\Default

You'll see keys that begin with MRU

Delete the entry you don't want.

 

Then go to the current user's My Documents and delete a hidden file called Default.rdp.