Actually you do not need to worry since if the icon are inactive for a long time, Windows will monitor it and automatically ask you to remove them.
If you wanna remove it immediately, then you have to check with the desktop under you "My document/Desktop" else you have to remove from registry ( it is not recommended until you really know what are you doing!!)