There are several solution, based on your Lync environment, but independent from the Outlook version.
First with Lync 2010 Client:
If you have Lync 2013 (15.0.4517.1005 or newer) or Lync 2010 (4.0.7577.4419 or newer) installed there is only one other option.
Lync 2013 Client Update May 2014: http://support.microsoft.com/kb/2880980
- Lync Server 2010
Upgrade your system with CU5
- Lync Server 2010 CU 5 and Lync 2013 (any CU)
You must change your Lync Client Policy
Set-CsClientPolicy -Identity Global -EnableExchangeContactSync $False
another option is, since not all clients are affected, you build another ClientPolicy with this option disabled.
New-CsClientPolicy -Identity OutlookContact -EnableExchangeContactSync $False
Deleting the Contacts:
After you have changed this setting and they are applied to the user, he is able to delete the duplicated contact in Outlook.
If you still facing problems deleting the contact, go to Outlook Web Apps and delete them in a browser.