Removing arrows and Icons from your Adsense Ads is possible using the "Allow and Block Ads" setting in your Adsense acounts.
Adsense terms these additional Arrows, Icons and Favicons as Enhanced Text Ads but these were disrupting my customised look of the ads. To remove the arrows and favicons you need to
login to your Adsense account >> Allow and Block Ads >> Ads serving >> Enhanced Text Ads
According to Adsense you
Choose whether or not you want to display text ads with performance-enhancing features such as product images, advertiser images, or clickable arrow icons, etc.. Blocking the Enhanced Text Ads will remove those unwanted arrows and icons from Adsense ads and show them as simple textual ads.