Mani Shankar Aiyar's controversial remark on PM Modi likely to effect Congress during polls