Love Island's Connor talks about returning from Casa Amour alone: "It did hurt me...I felt like my experience was over"