He wants to apologize for the way things ended. Things moved too fast which kinda scared him. He barely knew you and just wanted things to slow down, but accidentally put it all to a screeching halt. You end up on his mind every once in awhile, but biased towards the reasons he left.
Sounds like he should forget the past and move on or try to make up for it by starting all over.