In Episode 505, we saw Ghost at an all-time low. He got drunk at the club and missed Tariq's birthday celebration. Bad enough this is the first birthday he had without his twin Raina. When he finally showed up, he was drunk and out of his mind. He was so drunk that he did not realize that the cake had his daughter's name on it. And then he was ready to fight his own son after Tariq hit him with some cold hard truth. That would lead to Ghost seeking help from a minister.
During his one-on-one with the minister, Ghost comes to the realization that he is the one at fault for everything. Of course, he tried to blame everyone else for everything that is going on around him, but when you look at it, it is his fault. It took another man to point that out to him.
So who should a broken man go to? His wife right? But what if his wife is a major cause for him being broken? Instead, he goes to Angela. I have always said, Ghost wanted to become Jamie and when he was with Angela he can be Jamie. With Tasha, he will always be Ghost. She always wanted to be the biggest damn drug dealer in New York. While Angela did not know that side of him, she saw the man that he wanted to be, legit. Remember, Angela was his first love. He entered the street life with hopes of getting out. His biggest fault was opening one chapter before closing the other.