Another way of releasing guilt is by expressing your feelings in a diary or a journal. In fact, working your thought out on paper is like self-therapy. Afterwards, you will feel you released a big burden, especially if you are too embarrassed or upset to let others know or to talk about it.