Guilt is one of the biggest killers of joy. We have an idea in our heads about how we should act, how we ought to be. Some of those ideas are right, some are wrong. Some developed because we decided what type of life we wanted to live. Some were instilled by our parents, teachers or friends. Either because they wanted us to have a certain type of life or because it fortified their own idea of the world.
The next time guilt is taking the joy out of something it may be worth it to ask, "where did this guilt come from?" "Is this guilt guiding me to the life I want to live? or is it taking the joy out of that life?"
You may find out that guilt isn't your own.