What is the difference between saying ‘Holy Ghost’ and ‘Holy Spirit’?

The short answer: there is no fundamental difference.

Like many languages English has more than one word for the same thing.  The word ‘ghost’ comes from the German word ‘geist’ and the word ‘spirit’ comes from the Latin word ‘spiritus.’

The choice to use ‘spirit’ in many places where ‘ghost’ was used in the past may have more to do with popular understanding of these two words than anything else.  If Christians start thinking that a ghost is something to be feared then perhaps ‘Holy Spirit’ is a better choice when discussing the God who is love.