Inmate or visitor?

It's not really a punctuation post, but I realised the other day how hard it must be to learn English.

Take for example the huge difference in the following sentences by the addition of one letter:

Have you ever been in gaol?

Have you ever been in a gaol?