Spell check for unicode typography

Question

I’m using more and more Unicode in writing normal text, for instance I write “I’m” and not “I'm”, because it looks nice without having to trust some conversion when producing the output, and why not! (I have configured the keyboard to type, e.g., apostrophes and quotes easily.)

But this causes problems with spell checking in Emacs using flyspell/ispell, which recognizes, e.g., “isn't” buy not “isn’t”.

Any ideas of how to fix this, so that words using U+2019 for apostrophe (as recommended in section 6.2 of the Unicode standard) are accepted? I am fine with writing some elisp code if necessary, but I don’t know where to start attacking this.