There is a distinct difference, however whether it's perceivable, and how much, depends on many factors.
Unfortunately I can't access the full articles, however here's the abstract of a very relevant article, emphasis mine.
Mp3 compression is commonly used to reduce the size of digital music files but introduces a number of potentially audible artifacts, especially at low bitrates. We investigated whether listeners prefer CD quality to mp3 files at various bitrates (96 kb/s to 320 kb/s), and whether this preference is affected by musical genre. Thirteen trained listeners completed an A/B comparison task judging CD quality and compressed files. Listeners significantly preferred CD quality to mp3 files up to 192 kb/s for all musical genres. In addition, we observed a significant effect of expertise (sound engineers vs. musicians) and musical genres (electric v.s acoustic music).
So, this study finds that the answer is that above 192Kbit/s, further gains depend on the genre and the training of the listener.
Furthermore another study finds that perceived quality depends on whether you are using headphones or not:
The impact of using loudspeaker versus headphone playback on the subjective quality of compressed audio is investigated. It is shown that reverberation and to a lesser extent cross-talk, which both are introduced naturally in loudspeaker playback, can effectively hide coding artifacts.
This other paper describes the differences between different bitrates and different ways of testing. In all cases it shows a very minor difference between 192Kbit/s and 256Kbit/s and basically no difference between 256Kbit/s and 320Kbit/s.
As you can see, above 192Kbit it becomes quite hard to tell the difference.