The iTunes Party Shuffle playlist quit working properly. It would show all songs in the library. Refreshing or changing settings did not fix. I found the fix here. Highlight all of the songs in the Party Shuffle playlist and delete them. (It does not delete your songs.) And then hit refresh.

Ask Ein: Party Shuffle shows every song in my iTunes library