The quick answer is yes. The adapter plates that come with the speakers are not really ideal. They too have to be shaved down substantially to fit of coarse. A superior choice would be to make some adapter plates out of wood or solid ABS plastic in the proper thickness needed.
The problem is the speaker basket / structure / spider hits inside of the speaker enclosure before the speaker sits flat so a spacer of some sort is needed.