More precisely check for the qualifiers that enable the vulkan memory model for buffer references.

Not all memory qualifiers necessarily need the vulkan memory model, e.g. volatile/restrict that correspond to core SPIR-V features do not.