This makes v3d match vc4's destroy path. Fixes: e113b21c ("v3d: Add renderonly support.") (cherry picked from commit 486b181f)