Archive and suite lookups aren't using available indexes
I looked at pg_stat_activity while a moderately-sized workflow was being orchestrated, and I happened to notice that we're spending a fair bit of time in poorly-indexed CollectionItem queries. For example:
debusine=> EXPLAIN (ANALYZE, BUFFERS) SELECT "db_collectionitem"."id", "db_collectionitem"."name", "db_collectionitem"."child_type", "db_collectionitem"."category", "db_collectionitem"."parent_collection_id", "db_collectionitem"."parent_category", "db_collectionitem"."collection_id", "db_collectionitem"."artifact_id", "db_collectionitem"."data", "db_collectionitem"."created_at", "db_collectionitem"."created_by_user_id", "db_collectionitem"."created_by_workflow_id", "db_collectionitem"."removed_at", "db_collectionitem"."removed_by_user_id", "db_collectionitem"."removed_by_workflow_id", ("db_collectionitem"."data" ->> 'package') AS "package", ("db_collectionitem"."data" ->> 'version') AS "version" FROM "db_collectionitem" WHERE ("db_collectionitem"."parent_collection_id" = 3 AND "db_collectionitem"."removed_at" IS NULL AND "db_collectionitem"."child_type" = 'a' AND "db_collectionitem"."category" = 'debian:source-package' AND ("db_collectionitem"."data" ->> 'package') = 'sip6' AND ("db_collectionitem"."data" ->> 'version') = '6.15.1-1');
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Gather (cost=7643.72..273735.35 rows=1 width=407) (actual time=97.396..104.225 rows=1 loops=1)
Workers Planned: 2
Workers Launched: 2
Buffers: shared read=36528 written=2
-> Parallel Bitmap Heap Scan on db_collectionitem (cost=6643.72..272735.25 rows=1 width=407) (actual time=79.104..82.316 rows=0 loops=3)
Recheck Cond: (parent_collection_id = 3)
Filter: ((removed_at IS NULL) AND ((child_type)::text = 'a'::text) AND ((category)::text = 'debian:source-package'::text) AND ((data ->> 'package'::text) = 'sip6'::text) AND ((data ->> 'version'::text) = '6.15.1-1'::text))
Rows Removed by Filter: 63457
Heap Blocks: exact=18409
Buffers: shared read=36528 written=2
-> Bitmap Index Scan on db_collectionitem_parent_collection_id_7b1febd7 (cost=0.00..6643.72 rows=194039 width=0) (actual time=8.908..8.908 rows=190372 loops=1)
Index Cond: (parent_collection_id = 3)
Buffers: shared read=376
Planning:
Buffers: shared read=1
Planning Time: 0.209 ms
JIT:
Functions: 18
Options: Inlining false, Optimization false, Expressions true, Deforming true
Timing: Generation 1.637 ms (Deform 0.882 ms), Inlining 0.000 ms, Optimization 1.160 ms, Emission 15.706 ms, Total 18.503 ms
Execution Time: 104.817 ms
(21 rows)
Simply adding a parent_category filter allows PostgreSQL to notice that there's a matching index:
debusine=> EXPLAIN (ANALYZE, BUFFERS) SELECT "db_collectionitem"."id", "db_collectionitem"."name", "db_collectionitem"."child_type", "db_collectionitem"."category", "db_collectionitem"."parent_collection_id", "db_collectionitem"."parent_category", "db_collectionitem"."collection_id", "db_collectionitem"."artifact_id", "db_collectionitem"."data", "db_collectionitem"."created_at", "db_collectionitem"."created_by_user_id", "db_collectionitem"."created_by_workflow_id", "db_collectionitem"."removed_at", "db_collectionitem"."removed_by_user_id", "db_collectionitem"."removed_by_workflow_id", ("db_collectionitem"."data" ->> 'package') AS "package", ("db_collectionitem"."data" ->> 'version') AS "version" FROM "db_collectionitem" WHERE ("db_collectionitem"."parent_collection_id" = 3 AND "db_collectionitem"."removed_at" IS NULL AND "db_collectionitem"."child_type" = 'a' AND "db_collectionitem"."parent_category" = 'debian:suite' AND "db_collectionitem"."category" = 'debian:source-package' AND ("db_collectionitem"."data" ->> 'package') = 'sip6' AND ("db_collectionitem"."data" ->> 'version') = '6.15.1-1');
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------
Index Scan using db_ci_suite_source_idx on db_collectionitem (cost=0.42..8.45 rows=1 width=407) (actual time=0.051..0.052 rows=1 loops=1)
Index Cond: ((parent_collection_id = 3) AND ((data ->> 'package'::text) = 'sip6'::text) AND ((data ->> 'version'::text) = '6.15.1-1'::text))
Filter: (removed_at IS NULL)
Buffers: shared read=4
Planning:
Buffers: shared read=1
Planning Time: 0.220 ms
Execution Time: 0.067 ms
(8 rows)