Add ListPackageCustomSchemas gRPC endpoint to opm serve

[perdasilva]

Summary

Adds a streaming gRPC endpoint ListPackageCustomSchemas(schema, packageName) to opm serve for retrieving raw FBC blobs of non-standard (custom) schemas.

Design

• packageName is optional — empty string queries custom schema blobs with no package association; non-empty queries blobs scoped to that package
• Streaming — multiple blobs per (schema, packageName) are supported; the caller decides whether multiple results are expected or an error
• Content-hash keying — blobs are stored under (schema, packageName, FNV-64a), providing natural deduplication of identical blobs without build-time counters or ordering state
• Standard schemas excluded — olm.package, olm.channel, olm.bundle, olm.deprecation are served by existing typed RPCs
• Input validation — path traversal (/, \, ..) rejected at both server (codes.InvalidArgument) and cache layers (defense in depth)
• Cache digest backward-compatible — ComputeDigest hashes all cache contents agnostically, so caches built by this version are usable by older opm versions without digest mismatch

Changes

Area	Files
Proto	registry.proto — new ListPackageCustomSchemas streaming RPC
Cache	cache.go, meta_key.go — ListPackageCustomSchemas on Cache interface, metaKey validation
Backends	pogrebv1.go, json.go — PutMeta/SendMetas with content-hash keys
Server	server.go — streaming handler with input validation
Tests	meta_key_test.go — validation edge cases; cache_test.go, server_test.go — integration
Tooling	Makefile, ensure-protoc.sh — well-known proto includes; protoc v5.27.0, protoc-gen-go v1.36.11

Test plan

• go build ./...
• go test ./pkg/cache/... — storage, retrieval, multiple blobs per key, packageless blobs, empty results, validation edge cases (both backends)
• go test ./pkg/server/... — end-to-end gRPC streaming, packageless queries, empty results, InvalidArgument validation
• go test ./pkg/client/... — stub compatibility

🤖 Generated with Claude Code

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kevinrizza for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov]

Codecov Report

❌ Patch coverage is 69.36937% with 68 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.79%. Comparing base (210e5d6) to head (f45c501).
⚠️ Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/api/registry.pb.go	72.38%	24 Missing and 5 partials ⚠️
pkg/api/registry_grpc.pb.go	58.33%	6 Missing and 4 partials ⚠️
pkg/server/server.go	54.54%	5 Missing and 5 partials ⚠️
pkg/cache/json.go	64.00%	5 Missing and 4 partials ⚠️
pkg/cache/cache.go	53.84%	3 Missing and 3 partials ⚠️
pkg/cache/pogrebv1.go	80.00%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1981      +/-   ##
==========================================
+ Coverage   57.90%   58.79%   +0.88%     
==========================================
  Files         140      141       +1     
  Lines       13441    13373      -68     
==========================================
+ Hits         7783     7862      +79     
+ Misses       4470     4306     -164     
- Partials     1188     1205      +17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

Adds support for streaming per-package “custom schema” FBC blobs over gRPC, backed by new cache storage primitives in both pogreb and JSON cache backends. This extends the opm serve gRPC surface to expose non-standard schema content without introducing new typed protobuf messages.

Changes:

• Adds ListPackageCustomSchemas(schema, packageName) server-streaming gRPC RPC returning google.protobuf.Struct.
• Extends cache backends with meta blob storage/retrieval keyed by (schema, packageName) and updates cache build to persist non-standard schemas.
• Updates protoc/tooling wiring to include protobuf well-known types (struct.proto) during codegen.

Reviewed changes

Copilot reviewed 13 out of 15 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
scripts/ensure-protoc.sh	Downloads protoc include files so well-known protos (e.g., struct.proto) can be imported during codegen.
README.md	Documents the new gRPC method in the public list of endpoints.
pkg/server/server.go	Implements the new gRPC streaming handler and converts stored JSON blobs into structpb.Struct.
pkg/server/server_test.go	Adds integration-style gRPC tests for streaming results, empty results, and invalid-argument behavior.
pkg/client/client_test.go	Updates the client stub interface to include the new RPC.
pkg/cache/pogrebv1.go	Adds meta blob Put/Send support in the pogreb backend.
pkg/cache/meta_key.go	Introduces meta key validation helpers for schema/packageName components.
pkg/cache/json.go	Adds meta blob directory layout and Put/Send support in the JSON backend.
pkg/cache/cache.go	Extends cache interface + build pipeline to store non-standard schema blobs and expose query method.
pkg/cache/cache_test.go	Adds unit tests covering custom schema storage/retrieval and packageless blob skipping.
pkg/api/registry.proto	Adds the new RPC and request message; imports google/protobuf/struct.proto.
pkg/api/registry.pb.go	Regenerated protobuf Go types (incl. new request type).
pkg/api/registry_grpc.pb.go	Regenerated gRPC service stubs (incl. new streaming method).
Makefile	Updates protoc include paths used by make codegen.
AGENTS.md	Documents the new gRPC method in internal agent docs.

Files not reviewed (2)

• pkg/api/registry.pb.go: Language not supported
• pkg/api/registry_grpc.pb.go: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 15 changed files in this pull request and generated 4 comments.

Files not reviewed (2)

• pkg/api/registry.pb.go: Language not supported
• pkg/api/registry_grpc.pb.go: Language not supported

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 16 changed files in this pull request and generated 2 comments.

Files not reviewed (2)

• pkg/api/registry.pb.go: Language not supported
• pkg/api/registry_grpc.pb.go: Language not supported

Comments suppressed due to low confidence (1)

pkg/cache/cache.go:323

• In Build’s WalkMetasFS callback, custom-schema metas with an empty packageName are still appended to byPackageReaders[""] and later written into the package index via pkgs[pkgName] = pkgIndex[pkgName]. This will cause ListPackages to include an empty package name when packageless custom schema blobs exist. Consider short-circuiting after storing the meta (PutMeta) for non-standard schemas when packageName == "" so they don’t participate in package index construction (and don’t create a pkgs[""] entry).

		switch meta.Schema {
		case declcfg.SchemaPackage, declcfg.SchemaChannel, declcfg.SchemaBundle, declcfg.SchemaDeprecation:
		default:
			mk, err := newValidatedMetaKey(meta.Schema, packageName)
			if err != nil {
				return fmt.Errorf("invalid custom schema meta: %v", err)
			}
			if err := c.backend.PutMeta(ctx, mk, meta.Blob); err != nil {
				return fmt.Errorf("store custom schema meta %v: %v", mk, err)
			}
		}
		if _, err := tmpFile.Write(meta.Blob); err != nil {
			return err
		}
		sr := io.NewSectionReader(tmpFile, offset, int64(len(meta.Blob)))
		byPackageReaders[packageName] = append(byPackageReaders[packageName], sr)
		offset += int64(len(meta.Blob))