Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,21 @@
# Changelog

## Unreleased

### Added

- Per-environment log-view scoping: `log_view` (with optional `log_bucket`, `log_location`) confines queries to a single Cloud Logging log view instead of the whole project. When `log_view` is unset, behavior is unchanged (project scope).
- `fx-ci-scoped` / `community-tc-scoped` / `staging-scoped` example environments, intended for untrusted agents/containers: a token minted from the narrow `tc-logview-reader` service account can read only TaskCluster's per-namespace tenant log bucket. Under `-v`, the active scope is printed as `Scope: <view resource>`.

### Changed

- `TASKCLUSTER_ROOT_URL` auto-detection now ignores log-view-scoped environments (they share a `root_url` with their broad counterpart). Scoped envs must be selected explicitly with `--env`; auto-detect resolves deterministically to the broad env.
- Result cache keys now include the environment's project and log-view scope, so scoped and broad environments no longer share cache entries.

### Notes

- Infrastructure presets (`k8s.*`, `cloudsql.*`) are not available on `*-scoped` envs — those logs live in other projects/buckets; querying one now returns a clear error (instead of silently hitting the wrong project). Use the broad envs with your own ADC.

## v1.3.1 - 2026-05-29

### Changed
Expand Down
63 changes: 63 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,32 @@ environments:
cluster: "taskcluster-dev"
root_url: "https://tc.dev.taskcluster.mozgcp.net"
# key_path omitted — uses ADC by default

# Scoped envs for untrusted agents — query a single log view, not the project.
fx-ci-scoped:
project_id: "moz-fx-taskcluster-prod"
cluster: "webservices-high-prod"
namespace: "taskcluster-prod"
log_bucket: "gke-taskcluster-prod-log-bucket"
log_location: "global"
log_view: "_AllLogs"
root_url: "https://firefox-ci-tc.services.mozilla.com"
community-tc-scoped:
project_id: "moz-fx-taskcluster-prod"
cluster: "webservices-high-prod"
namespace: "taskcluster-communitytc"
log_bucket: "gke-taskcluster-communitytc-log-bucket"
log_location: "global"
log_view: "_AllLogs"
root_url: "https://community-tc.services.mozilla.com"
staging-scoped:
project_id: "moz-fx-webservices-high-nonpro"
cluster: "webservices-high-nonprod"
namespace: "taskcluster-stage"
log_bucket: "gke-taskcluster-stage-log-bucket"
log_location: "global"
log_view: "_AllLogs"
root_url: "https://stage.taskcluster.nonprod.cloudops.mozgcp.net"
```

### 2. Authenticate to GCP
Expand Down Expand Up @@ -106,6 +132,43 @@ When `TC_LOGVIEW_ACCESS_TOKEN` is set it takes priority over `key_path` and ADC.

Re-mint and re-inject the token before it expires for long-running containers. An expired token surfaces as a `401` from the query path with a hint pointing back at the re-mint command.

#### Scoping a token to TaskCluster logs only (`*-scoped` envs)

An access token minted from your own ADC inherits your privileges — too broad for an untrusted agent. To hand an agent a token that can read **only** TaskCluster's logs, mint it from a dedicated, narrowly-scoped reader service account and point tc-logview at a **log view** instead of the whole project.

This is what the `*-scoped` environments are for. They set three optional fields that confine every query to a single Cloud Logging log view:

| Field | Meaning | Default |
|---|---|---|
| `log_view` | View ID; when set, queries are scoped to this view (otherwise project scope) | — (project scope) |
| `log_bucket` | Bucket holding the view | `_Default` |
| `log_location` | Bucket location | `global` |

The `tc-logview-reader` service account is granted `roles/logging.viewAccessor` on exactly that view (a per-namespace tenant log bucket), so a token minted from it can read nothing else:

```bash
# Host: mint a narrow, short-lived token by impersonating the reader SA.
TOKEN=$(gcloud auth print-access-token \
--impersonate-service-account=tc-logview-reader@moz-fx-taskcluster-prod.iam.gserviceaccount.com)

# Agent/container: scoped env queries only the TaskCluster log view.
TC_LOGVIEW_ACCESS_TOKEN=$TOKEN tc-logview query -e fx-ci-scoped --type monitor.error --since 1h
```

The same `tc-logview-reader` service account backs all scoped envs:

| Scoped env | Project | Log bucket |
|---|---|---|
| `fx-ci-scoped` | `moz-fx-taskcluster-prod` | `gke-taskcluster-prod-log-bucket` |
| `community-tc-scoped` | `moz-fx-taskcluster-prod` | `gke-taskcluster-communitytc-log-bucket` |
| `staging-scoped` | `moz-fx-webservices-high-nonpro` | `gke-taskcluster-stage-log-bucket` |

The reader SA lives in the prod project; for `staging-scoped` it's granted `viewAccessor` on the stage bucket in the nonprod project, so the same impersonation command works for all three.

> Infrastructure presets (`k8s.*`, `cloudsql.*`) are **not** available on `*-scoped` envs — those logs live in other projects/buckets the reader SA can't see. Use the broad envs (with your own ADC) for infra queries.

> Auto-detection via `TASKCLUSTER_ROOT_URL` always resolves to the broad env (scoped envs share its `root_url`); select a scoped env explicitly with `-e`.

### 3. Sync references

```bash
Expand Down
30 changes: 30 additions & 0 deletions cmd/config_init.go
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,36 @@ environments:
cluster: "taskcluster-dev"
root_url: "https://tc.dev.taskcluster.mozgcp.net"
# key_path omitted — uses ADC by default

# Scoped environments for untrusted agents/containers. These query a single
# Cloud Logging log view (a per-namespace tenant bucket) instead of the whole
# project, so an injected TC_LOGVIEW_ACCESS_TOKEN minted from the narrow
# tc-logview-reader service account can read ONLY TaskCluster service logs.
# Infra/k8s/CloudSQL presets are not available here — use the broad envs above.
fx-ci-scoped:
project_id: "moz-fx-taskcluster-prod"
cluster: "webservices-high-prod"
namespace: "taskcluster-prod"
log_bucket: "gke-taskcluster-prod-log-bucket"
log_location: "global"
log_view: "_AllLogs"
root_url: "https://firefox-ci-tc.services.mozilla.com"
community-tc-scoped:
project_id: "moz-fx-taskcluster-prod"
cluster: "webservices-high-prod"
namespace: "taskcluster-communitytc"
log_bucket: "gke-taskcluster-communitytc-log-bucket"
log_location: "global"
log_view: "_AllLogs"
root_url: "https://community-tc.services.mozilla.com"
staging-scoped:
project_id: "moz-fx-webservices-high-nonpro"
cluster: "webservices-high-nonprod"
namespace: "taskcluster-stage"
log_bucket: "gke-taskcluster-stage-log-bucket"
log_location: "global"
log_view: "_AllLogs"
root_url: "https://stage.taskcluster.nonprod.cloudops.mozgcp.net"
`
if err := os.WriteFile(cfgPath, []byte(example), 0o644); err != nil {
return fmt.Errorf("writing config: %w", err)
Expand Down
21 changes: 17 additions & 4 deletions cmd/query.go
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,14 @@ func runQuery(cmd *cobra.Command, args []string) error {
}

if preset != nil {
// Infra presets (k8s.*, cloudsql.*) read logs that live outside the scoped
// log view (other projects/buckets), so they are not available on
// log-view-scoped environments. Fail loud rather than silently querying the
// wrong project (or relying on an IAM denial).
if env.LogViewResource() != "" {
return fmt.Errorf("preset %q is not available on log-view-scoped environments (those with log_view set); use a non-scoped environment for k8s/CloudSQL presets", preset.Name)
}

// Preset mode: use preset's filter and field mappings
presetFilter := preset.Filter
skipCluster := false
Expand Down Expand Up @@ -136,7 +144,7 @@ func runQuery(cmd *cobra.Command, args []string) error {

// Query, cache, format — same as TC path
resultsCache := cache.New(filepath.Join(config.CacheDir(), "results"), resultsCacheTTL)
cacheKey := resultsCache.Key(env.Cluster, fromTime.Format(time.RFC3339), toTime.Format(time.RFC3339), filterStr)
cacheKey := resultsCache.Key(env.Cluster, env.ProjectID, env.LogViewResource(), fromTime.Format(time.RFC3339), toTime.Format(time.RFC3339), filterStr)

var rawEntries []map[string]interface{}
var totalCount int
Expand Down Expand Up @@ -169,7 +177,9 @@ func runQuery(cmd *cobra.Command, args []string) error {
defer client.Close()

logInfo("Querying GCP Cloud Logging...")
result, err := client.Query(ctx, filterStr, queryLimit+queryOffset)
// Presets (k8s/CloudSQL) query at project scope; they are not served
// by a tenant log view, so resourceName is intentionally empty.
result, err := client.Query(ctx, filterStr, "", queryLimit+queryOffset)
if err != nil {
wrapped := fmt.Errorf("querying logs (auth=%s): %w", gcp.AuthModeLabel(auth), err)
if hint := authHint(err, auth); hint != "" {
Expand Down Expand Up @@ -291,7 +301,7 @@ func runQuery(cmd *cobra.Command, args []string) error {

// Check results cache
resultsCache := cache.New(filepath.Join(config.CacheDir(), "results"), resultsCacheTTL)
cacheKey := resultsCache.Key(env.Cluster, fromTime.Format(time.RFC3339), toTime.Format(time.RFC3339), filterStr)
cacheKey := resultsCache.Key(env.Cluster, env.ProjectID, env.LogViewResource(), fromTime.Format(time.RFC3339), toTime.Format(time.RFC3339), filterStr)

var rawEntries []map[string]interface{}
var totalCount int
Expand Down Expand Up @@ -320,8 +330,11 @@ func runQuery(cmd *cobra.Command, args []string) error {
}
defer client.Close()

if rn := env.LogViewResource(); rn != "" {
logInfo("Scope: %s", rn)
}
logInfo("Querying GCP Cloud Logging...")
result, err := client.Query(ctx, filterStr, queryLimit+queryOffset)
result, err := client.Query(ctx, filterStr, env.LogViewResource(), queryLimit+queryOffset)
if err != nil {
wrapped := fmt.Errorf("querying logs (auth=%s): %w", gcp.AuthModeLabel(auth), err)
if hint := authHint(err, auth); hint != "" {
Expand Down
21 changes: 17 additions & 4 deletions cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -124,12 +124,25 @@ func resolveEnv() (*config.Environment, error) {
}
rootURL := os.Getenv("TASKCLUSTER_ROOT_URL")
if rootURL != "" {
// Auto-detection deliberately ignores log-view-scoped environments:
// multiple envs (broad + scoped) can share a root_url, and a scoped env
// uses different credentials/scope. Scoped access must be opted into
// explicitly with --env, so detection resolves to the broad env only.
var scopedMatch string
for name, env := range cfg.Environments {
if env.RootURL == rootURL {
e := env
logInfo("Auto-detected environment: %s", name)
return &e, nil
if env.RootURL != rootURL {
continue
}
if env.LogViewResource() != "" {
scopedMatch = name
continue
}
e := env
logInfo("Auto-detected environment: %s", name)
return &e, nil
}
if scopedMatch != "" {
return nil, fmt.Errorf("TASKCLUSTER_ROOT_URL=%q matches only the scoped environment %q; select it explicitly with --env", rootURL, scopedMatch)
}
return nil, fmt.Errorf("TASKCLUSTER_ROOT_URL=%q does not match any environment", rootURL)
}
Expand Down
50 changes: 50 additions & 0 deletions cmd/root_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import (
"strings"
"testing"

"github.com/taskcluster/tc-logview/internal/config"
"github.com/taskcluster/tc-logview/internal/gcp"
)

Expand Down Expand Up @@ -53,6 +54,55 @@ func TestAuthModeMessage(t *testing.T) {
}
}

func TestResolveEnvAutoDetectSkipsScoped(t *testing.T) {
prevCfg, prevFlag := cfg, envFlag
t.Cleanup(func() { cfg, envFlag = prevCfg, prevFlag })

envFlag = ""
t.Setenv("TASKCLUSTER_ROOT_URL", "https://firefox-ci-tc.services.mozilla.com")
cfg = &config.Config{Environments: map[string]config.Environment{
"fx-ci": {
ProjectID: "moz-fx-webservices-high-prod",
RootURL: "https://firefox-ci-tc.services.mozilla.com",
},
"fx-ci-scoped": {
ProjectID: "moz-fx-taskcluster-prod",
RootURL: "https://firefox-ci-tc.services.mozilla.com",
LogBucket: "gke-taskcluster-prod-log-bucket",
LogLocation: "global",
LogView: "_AllLogs",
},
}}

// Map iteration is randomized; a nondeterministic resolveEnv would eventually
// pick the scoped env. Run enough times to catch that.
for i := 0; i < 50; i++ {
env, err := resolveEnv()
if err != nil {
t.Fatalf("resolveEnv: %v", err)
}
if env.ProjectID != "moz-fx-webservices-high-prod" || env.LogViewResource() != "" {
t.Fatalf("auto-detect selected scoped env (project=%s, view=%q); want broad env",
env.ProjectID, env.LogViewResource())
}
}
}

func TestResolveEnvScopedOnlyRequiresExplicit(t *testing.T) {
prevCfg, prevFlag := cfg, envFlag
t.Cleanup(func() { cfg, envFlag = prevCfg, prevFlag })

envFlag = ""
t.Setenv("TASKCLUSTER_ROOT_URL", "https://example.com")
cfg = &config.Config{Environments: map[string]config.Environment{
"only-scoped": {ProjectID: "p", RootURL: "https://example.com", LogView: "_AllLogs"},
}}

if _, err := resolveEnv(); err == nil {
t.Fatal("expected error when only a scoped env matches TASKCLUSTER_ROOT_URL")
}
}

func TestAuthHint(t *testing.T) {
adcHintFragment := "gcloud auth application-default login"
tokenHintFragment := "may be expired"
Expand Down
35 changes: 29 additions & 6 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,16 @@ import (
)

type Environment struct {
ProjectID string `yaml:"project_id"`
ProjectID string `yaml:"project_id"`
CloudSQLProjectID string `yaml:"cloudsql_project_id,omitempty"`
Cluster string `yaml:"cluster"`
Namespace string `yaml:"namespace,omitempty"`
RootURL string `yaml:"root_url"`
KeyPath string `yaml:"key_path,omitempty"`
CloudSQLInstance string `yaml:"cloudsql_instance,omitempty"`
Cluster string `yaml:"cluster"`
Namespace string `yaml:"namespace,omitempty"`
RootURL string `yaml:"root_url"`
KeyPath string `yaml:"key_path,omitempty"`
CloudSQLInstance string `yaml:"cloudsql_instance,omitempty"`
LogBucket string `yaml:"log_bucket,omitempty"`
LogLocation string `yaml:"log_location,omitempty"`
LogView string `yaml:"log_view,omitempty"`
}

// CloudSQLProject returns the GCP project that holds the CloudSQL instance.
Expand All @@ -28,6 +31,26 @@ func (e Environment) CloudSQLProject() string {
return e.ProjectID
}

// LogViewResource returns the fully-qualified Cloud Logging log view resource
// name to scope queries to, or "" to query at project scope (the default,
// unchanged behavior). A non-empty LogView is the trigger; LogBucket defaults
// to "_Default" and LogLocation to "global" when unset.
func (e Environment) LogViewResource() string {
if e.LogView == "" {
return ""
}
bucket := e.LogBucket
if bucket == "" {
bucket = "_Default"
}
location := e.LogLocation
if location == "" {
location = "global"
}
return fmt.Sprintf("projects/%s/locations/%s/buckets/%s/views/%s",
e.ProjectID, location, bucket, e.LogView)
}

type Config struct {
Environments map[string]Environment `yaml:"environments"`
}
Expand Down
Loading
Loading