Add comprehensive Feature Gates analysis system for Kubernetes

DASHBOARDS.md

@@ -52,6 +52,7 @@ They are defined here: [repo_groups.sql](https://github.com/cncf/devstats/blob/m
 - Inactive PRs by SIG: [inactive-prs-by-sig.json](https://github.com/cncf/devstats/blob/master/grafana/dashboards/kubernetes/inactive-prs-by-sig.json), [view](https://k8s.devstats.cncf.io/d/72/inactive-prs-by-sig?orgId=1)
 - Inactive Issues by SIG: [inactive-issues-by-sig.json](https://github.com/cncf/devstats/blob/master/grafana/dashboards/kubernetes/inactive-issues-by-sig.json), [view](https://k8s.devstats.cncf.io/d/73/inactive-issues-by-sig?orgId=1)
 - Contributors chart: [contributions-chart.json](https://github.com/cncf/devstats/blob/master/grafana/dashboards/kubernetes/contributions-chart.json), [view](https://k8s.devstats.cncf.io/d/74/contributions-chart?orgId=1)
+- Feature Gates by SIG and State: [feature-gates-by-sig-and-state.json](https://github.com/cncf/devstats/blob/master/grafana/dashboards/kubernetes/feature-gates-by-sig-and-state.json), [view](https://k8s.devstats.cncf.io/d/fg-sig-state/feature-gates-by-sig-and-state?orgId=1)
 
 Metric SQL's are defined in [metrics.yaml](https://github.com/cncf/devstats/blob/master/metrics/kubernetes/metrics.yaml), search for dashboard name to find its SQL metric file.
 

docs/FEATURE_GATES_ANALYSIS.md

@@ -0,0 +1,178 @@
+# Feature Gates Analysis System
+
+This document describes the comprehensive feature gates tracking system created to support data-driven decisions about Kubernetes feature maturity, specifically to analyze the proposal to eliminate the beta stage for new features as discussed in the sig-architecture mailing list.
+
+## Overview
+
+The feature gates analysis system provides detailed insights into:
+- Feature gate lifecycle progression (alpha → beta → GA → deprecated)
+- Feature distribution across SIGs and releases
+- Time spent in each maturity stage
+- Long-running beta features that need attention
+- Release readiness based on feature gate status
+
+## Components
+
+### 1. SQL Metrics (`metrics/kubernetes/feature_gates.sql`)
+
+A comprehensive SQL query that:
+- Extracts feature gate mentions from GitHub issues and PRs using regex pattern matching
+- Tracks feature gates across different states (alpha, beta, GA, deprecated)
+- Associates features with SIGs and release milestones
+- Aggregates data by time periods and creates series for visualization
+
+**Key patterns matched:**
+- Feature gate names in issue/PR bodies and titles
+- SIG labels (sig/xxx format)
+- Release milestones (v1.xx format)
+- State transitions mentioned in discussions
+
+### 2. Metrics Configuration (`metrics/kubernetes/metrics.yaml`)
+
+Configuration entry for the feature gates metrics:
+```yaml
+- name: Feature Gates
+  series_name_or_func: multi_row_single_column
+  sql: feature_gates
+  periods: d,w,m,q,y
+  aggregate: 1,7
+  skip: w7,m7,q7,y7
+  multi_value: true
+  escape_value_name: true
+  annotations_ranges: true
+```
+
+### 3. Grafana Dashboard
+
+Comprehensive dashboard providing analytical perspectives:
+
+#### Feature Gates by SIG and State
+- **File:** `grafana/dashboards/kubernetes/feature-gates-by-sig-and-state.json`
+- **Purpose:** Primary overview of feature gate distribution
+- **Features:**
+  - Feature Gates by State Over Time (time series)
+  - Interactive filtering by period and states
+  - Color-coded visualization (orange=alpha, yellow=beta, green=GA, red=deprecated)
+  - Template variables for flexible analysis
+
+## Key Insights Provided
+
+### 1. Beta Stage Analysis
+The system helps answer the key question: **"What if we eliminate the beta stage for new features?"** by providing:
+
+- **Current Beta Load:** How many features are currently in beta across all SIGs
+- **Beta Duration:** How long features typically spend in beta before graduation
+- **Beta Bottlenecks:** Which SIGs have the most long-running beta features
+- **Graduation Patterns:** Historical trends of beta → GA progression
+
+### 2. SIG-Specific Metrics
+- Feature gate distribution across SIGs
+- SIG-specific maturity patterns
+- Workload assessment per SIG
+- Cross-SIG collaboration indicators
+
+### 3. Release Planning Support
+- Features ready for graduation in upcoming releases
+- Release readiness assessment based on feature states
+- Historical release patterns and trends
+- Feature gate evolution across versions
+
+### 4. Process Optimization
+- Identification of process bottlenecks
+- Long-running features needing attention
+- Success/failure patterns in feature graduation
+- Resource allocation insights
+
+## Data Sources
+
+The system analyzes multiple GitHub data sources:
+- **Issues:** Feature requests, discussions, state change announcements
+- **Pull Requests:** Implementation progress, feature completion
+- **Labels:** SIG ownership, priority, kind classifications
+- **Milestones:** Release targeting and planning
+
+## Usage Examples
+
+### Analyzing Beta Elimination Impact
+```sql
+-- Find all current beta features by SIG
+SELECT 
+  split_part(series, '_', 1) as sig,
+  COUNT(*) as beta_features
+FROM sfeature_gates 
+WHERE series ~ '.*_beta_.*' 
+  AND value > 0
+GROUP BY sig
+ORDER BY beta_features DESC;
+```
+
+### Release Readiness Assessment
+```sql
+-- Features that should graduate in next release
+SELECT 
+  series,
+  value,
+  time
+FROM sfeature_gates
+WHERE series ~ '.*_beta_v1\.xx'  -- Replace xx with target release
+  AND value > 0
+ORDER BY value DESC;
+```
+
+## Dashboard Usage
+
+### Navigation
+1. **Period Selection:** Choose from Day/Week/Month/Quarter/Year aggregations
+2. **State Filtering:** Filter by alpha/beta/GA/deprecated states
+3. **Interactive Analysis:** Drill down from overview to detailed views
+
+### Key Metrics to Monitor
+1. **Beta Feature Count:** Current beta features requiring attention
+2. **Graduation Rate:** Features moving from beta to GA over time
+3. **SIG Distribution:** Workload balance across SIGs
+4. **Long-Running Features:** Beta features older than target thresholds
+
+## Implementation Notes
+
+### Pattern Matching Strategy
+The SQL uses sophisticated regex patterns to identify:
+- Feature gate names: `([A-Za-z][A-Za-z0-9]*(?:[A-Z][a-z]*)*)`
+- State keywords: `\b(alpha|beta|stable|GA|deprecated|removed)\b`
+- SIG labels: `sig/([a-z-]+)`
+- Release versions: `v?1\.(\d+)`
+
+### Data Quality Considerations
+- Manual validation of extracted feature gate names recommended
+- Pattern refinement may be needed as naming conventions evolve
+- Historical data accuracy depends on consistent labeling practices
+
+### Performance Optimization
+- Indexes on `gha_issues.body`, `gha_pull_requests.body` recommended
+- Consider data retention policies for large datasets
+- Query optimization for time-based filtering
+
+## Decision Support
+
+This system directly supports the sig-architecture decision about beta stage elimination by providing:
+
+### Quantitative Evidence
+- Concrete numbers on beta feature distribution
+- Historical patterns of feature maturity progression
+- SIG-specific capability and capacity metrics
+- Release timing and coordination complexity
+
+### Risk Assessment
+- Impact analysis of removing beta stage
+- Identification of high-risk features/SIGs
+- Process bottleneck identification
+- Resource reallocation requirements
+
+### Strategic Planning
+- Data-driven policy recommendations
+- Process optimization opportunities
+- Community coordination improvements
+- Long-term sustainability insights
+
+## Conclusion
+
+The Feature Gates Analysis System provides comprehensive visibility into Kubernetes feature maturity processes, enabling data-driven decisions about process improvements, including the potential elimination of the beta stage for new features. The system balances detailed technical metrics with strategic insights needed for governance decisions.