Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,7 @@ lint:
@golangci-lint run

run:
@go run example/main.go
@go run example/main.go

mod:
@go mod tidy
83 changes: 81 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,21 @@ A comprehensive Go utility package providing common application framework compon
- **Database Monitoring**: Connection tracking and replica status
- **API Exposure**: Public/private API configuration tracking

### HTTP Client
- **Pre-configured Client**: HTTP client with automatic metrics and internal headers
- **Transport Wrappers**: Composable transport layers for metrics and header injection
- **Caller Tracking**: Context-based caller identification for metrics

### Request Management
- **X-Request-ID**: Automatic generation, validation, and propagation
- **Context Integration**: Request ID stored in context for easy access
- **Header Management**: CORS and custom header support

### Context Utilities
- **Metadata Context**: Store and retrieve request metadata (IP, User-Agent, Platform, Version, Country)
- **Debug Support**: Debug ID and SQL group tracking for logging
- **Integration Support**: Sentry Hub and method tracking

## Usage Examples

Please see example/main.go
Expand Down Expand Up @@ -86,6 +101,22 @@ func callInternalService() {
}
```

### HTTP Client with Metrics

```go
// Create pre-configured HTTP client for internal service calls
client := appkit.NewHTTPClient("externalsrv", appkit.Version(), 30*time.Second)

// Set caller name in context for metrics tracking
ctx := appkit.NewCallerNameContext(context.Background(), "externalsrv")

req, _ := http.NewRequestWithContext(ctx, http.MethodGet, "http://api-service/endpoint", nil)
resp, _ := client.Do(req)

// Usage with clients generated by vmkteam/rpcgen
arithsrvClient := arithsrv.NewClient(arithsrvUrl, appkit.NewHTTPClient("arithsrv", appkit.Version(), 5*time.Second))
```

## API Reference

### Core Functions
Expand All @@ -105,19 +136,67 @@ func callInternalService() {
- `HTTPMetrics(serverName string) echo.MiddlewareFunc` - Prometheus HTTP metrics middleware
- `MetadataManager` - Service metadata configuration and metrics

### HTTP Client

- `NewHTTPClient(appName, version string, timeout time.Duration) *http.Client` - Creates HTTP client with metrics and internal headers
- `WithMetricsTransport(base http.RoundTripper) http.RoundTripper` - Wraps transport with Prometheus metrics
- `WithHeadersTransport(base http.RoundTripper, headers http.Header) http.RoundTripper` - Wraps transport to inject headers for each request
- `NewCallerNameContext(ctx context.Context, callerName string) context.Context` - Creates context with caller name for metrics
- `CallerNameFromContext(ctx context.Context) string` - Retrieves caller name from context

### Request ID & HTTP Handlers

- `XRequestIDFromContext(ctx context.Context) string` - Retrieves X-Request-ID from context
- `NewXRequestIDContext(ctx context.Context, requestID string) context.Context` - Creates context with X-Request-ID
- `SetXRequestIDFromCtx(ctx context.Context, req *http.Request)` - Adds X-Request-ID from context to request headers
- `CORS(next http.Handler, headers ...string) http.Handler` - CORS middleware for HTTP handlers
- `XRequestID(next http.Handler) http.Handler` - X-Request-ID middleware for HTTP handlers
- `EchoHandler(next http.Handler) echo.HandlerFunc` - Wraps HTTP handler as Echo handler
- `EchoSentryHubContext() echo.MiddlewareFunc` - Echo middleware to apply Sentry hub to context
- `EchoIPContext() echo.MiddlewareFunc` - Echo middleware to apply client IP to context

### Context Utilities

- `NewDebugIDContext(ctx context.Context, debugID uint64) context.Context` - Creates context with debug ID
- `DebugIDFromContext(ctx context.Context) uint64` - Retrieves debug ID from context
- `NewSQLGroupContext(ctx context.Context, group string) context.Context` - Creates context with SQL group for debug logging
- `SQLGroupFromContext(ctx context.Context) string` - Retrieves SQL group from context
- `NewSentryHubContext(ctx context.Context, sentryHub *sentry.Hub) context.Context` - Creates context with Sentry Hub
- `SentryHubFromContext(ctx context.Context) (*sentry.Hub, bool)` - Retrieves Sentry Hub from context
- `NewIPContext(ctx context.Context, ip string) context.Context` - Creates context with IP address
- `IPFromContext(ctx context.Context) string` - Retrieves IP address from context
- `NewUserAgentContext(ctx context.Context, ua string) context.Context` - Creates context with User-Agent
- `UserAgentFromContext(ctx context.Context) string` - Retrieves User-Agent from context
- `NewNotificationContext(ctx context.Context) context.Context` - Creates context with JSONRPC2 notification flag
- `NotificationFromContext(ctx context.Context) bool` - Retrieves JSONRPC2 notification flag from context
- `NewIsDevelContext(ctx context.Context, isDevel bool) context.Context` - Creates context with isDevel flag
- `IsDevelFromContext(ctx context.Context) bool` - Retrieves isDevel flag from context
- `NewPlatformContext(ctx context.Context, platform string) context.Context` - Creates context with platform
- `PlatformFromContext(ctx context.Context) string` - Retrieves platform from context
- `NewVersionContext(ctx context.Context, version string) context.Context` - Creates context with version
- `VersionFromContext(ctx context.Context) string` - Retrieves version from context
- `NewCountryContext(ctx context.Context, country string) context.Context` - Creates context with country
- `CountryFromContext(ctx context.Context) string` - Retrieves country from context
- `NewMethodContext(ctx context.Context, method string) context.Context` - Creates context with method
- `MethodFromContext(ctx context.Context) string` - Retrieves method from context

## Dependencies

- [Echo](https://echo.labstack.com/) - High performance HTTP framework
- [Prometheus](https://prometheus.io/) - Metrics collection and monitoring
- [Sentry](https://sentry.io/) - Error tracking and monitoring
- [zenrpc-middleware](https://github.com/vmkteam/zenrpc-middleware) - RPC middleware utilities

## Metrics Exported

### HTTP Metrics
### HTTP Server Metrics
- `app_http_requests_total` - Total HTTP requests by method/path/status
- `app_http_responses_duration_seconds` - Response time distribution

### HTTP Client Metrics
- `app_http_client_requests_total` - Total client requests by code/method/caller/origin
- `app_http_client_responses_duration_seconds` - Client response time distribution
- `app_http_client_requests_inflight` - Current inflight client requests by caller/origin

### Service Metadata Metrics
- `app_metadata_service` - Service configuration information
- `app_metadata_db_connections_total` - Database connection counts
Expand Down
2 changes: 2 additions & 0 deletions appkit.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ import (
"runtime/debug"
)

type contextKey string

// Version returns app version from VCS info.
func Version() string {
result := "devel"
Expand Down
132 changes: 132 additions & 0 deletions client.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
package appkit

import (
"context"
"net/http"
"strconv"
"sync"
"time"

"github.com/prometheus/client_golang/prometheus"
)

const (
ctxCallerName contextKey = "callerName"
)

var (
clientMetricsOnce sync.Once
clientRequests = prometheus.NewCounterVec(
prometheus.CounterOpts{
Namespace: "app",
Subsystem: "http_client",
Name: "requests_total",
Help: "Requests count by code/method/client/origin.",
},
[]string{"code", "method", "caller", "origin"},
)
clientDurations = prometheus.NewSummaryVec(
prometheus.SummaryOpts{
Namespace: "app",
Subsystem: "http_client",
Name: "responses_duration_seconds",
Help: "Response time by code/method/client/origin.",
},
[]string{"code", "method", "caller", "origin"},
)
clientInflights = prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Namespace: "app",
Subsystem: "http_client",
Name: "requests_inflight",
Help: "Gauge for inflight requests.",
},
[]string{"caller", "origin"},
)
)

// NewCallerNameContext creates new context with caller name.
func NewCallerNameContext(ctx context.Context, callerName string) context.Context {
return context.WithValue(ctx, ctxCallerName, callerName)
}

// CallerNameFromContext returns caller name from context.
func CallerNameFromContext(ctx context.Context) string {
r, _ := ctx.Value(ctxCallerName).(string)
return r
}

type metricsRoundTripper struct {
base http.RoundTripper
}

func (m *metricsRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
var origin string
if req.URL != nil {
origin = req.URL.Scheme + "://" + req.URL.Host
}
labels := prometheus.Labels{
"caller": CallerNameFromContext(req.Context()),
"origin": origin,
}

start := time.Now()
clientInflights.With(labels).Inc()
resp, err := m.base.RoundTrip(req)
clientInflights.With(labels).Dec()
duration := time.Since(start).Seconds()

labels["method"] = req.Method
if resp != nil {
labels["code"] = strconv.Itoa(resp.StatusCode)
}

clientRequests.With(labels).Inc()
clientDurations.With(labels).Observe(duration)

return resp, err
}

// WithMetricsTransport wraps http transport, adds client metrics tracking.
func WithMetricsTransport(base http.RoundTripper) http.RoundTripper {
clientMetricsOnce.Do(func() {
prometheus.MustRegister(clientRequests, clientDurations, clientInflights)
})
return &metricsRoundTripper{base: base}
}

type headerRoundTripper struct {
base http.RoundTripper
headers http.Header
}

func (h *headerRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
reqClone := req.Clone(req.Context())

for key, values := range h.headers {
for _, value := range values {
reqClone.Header.Add(key, value)
}
}

return h.base.RoundTrip(reqClone)
}

// WithHeadersTransport wraps http transport, adds provided headers for each request.
func WithHeadersTransport(base http.RoundTripper, headers http.Header) http.RoundTripper {
return &headerRoundTripper{
base: base,
headers: headers,
}
}

// NewHTTPClient returns http client with metrics and headers for internal service calls.
func NewHTTPClient(appName, version string, timeout time.Duration) *http.Client {
transport := WithHeadersTransport(http.DefaultTransport, NewInternalHeaders(appName, version))
transport = WithMetricsTransport(transport)

return &http.Client{
Timeout: timeout,
Transport: transport,
}
}
3 changes: 1 addition & 2 deletions echo.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ import (
sentryecho "github.com/getsentry/sentry-go/echo"
"github.com/labstack/echo/v4"
"github.com/prometheus/client_golang/prometheus"
zm "github.com/vmkteam/zenrpc-middleware"
)

const DefaultServerName = "default"
Expand All @@ -32,7 +31,7 @@ func NewEcho() *echo.Echo {
}))

// use zenrpc middlewares
e.Use(zm.EchoIPContext(), zm.EchoSentryHubContext())
e.Use(EchoIPContext(), EchoSentryHubContext())

return e
}
Expand Down
14 changes: 2 additions & 12 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,17 @@ module github.com/vmkteam/appkit
go 1.24.5

require (
github.com/getsentry/sentry-go v0.35.3
github.com/getsentry/sentry-go/echo v0.35.3
github.com/labstack/echo/v4 v4.13.4
github.com/prometheus/client_golang v1.23.2
github.com/vmkteam/zenrpc-middleware v1.2.2
github.com/vmkteam/zenrpc/v2 v2.2.12
)

require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/getsentry/sentry-go v0.35.3 // indirect
github.com/go-pg/pg/v10 v10.15.0 // indirect
github.com/go-pg/zerochecker v0.2.0 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/labstack/gommon v0.4.2 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
Expand All @@ -25,19 +22,12 @@ require (
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.66.1 // indirect
github.com/prometheus/procfs v0.17.0 // indirect
github.com/tmthrgd/go-hex v0.0.0-20190904060850-447a3041c3bc // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/fasttemplate v1.2.2 // indirect
github.com/vmihailenco/bufpool v0.1.11 // indirect
github.com/vmihailenco/msgpack/v5 v5.4.1 // indirect
github.com/vmihailenco/tagparser v0.1.2 // indirect
github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
github.com/vmkteam/zenrpc/v2 v2.2.12 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
golang.org/x/crypto v0.42.0 // indirect
golang.org/x/net v0.44.0 // indirect
golang.org/x/sys v0.36.0 // indirect
golang.org/x/text v0.29.0 // indirect
google.golang.org/protobuf v1.36.9 // indirect
mellium.im/sasl v0.3.2 // indirect
)
Loading