Fea: all memory load apps#228
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds/combines three “memory load” sample applications (MySQL-backed HTTP API, MongoDB-backed HTTP API, and an in-memory gRPC API) along with docker-compose + k6 load scenarios, to provide consistent load-testing examples across storage/protocol variants.
Changes:
- Added MySQL and Mongo HTTP API servers with shared endpoint surface (customers/products/orders/analytics/large-payloads).
- Added an in-memory gRPC LoadTestService (proto + generated stubs + server + store) and a k6 gRPC load scenario.
- Added docker-compose + Dockerfiles + env examples and keploy config for the new sample apps.
Reviewed changes
Copilot reviewed 40 out of 43 changed files in this pull request and generated 18 comments.
Show a summary per file
| File | Description |
|---|---|
| go-memory-load-mysql/loadtest/scenario.js | k6 HTTP scenario for MySQL API (mixed API + large-payload cycle). |
| go-memory-load-mysql/keploy.yml | Keploy config for MySQL sample. |
| go-memory-load-mysql/internal/store/store.go | MySQL-backed store: CRUD/order tx + analytics + large payload support. |
| go-memory-load-mysql/internal/store/models.go | MySQL API request/response models. |
| go-memory-load-mysql/internal/httpapi/server.go | HTTP routing, JSON helpers, logging/recovery middleware for MySQL API. |
| go-memory-load-mysql/internal/database/mysql.go | MySQL connection + runtime schema creation helpers. |
| go-memory-load-mysql/internal/config/config.go | Env-based config loader for MySQL sample. |
| go-memory-load-mysql/go.sum | Go dependency lockfile for MySQL sample. |
| go-memory-load-mysql/go.mod | Go module definition for MySQL sample. |
| go-memory-load-mysql/docker-compose.yml | Compose stack for MySQL DB + API + k6 runner. |
| go-memory-load-mysql/cmd/api/main.go | MySQL API entrypoint wiring config/db/schema/server. |
| go-memory-load-mysql/Dockerfile | Container build for MySQL API. |
| go-memory-load-mysql/.env.example | Example env for MySQL API. |
| go-memory-load-mysql/.dockerignore | Docker build context exclusions for MySQL sample. |
| go-memory-load-mongo/loadtest/scenario.js | k6 HTTP scenario for Mongo API (mixed API + large-payload cycle). |
| go-memory-load-mongo/keploy.yml | Keploy config for Mongo sample. |
| go-memory-load-mongo/internal/store/store.go | Mongo-backed store: CRUD + analytics + large payload support. |
| go-memory-load-mongo/internal/store/models.go | Mongo API request/response models (with bson tags). |
| go-memory-load-mongo/internal/httpapi/server.go | HTTP routing, JSON helpers, logging/recovery middleware for Mongo API. |
| go-memory-load-mongo/internal/database/mongo.go | Mongo connection helper with retry/ping. |
| go-memory-load-mongo/internal/config/config.go | Env-based config loader for Mongo sample. |
| go-memory-load-mongo/go.sum | Go dependency lockfile for Mongo sample. |
| go-memory-load-mongo/go.mod | Go module definition for Mongo sample. |
| go-memory-load-mongo/docker-compose.yml | Compose stack for Mongo DB + API + k6 runner. |
| go-memory-load-mongo/cmd/api/main.go | Mongo API entrypoint wiring config/db/indexes/server. |
| go-memory-load-mongo/Dockerfile | Container build for Mongo API. |
| go-memory-load-mongo/.env.example | Example env for Mongo API. |
| go-memory-load-mongo/.dockerignore | Docker build context exclusions for Mongo sample. |
| go-memory-load-grpc/loadtest/scenario.js | k6 gRPC scenario covering CRUD/analytics/large-payload roundtrip. |
| go-memory-load-grpc/keploy.yml | Keploy config for gRPC sample. |
| go-memory-load-grpc/internal/store/store.go | In-memory store implementation for gRPC server. |
| go-memory-load-grpc/internal/grpcapi/server.go | gRPC service implementation + mapping to proto types. |
| go-memory-load-grpc/internal/config/config.go | Env-based config for gRPC sample (HTTP health + gRPC ports). |
| go-memory-load-grpc/go.sum | Go dependency lockfile for gRPC sample. |
| go-memory-load-grpc/go.mod | Go module definition for gRPC sample. |
| go-memory-load-grpc/docker-compose.yml | Compose stack for gRPC API + k6 runner. |
| go-memory-load-grpc/cmd/api/main.go | gRPC + HTTP health server entrypoint. |
| go-memory-load-grpc/api/proto/loadtest_grpc.pb.go | Generated gRPC stubs for LoadTestService. |
| go-memory-load-grpc/api/proto/loadtest.proto | Service and message definitions for load test API. |
| go-memory-load-grpc/api/proto/loadtest.pb.go | Generated proto message code. |
| go-memory-load-grpc/Dockerfile | Container build for gRPC API. |
| go-memory-load-grpc/.env.example | Example env for gRPC sample. |
| go-memory-load-grpc/.dockerignore | Docker build context exclusions for gRPC sample. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| case <-ctx.Done(): | ||
| _ = client.Disconnect(context.Background()) | ||
| return nil, nil, fmt.Errorf("ping mongo: %w", ctx.Err()) | ||
| case <-time.After(2 * time.Second): | ||
| } | ||
| } | ||
|
|
||
| _ = client.Disconnect(context.Background()) | ||
| return nil, nil, fmt.Errorf("ping mongo after retries: %w", pingErr) |
There was a problem hiding this comment.
client.Disconnect(...) errors are ignored on failure paths. Since errcheck is enabled repo-wide, this will be flagged if this module is linted; handle the disconnect error or add an explicit suppression with rationale.
| } | ||
| defer db.Close() |
There was a problem hiding this comment.
defer db.Close() ignores the returned error. With errcheck enabled (and default excludes off), this will be reported; consider checking the error (e.g., via a deferred func) or adding an explicit, justified suppression.
| module loadtestmysqlapi | ||
|
|
||
| go 1.26 | ||
|
|
||
| require github.com/go-sql-driver/mysql v1.9.2 |
There was a problem hiding this comment.
This new Go module directory is not included in the golangci-lint GitHub Actions matrix (.github/workflows/golangci-lint.yml), so CI will not lint it. Add go-memory-load-mysql to the workflow matrix so new code is checked consistently.
| var items []OrderItem | ||
| var totalCents int32 | ||
| for _, inp := range inputs { | ||
| p, ok := s.products[inp.ProductID] | ||
| if !ok { | ||
| return nil, fmt.Errorf("product %s: %w", inp.ProductID, ErrNotFound) | ||
| } | ||
| if p.InventoryCount < inp.Quantity { | ||
| return nil, fmt.Errorf("product %s: %w", inp.ProductID, ErrOutOfStock) | ||
| } | ||
| p.InventoryCount -= inp.Quantity | ||
| line := inp.Quantity * p.PriceCents | ||
| items = append(items, OrderItem{ | ||
| ProductID: p.ID, | ||
| SKU: p.SKU, | ||
| Name: p.Name, | ||
| Category: p.Category, | ||
| Quantity: inp.Quantity, | ||
| UnitPriceCents: p.PriceCents, | ||
| LineTotalCents: line, | ||
| }) | ||
| totalCents += line | ||
| } | ||
|
|
||
| if orderStatus == "" { | ||
| orderStatus = "pending" | ||
| } | ||
| fingerprint := orderFingerprint(inputs) | ||
| orderID := contentID(customerID, fingerprint, orderStatus) | ||
| // Idempotent: if an identical order already exists, return it without | ||
| // re-decrementing inventory (handles duplicate keploy replay calls). | ||
| if existing, ok := s.orders[orderID]; ok { | ||
| return existing, nil | ||
| } |
There was a problem hiding this comment.
CreateOrder() claims idempotency for duplicate replays, but the inventory is decremented while building items before checking whether the order already exists. If the same order is replayed, inventory will be decremented again even though you return the existing order. Compute orderID/check existence before mutating inventory (or restore inventory on duplicate).
| <-ctx.Done() | ||
| log.Println("shutting down…") | ||
| grpcServer.GracefulStop() | ||
| _ = httpServer.Shutdown(context.Background()) |
There was a problem hiding this comment.
httpServer.Shutdown(...) returns an error but it's currently ignored. With errcheck enabled in this repo, this will be flagged; handle or explicitly suppress the error (with justification) so shutdown failures aren't silently lost.
| _ = httpServer.Shutdown(context.Background()) | |
| if err := httpServer.Shutdown(context.Background()); err != nil { | |
| log.Printf("HTTP server shutdown failed: %v; check for stuck in-flight requests or connection cleanup issues", err) | |
| } |
| func newID() string { | ||
| // Generate a UUID v4-style string using random bytes from crypto/sha256 as a seed | ||
| // fallback: use time + rand; for a load-test, a simple unique ID is sufficient. | ||
| raw := sha256.Sum256([]byte(fmt.Sprintf("%d", time.Now().UnixNano()))) | ||
| b := raw[:] | ||
| // Format as UUID v4 |
There was a problem hiding this comment.
The comment above newID() says it uses "random bytes" / UUID v4-style generation, but the implementation is a SHA-256 of the current timestamp. Either update the comment to match the deterministic time-based approach or switch to a true random/uuid source if unpredictability is intended.
| rowsAffected, _ := result.RowsAffected() | ||
| if rowsAffected == 0 { | ||
| // Either product not found or insufficient inventory. |
There was a problem hiding this comment.
RowsAffected() returns (int64, error); the error is currently ignored. With errcheck enabled in this repo, this will be flagged and can also hide driver issues. Capture and handle the error from result.RowsAffected() before using rowsAffected.
| } else if attempt == maxAttempts { | ||
| db.Close() | ||
| return nil, fmt.Errorf("mysql did not become ready after %d attempts: %w", maxAttempts, pingErr) | ||
| } | ||
| select { | ||
| case <-ctx.Done(): | ||
| db.Close() | ||
| return nil, ctx.Err() |
There was a problem hiding this comment.
db.Close() errors are ignored in the retry/error paths. Since the repo enables errcheck and does not exclude Close() errors, this will fail lint and also risks missing cleanup failures. Handle the error from db.Close() (or explicitly document/suppress it with a justified //nolint).
| module loadtestmongoapi | ||
|
|
||
| go 1.26 | ||
|
|
||
| require go.mongodb.org/mongo-driver/v2 v2.2.1 | ||
|
|
There was a problem hiding this comment.
This new Go module directory is not included in the golangci-lint GitHub Actions matrix (.github/workflows/golangci-lint.yml), so CI will not lint it. Add go-memory-load-mongo to the workflow matrix so new code is checked consistently.
| module loadtestgrpcapi | ||
|
|
||
| go 1.26 | ||
|
|
||
| require ( | ||
| google.golang.org/grpc v1.73.0 | ||
| google.golang.org/protobuf v1.36.6 | ||
| ) |
There was a problem hiding this comment.
This new Go module directory is not included in the golangci-lint GitHub Actions matrix (.github/workflows/golangci-lint.yml), so CI will not lint it. Add go-memory-load-grpc to the workflow matrix so new code is checked consistently.
Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
…avior Reverts commits bddd9bd, ad3c92c, 531f930, c2cb815 which changed: - store.go: content-derived UUIDs instead of random UUIDs - scenario.js: read-only VU phase, unique payload builder - docker-compose.yml: interpolateParams removal These changes caused all 3 MySQL CI variants to fail and take 1hr+. Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
…tamination" This reverts commit 538d830.
…eplay Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
- Set fallBackOnMiss: true so BodySkipped large-payload mocks fall back to real DB - Add globalNoise for all VU-variable fields (id, timestamps, emails, prices, statuses etc.) so concurrent-VU FIFO mock collisions don't fail assertions - payload_size_bytes intentionally not noised — always deterministic
- Add average_order_value_cents and lifetime_value_cents to globalNoise - Move GET /orders search out of VU loop into teardown so DB is settled before the call, producing one deterministic mock instead of many non-deterministic ones with empty/populated FIFO collision
… fields - Remove order search and top-products from VU loop — both return non-deterministic results under concurrent load (FIFO collisions) - Add teardown() with order search (5 customers) and top-products so DB is settled before calls — one mock each, deterministic replay - Add average_order_value_cents and lifetime_value_cents to globalNoise
…mysql and mongo - Set strictMockWindow: true for both MySQL and MongoDB (correct default; if errors surface, they should be debugged and reported as Keploy bugs) - Add Content-Length to header noise for both pipelines to fix 1-byte mismatches when FIFO mocks have different-length integer field values - Add average_order_value_cents and lifetime_value_cents to globalNoise - Set fallBackOnMiss: true for both pipelines Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ue, expose mongo 500 error for root cause tracing
Removes the CI workaround that disabled large-payload testing for MongoDB. The CI script now sets LARGE_PAYLOAD_SIZES_MB=1, matching the mysql pipeline. This lets the memory guard trigger during recording and keeps test-case count in line with MySQL (~300 TCs). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
Add sleep(5) at the start of teardown() so memory pressure clears before the teardown analytics and order-search requests run; without the delay, those SQL mocks are skipped under pressure and the TCs fail with closest_mock="" in replay. Clamp GetLargePayload's returned payload to exactly PayloadSizeBytes bytes; MySQL LONGTEXT retrieval can return ±1 byte across binary versions, causing the response body to diverge by one byte between record_latest and replay_build runs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
…apps
Error handling:
- Handle client.Disconnect() errors in mongo database.Open() retry paths
and cmd/api/main.go shutdown instead of silently ignoring them
- Wrap db.Close() in deferred func with error logging in mysql cmd/api
- Log httpServer.Shutdown() error in grpc cmd/api instead of _ = ...
- Capture and propagate result.RowsAffected() error in mysql store
gRPC store logic:
- Fix CreateOrder() idempotency: compute orderID and check existence
before mutating inventory so duplicate replay calls do not double-
decrement stock
- Fix TopProducts() OrdersCount: track distinct order IDs per product
instead of incrementing per item (matches COUNT(DISTINCT) semantics
used by MySQL and Mongo implementations)
Mongo store logic:
- Fix FindOne() error classification in CreateOrder(): distinguish
ErrNoDocuments (product not found) from transient DB errors so
timeouts are not misreported as ErrNotFound
- Fix lifetime_value_cents overcounting in GetCustomerSummary(): collect
per-order totals with $addToSet{id,cents} before the $unwind stage
and sum them in Go so total_cents is counted once per order
k6 scenario.js:
- Add bootstrap guard in setup() for mysql and mongo: throw a clear error
when customer or product creation completely fails rather than crashing
with undefined.id inside randomItem([])
- Add per-iteration guard in default(): skip the VU iteration rather than
panicking if setup returned an empty customers array
CI:
- Add go-memory-load-grpc, go-memory-load-mongo, go-memory-load-mysql to
the golangci-lint workflow matrix so new modules are linted on every PR
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
grpc: - Handle fmt.Fprintln return error in healthz handler (errcheck) - Add package comments to config, grpcapi, and store packages (revive) mongo: - Replace ineffectual message := "internal server error" with var message string; the initial value was always overwritten in every switch case before use (ineffassign) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
In Go MongoDB driver v2, nested documents inside aggregation results decode as bson.D (ordered key-value slices), not bson.M (maps). The GetCustomerSummary pipeline collected per-order totals via \$addToSet into order_totals subdocuments, then type-asserted each element as bson.M to extract the 'cents' field. Because the assertion always failed silently, lifetime_value_cents stayed at zero even when orders_count was non-zero. Switch from decoding into bson.M and manually type-asserting to decoding into a concrete anonymous struct. The BSON decoder maps field names directly, eliminating the bson.D/bson.M ambiguity entirely. Apply the same pattern to category_spend to fix the favourite-category calculation for the same reason. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
grpc: cmd/api/main.go was missing a // Package main ... doc comment required by the revive package-comments rule. mongo: store.go had misaligned struct field tags in the anonymous result struct added to GetCustomerSummary; gofmt removed the extra spaces used to align OrdersCount/OrderTotals. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
…order search
The MySQL recorder skips mock capture while memoryguard.IsRecordingPaused()
is true (per-packet pressure checks added in recorder/query.go). After the
VU phase Keploy holds all accumulated mocks in memory and needs time to flush
them and let GC reclaim enough to drop below the 60 % resume threshold. The
previous sleep(5) was too short when a second memory-pressure burst coincided
with the start of teardown, leaving the analytics and order-search MySQL mocks
uncaptured. Increasing to sleep(20) gives the GC sufficient margin.
The teardown order-search loop previously queried per bootstrap customer
(customer_id=${customer.id}). Those IDs are derived from emails containing
Date.now() + Math.random(); if any customer-creation HTTP mock was dropped
during a pressure window (before the syncMock multi-window fix), the mock
responses replayed by Keploy could carry different IDs than the recording,
making the SQL args non-deterministic across runs. Replacing with five
offset-based paginated queries (LIMIT 10 OFFSET 0/10/20/30/40) gives each
a fixed, parameter-free SQL text that is identical across every record and
replay session.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
… search
The MongoDB recorder (integrations-tmp/pkg/mongo/v2/encode.go) skips mock
capture on a per-packet basis while memoryguard.IsRecordingPaused() is true.
Without any sleep the Mongo teardown ran immediately after the VU phase,
potentially while Keploy was still under memory pressure, leaving analytics
and order-search Mongo mocks uncaptured and causing replay to fail with
"no matching mock found". Adding sleep(20) gives the GC sufficient time to
drain accumulated per-test mocks and drop below the 60 % resume threshold.
The teardown order-search loop previously queried per bootstrap customer
(customer_id=${customer.id}). Replacing with five offset-based paginated
queries (LIMIT 10 OFFSET 0/10/20/30/40) makes the query parameters fully
deterministic across every recording and replay run, eliminating any risk of
BSON filter mismatches caused by customer IDs that depended on dropped
customer-creation mocks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
…cle before teardown MySQL connections acquired during the VU phase (ramping-vus, up to 12 VUs, ~105s) accumulate thousands of SQL commands. With the previous 2-minute idle timeout, those connections remained alive through the 20-second teardown sleep and were reused for the analytics and order-search queries. At replay, the keploy replayer serves all accumulated VU-phase SQL commands on the reused connection before reaching the teardown query's SQL, which has no matching mock — causing 6 TCs to fail every run. Reducing SetConnMaxIdleTime to 5s ensures all VU-phase connections time out and are closed within the 20-second teardown window. Teardown queries then open fresh connections with a clean, minimal SQL history that the replayer can match exactly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
…ool recycle before teardown" This reverts commit 69f5893.
…mples
Default ramps reduced for symmetry with the matching CI env-var
values in keploy/keploy's go_memory_load_* workflow scripts. Both
sides now use the same load profile:
go-memory-load (gin-mongo):
MIXED_API_VU_STAGE_TARGETS default: [20,40,80,30] → [2,3,4,2]
LARGE_PAYLOAD_STAGE_TARGETS default: unchanged (1,2,1)
go-memory-load-mysql:
MIXED_API_VU_STAGE_TARGETS default: [20,40,80,30] → [2,3,4,2]
LARGE_PAYLOAD_STAGE_TARGETS default: unchanged (1,2,1)
go-memory-load-mongo:
MIXED_API_VU_STAGE_TARGETS default: [20,40,80,30] → [2,3,4,2]
LARGE_PAYLOAD_STAGE_TARGETS default: unchanged (1,2,1)
go-memory-load-grpc:
K6_VUS default: 20 → 3 (constant-vus model)
K6_DURATION default: unchanged (120s)
Why: keploy/keploy CI's rate-mismatch investigation (PR #4107)
proved that the recorder's mock-emit rate at the previous load
(~14k mocks/sec peak under 12+ concurrent VUs) exceeded the host
CLI's YAML-write disk throughput (~2k/sec) by ~7x. With unbuffered
host channels the resulting backlog overflowed kernel TCP buffers
on SIGINT (proven: 64% silent loss between agent encode and host
decode); with buffered channels the same backlog back-pressured
into the application and deadlocked docker compose (proven: 30+
min lane hangs).
These memory-load samples are designed to validate that keploy's
memory-pressure feature fires across the mysql/mongo/grpc parsers.
At 4 peak VUs + 2 concurrent 1 MB large-payload uploads, the agent
still spikes memory enough to trigger 2-3 pressure events per run
— the load profile the lane was meant to exercise — without
overrunning the pipeline's sustainable throughput. The 1 MB
LARGE_PAYLOAD ramp is the actual pressure trigger and is
deliberately preserved.
Local runs (no env overrides) now match the CI defaults. CI env
overrides in keploy/keploy track the same values for clarity.
Signed-off-by: Harshit Pathak <harshit07pathak@gmail.com>
Comnined all 3 (mongo , sql, grpc) sample apps