Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/golangci-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ jobs:
fail-fast: false
matrix:
working-directory:
- aerospike-tls
- book-store-inventory
- connect-tunnel
- dns-dedup
Expand Down
11 changes: 11 additions & 0 deletions aerospike-tls/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
FROM golang:1.26 AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY main.go ./
RUN CGO_ENABLED=0 go build -o /out/aerospike-tls-sample .

FROM gcr.io/distroless/static-debian12
COPY --from=build /out/aerospike-tls-sample /aerospike-tls-sample
EXPOSE 8080
ENTRYPOINT ["/aerospike-tls-sample"]
140 changes: 140 additions & 0 deletions aerospike-tls/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# aerospike-tls — Aerospike-Go sample with Keploy record/replay

A small Go HTTP service that talks to Aerospike CE over the clear-text
service port (3000) using `aerospike-client-go/v7`. The sample is
recorded and replayed end-to-end with Keploy: bundled scripts spin
up the dependency, drive the API with `curl`, record the resulting
Aerospike traffic, and replay it deterministically against captured
mocks.

What the sample demonstrates:

* **Keploy records binary Aerospike protocol traffic** — Info,
AS_MSG (single-record PUT/GET/TOUCH/DELETE), BATCH_READ/WRITE,
SCAN, QUERY, UDF, CDT — and replays them from `mocks.yaml`
without needing the real cluster.
* **Replay stays deterministic at any concurrency the app exposes** —
single-client `/parallel`, multi-client round-robin, and per-request
fresh-client construction all pass cleanly.
* **A pipeline-friendly shape**. Three `scripts/script-{1,2,3}.sh`
entry points each record and replay one test-set independently,
so CI can call them as separate jobs (or as one matrix).

## Layout

```
aerospike-tls/
├── main.go # the HTTP service
├── go.mod / go.sum
├── aerospike-conf/
│ └── aerospike.conf # CE config: clear-text on 3000
├── docker-compose.yml # Aerospike CE + the sample
├── Dockerfile # builds the sample binary for compose
├── keploy.yml # Keploy CLI config (command, ports)
└── scripts/
├── common.sh # shared helpers (boot, build, record, replay)
├── script-1.sh # records + replays test-set-0 (CRUD)
├── script-2.sh # records + replays test-set-1 (/parallel)
└── script-3.sh # records + replays test-set-2 (/multiclient + /freshclient)
```

There is no committed `keploy/` directory — the scripts produce it
from scratch every run. That keeps the repo lean and means every CI
run validates the full record-then-replay loop.

## Endpoints

| Method | Path | What it does |
| ------ | -------------------------- | ---------------------------------------------------------------------------- |
| GET | `/health` | `info "build" + "namespaces"` |
| POST | `/put` | single-record PUT |
| GET | `/get/{key}` | single-record GET |
| POST | `/batch/put` | sequential write loop |
| GET | `/batch/get?k=a&k=b` | BATCH_READ |
| POST | `/scan` | full namespace scan |
| POST | `/query` | secondary-index range query |
| POST | `/udf` | UDF_EXECUTE |
| POST | `/cdt/list/append` | CDT list append |
| POST | `/cdt/map/put` | CDT map put |
| POST | `/touch/{key}` | TOUCH |
| DELETE | `/key/{key}` | DELETE |
| POST | `/parallel?n=N&prefix=P` | fans out N goroutines, each PUT+GET on a unique key — **one shared client** |
| POST | `/multiclient?n=N&prefix=P`| same, but round-robins across **4 pre-built `*as.Client` instances** |
| POST | `/freshclient?n=N&prefix=P`| **each goroutine builds its own `*as.Client`** inside the request |

## Run it manually

```bash
# 1) Boot Aerospike CE on clear-text 3000.
docker compose up -d aerospike

# 2) Build + run the sample.
go build -o aerospike-tls .
./aerospike-tls

# 3) Hit it.
curl -s localhost:8080/health
curl -s -XPOST localhost:8080/put -d '{"key":"alice","bins":{"age":30}}'
curl -s localhost:8080/get/alice
curl -s -XPOST 'localhost:8080/parallel?n=24&prefix=run1'
curl -s -XPOST 'localhost:8080/multiclient?n=24&prefix=mc1'
curl -s -XPOST 'localhost:8080/freshclient?n=8&prefix=fc1'
```

## Record + replay with the scripts

```bash
# Each script is self-contained: brings up Aerospike, builds, records,
# replays. Exit code is non-zero if any case fails on replay.
sudo ./scripts/script-1.sh # test-set-0: single-endpoint CRUD
sudo ./scripts/script-2.sh # test-set-1: /parallel n = 4..24
sudo ./scripts/script-3.sh # test-set-2: /multiclient + /freshclient
```

Pipeline-friendly knobs (env vars):

| Var | Default | What it does |
|--------------|---------------|---------------------------------------------------------------|
| `KEPLOY` | `sudo keploy` | binary + auth invocation. Override to `keploy` if root |
| `PORT` | `8090` | HTTP port the recorded sample listens on |
| `LOG_DIR` | `/tmp` | where to drop the keploy record log |
| `SKIP_DOCKER`| (unset) | `=1` skips `docker compose up -d aerospike` (already running) |
| `SKIP_BUILD` | (unset) | `=1` skips `go build` (binary already in place) |

A typical CI job looks like:

```yaml
- run: docker compose up -d aerospike
- run: go build -o aerospike-tls .
- run: SKIP_DOCKER=1 SKIP_BUILD=1 ./scripts/script-1.sh
- run: SKIP_DOCKER=1 SKIP_BUILD=1 ./scripts/script-2.sh
- run: SKIP_DOCKER=1 SKIP_BUILD=1 ./scripts/script-3.sh
```

## Concurrency notes — what makes replay deterministic

Mocked replay through Keploy is roughly 20× faster than real
Aerospike for the same op. A burst of N concurrent goroutines on a
cold client pool then races to open N fresh sockets, and the
goroutine that loses the race surfaces as `MAX_RETRIES_EXCEEDED` at
the application — even though every peer in the same burst succeeds.

`main.go` paints over this with four layered changes; together they
make `/parallel?n=24`, `/multiclient?n=24`, and `/freshclient?n=8`
replay clean on every run:

1. **Sized pool** — `ClientPolicy.ConnectionQueueSize = 256`,
`OpeningConnectionThreshold = 16`.
2. **Tolerant per-op policy** — `parallelWritePolicy` and
`parallelReadPolicy` set `SocketTimeout 10s`, `TotalTimeout 30s`,
`MaxRetries 10`, `SleepBetweenRetries 5ms`.
3. **Two-phase warmup** on the main client at startup: a sequential
prelude that walks the cluster past cold-start latencies,
followed by a parallel fill that puts idle connections in the
pool before the HTTP server accepts the first request.
4. **App-level retry wrapper** (`parallelDo`) around each PUT and
GET in `/parallel`, `/multiclient`, and `/freshclient`.

`/multiclient`'s extra clients are deliberately NOT warmed at
startup — a hundred concurrent dials at boot can stall a record-time
proxy. The retry wrapper covers their first burst instead.
44 changes: 44 additions & 0 deletions aerospike-tls/aerospike-conf/aerospike.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Aerospike CE config — clear-text on port 3000.

service {
proto-fd-max 15000
cluster-name aerospike-sample
}

logging {
console {
context any info
}
}

network {
service {
address any
port 3000
}

heartbeat {
mode mesh
address local
port 3002
interval 150
timeout 10
}

fabric {
address local
port 3001
}

info {
port 3003
}
}

namespace test {
replication-factor 1
default-ttl 0
storage-engine memory {
data-size 1G
}
}
43 changes: 43 additions & 0 deletions aerospike-tls/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Aerospike CE on clear-text 3000, exposed to the host. The sample
# dials it directly — no TLS terminator in the path.
services:
aerospike:
image: aerospike/aerospike-server:7.2.0.1
container_name: aerospike
networks:
- aerospike-net
ports:
- "3000:3000"
volumes:
- ./aerospike-conf/aerospike.conf:/etc/aerospike/aerospike.conf:ro
entrypoint: ["/usr/bin/asd", "--foreground", "--config-file", "/etc/aerospike/aerospike.conf"]
command: []
ulimits:
nofile:
soft: 65536
hard: 65536
healthcheck:
test: ["CMD", "asinfo", "-h", "127.0.0.1", "-p", "3000", "-v", "build"]
interval: 5s
timeout: 3s
retries: 20

sample:
build:
context: .
dockerfile: Dockerfile
container_name: aerospike-sample
depends_on:
aerospike:
condition: service_healthy
environment:
AEROSPIKE_HOST: aerospike
AEROSPIKE_PORT: "3000"
LISTEN: ":8080"
ports:
- "8080:8080"
networks:
- aerospike-net

networks:
aerospike-net:
16 changes: 16 additions & 0 deletions aerospike-tls/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
module github.com/keploy/samples-go/aerospike-tls

go 1.26

require github.com/aerospike/aerospike-client-go/v7 v7.7.3

require (
github.com/yuin/gopher-lua v1.1.1 // indirect
golang.org/x/net v0.26.0 // indirect
golang.org/x/sync v0.7.0 // indirect
golang.org/x/sys v0.21.0 // indirect
golang.org/x/text v0.16.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240711142825-46eb208f015d // indirect
google.golang.org/grpc v1.63.3 // indirect
google.golang.org/protobuf v1.34.2 // indirect
)
34 changes: 34 additions & 0 deletions aerospike-tls/go.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
github.com/aerospike/aerospike-client-go/v7 v7.7.3 h1:8uU+GvHm6VQ0WaTIUorWTmEPEnZA1XuUdq6zFHCXYL0=
github.com/aerospike/aerospike-client-go/v7 v7.7.3/go.mod h1:STlBtOkKT8nmp7iD+sEkr/JGEOu+4e2jGlNN0Jiu2a4=
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/pprof v0.0.0-20240711041743-f6c9dda6c6da h1:xRmpO92tb8y+Z85iUOMOicpCfaYcv7o3Cg3wKrIpg8g=
github.com/google/pprof v0.0.0-20240711041743-f6c9dda6c6da/go.mod h1:K1liHPHnj73Fdn/EKuT8nrFqBihUSKXoLYU0BuatOYo=
github.com/onsi/ginkgo/v2 v2.16.0 h1:7q1w9frJDzninhXxjZd+Y/x54XNjG/UlRLIYPZafsPM=
github.com/onsi/ginkgo/v2 v2.16.0/go.mod h1:llBI3WDLL9Z6taip6f33H76YcWtJv+7R3HigUjbIBOs=
github.com/onsi/gomega v1.32.0 h1:JRYU78fJ1LPxlckP6Txi/EYqJvjtMrDC04/MM5XRHPk=
github.com/onsi/gomega v1.32.0/go.mod h1:a4x4gW6Pz2yK1MAmvluYme5lvYTn61afQ2ETw/8n4Lg=
github.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M=
github.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw=
golang.org/x/net v0.26.0 h1:soB7SVo0PWrY4vPW/+ay0jKDNScG2X9wFeYlXIvJsOQ=
golang.org/x/net v0.26.0/go.mod h1:5YKkiSynbBIh3p6iOc/vibscux0x38BZDkn8sCUPxHE=
golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.21.0 h1:rF+pYz3DAGSQAxAu1CbC7catZg4ebC4UIeIhKxBZvws=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.16.0 h1:a94ExnEXNtEwYLGJSIUxnWoxoRz/ZcCsV63ROupILh4=
golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240711142825-46eb208f015d h1:JU0iKnSg02Gmb5ZdV8nYsKEKsP6o/FGVWTrw4i1DA9A=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240711142825-46eb208f015d/go.mod h1:Ue6ibwXGpU+dqIcODieyLOcgj7z8+IcskoNIgZxtrFY=
google.golang.org/grpc v1.63.3 h1:FGVegD7MHo/zhaGduk/R85WvSFJ+si70UQIJ0fg+BiU=
google.golang.org/grpc v1.63.3/go.mod h1:5FFeE/YiGPD2flWFCrCx8K3Ay7hALATnKiI8U3avIuw=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
Loading
Loading