Skip to content

Commit c39a956

Browse files
committed
Simplify env setup: use .env.local with Bun native loading
Refactor environment variable management to leverage Bun native .env loading: - Replace .infisical-cache with .env.local as the secrets file - Use .env.development.local for worktree port overrides (highest precedence) - Simplify .bin/bun wrapper: sync Infisical to .env.local, let Bun load natively - Add create_env_symlinks() for subdirectory .env.local access - Simplify CONTRIBUTING.md: manual .env.local setup as default path - Move Infisical to optional team setup in INFISICAL_SETUP_GUIDE.md - Add "bun run setup" script for quick .env.local creation - Update knowledge.md with correct env file hierarchy - Fix Next.js dev server: use PORT env var instead of shell expansion - Fix init-worktree.ts to set PORT to webPort for Next.js
1 parent bada375 commit c39a956

File tree

9 files changed

+283
-305
lines changed

9 files changed

+283
-305
lines changed

.bin/bun

Lines changed: 154 additions & 203 deletions
Large diffs are not rendered by default.

.gitignore

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,10 @@
22
.idea/
33
.vercel
44

5+
# Environment files (secrets) - Bun loads .env.* files natively
56
.env
67
.env.*
78
!.env.example
8-
.env.worktree
9-
.worktree-env.sh
109

1110
# ctags
1211
tags
@@ -29,7 +28,5 @@ debug/
2928
.nx/cache
3029
.nx/workspace-data
3130

32-
# Infisical cache (contains secrets, should not be committed)
33-
.infisical-cache
34-
.infisical-cache.tmp
31+
# Infisical config (user-specific)
3532
.infisical.json

CONTRIBUTING.md

Lines changed: 21 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -9,74 +9,46 @@ Hey there! 👋 Thanks for wanting to contribute to Codebuff. Whether you're squ
99
Before you begin, you'll need to install a few tools:
1010

1111
1. **Bun** (our primary package manager): Follow the [Bun installation guide](https://bun.sh/docs/installation)
12-
2. **direnv**: This manages environment variables automatically
13-
- macOS: `brew install direnv`
14-
- Ubuntu/Debian: `sudo apt install direnv`
15-
- Other systems: See [direnv installation guide](https://direnv.net/docs/installation.html)
16-
3. **Docker**: Required for the web server database
17-
4. **Infisical CLI**: For secrets management
18-
```bash
19-
npm install -g @infisical/cli
20-
```
12+
2. **Docker**: Required for the web server database
2113

2214
### Setting Up Your Development Environment
2315

24-
1. **Hook direnv into your shell** (one-time setup):
25-
26-
- For zsh: `echo 'eval "$(direnv hook zsh)"' >> ~/.zshrc && source ~/.zshrc`
27-
- For bash: `echo 'eval "$(direnv hook bash)"' >> ~/.bashrc && source ~/.bashrc`
28-
- For fish: `echo 'direnv hook fish | source' >> ~/.config/fish/config.fish && source ~/.config/fish/config.fish`
29-
30-
2. **Restart your shell**: Run `exec $SHELL` or restart your terminal
31-
32-
3. **Clone the repository**:
16+
1. **Clone the repository**:
3317

3418
```bash
3519
git clone https://github.com/CodebuffAI/codebuff.git
3620
cd codebuff
3721
```
3822

39-
4. **Set up secrets management**:
23+
2. **Set up environment variables**:
4024

4125
```bash
42-
npm install -g @infisical/cli
43-
infisical init
44-
infisical login
45-
# Select "US" region when prompted
26+
# Copy the example file
27+
cp .env.example .env.local
28+
29+
# Edit .env.local and update DATABASE_URL to match Docker:
30+
# DATABASE_URL=postgresql://manicode_user_local:secretpassword_local@localhost:5432/manicode_db_local
4631
```
4732

48-
Follow the [Infisical Setup Guide](./INFISICAL_SETUP_GUIDE.md) for detailed setup instructions.
49-
50-
Load all environment variables at once:
51-
52-
```bash
53-
infisical secrets set --file .env.example
54-
infisical secrets set DATABASE_URL=postgresql://manicode_user_local:secretpassword_local@localhost:5432/manicode_db_local
55-
```
56-
57-
5. **Configure environment**:
58-
59-
```bash
60-
direnv allow
61-
```
33+
> **Team members**: For shared secrets management, see the [Infisical Setup Guide](./INFISICAL_SETUP_GUIDE.md).
6234
63-
6. **Install dependencies**:
35+
3. **Install dependencies**:
6436

6537
```bash
6638
bun install
6739
```
6840

69-
7. **Setup a Github OAuth app**
41+
4. **Setup a Github OAuth app**
7042

7143
1. Follow these instructions to set up a [Github OAuth app](https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/creating-an-oauth-app)
72-
2. Find your app's Github client ID and secret access key and set them in Infisical:
44+
2. Add your Github client ID and secret to `.env.local`:
7345

7446
```bash
75-
infisical secrets set CODEBUFF_GITHUB_ID=<your-github-app-id-here>
76-
infisical secrets set CODEBUFF_GITHUB_SECRET=<your-github-app-secret-here>
47+
CODEBUFF_GITHUB_ID=<your-github-app-id-here>
48+
CODEBUFF_GITHUB_SECRET=<your-github-app-secret-here>
7749
```
7850

79-
8. **Start development services**:
51+
5. **Start development services**:
8052

8153
**Option A: All-in-one (recommended)**
8254
```bash
@@ -99,7 +71,7 @@ Before you begin, you'll need to install a few tools:
9971

10072
**Note**: CLI requires both backend and web server running for authentication.
10173

102-
9. **Giving yourself credits**:
74+
6. **Giving yourself credits**:
10375

10476
1. Log into Codebuff at [http://localhost:3000/login](http://localhost:3000/login)
10577

@@ -115,7 +87,7 @@ Before you begin, you'll need to install a few tools:
11587

11688
Now, you should be able to run the CLI commands locally from within the `codebuff` directory.
11789

118-
10. **Running in other directories**:
90+
7. **Running in other directories**:
11991

12092
In order to run the CLI from other directories, you need to first publish the agents to the database.
12193

@@ -271,14 +243,12 @@ Level up the web interface in `web/` with better agent management, project templ
271243

272244
**Setup issues?**
273245

274-
- **direnv problems?** Make sure it's hooked into your shell, run `direnv allow`, and restart your terminal
275246
- **Script errors?** Double-check you're using bun for all commands
276-
- **Infisical issues?** See our [Infisical Setup Guide](./INFISICAL_SETUP_GUIDE.md) for step-by-step instructions
277247
- **Database connection errors?** If you see `password authentication failed for user "postgres"` errors:
278-
1. Ensure DATABASE_URL uses the correct credentials: `postgresql://manicode_user_local:secretpassword_local@localhost:5432/manicode_db_local`
279-
2. Update both your local `.env` file and Infisical secret: `infisical secrets set DATABASE_URL=postgresql://manicode_user_local:secretpassword_local@localhost:5432/manicode_db_local`
280-
3. Run the database migration: `infisical run -- bun run db:migrate`
281-
4. Restart your development services
248+
1. Ensure DATABASE_URL in `.env.local` uses the correct credentials: `postgresql://manicode_user_local:secretpassword_local@localhost:5432/manicode_db_local`
249+
2. Run the database migration: `bun run db:migrate`
250+
3. Restart your development services
251+
- **Using Infisical?** See the [Infisical Setup Guide](./INFISICAL_SETUP_GUIDE.md) for team secrets management
282252
- **Empty Agent Store in dev mode?** This is expected behavior - agents from `.agents/` directory need to be published to the database to appear in the marketplace
283253

284254
**Questions?** Jump into our [Discord community](https://codebuff.com/discord) - we're friendly and always happy to help!

INFISICAL_SETUP_GUIDE.md

Lines changed: 52 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,51 +1,80 @@
1-
# Simple Infisical Setup for Codebuff
1+
# Infisical Setup Guide
22

3-
Quick 5-step setup to get Codebuff running with Infisical secrets.
3+
This guide is for **team/advanced secrets management** using [Infisical](https://infisical.com/).
4+
5+
> **Personal development?** Just copy `.env.example` to `.env.local` and fill in your values. See [CONTRIBUTING.md](./CONTRIBUTING.md) for the full setup guide.
6+
7+
## Why Infisical?
8+
9+
- **Team sync**: Share secrets securely across team members
10+
- **Auto-refresh**: The `.bin/bun` wrapper automatically syncs secrets to `.env.local`
11+
- **No manual copying**: Secrets stay up-to-date with a 15-minute cache
12+
13+
## Prerequisites
14+
15+
- **direnv**: Required for the bun wrapper to work
16+
- macOS: `brew install direnv`
17+
- Ubuntu/Debian: `sudo apt install direnv`
18+
- [Hook it into your shell](https://direnv.net/docs/hook.html)
419

520
## Setup Steps
621

7-
### 1. Install & Setup
22+
### 1. Enable the bun wrapper
23+
```bash
24+
direnv allow
25+
```
26+
This adds `.bin/` to your PATH so the Infisical sync wrapper runs automatically.
27+
28+
### 2. Install & Login
829
```bash
930
npm install -g @infisical/cli
1031
infisical init
1132
infisical login
1233
# Select "US" region when prompted
13-
infisical secrets # Verify setup works
1434
```
1535

16-
### 2. Browser Login
36+
### 3. Browser Login
1737
- Browser opens automatically to https://app.infisical.com
18-
- Login with **any email** (Gmail, etc.)
19-
- Fill in any **name** and **organization name** when asked
38+
- Login with your email
39+
- Select or create your organization and project
2040
- Copy the token from browser and paste in terminal
2141

22-
### 3. Select Project
23-
- Use arrow keys to choose your organization
24-
- Select or create a Codebuff project
25-
26-
### 4. Load Environment Variables
42+
### 4. Load Initial Secrets
2743
```bash
28-
# Load all variables at once
44+
# Load all variables from .env.example as a starting point
2945
infisical secrets set --file .env.example
3046

31-
# IMPORTANT: Fix the database password separately
47+
# Fix the database password to match Docker
3248
infisical secrets set DATABASE_URL=postgresql://manicode_user_local:secretpassword_local@localhost:5432/manicode_db_local
3349
```
3450

35-
### 5. Done! Run Codebuff
51+
### 5. Run Codebuff
3652
```bash
37-
bun run dev # Starts everything automatically
53+
bun run dev # Secrets auto-sync to .env.local
3854
```
3955

40-
## Common Issues & Quick Fixes
56+
## How It Works
4157

42-
**Token won't paste?** → Right-click → paste
43-
**Database error?** → Run the DATABASE_URL command above
44-
**Can't navigate menus?** → Use arrow keys ↓ ↑
58+
1. The `.bin/bun` wrapper checks if `.env.local` needs refreshing
59+
2. If stale (>15 min) or missing, it syncs from Infisical
60+
3. Bun then loads `.env.local` automatically
4561

46-
## That's It!
62+
## Common Issues
4763

48-
The `.env.example` file contains all the dummy values needed for development. Only the database password needs to be fixed to match Docker's password.
64+
| Problem | Solution |
65+
|---------|----------|
66+
| Token won't paste | Right-click → paste |
67+
| Session expired | Run `infisical login` again |
68+
| Can't navigate menus | Use arrow keys ↓ ↑ |
69+
| Infisical not working | Fall back to manual `.env.local` |
4970

50-
You will need to populate all the secrets in the Infisical UI at https://app.infisical.com. You can provide dummy values for the secrets if you get an error about missing secrets, but you will need to update them with real values in order to use the associated feature.
71+
## Updating Secrets
5172

73+
```bash
74+
# Set a single secret
75+
infisical secrets set MY_API_KEY=abc123
76+
77+
# Delete the local cache to force refresh
78+
rm .env.local
79+
bun run dev # Re-syncs from Infisical
80+
```

knowledge.md

Lines changed: 34 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -366,33 +366,52 @@ Important constants are centralized in `common/src/constants.ts`:
366366

367367
## Environment Variables
368368

369-
This project uses [Infisical](https://infisical.com/) for secret management. All secrets are injected at runtime.
369+
This project uses standard `.env.*` files for environment configuration. Bun natively loads these files automatically.
370370

371-
### Release Process
371+
### File Hierarchy
372372

373-
The release mechanism uses the `CODEBUFF_GITHUB_TOKEN` environment variable directly. The old complex GitHub App token generation system has been removed in favor of using a simple personal access token or the infisical-managed token.
373+
Bun loads `.env` files in this order (highest precedence last):
374374

375-
Environment variables are defined and validated in `packages/internal/src/env.ts`. This module provides type-safe `env` objects for use throughout the monorepo.
375+
1. `.env.local` - Main secrets file synced from Infisical (gitignored)
376+
2. `.env.development.local` - Worktree-specific overrides like ports (gitignored, highest precedence)
377+
378+
**Note**: Bun also supports `.env` and `.env.development`, but this project only uses `.env.local` and `.env.development.local`.
379+
380+
### Infisical Integration
381+
382+
Infisical is used as a background sync mechanism to populate `.env.local`:
376383

377-
### Bun Wrapper Script
384+
- The `.bin/bun` wrapper syncs secrets from Infisical to `.env.local`
385+
- Caching: 15-minute TTL (configurable via `INFISICAL_CACHE_TTL`)
386+
- Cache invalidates when `.infisical.json` is modified or TTL expires
387+
- If Infisical is not set up, existing `.env.local` is used directly
378388

379-
The `.bin/bun` script automatically wraps bun commands with infisical when secrets are needed. It prevents nested infisical calls by checking for `NEXT_PUBLIC_INFISICAL_UP` environment variable, ensuring infisical runs only once at the top level while nested bun commands inherit the environment variables.
389+
**Setup Options**:
390+
1. **With Infisical**: Run `infisical login`, secrets auto-sync to `.env.local`
391+
2. **Without Infisical**: Copy `.env.example` to `.env.local` and fill in values
380392

381-
**Worktree Support**: The wrapper automatically detects and loads `.env.worktree` files when present, allowing worktrees to override Infisical environment variables (like ports) for local development. This enables multiple worktrees to run simultaneously on different ports without conflicts.
393+
### Worktree Support
382394

383-
The wrapper also loads environment variables in the correct precedence order:
395+
For git worktrees running on different ports:
384396

385-
1. Infisical secrets are loaded first (if needed)
386-
2. `.env.worktree` is loaded second to override any conflicting variables
387-
3. This ensures worktree-specific overrides (like custom ports) always take precedence over cached Infisical defaults
397+
- Create `.env.development.local` in the worktree with port overrides
398+
- Bun loads `.env.development.local` with highest precedence, so it overrides ports from `.env.local`
399+
- The `scripts/init-worktree.ts` script creates this file automatically
388400

389-
The wrapper looks for `.env.worktree` in the project root directory, making it work consistently regardless of the current working directory when bun commands are executed.
401+
### Bun Wrapper Script (`.bin/bun`)
390402

391-
**Performance Optimizations**: The wrapper uses `--silent` flag with Infisical to reduce CLI output overhead and sets `INFISICAL_DISABLE_UPDATE_CHECK=true` to skip version checks for faster startup times.
403+
The wrapper's role is simple: ensure `.env.local` is synced from Infisical before running bun.
392404

393-
**Infisical Caching**: The wrapper implements robust caching of environment variables in `.infisical-cache` with a 15-minute TTL (configurable via `INFISICAL_CACHE_TTL`). This reduces startup time from ~1.2s to ~0.16s (87% improvement). The cache uses `infisical export` which outputs secrets directly in `KEY='value'` format, ensuring ONLY Infisical-managed secrets are cached (no system environment variables). Multi-line secrets like RSA private keys are handled correctly using `source` command. Cache automatically invalidates when `.infisical.json` is modified or after TTL expires. Uses subshell execution to avoid changing the main shell's working directory.
405+
- Checks `NEXT_PUBLIC_INFISICAL_UP` to prevent nested syncs
406+
- Uses `INFISICAL_DISABLE_UPDATE_CHECK=true` for faster startup
407+
- 10-second timeout on Infisical commands to prevent hangs
408+
- Falls back gracefully if Infisical is not available
394409

395-
**Session Validation**: The wrapper detects expired Infisical sessions using `infisical export` with a robust 10-second timeout implementation that works cross-platform (macOS and Linux). Uses background processes with polling to prevent hanging on interactive prompts. Valid sessions output environment variables in `KEY='value'` format, while expired sessions either output interactive prompts or timeout. Provides clear error messages directing users to run `infisical login`.
410+
### Release Process
411+
412+
The release mechanism uses the `CODEBUFF_GITHUB_TOKEN` environment variable directly.
413+
414+
Environment variables are defined and validated in `packages/internal/src/env.ts`. This module provides type-safe `env` objects for use throughout the monorepo.
396415

397416
## Python Package
398417

packages/internal/src/knowledge.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ All environment variables are defined and validated in `env.ts`:
1616

1717
- Server variables: API keys, database URLs, service credentials
1818
- Client variables: Public configuration values
19-
- Uses Infisical for secret management in development
19+
- Loaded from `.env.local` (manually created or synced from Infisical)
2020

2121
## Current Integrations
2222

scripts/cleanup-worktree.sh

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,13 @@ cleanup_worktree() {
2929
fi
3030

3131
# Clean up any worktree-specific files that might be left behind
32+
if [ -f "$WORKTREE_PATH/.env.development.local" ]; then
33+
rm -f "$WORKTREE_PATH/.env.development.local" 2>/dev/null || true
34+
fi
35+
# Also clean up legacy .env.development and .env.worktree if present
36+
if [ -f "$WORKTREE_PATH/.env.development" ]; then
37+
rm -f "$WORKTREE_PATH/.env.development" 2>/dev/null || true
38+
fi
3239
if [ -f "$WORKTREE_PATH/.env.worktree" ]; then
3340
rm -f "$WORKTREE_PATH/.env.worktree" 2>/dev/null || true
3441
fi

scripts/init-worktree.ts

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -206,17 +206,22 @@ async function checkGitBranchExists(branchName: string): Promise<boolean> {
206206
}
207207
}
208208

209-
function createEnvWorktreeFile(worktreePath: string, args: WorktreeArgs): void {
209+
function createEnvDevelopmentLocalFile(
210+
worktreePath: string,
211+
args: WorktreeArgs,
212+
): void {
210213
const envContent = `# Worktree-specific port overrides
211214
# Generated by init-worktree.ts for worktree: ${args.name}
212-
PORT=${args.backendPort}
215+
# This file has highest precedence in Bun's .env loading order, overriding .env.local
216+
# PORT is used by Next.js dev server
217+
PORT=${args.webPort}
213218
NEXT_PUBLIC_CODEBUFF_BACKEND_URL=localhost:${args.backendPort}
214219
NEXT_PUBLIC_WEB_PORT=${args.webPort}
215220
NEXT_PUBLIC_CODEBUFF_APP_URL=http://localhost:${args.webPort}
216221
`
217222

218-
writeFileSync(join(worktreePath, '.env.worktree'), envContent)
219-
console.log('Created .env.worktree with port configurations')
223+
writeFileSync(join(worktreePath, '.env.development.local'), envContent)
224+
console.log('Created .env.development.local with port configurations')
220225
}
221226

222227
function copyInfisicalConfig(worktreePath: string): void {
@@ -308,9 +313,9 @@ async function main(): Promise<void> {
308313
console.log(`Web port: ${args.webPort}`)
309314

310315
// Create configuration files
311-
createEnvWorktreeFile(worktreePath, args)
316+
createEnvDevelopmentLocalFile(worktreePath, args)
312317
copyInfisicalConfig(worktreePath)
313-
// Note: .bin/bun wrapper now automatically loads .env.worktree
318+
// Note: Bun loads .env.development.local with highest precedence, overriding .env.local
314319

315320
// Run direnv allow
316321
await runDirenvAllow(worktreePath)

web/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
}
1111
},
1212
"scripts": {
13-
"dev": "next dev -p ${NEXT_PUBLIC_WEB_PORT:-3000}",
13+
"dev": "next dev",
1414
"build": "next build 2>&1 | sed '/Contentlayer esbuild warnings:/,/^]/d' && bun run scripts/prebuild-agents-cache.ts",
1515
"start": "next start",
1616
"preview": "bun run build && bun run start",

0 commit comments

Comments
 (0)