-
Notifications
You must be signed in to change notification settings - Fork 0
feat: [#272] Add HTTPS support with Caddy for all HTTP services #273
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
- Add Context Data Preparation Pattern section to template architecture docs - Explain why templates receive pre-processed data instead of raw domain objects - Document port extraction example: handled in Rust, not Tera filters - Update issue spec to use port-as-integer pattern in Caddyfile template - Remove extract_port Tera filter from implementation plan (not needed) This follows existing patterns (e.g., PrometheusContext receives api_port: u16) and maintains consistency with the codebase's approach to template rendering.
- Add Caddyfile.tera template with conditional service blocks - Update docker-compose.yml.tera with Caddy service configuration - Add proxy_network and caddy volumes - Add caddy.md documentation for template usage - Update template-system-architecture.md with directory organization rule - Update issue progress tracking
- Create CaddyProjectGenerator following Project Generator pattern - Create CaddyContext with pre-processed data for template rendering - Create CaddyService struct with domain and port fields - Create CaddyfileRenderer for Caddyfile template processing - Add 14 unit tests covering all HTTPS scenarios - Update issue progress tracking (Phase 3 mostly complete)
Implement Caddy reverse proxy with TLS termination for automatic HTTPS: Phase 1 - Template Creation: - Create Caddyfile.tera template with conditional service blocks - Create caddy.md documentation for template variables - Update docker-compose.yml.tera with Caddy service block - Register templates in CaddyProjectGenerator with 14 unit tests Phase 2 - Configuration DTOs: - Add HttpsSection DTO with admin_email and use_staging fields - Add TlsSection DTO with domain field for service-specific TLS - Extend HttpApiSection, HttpTrackerSection, GrafanaSection with optional tls - Implement validation (has_any_tls_configured, https/tls consistency) - Add Email type in src/shared/email.rs for email format validation - Add DomainName type in src/shared/domain_name.rs for domain validation Phase 3 - Template Rendering Integration: - Create RenderCaddyTemplatesStep for template rendering - Create DeployCaddyConfigStep for Ansible deployment - Create deploy-caddy-config.yml Ansible playbook - Add RenderCaddyTemplates and DeployCaddyConfigToRemote to ReleaseStep enum - Integrate CaddyContext into Docker Compose template rendering - Add CaddyConfigDeployment error variant with actionable help text Manual E2E testing verified: - HTTPS endpoints working for API, Grafana, and HTTP trackers - HTTP→HTTPS redirect (308 Permanent Redirect) - HTTP/2 and HTTP/3 enabled - Caddy Local CA for .local domains Work in progress - remaining phases: - Phase 4: Security workflow updates - Phase 5: Documentation - Phase 6: Automated E2E tests - Phase 7: Schema generation - Phase 8: ADR creation
…S flags in Rust - Refactor TrackerPorts to use constructor with pre-computed flags - Remove HttpTrackerPort struct, use Vec<u16> for ports without TLS - Add needs_ports_section flag to simplify template conditionals - Add http_api_has_tls and grafana_has_tls flags - Filter TLS-enabled ports in Rust instead of Tera template - Simplify docker-compose.yml.tera with cleaner conditionals - Update all tests to use TrackerPorts::new() constructor - Document manual E2E testing procedure in issue spec
…config - Add x-defaults anchor with common service settings (tty, restart, logging) - Apply anchor to all services: caddy, tracker, prometheus, grafana, mysql - Remove ~30 lines of duplicated configuration
Move conditional network selection from Tera template to pre-computed Rust values. This follows the project pattern of keeping templates simple by computing all logic in Rust. Changes: - Rename TrackerPorts to TrackerServiceConfig (more descriptive) - Add networks: Vec<String> field to TrackerServiceConfig - Add compute_networks() method for metrics/database/proxy networks - Add has_prometheus, has_mysql, has_caddy parameters to constructor - Update template to iterate over tracker.networks list - Rename context field from 'ports' to 'tracker' - Keep TrackerPorts type alias for backward compatibility - Update all tests to use new 7-argument constructor
…pose templates - Create CaddyServiceConfig in docker-compose context module for docker-compose.yml.tera - CaddyContext remains in caddy module for Caddyfile.tera - Simplify builder to use boolean flag instead of full CaddyContext - Update docker-compose template to use new caddy variable with network iteration - Rename caddy_config volume to caddy_config_vol to avoid naming conflict
…figuration - Create MysqlServiceConfig in docker-compose context module - Follow same pattern as CaddyServiceConfig (networks field only) - Update docker-compose template to use mysql variable for network iteration - Ensures consistency across all service configurations
- Use Tera whitespace trimming ({%- and -%}) to reduce excessive blank lines
- Generated docker-compose.yml now has cleaner, more readable formatting
- Single blank lines between services, compact network/volume sections
Prevent accidental commits of Rust compiler ICE dump files
|
This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation. |
- Add caddy:2.10 to third-party images matrix in CI - Add SARIF upload step for Caddy vulnerability scanning - Create security scan documentation for Caddy image - Document 4 known vulnerabilities (3 HIGH, 1 CRITICAL) in Go dependencies
ef56958 to
704f153
Compare
- Show HTTPS URLs with configured domains for TLS-enabled services - Separate HTTP trackers into HTTPS (via Caddy) and direct access groups - Display API endpoint with HTTPS indicator when TLS is configured - Add /etc/hosts hint with all TLS domains and instance IP - Show note about internal ports not being directly accessible with TLS - Add Grafana HTTPS URL when TLS is configured
… handling - Task 7.3: Add TLS support for health check API - Task 7.4: Handle localhost-bound services (validation + show command) - Mark task 7.2 as complete
Task 7.3: Health Check API TLS Support - Add TLS configuration to HealthCheckApiConfig domain model - Add HealthCheckApiSection DTO with TLS support - Update ServiceInfo to include health_check_uses_https flag - Add health_check_api service to CaddyContext - Update Caddyfile.tera template for health check reverse proxy - Add TrackerContext health_check_api_bind_address field - Update tracker.toml.tera with [health_check_api] section - Update show command views to display HTTPS indicator for health check - Add tests for health check TLS configuration
- Validation occurs in domain layer during DTO-to-domain conversion - Grafana excluded (bind address hardcoded at port 3000) - Localhost detection: 127.0.0.1 and ::1 only - Add is_localhost_only field to ServiceInfo (not message in URL) - Show 'Internal only' for localhost services (never hide)
…dation (Task 7.4) - Add LocalhostWithTls error to reject localhost + TLS combinations in domain layer - Add is_localhost helper function for detecting 127.0.0.1 and ::1 - Add is_localhost_only fields to ServiceInfo for API, health check, HTTP trackers - Update show command to display "Internal only" for localhost services - Add SSH tunnel hint for localhost-only services - Validation logic in TrackerConfig::validate() to avoid duplication
e40e3ff to
5334429
Compare
- Add subtask 7.5 to fix on_reverse_proxy tracker config bug - Replace misleading 'tls' object with 'domain' + 'use_tls_proxy' fields - Document incremental implementation plan (one service at a time) - Add before/after examples using full manual-https-test.json - Document dependency rule: use_tls_proxy implies on_reverse_proxy - Document future compatibility with per-tracker on_reverse_proxy
- Document issue discovered during HTTPS/Caddy implementation - Explain why per-HTTP-tracker on_reverse_proxy is needed - Propose solution with backward-compatible configuration - Reference deployer workaround (all-or-nothing proxy rule)
- Add 'How to Reproduce' section to task 7.5 - Document step-by-step verification of the problem - Include actual error message from tracker - Link to upstream issue torrust/torrust-tracker#1640
Summary
Add HTTPS support with Caddy reverse proxy for automatic TLS termination on all HTTP services (Tracker API, HTTP Trackers, Grafana).
Closes #272
What's Implemented
Phase 1: Template Creation ✅
templates/caddy/Caddyfile.terawith conditional service blocksdocs/contributing/templates/caddy.mddocumenting template variablestemplates/docker-compose/docker-compose.yml.terawith Caddy service blockCaddyProjectGeneratorwith 14 unit testsPhase 2: Configuration DTOs ✅
HttpsSectionDTO withadmin_emailanduse_stagingfieldsTlsSectionDTO withdomainfield for service-specific TLSHttpApiSection,HttpTrackerSection,GrafanaSectionwith optionaltlshas_any_tls_configured, https/tls consistency)Emailtype insrc/shared/email.rsfor email format validationDomainNametype insrc/shared/domain_name.rsfor domain validationPhase 3: Template Rendering Integration ✅
RenderCaddyTemplatesStepfor template renderingDeployCaddyConfigStepfor Ansible deploymentdeploy-caddy-config.ymlAnsible playbookRenderCaddyTemplatesandDeployCaddyConfigToRemotetoReleaseStepenumCaddyContextinto Docker Compose template renderingCaddyConfigDeploymenterror variant with actionable help textPhase 6: E2E Testing (Partial) ✅
Manual E2E testing verified:
.localdomainsWhat's Remaining (Work in Progress)
docs/user-guide/https-setup.md)Configuration Example
{ "https": { "admin_email": "admin@example.com", "use_staging": true }, "tracker": { "http_api": { "bind_address": "0.0.0.0:1212", "admin_token": "secret", "tls": { "domain": "api.tracker.local" } }, "http_trackers": [ { "bind_address": "0.0.0.0:7070", "tls": { "domain": "http1.tracker.local" } } ] }, "grafana": { "admin_user": "admin", "admin_password": "admin", "tls": { "domain": "grafana.tracker.local" } } }Testing
Notes for Reviewers
This is a draft PR - the implementation is functional but there are still remaining phases to complete. Early feedback on the architecture and approach is welcome.
Key files to review:
templates/caddy/Caddyfile.tera- Caddy templatesrc/application/command_handlers/create/config/https.rs- HTTPS configuration DTOssrc/application/command_handlers/release/handler.rs- Release workflow integrationsrc/infrastructure/templating/caddy/- Caddy template rendering