Added DuckDB as log backend#1656
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds DuckDB as a new log backend for the DNS Server, providing an embedded columnar database option for storing and querying DNS request logs. DuckDB is designed for OLAP workloads and offers fast appending and querying capabilities similar to SQLite but optimized for analytical queries.
Key changes:
- Adds a new QueryLogsDuckDBApp application that implements IDnsQueryLogger and IDnsQueryLogs interfaces
- Uses channel-based batching for efficient bulk inserts with a capacity of 20,000 entries and batch size of 1,000
- Creates a comprehensive database schema with indexes on all queryable fields
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 16 comments.
| File | Description |
|---|---|
| DnsServer.sln | Adds QueryLogsDuckDBApp project to the solution with appropriate build configurations and folder nesting |
| Apps/QueryLogsDuckDBApp/dnsApp.config | Provides default configuration enabling logging with a local database path |
| Apps/QueryLogsDuckDBApp/QueryLogsDuckDBApp.csproj | Defines project structure targeting .NET 9.0 with DuckDB.NET.Data.Full package dependency |
| Apps/QueryLogsDuckDBApp/App.cs | Implements the main application logic for logging DNS queries to DuckDB and querying the stored logs |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Thanks for the PR. Will check it soon. |
|
I see that DuckDB is added as a plugin. Wouldn't it be better if DuckDB is used for the internal logs and as a default for query logs instead of the current log files? It will make the internal logs more performant, especially if queries are logged in the system log |
|
Hi @Shaaan, What do you mean by internal logs? The system logs that Technitium DNS Server stores in |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 22 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
zbalkan
force-pushed
the
app/querylogsduckdbapp
branch
from
0bb1780 to
72cb9a1
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 10 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 14 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 8 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This reverts commit 07b2a33.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 11 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Ready for review. |
3 participants
DuckDB is an embedded columnar database, designed for OLAP loads. It is very fast in appending and querying. The usage is similar to SQLite. The ability to extend via plugins is the biggest plus.
I do have an internal use case for this, but a generic app may help others as well.