Skip to content

Move input coroutine management variables to TLS #11576

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
runderwo wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Move input coroutine management variables to TLS #11576

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
afe4472
input: move input coroutine management to TLS
runderwo Mar 17, 2026
5619c39
Merge remote-tracking branch 'upstream/master'
runderwo Mar 30, 2026
7a4f4f4
input: move TLS destructor to .c module
runderwo Mar 30, 2026
1f0e285
input: fix typo in pointer type
runderwo Mar 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 5 additions & 17 deletions include/fluent-bit/flb_input.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -580,7 +580,8 @@ struct flb_libco_in_params {
};

extern pthread_key_t libco_in_param_key;
extern struct flb_libco_in_params libco_in_param;
extern pthread_once_t libco_in_param_key_once;
void libco_in_param_key_init_func(void);
void flb_input_coro_prepare_destroy(struct flb_input_coro *input_coro);

static FLB_INLINE void input_params_set(struct flb_coro *coro,
Expand Down Expand Up @@ -615,14 +616,8 @@ static FLB_INLINE void input_pre_cb_collect(void)
struct flb_libco_in_params *params;

params = pthread_getspecific(libco_in_param_key);
if (params == NULL) {
params = flb_calloc(1, sizeof(struct flb_libco_in_params));
if (params == NULL) {
flb_errno();
return;
}
pthread_setspecific(libco_in_param_key, params);
}
flb_bug(params == NULL);

Comment on lines +619 to +620
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Mar 17, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

nl -ba include/fluent-bit/flb_input.h | sed -n '585,670p'

Repository: fluent/fluent-bit

Length of output: 103

🏁 Script executed:

#!/bin/bash
# Read lines 585-670 from flb_input.h
cat -n include/fluent-bit/flb_input.h | sed -n '585,670p'

Repository: fluent/fluent-bit

Length of output: 3099

🏁 Script executed:

#!/bin/bash
# Check the extern declarations and __thread usage
head -100 include/fluent-bit/flb_input.h | cat -n

Repository: fluent/fluent-bit

Length of output: 4123

🏁 Script executed:

#!/bin/bash
# Search for libco_in_param_key declaration
rg "libco_in_param_key" include/fluent-bit/flb_input.h -B 2 -A 2

Repository: fluent/fluent-bit

Length of output: 908

🏁 Script executed:

#!/bin/bash
# Search for libco_in_param_key definition in the source files
rg "libco_in_param_key" src/ --type c -B 1 -A 1 | head -50

Repository: fluent/fluent-bit

Length of output: 204

Eliminate the unsafe assertion by propagating allocation/setup errors.

The assertion at line 617 assumes params is set, but input_params_set() returns void and silently fails if flb_calloc() or pthread_setspecific() fails. When the coroutine resumes and calls input_pre_cb_collect(), the assertion aborts instead of failing cleanly. Additionally, the __thread declaration of pthread_key_t is incorrect—pthread keys are global identifiers, not thread-local variables—and pthread_key_create() should not be called per-coroutine.

Required changes:

  1. Make input_params_set() return an error code (e.g., int with 0 for success)
  2. Check the return value in flb_input_coro_collect() and unwind coroutine creation on failure
  3. Replace the assertion in input_pre_cb_collect() with proper error handling or a fatal log if params is NULL (after confirming bootstrap succeeded)
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@include/fluent-bit/flb_input.h` around lines 617 - 618, Change
input_params_set() to return an int error code (0 on success, non-zero on
failure) and propagate allocation/pthread_setspecific errors from flb_calloc and
pthread_setspecific instead of silently swallowing them; update callers (notably
flb_input_coro_collect()) to check the return value and on failure
unwind/cleanup coroutine creation and any allocated resources. Remove the unsafe
flb_bug(params == NULL) assertion in input_pre_cb_collect() and replace it with
proper error handling (return an error/fatal log if params is NULL after
successful bootstrap) so coroutine resume won’t abort; also stop treating
pthread_key_t as __thread—ensure pthread_key_create() is done once globally (not
per-coroutine) and use pthread_setspecific/pthread_getspecific for per-coroutine
storage. Ensure references to input_params_set, flb_calloc, pthread_setspecific,
flb_input_coro_collect, input_pre_cb_collect, pthread_key_t, and
pthread_key_create are updated accordingly.

Copy link
Copy Markdown
Contributor Author

@runderwo runderwo Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cosmo0920 Is this an acceptable way to bail out while surfacing the problem in this rare condition? Alternatives would seem to require significant refactoring.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!

coll = params->coll;
config = params->config;
coro = params->coro;
Expand All @@ -636,13 +631,6 @@ static FLB_INLINE void flb_input_coro_resume(struct flb_input_coro *input_coro)
flb_coro_resume(input_coro->coro);
}

static void libco_in_param_key_destroy(void *data)
{
struct flb_libco_inparams *params = (struct flb_libco_inparams*)data;

flb_free(params);
}

static FLB_INLINE
struct flb_input_coro *flb_input_coro_collect(struct flb_input_collector *coll,
struct flb_config *config)
Expand All @@ -656,7 +644,7 @@ struct flb_input_coro *flb_input_coro_collect(struct flb_input_collector *coll,
return NULL;
}

pthread_key_create(&libco_in_param_key, libco_in_param_key_destroy);
pthread_once(&libco_in_param_key_once, libco_in_param_key_init_func);

coro = input_coro->coro;
if (!coro) {
Expand Down
13 changes: 12 additions & 1 deletion src/flb_input.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,19 @@
#include <fluent-bit/flb_chunk_trace.h>
#endif /* FLB_HAVE_CHUNK_TRACE */

struct flb_libco_in_params libco_in_param;
pthread_key_t libco_in_param_key;
pthread_once_t libco_in_param_key_once = PTHREAD_ONCE_INIT;

static void libco_in_param_key_destroy(void *data)
{
struct flb_libco_in_params *params = (struct flb_libco_in_params*)data;

flb_free(params);
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.

void libco_in_param_key_init_func(void) {
pthread_key_create(&libco_in_param_key, libco_in_param_key_destroy);
}

#define protcmp(a, b) strncasecmp(a, b, strlen(a))

Expand Down
Loading