Add dedup Param to apiClient

[Teifun2]

With #3373 it was discussed that the dedup option can be used to prevent deduplication.
This option was added to the server with #687

Exposing this would help other client use this query param properly.

[github-actions]

@Teifun2: There are no 'kind' label on this PR. You need a 'kind' label to generate the release automatically.

• /kind feature
• /kind enhancement
• /kind refactoring
• /kind fix
• /kind chore
• /kind dependencies

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[github-actions]

@Teifun2: There are no area labels on this PR. You can add as many areas as you see fit.

• /area agent
• /area local-api
• /area cscli
• /area appsec
• /area security
• /area configuration

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[Teifun2]

/kind enhancement
/area local-api

[blotus]

Hello,

I'm not sure I understand the goal of the PR ?
There's already a dedup parameter present in LAPI.

Here, you have just added it to the CAPI swagger and to the queries made to CAPI, but this won't have any impact: CAPI does not support deduplicating decisions, and never will: CAPI only serves blocklists content, which will always have unique IPs.

[lluisd]

Hello,

I'm not sure I understand the goal of the PR ? There's already a dedup parameter present in LAPI.

Here, you have just added it to the CAPI swagger and to the queries made to CAPI, but this won't have any impact: CAPI does not support deduplicating decisions, and never will: CAPI only serves blocklists content, which will always have unique IPs.

To give context this comes from my unifi bouncer issue (Teifun2/cs-unifi-bouncer#77) where I explain it.

The problem is that I use 2 crowdsec bouncers. The first one is the firewall of my Unifi router for crowdsec/cli origins only because if I use all then my unifi router crashes. Then I use the CAPI/lists origins on the traefik bouncer as the second layer.

But I noticed I was losing some crowdsec decisions on my Unifi bouncer and I realized this was because they were already in CAPI or lists. Then I realize that using the dedup=false query param like that /v1/decisions/stream?additional_pull=false&community_pull=false&origins=crowdsec%2Ccscli&dedup=false I can get them, but this can only be used by calling the api manually, not by using the github.com/crowdsecurity/crowdsec package like its done in https://github.com/Teifun2/cs-unifi-bouncer repo.

[blotus]

Ok, got it, thanks for the clarification.

If you don't mind, I'll try to make the changes myself this week, as I'd like to clean up some things as well (for example, there's no good reason for calls to LAPI to have the additional_pull and community_pull parameters set, as they are only relevant for CAPI)