Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:

- name: Install hatch
run: |
pip install hatch==1.14.0
pip install hatch
pip install .

- name: Run tests
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,6 @@ jobs:
- name: Install click
run: pip install click==8.1.8
- name: Install hatch
run: pip install hatch==1.14.0
run: pip install hatch
- name: Build docs
run: hatch run docs:build
2 changes: 1 addition & 1 deletion .github/workflows/style.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
run: pip install click===8.1.8

- name: Install hatch
run: pip install hatch==1.14.0
run: pip install hatch

- name: Run style checks
run: hatch run style:check
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,14 @@
how a consumer would use the library or CLI tool (e.g. adding unit tests, updating documentation, etc) are not captured
here.

## Unreleased

### Added
- Added the `sdk.file-events.v2.search_groups` method to get approximate aggregate file event counts by a given grouping term.
- Added the `GroupingEventQuery` class, used to make these queries.
- Added the cli command `incydr file-events search-groups` to get approximate aggregate file event counts by a given grouping term.


## 2.11.0 - 2026-02-10

### Added
Expand Down
8 changes: 8 additions & 0 deletions docs/sdk/clients/file_event_queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,14 @@ Use the `EventQuery` class to create a query for searching and filtering file ev
:docstring:
:members: equals not_equals exists does_not_exist greater_than less_than matches_any is_any is_none date_range subquery

## GroupingEventQuery Class

Use the `GroupingEventQuery` class to create a query for searching for approximate event counts, grouped by a term called the `grouping_term`. `GroupingEventQuery` supports all of the same operators as `EventQuery`, with the addition of `group_by` and `maximum_size`, which can be used to control the grouping term and the maximum size of the response.

::: _incydr_sdk.queries.file_events.GroupingEventQuery
:docstring:
:members: group_by maximum_size

## Query Building

The `EventQuery` class can be imported directly from the `incydr` module.
Expand Down
10 changes: 10 additions & 0 deletions docs/sdk/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,16 @@ Devices has been replaced by [Agents](#agents).
::: incydr.models.SavedSearch
:docstring:

### `GroupedFileEventResponse` model

::: incydr.models.GroupedFileEventResponse
:docstring:

### `FileEventGroup` model

::: incydr.models.FileEventGroup
:docstring:

## Roles
---

Expand Down
118 changes: 115 additions & 3 deletions src/_incydr_cli/cmds/file_events.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,10 @@
from _incydr_sdk.enums.file_events import RiskIndicators
from _incydr_sdk.enums.file_events import RiskSeverity
from _incydr_sdk.file_events.models.event import FileEventV2
from _incydr_sdk.file_events.models.response import FileEventGroup
from _incydr_sdk.file_events.models.response import SavedSearch
from _incydr_sdk.queries.file_events import EventQuery
from _incydr_sdk.queries.file_events import GroupingEventQuery
from _incydr_sdk.utils import model_as_card


Expand Down Expand Up @@ -100,14 +102,15 @@ def search(
elif advanced_query:
if not isinstance(advanced_query, str):
advanced_query = advanced_query.read()
query = EventQuery.parse_raw(advanced_query)
query = EventQuery.model_validate_json(advanced_query)
else:
if not start:
raise BadOptionUsage(
"start",
"--start option required if not using --saved-search or --advanced-query options.",
)
query = _create_query(
cls=EventQuery,
start=start,
end=end,
event_action=event_action,
Expand Down Expand Up @@ -191,6 +194,115 @@ def yield_all_events(q: EventQuery):
console.print("No results found.")


@file_events.command(cls=IncydrCommand)
@click.option(
"--group-by",
default=None,
help="(required) The term by which approximate counts will be grouped. Example: `user.email`.",
required=True,
)
@table_format_option
@columns_option
@output_options
@advanced_query_option
@saved_search_option
@event_filter_options
@logging_options
def search_groups(
format_: TableFormat,
columns: Optional[str],
output: Optional[str],
certs: Optional[str],
ignore_cert_validation: Optional[bool],
advanced_query: Optional[Union[str, File]],
saved_search: Optional[str],
start: Optional[str],
end: Optional[str],
event_action: Optional[str],
username: Optional[str],
md5: Optional[str],
sha256: Optional[str],
source_category: Optional[str],
destination_category: Optional[str],
file_name: Optional[str],
file_directory: Optional[str],
file_category: Optional[str],
risk_indicator: Optional[RiskIndicators],
risk_severity: Optional[RiskSeverity],
risk_score: Optional[int],
group_by: Optional[str],
):
"""
Retrieve approximate aggregated file event counts. Various options are provided to filter query results.

Use the `--saved-search` or the `--advanced-query` option if the available filters don't satisfy your requirements.

Results will be output to the console by default, use the `--output` option to send data to a server.

This method returns approximate counts, grouped by the provided term. To obtain full event details, use the `search` method.
"""
if output:
format_ = TableFormat.json_lines

client = Client()

if saved_search:
saved_search = client.file_events.v2.get_saved_search(saved_search)
query = GroupingEventQuery.from_saved_search(saved_search)
elif advanced_query:
if not isinstance(advanced_query, str):
advanced_query = advanced_query.read()
query = GroupingEventQuery.model_validate_json(advanced_query)
else:
if not start:
raise BadOptionUsage(
"start",
"--start option required if not using --saved-search or --advanced-query options.",
)
query = _create_query(
cls=GroupingEventQuery,
start=start,
end=end,
event_action=event_action,
username=username,
md5=md5,
sha256=sha256,
source_category=source_category,
destination_category=destination_category,
file_name=file_name,
file_directory=file_directory,
file_category=file_category,
risk_indicator=risk_indicator,
risk_severity=risk_severity,
risk_score=risk_score,
)

query.group_by(group_by).maximum_size(10000)

groups = client.file_events.v2.search_groups(query).groups or []

if output:
logger = get_server_logger(output, certs, ignore_cert_validation)
for group in groups:
logger.info(json.dumps(group.dict()))
return

if format_ == TableFormat.csv:
render.csv(FileEventGroup, groups, columns=columns, flat=True)
elif format_ == TableFormat.table:
render.table(FileEventGroup, groups, columns=columns, flat=False)
else:
printed = False
for group in groups:
printed = True
if format_ == TableFormat.json_pretty:
console.print_json(data=group)
else:
click.echo(json.dumps(group.dict()))
if not printed:
console.print("No results found.")


@file_events.command()
@click.argument("checkpoint-name")
def clear_checkpoint(checkpoint_name: str):
Expand Down Expand Up @@ -262,8 +374,8 @@ def list_saved_searches(
}


def _create_query(**kwargs):
query = EventQuery(start_date=kwargs["start"], end_date=kwargs["end"])
def _create_query(cls, **kwargs):
query = cls(start_date=kwargs["start"], end_date=kwargs["end"])
for k, v in kwargs.items():
if v:
if k in ["start", "end"]:
Expand Down
24 changes: 24 additions & 0 deletions src/_incydr_sdk/file_events/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,10 @@

from ..exceptions import IncydrException
from .models.response import FileEventsPage
from .models.response import GroupedFileEventResponse
from .models.response import SavedSearch
from _incydr_sdk.queries.file_events import EventQuery
from _incydr_sdk.queries.file_events import GroupingEventQuery


class InvalidQueryException(IncydrException):
Expand Down Expand Up @@ -74,6 +76,28 @@ def search(self, query: EventQuery) -> FileEventsPage:
query.page_token = page.next_pg_token
return page

def search_groups(self, query: GroupingEventQuery) -> GroupedFileEventResponse:
"""
Search for file event counts by a grouping term.

**Parameters**:

* **query**: `GroupingEventQuery` (required) - The query object to group file events by a given field.

**Returns**: A [`GroupedFileEventResponse`][groupedfileeventresponse-model] object."""
self._mount_retry_adapter()

try:
response = self._parent.session.post(
"/v2/file-events/grouping", json=query.dict()
)
except HTTPError as err:
if err.response.status_code == 400:
raise InvalidQueryException(query=query, exception=err)
raise err
response = GroupedFileEventResponse.parse_response(response)
return response

def list_saved_searches(self) -> List[SavedSearch]:
"""
Get all saved searches.
Expand Down
37 changes: 37 additions & 0 deletions src/_incydr_sdk/file_events/models/response.py
Original file line number Diff line number Diff line change
Expand Up @@ -218,3 +218,40 @@ class SavedSearch(ResponseModel):
description="Search term for sorting.",
examples=["event.id"],
)


class FileEventGroup(ResponseModel):
"""A model representing a single group in a grouped response.

**Fields:**

* **value**: `str` - The value of the term for this group.
* **doc_count**: `int` - The approximate count of hits matching this value for your query.
"""

value: Optional[str] = Field(
None, description="The value of the term for this group."
)
doc_count: Optional[int] = Field(
None,
description="The approximate count of hits matching this value for your query.",
alias="docCount",
)


class GroupedFileEventResponse(ResponseModel):
"""A model representing a response of grouped file events.

**Fields:**

* **groups**: `List[FileEventGroup]` - A list of file event counts by grouping term and doc count.
* **problems**: `List[QueryProblem]` - List of problems in the request. A problem with a search request could be an invalid filter value, an operator that can't be used on a term, etc.
"""

groups: Optional[List[FileEventGroup]] = Field(
None, description="A list of file event counts by grouping term and doc count."
)
problems: Optional[List[QueryProblem]] = Field(
None,
description="List of problems in the request. A problem with a search request could be an invalid filter value, an operator that can't be used on a term, etc.",
)
60 changes: 48 additions & 12 deletions src/_incydr_sdk/queries/file_events.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,23 +106,13 @@ class Query(Model):
srtKey: EventSearchTerm = "event.id"


class EventQuery(Model):
class BaseEventQuery(Model):
"""
Class to build a file event query. Use the class methods to attach additional filter operators.

**Parameters**:

* **start_date**: `int`, `float`, `str`, `datetime`, `timedelta` - Start of the date range to query for events. Defaults to None.
* **end_date**: `int`, `float`, `str`, `datetime` - End of the date range to query for events. Defaults to None.
Base class used for EventQuery and GroupingEventQuery.
"""

group_clause: str = Field("AND", alias="groupClause")
groups: Optional[List[FilterGroup]]
page_num: int = Field(1, alias="pgNum")
page_size: Annotated[int, Field(le=10000)] = Field(100, alias="pgSize")
page_token: Optional[str] = Field("", alias="pgToken")
sort_dir: str = Field("asc", alias="srtDir")
sort_key: EventSearchTerm = Field("event.id", alias="srtKey")
model_config = ConfigDict(
validate_assignment=True,
use_enum_values=True,
Expand Down Expand Up @@ -424,6 +414,52 @@ def from_saved_search(cls, saved_search: SavedSearch):
return query


class EventQuery(BaseEventQuery):
"""
Class to build a file event query. Use the class methods to attach additional filter operators.

**Parameters**:

* **start_date**: `int`, `float`, `str`, `datetime`, `timedelta` - Start of the date range to query for events. Defaults to None.
* **end_date**: `int`, `float`, `str`, `datetime` - End of the date range to query for events. Defaults to None.
"""

page_num: int = Field(1, alias="pgNum")
page_size: Annotated[int, Field(le=10000)] = Field(100, alias="pgSize")
page_token: Optional[str] = Field("", alias="pgToken")
sort_dir: str = Field("asc", alias="srtDir")
sort_key: EventSearchTerm = Field("event.id", alias="srtKey")


class GroupingEventQuery(BaseEventQuery):
"""
Class to build a file event query for use in grouped searches, which return aggregated counts. The `grouping_term` parameter determines by what term the result will be aggregated.

Use the class methods to attach additional filter operators.

**Parameters**:

* **start_date**: `int`, `float`, `str`, `datetime`, `timedelta` - Start of the date range to query for events. Defaults to None.
* **end_date**: `int`, `float`, `str`, `datetime` - End of the date range to query for events. Defaults to None.
* **grouping_term**: `str` - The search term to use to form the groups
* **size**: `int` - The maximum number of groups that will be returned for this query. Default value is 1000. Maximum possible value is 10,000.
"""

grouping_term: str = Field("", alias="groupingTerm")
size: int = Field(1000)

def group_by(self, grouping_term: str):
"""Sets the grouping term for this query. When the query is run, it will group events by this term. For example, to group by file category,
set the term to `file.category`"""
self.grouping_term = grouping_term
return self

def maximum_size(self, size: int):
"""Sets the maximum number of groups that will be returned for this query. Defaults to 1000. Maximum possible value supported by the API is 10000."""
self.size = size
return self


def _create_date_range_filter_group(start_date, end_date, term=None):
def _validate_duration_str(iso_duration_str):
try:
Expand Down
Loading
Loading