Skip to content

python(feat): Plumb bytes through sift-py and sift_client.#304

Merged
ian-sift merged 4 commits intomainfrom
fd-195-add-bytes-type-to-sift-stack-py
Aug 29, 2025
Merged

python(feat): Plumb bytes through sift-py and sift_client.#304
ian-sift merged 4 commits intomainfrom
fd-195-add-bytes-type-to-sift-stack-py

Conversation

@ian-sift
Copy link
Copy Markdown
Contributor

@ian-sift ian-sift commented Aug 29, 2025

Verification:

Setup

Run the updated examples/ingestion_with_python_config/main.py to create a run w/ bytes type data

Results

    run = sift.runs.get(run_id="d24f66d7-723f-4fa3-b546-255df298af8c")
    nostromo = run.assets[0]
    channels = nostromo.channels(run_id=run.id_)
    for channel in channels:
        if channel.name == "raw_bin":
            print(channel.description)
            data = channel.data(run_id=run.id_)
            channel_name, df = next(iter(data.items()))
            for i, (time, value) in enumerate(df.iterrows()):
                print(i, time, value[channel_name].decode("utf-8"))
                if i > 10:
                    break

Yielded:

Example of binary encoded data (binary string encoding of time in seconds)
0 2025-08-29 22:11:15.846012+00:00 1756505475.846039
1 2025-08-29 22:11:16.512679+00:00 1756505476.512702
2 2025-08-29 22:11:17.179542+00:00 1756505477.1795468
3 2025-08-29 22:11:17.846210+00:00 1756505477.846234
4 2025-08-29 22:11:18.512880+00:00 1756505478.512905
5 2025-08-29 22:11:19.179542+00:00 1756505479.179546
6 2025-08-29 22:11:19.846212+00:00 1756505479.8462431
7 2025-08-29 22:11:20.512880+00:00 1756505480.512904
8 2025-08-29 22:11:21.179543+00:00 1756505481.1795478
9 2025-08-29 22:11:21.846213+00:00 1756505481.846234
10 2025-08-29 22:11:22.512880+00:00 1756505482.5129018
11 2025-08-29 22:11:23.179544+00:00 1756505483.1795478

Rerun for yaml ingestion:

    yaml_run = sift.runs.get(run_id="c7839e77-1267-49d6-a05f-b58951bc5031")
    nostromo = yaml_run.assets[0]
    channels = nostromo.channels(run_id=yaml_run.id_)
    for channel in channels:
        if channel.name == "raw_bin":
            print(channel.name)
            print(channel.description)
            data = channel.data(run_id=yaml_run.id_)
            channel_name, df = next(iter(data.items()))
            print(channel_name)
            for i, (time, value) in enumerate(df.iterrows()):
                print(i, time, value[channel_name].decode("utf-8"))
                if i > 10:
                    break
               
=>
               
Example of binary encoded data (binary string encoding of time in seconds)
raw_bin
0 2025-08-29 23:37:10.927584+00:00 2025-08-29 23:37:10.927584+00:00
1 2025-08-29 23:37:11.594251+00:00 2025-08-29 23:37:11.594251+00:00
2 2025-08-29 23:37:12.261125+00:00 2025-08-29 23:37:12.261125+00:00
3 2025-08-29 23:37:12.927793+00:00 2025-08-29 23:37:12.927793+00:00
4 2025-08-29 23:37:13.594460+00:00 2025-08-29 23:37:13.594460+00:00
5 2025-08-29 23:37:14.261125+00:00 2025-08-29 23:37:14.261125+00:00
6 2025-08-29 23:37:14.927795+00:00 2025-08-29 23:37:14.927795+00:00
7 2025-08-29 23:37:15.594463+00:00 2025-08-29 23:37:15.594463+00:00
8 2025-08-29 23:37:16.261137+00:00 2025-08-29 23:37:16.261137+00:00
9 2025-08-29 23:37:16.927806+00:00 2025-08-29 23:37:16.927806+00:00
10 2025-08-29 23:37:17.594476+00:00 2025-08-29 23:37:17.594476+00:00
11 2025-08-29 23:37:18.261138+00:00 2025-08-29 23:37:18.261138+00:00

@ian-sift ian-sift requested a review from alexluck-sift August 29, 2025 22:52
Copy link
Copy Markdown
Collaborator

@alexluck-sift alexluck-sift left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure to update the yaml parsing to support bytes type as well: https://github.com/sift-stack/sift/blob/main/python/lib/sift_py/yaml/channel.py#L193

@ian-sift ian-sift merged commit d21ade2 into main Aug 29, 2025
10 checks passed
@ian-sift ian-sift deleted the fd-195-add-bytes-type-to-sift-stack-py branch August 29, 2025 23:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants