Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 6 additions & 19 deletions .blackboxrules
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
<!-- BEGIN_PIPELEX_RULES -->
# Pipelex Coding Rules

## Guide to write or edit pipelines using the Pipelex language in .plx files

- Always first write your "plan" in natural language, then transcribe it in pipelex.
Expand Down Expand Up @@ -39,10 +41,10 @@ ConceptName = "Description of the concept"
- Use PascalCase for concept names
- Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex
- Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents
- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page)
- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page, JSON)

**Native Concepts:**
Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate.
Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`, `JSON`. Use these directly or refine them when appropriate.

**Refining Native Concepts:**
To create a concept that specializes a native concept without adding fields:
Expand Down Expand Up @@ -70,7 +72,7 @@ output = "ConceptName"
The pipes will all have at least this base definition.
- `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition).
So If you have this error:
`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
variable='['invoice']'``
That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.

Expand Down Expand Up @@ -857,21 +859,6 @@ But don't write documentation unless asked explicitly to.

## Guide to execute a pipeline and write example code

### Prerequisites: Virtual Environment

**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.

For standard installations, the virtual environment is named `.venv`. Always check this first:

```bash
## Activate the virtual environment (standard installation)
source .venv/bin/activate # On macOS/Linux
## or
.venv\Scripts\activate # On Windows
```

If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.

### Example to execute a pipeline with text output

```python
Expand Down Expand Up @@ -1146,7 +1133,7 @@ Presets are meant to record the choice of an llm with its hyper parameters (temp
Examples:
```toml
llm_to_engineer = { model = "base-claude", temperature = 1 }
llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
llm_to_extract_invoice = { model = "claude-4.5-sonnet", temperature = 0.1, max_tokens = "auto" }
```

The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
Expand Down
2 changes: 1 addition & 1 deletion .cursor/rules/llms.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ Presets are meant to record the choice of an llm with its hyper parameters (temp
Examples:
```toml
llm_to_engineer = { model = "base-claude", temperature = 1 }
llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
llm_to_extract_invoice = { model = "claude-4.5-sonnet", temperature = 0.1, max_tokens = "auto" }
```

The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
Expand Down
15 changes: 0 additions & 15 deletions .cursor/rules/run_pipelex.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -6,21 +6,6 @@ globs:
---
# Guide to execute a pipeline and write example code

## Prerequisites: Virtual Environment

**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.

For standard installations, the virtual environment is named `.venv`. Always check this first:

```bash
# Activate the virtual environment (standard installation)
source .venv/bin/activate # On macOS/Linux
# or
.venv\Scripts\activate # On Windows
```

If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.

## Example to execute a pipeline with text output

```python
Expand Down
6 changes: 3 additions & 3 deletions .cursor/rules/write_pipelex.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,10 @@ ConceptName = "Description of the concept"
- Use PascalCase for concept names
- Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex
- Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents
- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page)
- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page, JSON)

**Native Concepts:**
Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate.
Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`, `JSON`. Use these directly or refine them when appropriate.

**Refining Native Concepts:**
To create a concept that specializes a native concept without adding fields:
Expand Down Expand Up @@ -76,7 +76,7 @@ output = "ConceptName"
The pipes will all have at least this base definition.
- `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition).
So If you have this error:
`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
variable='['invoice']'``
That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.

Expand Down
25 changes: 6 additions & 19 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
<!-- BEGIN_PIPELEX_RULES -->
# Pipelex Coding Rules

## Guide to write or edit pipelines using the Pipelex language in .plx files

- Always first write your "plan" in natural language, then transcribe it in pipelex.
Expand Down Expand Up @@ -39,10 +41,10 @@ ConceptName = "Description of the concept"
- Use PascalCase for concept names
- Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex
- Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents
- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page)
- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page, JSON)

**Native Concepts:**
Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate.
Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`, `JSON`. Use these directly or refine them when appropriate.

**Refining Native Concepts:**
To create a concept that specializes a native concept without adding fields:
Expand Down Expand Up @@ -70,7 +72,7 @@ output = "ConceptName"
The pipes will all have at least this base definition.
- `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition).
So If you have this error:
`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
variable='['invoice']'``
That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.

Expand Down Expand Up @@ -857,21 +859,6 @@ But don't write documentation unless asked explicitly to.

## Guide to execute a pipeline and write example code

### Prerequisites: Virtual Environment

**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.

For standard installations, the virtual environment is named `.venv`. Always check this first:

```bash
## Activate the virtual environment (standard installation)
source .venv/bin/activate # On macOS/Linux
## or
.venv\Scripts\activate # On Windows
```

If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.

### Example to execute a pipeline with text output

```python
Expand Down Expand Up @@ -1146,7 +1133,7 @@ Presets are meant to record the choice of an llm with its hyper parameters (temp
Examples:
```toml
llm_to_engineer = { model = "base-claude", temperature = 1 }
llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
llm_to_extract_invoice = { model = "claude-4.5-sonnet", temperature = 0.1, max_tokens = "auto" }
```

The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
Expand Down
78 changes: 78 additions & 0 deletions .github/workflows/github-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
name: Create release

on:
push:
branches:
- main

jobs:
github-release:
name: Create GitHub Release
runs-on: ubuntu-latest
permissions:
contents: write # IMPORTANT: mandatory for making GitHub Releases

steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Extract version from pyproject.toml
id: get_version
run: |
VERSION=$(grep -m 1 'version = ' pyproject.toml | cut -d '"' -f 2)
echo "VERSION=$VERSION" >> $GITHUB_ENV
- name: Extract changelog notes for current version
id: get_changelog
run: |
VERSION=${{ env.VERSION }}
echo "Extracting changelog for version v$VERSION"

# Find the start of the current version section
START_LINE=$(grep -n "## \[v$VERSION\] - " CHANGELOG.md | cut -d: -f1)

if [ -z "$START_LINE" ]; then
echo "Warning: No changelog entry found for version v$VERSION"
echo "CHANGELOG_NOTES=" >> $GITHUB_ENV
exit 0
fi

# Find the start of the next version section (previous version)
NEXT_VERSION_LINE=$(tail -n +$((START_LINE + 1)) CHANGELOG.md | grep -n "^## \[v.*\] - " | head -1 | cut -d: -f1)

if [ -z "$NEXT_VERSION_LINE" ]; then
# No next version found, extract from current version till end of file
CHANGELOG_CONTENT=$(tail -n +$START_LINE CHANGELOG.md)
else
# Extract content from current version header to before next version
END_LINE=$((START_LINE + NEXT_VERSION_LINE - 1))
CHANGELOG_CONTENT=$(sed -n "$START_LINE,$((END_LINE - 1))p" CHANGELOG.md)
fi

# Clean up the content but preserve the blank line after the header
# First, get the header line and add a blank line after it
HEADER_LINE=$(echo "$CHANGELOG_CONTENT" | head -1)
CONTENT_LINES=$(echo "$CHANGELOG_CONTENT" | tail -n +2 | sed '/^$/d' | sed 's/^[[:space:]]*//' | sed 's/[[:space:]]*$//')

# Combine header + blank line + content
CHANGELOG_CONTENT=$(printf "%s\n\n%s" "$HEADER_LINE" "$CONTENT_LINES")

# Escape for GitHub Actions
echo "CHANGELOG_NOTES<<EOF" >> $GITHUB_ENV
echo "$CHANGELOG_CONTENT" >> $GITHUB_ENV
echo "EOF" >> $GITHUB_ENV
- name: Create GitHub Release
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
if [ -n "$CHANGELOG_NOTES" ]; then
gh release create "v$VERSION" \
--repo "$GITHUB_REPOSITORY" \
--title "v$VERSION" \
--notes "$CHANGELOG_NOTES"
else
gh release create "v$VERSION" \
--repo "$GITHUB_REPOSITORY" \
--title "v$VERSION" \
--notes "Release v$VERSION"
fi

82 changes: 11 additions & 71 deletions .github/workflows/version-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ on:
pull_request:
branches:
- main
- "release/v[0-9]+.[0-9]+.[0-9]+"
jobs:
version-check:
runs-on: ubuntu-latest
Expand All @@ -24,26 +23,17 @@ jobs:
echo "base_branch=$BASE_BRANCH" >> $GITHUB_OUTPUT
echo "source_branch=$SOURCE_BRANCH" >> $GITHUB_OUTPUT

# Check if target is a release branch
if [[ "$BASE_BRANCH" =~ ^release/v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "Target is a release branch"
echo "is_release_target=true" >> $GITHUB_OUTPUT
echo "target_release_version=${BASE_BRANCH#release/v}" >> $GITHUB_OUTPUT
echo "Extracted target release version: ${BASE_BRANCH#release/v}"
else
echo "Target is NOT a release branch"
echo "is_release_target=false" >> $GITHUB_OUTPUT
fi

# Check if source is a release branch
if [[ "$SOURCE_BRANCH" =~ ^release/v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "Source is a release branch"
echo "is_release_source=true" >> $GITHUB_OUTPUT
echo "source_release_version=${SOURCE_BRANCH#release/v}" >> $GITHUB_OUTPUT
echo "Extracted source release version: ${SOURCE_BRANCH#release/v}"
else
echo "Source is NOT a release branch"
echo "Source is NOT a release branch - skipping version check"
echo "is_release_source=false" >> $GITHUB_OUTPUT
echo "This workflow only runs for PRs from release branches to main"
exit 0
fi
echo "======================================="

Expand All @@ -61,74 +51,24 @@ jobs:
echo "version=$VERSION" >> $GITHUB_OUTPUT
echo "======================================="

# MAIN BRANCH CASE - Check that version is bumped
- name: Check version bump for main branch
if: steps.branch_info.outputs.base_branch == 'main'
# Check that version in pyproject.toml matches the source release branch name
- name: Check version matches release branch
run: |
echo "====== CHECKING VERSION BUMP FOR MAIN ======"
# Get version from main branch
git checkout ${{ github.event.pull_request.base.ref }}
MAIN_VERSION=$(grep '^version' pyproject.toml | sed -E 's/version = "(.*)"/\1/')

# Get version from PR branch
git checkout ${{ github.event.pull_request.head.sha }}
PR_VERSION="${{ steps.current_version.outputs.version }}"

echo "Main version: $MAIN_VERSION"
echo "PR version: $PR_VERSION"

# Check that PR version is greater than main version
if dpkg --compare-versions "$PR_VERSION" le "$MAIN_VERSION"; then
echo "❌ ERROR: PR version ($PR_VERSION) is not greater than main version ($MAIN_VERSION)."
echo "Please bump the version."
exit 1
else
echo "✅ Version was properly bumped from $MAIN_VERSION to $PR_VERSION"
fi
echo "======================================="

# TARGET RELEASE BRANCH CASE - Check that version matches release branch name
- name: Check version matches target release branch
if: steps.branch_info.outputs.is_release_target == 'true'
run: |
echo "====== CHECKING VERSION MATCH FOR TARGET RELEASE BRANCH ======"
# Extract version from branch name
RELEASE_VERSION="${{ steps.branch_info.outputs.target_release_version }}"

# Get version from PR branch
PR_VERSION="${{ steps.current_version.outputs.version }}"

echo "Target release branch version: $RELEASE_VERSION"
echo "PR version: $PR_VERSION"

# Check if versions match
if [[ "$PR_VERSION" != "$RELEASE_VERSION" ]]; then
echo "❌ ERROR: Version in pyproject.toml ($PR_VERSION) does not match target release branch version ($RELEASE_VERSION)"
exit 1
else
echo "✅ Version in pyproject.toml matches target release branch version"
fi
echo "======================================="

# SOURCE RELEASE BRANCH CASE - Check that version matches release branch name
- name: Check version matches source release branch
if: steps.branch_info.outputs.is_release_source == 'true'
run: |
echo "====== CHECKING VERSION MATCH FOR SOURCE RELEASE BRANCH ======"
echo "====== CHECKING VERSION MATCH FOR RELEASE BRANCH ======"
# Extract version from branch name
RELEASE_VERSION="${{ steps.branch_info.outputs.source_release_version }}"

# Get version from PR branch
PR_VERSION="${{ steps.current_version.outputs.version }}"

echo "Source release branch version: $RELEASE_VERSION"
echo "PR version: $PR_VERSION"
echo "Release branch version: $RELEASE_VERSION"
echo "pyproject.toml version: $PR_VERSION"

# Check if versions match
if [[ "$PR_VERSION" != "$RELEASE_VERSION" ]]; then
echo "❌ ERROR: Version in pyproject.toml ($PR_VERSION) does not match source release branch version ($RELEASE_VERSION)"
echo "❌ ERROR: Version in pyproject.toml ($PR_VERSION) does not match release branch version ($RELEASE_VERSION)"
exit 1
Comment on lines 67 to 70
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Version check fails all non-release PRs

When the source branch is not named like release/v*, the Get branch info step exits early after printing “skipping version check” and never populates source_release_version, but the Check version matches release branch block still runs unconditionally and compares the pyproject version against that empty string, triggering the exit-1 path. As a result every normal PR to main now fails this workflow instead of being skipped as intended.

Useful? React with 👍 / 👎.

else
echo "✅ Version in pyproject.toml matches source release branch version"
echo "✅ Version in pyproject.toml ($PR_VERSION) matches release branch version ($RELEASE_VERSION)"
fi
echo "======================================="
echo "======================================="
Loading
Loading