Skip to content

feat: Private git repositories#2076

Open
sstreichan wants to merge 24 commits into
volcengine:mainfrom
sstreichan:main
Open

feat: Private git repositories#2076
sstreichan wants to merge 24 commits into
volcengine:mainfrom
sstreichan:main

Conversation

@sstreichan
Copy link
Copy Markdown

@sstreichan sstreichan commented May 15, 2026

Description

Adds support for configuring and using Personal Access Tokens (PAT) to access private Git repositories (GitHub and GitLab) via ov add-resource. Users can now store host-to-token mappings in ovcli.conf and have tokens automatically injected into git URLs during resource import.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update

Changes Made

  • Added _cmd_configure_git_credentials command to interactively configure git credentials per host
  • Added _preprocess_add_resource_token to inject PATs into git URLs for ov add-resource
  • Updated main to support the new configure-git-credentials command
  • Introduced git_credentials mapping in ovcli.conf for host-to-token associations
  • Created git_credentials.py for token injection, retrieval, and URL masking logic
  • Added integration and unit tests for credential handling and token masking
  • Updated documentation with guidance on using PATs for private repositories

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this on the following platforms:
    • Linux
    • macOS
    • Windows

Checklist

  • My code follows the project's coding style
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Additional Notes

Tokens are masked in logs and CLI output. The credential store uses the existing ovcli.conf config file to avoid introducing a separate secrets file.

sebastian and others added 14 commits May 11, 2026 12:46
…ories

- Implemented `_cmd_configure_git_credentials` to handle user input for git credentials.
- Added `_preprocess_add_resource_token` to inject tokens into git URLs for `ov add-resource`.
- Updated `main` function to support new commands for configuring git credentials and processing tokens.
- Introduced `git_credentials` mapping in `ovcli.conf` to store host-to-token associations.
- Created `git_credentials.py` for managing git credential logic, including token injection and retrieval.
- Added integration and unit tests for the new functionality, ensuring proper token handling and masking.
- Updated documentation to include guidance on using personal access tokens for private repositories.
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
… rust_cli.py and test files

feat: add ruff as a linter dependency in pyproject.toml and uv.lock
Restrict file permissions for the credentials file to owner only.
…ories

- Implemented `_cmd_configure_git_credentials` to handle user input for git credentials.
- Added `_preprocess_add_resource_token` to inject tokens into git URLs for `ov add-resource`.
- Updated `main` function to support new commands for configuring git credentials and processing tokens.
- Introduced `git_credentials` mapping in `ovcli.conf` to store host-to-token associations.
- Created `git_credentials.py` for managing git credential logic, including token injection and retrieval.
- Added integration and unit tests for the new functionality, ensuring proper token handling and masking.
- Updated documentation to include guidance on using personal access tokens for private repositories.
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
… rust_cli.py and test files

feat: add ruff as a linter dependency in pyproject.toml and uv.lock
Restrict file permissions for the credentials file to owner only.
@github-actions
Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🏅 Score: 75
🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ Recommended focus areas for review

GitLab token injection is incorrect

The inject_token function uses the token directly as the username for all hosts, but GitLab requires oauth2:<token> as the userinfo (username oauth2, password token). This will cause authentication failures for private GitLab repositories.

def inject_token(url: str, token: str) -> str:
    """Inject a token into an HTTPS/HTTP git URL as URL userinfo.

    Transforms ``https://github.com/org/repo`` to
    ``https://token@github.com/org/repo``.

    SSH (``git@``, ``ssh://``) and ``git://`` URLs are returned unchanged —
    token injection is not applicable for those schemes.

    Args:
        url: The repository URL.
        token: The authentication token to embed.

    Returns:
        URL with token injected as userinfo, or the original URL unchanged.
    """
    if not url.startswith(("https://", "http://")):
        return url
    parsed = urlparse(url)
    # Build netloc with token as userinfo, replacing any pre-existing credentials.
    hostname = parsed.hostname or ""
    host_with_port = f"{hostname}:{parsed.port}" if parsed.port else hostname
    netloc_with_token = f"{token}@{host_with_port}"
    return parsed._replace(netloc=netloc_with_token).geturl()
GitLab token extraction from URL is incorrect

The _gitlab_zip_download function extracts the token from parsed.username, but for correctly formatted GitLab URLs (with oauth2:<token> userinfo), the token is in parsed.password. This will fail to use tokens injected for GitLab.

# Extract token from embedded URL userinfo (injected client-side), fall back to env.
token_from_url = parsed.username
gitlab_token = token_from_url or os.environ.get("GITLAB_TOKEN")

@github-actions
Copy link
Copy Markdown

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Use oauth2:token for GitLab URLs in inject_token

For GitLab (and non-GitHub hosts), Git expects the username to be "oauth2" when
using a personal access token over HTTPS. Update inject_token to use "oauth2:" for
hosts not containing "github" to ensure compatibility with GitLab.

openviking_cli/utils/git_credentials.py [39-62]

 def inject_token(url: str, token: str) -> str:
     """Inject a token into an HTTPS/HTTP git URL as URL userinfo.
 
     Transforms ``https://github.com/org/repo`` to
     ``https://token@github.com/org/repo``.
+
+    Transforms ``https://gitlab.com/group/repo`` to
+    ``https://oauth2:token@gitlab.com/group/repo``.
 
     SSH (``git@``, ``ssh://``) and ``git://`` URLs are returned unchanged —
     token injection is not applicable for those schemes.
 
     Args:
         url: The repository URL.
         token: The authentication token to embed.
 
     Returns:
         URL with token injected as userinfo, or the original URL unchanged.
     """
     if not url.startswith(("https://", "http://")):
         return url
     parsed = urlparse(url)
-    # Build netloc with token as userinfo, replacing any pre-existing credentials.
     hostname = parsed.hostname or ""
     host_with_port = f"{hostname}:{parsed.port}" if parsed.port else hostname
-    netloc_with_token = f"{token}@{host_with_port}"
+    
+    # Use token directly as username for GitHub, oauth2:token for others
+    if "github" in hostname.lower():
+        netloc_with_token = f"{token}@{host_with_port}"
+    else:
+        netloc_with_token = f"oauth2:{token}@{host_with_port}"
+    
     return parsed._replace(netloc=netloc_with_token).geturl()
Suggestion importance[1-10]: 5

__

Why: The current implementation uses the token directly as the username for all hosts, which works for GitLab, but the plan.md recommends using "oauth2:token" for non-GitHub hosts. This change improves compatibility with GitLab's documented authentication format, though it is not strictly required for functionality.

Low

@MaojiaSheng MaojiaSheng requested a review from myysy May 15, 2026 12:38
Comment thread openviking_cli/rust_cli.py Outdated
@@ -39,12 +39,84 @@ def _exec_binary(binary: str, argv: list[str]) -> None:
os.execv(binary, [binary] + argv)


def _cmd_configure_git_credentials(args: list[str]) -> int:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could move this code into Rust?

Comment thread openviking_cli/setup_wizard.py Outdated
@@ -1090,6 +1091,37 @@ def _wizard_server() -> dict[str, Any] | None:
return {"host": "0.0.0.0", "root_api_key": root_api_key}


def _wizard_git_credentials() -> None:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

openviking_cli/setup_wizard.py is for server side setup.
you can add this in command: ov config setup-cli

sstreichan and others added 8 commits May 22, 2026 11:09
… config to Rust

- Fix inject_token() to use oauth2:token@host for GitLab/non-GitHub hosts
- Fix token extraction in git_accessor.py to read parsed.password or parsed.username
- Add git_credentials field to Rust Config struct
- Create Rust git_credentials module with inject/mask/is_git_url/extract_url_host
- Extend ov config setup-cli with optional git credentials step
- Add ov config git-credentials Rust subcommand
- Auto-inject tokens from config in handle_add_resource
- Remove Python-side _cmd_configure_git_credentials from rust_cli.py
- Remove _wizard_git_credentials from setup_wizard.py
- Update tests for new GitLab oauth2:token format
… config to Rust

- Fix inject_token() to use oauth2:token@host for GitLab/non-GitHub hosts
- Fix token extraction in git_accessor.py to read parsed.password or parsed.username
- Add git_credentials field to Rust Config struct
- Create Rust git_credentials module with inject/mask/is_git_url/extract_url_host
- Extend ov config setup-cli with optional git credentials step
- Add ov config git-credentials Rust subcommand
- Auto-inject tokens from config in handle_add_resource
- Remove Python-side _cmd_configure_git_credentials from rust_cli.py
- Remove _wizard_git_credentials from setup_wizard.py
- Update tests for new GitLab oauth2:token format
Private git repositories
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

2 participants