Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ See the [REST API documentation](site:pulp_rust/restapi/) for detailed endpoint

- [Use Pulp as a pull-through cache](site:pulp_rust/docs/user/guides/pull-through-cache/) for crates.io or any Cargo sparse registry
- [Host a private Cargo registry](site:pulp_rust/docs/user/guides/private-registry/) for internal crates
- Publish crates with `cargo publish` and manage them with `cargo yank`
- Implements the [Cargo sparse registry protocol](https://doc.rust-lang.org/cargo/reference/registry-index.html#sparse-index) for compatibility with standard Cargo tooling
- Download crates on-demand to reduce disk usage
- Every operation creates a restorable snapshot with Versioned Repositories
Expand Down
62 changes: 56 additions & 6 deletions docs/user/guides/private-registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,6 @@ This guide walks you through setting up Pulp as a private Cargo registry for hos
crates. This is useful for organizations that need to distribute proprietary or internal-only
Rust packages.

!!! note
Package publishing support (`cargo publish`) is not yet available but is planned for an
upcoming release. In the meantime, content can be uploaded through the Pulp REST API.

## Create a Repository

```bash
Expand All @@ -16,13 +12,15 @@ pulp rust repository create --name my-crates

## Create a Distribution

A distribution makes the repository's content available to Cargo over HTTP.
A distribution makes the repository's content available to Cargo over HTTP. Set `--allow-uploads`
to enable publishing crates via `cargo publish`.

```bash
pulp rust distribution create \
--name my-crates \
--base-path my-crates \
--repository my-crates
--repository my-crates \
--allow-uploads
```

Your private registry is now served at `http://<pulp-host>/pulp/cargo/my-crates/`.
Expand All @@ -36,6 +34,58 @@ Add the private registry to your Cargo configuration. Create or edit `~/.cargo/c
index = "sparse+http://<pulp-host>/pulp/cargo/my-crates/"
```

## Authentication

State-changing operations (publishing, yanking, and unyanking) require an authorization token.
Configure the token for your registry in `~/.cargo/credentials.toml`:

```toml
[registries.my-crates]
token = "i_understand_that_pulp_rust_does_not_support_proper_auth_yet"
```

Alternatively, you can pass the token on the command line:

```bash
cargo publish --registry my-crates --token "i_understand_that_pulp_rust_does_not_support_proper_auth_yet"
```

!!! warning
This is a temporary stub token. Proper token-based authentication is planned for a future
release. The stub token exists to ensure that the authentication workflow is exercised and that
state-changing operations are not completely open.

Read-only operations (downloading crates, browsing the index) do not require a token.

## Publish a Crate

Once the registry is configured and a distribution with `--allow-uploads` exists, you can publish
crates using standard Cargo tooling:

```bash
cargo publish --registry my-crates
```

This uploads the crate to Pulp, which creates the artifact, content metadata, and a new repository
version. The crate is immediately available for download through the distribution.

Publishing the same crate version twice is rejected — crate versions are immutable, consistent
with crates.io behavior.

## Yank and Unyank

Yanking marks a crate version as unavailable for new dependency resolution, while still allowing
existing projects that already depend on it to continue downloading it. This matches the
[crates.io yank semantics](https://doc.rust-lang.org/cargo/reference/publishing.html#cargo-yank).

```bash
# Yank a version
cargo yank --registry my-crates --version 1.0.0 my-crate

# Unyank a version
cargo yank --registry my-crates --version 1.0.0 --undo my-crate
```

### Using the Private Registry as a Dependency Source

To depend on crates from your private registry, specify the registry in your `Cargo.toml`:
Expand Down
37 changes: 37 additions & 0 deletions pulp_rust/app/auth.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
"""Stub authentication for Cargo API endpoints.
This is a temporary placeholder — it validates the Authorization header against
a hardcoded token so that state-changing endpoints (publish, yank, unyank) are
not completely open. It will be replaced by proper token-based auth later.
"""

import functools
import json

from django.http import HttpResponse

STUB_TOKEN = "i_understand_that_pulp_rust_does_not_support_proper_auth_yet"


def require_cargo_token(view_method):
"""Decorator that validates the Cargo Authorization header against the stub token.
Returns a 403 with a Cargo-style JSON error if the token is missing or incorrect.
"""

@functools.wraps(view_method)
def wrapper(self, request, *args, **kwargs):
token = request.META.get("HTTP_AUTHORIZATION")
if token == STUB_TOKEN:
return view_method(self, request, *args, **kwargs)
if not token:
detail = "this endpoint requires an authorization token"
else:
detail = "invalid authorization token"
return HttpResponse(
json.dumps({"errors": [{"detail": detail}]}),
content_type="application/json",
status=403,
)

return wrapper
20 changes: 12 additions & 8 deletions pulp_rust/app/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@

logger = getLogger(__name__)

# Cache for the "dl" template from each registry's config.json.
# Keyed by index URL; effectively never changes for a given registry.
_dl_template_cache = {}


def _strip_sparse_prefix(url):
"""Strip the sparse+ prefix from a Cargo registry URL."""
Expand Down Expand Up @@ -194,9 +198,8 @@ class RustDependency(models.Model):
default="normal",
)

# @TODO: I suspect this isn't needed
# URL of alternative registry if dependency comes from a non-default registry
# Null means the dependency is from the same registry as the parent package
# URL of alternative registry if dependency comes from a non-default registry.
# Null means the dependency is from the same registry as the parent package.
registry = models.CharField(max_length=512, blank=True, null=True)

# Original crate name if the dependency was renamed
Expand Down Expand Up @@ -235,11 +238,12 @@ def get_remote_artifact_url(self, relative_path=None, request=None):
crate_name, version = _parse_crate_relative_path(relative_path)
index_url = _strip_sparse_prefix(self.url).rstrip("/")

# TODO: Cache the config.json response to avoid fetching it on every request.
config_url = f"{index_url}/config.json"
response = urllib.request.urlopen(config_url)
config = json.loads(response.read())
dl_template = config["dl"]
if index_url not in _dl_template_cache:
config_url = f"{index_url}/config.json"
response = urllib.request.urlopen(config_url, timeout=30)
config = json.loads(response.read())
_dl_template_cache[index_url] = config["dl"]
dl_template = _dl_template_cache[index_url]

if "{crate}" in dl_template or "{version}" in dl_template:
return dl_template.replace("{crate}", crate_name).replace("{version}", version)
Expand Down
6 changes: 6 additions & 0 deletions pulp_rust/app/urls.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
IndexRoot,
CargoIndexApiViewSet,
CargoDownloadApiView,
CargoMeApiView,
CargoPublishApiView,
)

Expand All @@ -15,6 +16,11 @@


urlpatterns = [
path(
CRATES_IO_URL + "me",
CargoMeApiView.as_view(),
name="cargo-me-api",
),
path(
CRATES_IO_URL + "api/v1/crates/new",
CargoPublishApiView.as_view(),
Expand Down
63 changes: 44 additions & 19 deletions pulp_rust/app/views.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import json
import logging
import os
import struct
import tempfile
import urllib.request
import urllib.error
Expand Down Expand Up @@ -29,6 +31,7 @@
RustPackageYank,
_strip_sparse_prefix,
)
from pulp_rust.app.auth import require_cargo_token
from pulp_rust.app.tasks import (
ayank_package,
aunyank_package,
Expand Down Expand Up @@ -76,7 +79,7 @@ def get_distribution(repo):
try:
return distro_qs.get(base_path=repo, pulp_domain=get_domain())
except ObjectDoesNotExist:
raise Http404(f"No RustDistribution found for base_path {repo}") # TODO: broken
raise Http404(f"No RustDistribution found for base_path {repo}")

@staticmethod
def get_repository_version(distribution):
Expand Down Expand Up @@ -180,7 +183,7 @@ def retrieve(self, request, path, **kwargs):
index_url = _strip_sparse_prefix(remote.url).rstrip("/")
upstream_url = f"{index_url}/{path}"
try:
response = urllib.request.urlopen(upstream_url)
response = urllib.request.urlopen(upstream_url, timeout=30)
return HttpResponse(response.read(), content_type="text/plain")
except urllib.error.HTTPError as e:
if e.code == 404:
Expand Down Expand Up @@ -258,6 +261,23 @@ def retrieve(self, request, repo):
return HttpResponse(json.dumps(data), content_type="application/json")


class CargoMeApiView(APIView):
"""
Auth verification endpoint for ``cargo login``.

Cargo calls GET /me after login to verify the token is valid.
See: https://doc.rust-lang.org/cargo/reference/registry-web-api.html
"""

authentication_classes = []
permission_classes = []
renderer_classes = [JSONRenderer]

@require_cargo_token
def get(self, request, **kwargs):
return HttpResponse(json.dumps({"ok": True}), content_type="application/json")


class CargoPublishApiView(APIView):
"""
View for Cargo's crate publish endpoint (PUT /api/v1/crates/new).
Expand All @@ -268,9 +288,8 @@ class CargoPublishApiView(APIView):
See: https://doc.rust-lang.org/cargo/reference/registry-web-api.html#publish
"""

# TODO: Authentication/authorization is not yet implemented.
# All users with network access can publish. In production, this should
# require a valid token and verify crate ownership.
# Authentication uses a stub token via @require_cargo_token decorator.
# TODO: Replace with proper per-user token auth and RBAC integration.
authentication_classes = []
permission_classes = []
renderer_classes = [JSONRenderer]
Expand All @@ -288,6 +307,7 @@ def _error_response(detail, status=400):
status=status,
)

@require_cargo_token
def put(self, request, **kwargs):
"""
Handle ``cargo publish`` requests.
Expand All @@ -308,7 +328,7 @@ def put(self, request, **kwargs):

try:
metadata, crate_bytes = parse_cargo_publish_body(request.body)
except Exception:
except (struct.error, json.JSONDecodeError, UnicodeDecodeError):
return self._error_response("invalid publish request body")

name = metadata.get("name")
Expand All @@ -327,17 +347,20 @@ def put(self, request, **kwargs):
tmp.write(crate_bytes)
tmp.close()

task = dispatch(
apublish_package,
exclusive_resources=[distro.repository],
immediate=True,
kwargs={
"repository_pk": str(distro.repository.pk),
"metadata": metadata,
"crate_path": tmp.name,
},
)
has_task_completed(task)
try:
task = dispatch(
apublish_package,
exclusive_resources=[distro.repository],
immediate=True,
kwargs={
"repository_pk": str(distro.repository.pk),
"metadata": metadata,
"crate_path": tmp.name,
},
)
has_task_completed(task)
finally:
os.unlink(tmp.name)

return HttpResponse(
json.dumps(
Expand All @@ -363,7 +386,7 @@ class CargoDownloadApiView(APIView):
permission_classes = []
renderer_classes = [PlainTextRenderer, JSONRenderer]

def get_full_path(self, base_path, pulp_domain=None): # TODO: replace with ApiMixin?
def get_full_path(self, base_path, pulp_domain=None):
if settings.DOMAIN_ENABLED:
domain = pulp_domain or get_domain()
return f"{domain.name}/{base_path}"
Expand Down Expand Up @@ -393,10 +416,11 @@ def get(self, request, name, version, rest, **kwargs):
relative_path = f"{name}/{name}-{version}.crate"
return self.redirect_to_content_app(distro, relative_path, request)
elif rest == "readme":
raise NotImplementedError("Readme endpoint is not yet implemented")
raise Http404("Readme endpoint is not yet implemented")
else:
raise Http404(f"Unknown action: {rest}")

@require_cargo_token
def delete(self, request, name, version, rest, **kwargs):
"""
Responds to DELETE requests for yanking crate versions.
Expand Down Expand Up @@ -436,6 +460,7 @@ def delete(self, request, name, version, rest, **kwargs):
has_task_completed(task)
return HttpResponse(json.dumps({"ok": True}), content_type="application/json")

@require_cargo_token
def put(self, request, name, version, rest, **kwargs):
"""
Responds to PUT requests for unyanking crate versions.
Expand Down
Loading