Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* @RomanR-dev
29 changes: 29 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Python Tests with Tox

on:
pull_request:
branches: [ master ]

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.11']

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install tox
run: |
python -m pip install --upgrade pip
pip install tox tox-gh-actions

- name: Run tox
run: tox
137 changes: 78 additions & 59 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,101 +1,120 @@
## python-logging-loki-v2
# 🚀 python-logging-loki-v2

# [Based on: https://github.com/GreyZmeem/python-logging-loki.]
===================
> Modern Python logging handler for Grafana Loki

[![PyPI version](https://img.shields.io/pypi/v/python-logging-loki-v2.svg)](https://pypi.org/project/python-logging-loki-v2/)
[![Python version](https://img.shields.io/badge/python-3.6%20%7C%203.7%20%7C%203.8-blue.svg)](https://www.python.org/)
[![License](https://img.shields.io/pypi/l/python-logging-loki.svg)](https://opensource.org/licenses/MIT)
[![Python](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/)

[//]: # ([![Build Status](https://travis-ci.org/GreyZmeem/python-logging-loki.svg?branch=master)](https://travis-ci.org/GreyZmeem/python-logging-loki))
Send Python logs directly to [Grafana Loki](https://grafana.com/loki) with minimal configuration.

Python logging handler for Loki.
https://grafana.com/loki
---

New
===========
0.4.0: support to headers (ability to pass tenants for multi tenant loki configuration)
## ✨ Features

- 📤 **Direct Integration** - Send logs straight to Loki
- 🔐 **Authentication Support** - Basic auth and custom headers
- 🏷️ **Custom Labels** - Flexible tagging system
- ⚡ **Async Support** - Non-blocking queue handler included
- 🔒 **SSL Verification** - Configurable SSL/TLS settings
- 🎯 **Multi-tenant** - Support for Loki multi-tenancy

---

## 📦 Installation

Installation
============
```bash
pip install python-logging-loki-v2
```

Usage
=====
---

## 🎯 Quick Start

### Basic Usage

```python
import logging
import logging_loki


handler = logging_loki.LokiHandler(
url="https://my-loki-instance/loki/api/v1/push",
tags={"application": "my-app"},
url="https://loki.example.com/loki/api/v1/push",
tags={"app": "my-application"},
auth=("username", "password"),
version="2",
verify_ssl=True
version="2"
)

logger = logging.getLogger("my-logger")
logger = logging.getLogger("my-app")
logger.addHandler(handler)
logger.error(
"Something happened",
extra={"tags": {"service": "my-service"}},
)
logger.info("Application started", extra={"tags": {"env": "production"}})
```

Example above will send `Something happened` message along with these labels:
- Default labels from handler
- Message level as `serverity`
- Logger's name as `logger`
- Labels from `tags` item of `extra` dict
### Async/Non-blocking Mode

The given example is blocking (i.e. each call will wait for the message to be sent).
But you can use the built-in `QueueHandler` and` QueueListener` to send messages in a separate thread.
For high-throughput applications, use the queue handler to avoid blocking:

```python
import logging.handlers
import logging_loki
from multiprocessing import Queue


queue = Queue(-1)
handler = logging.handlers.QueueHandler(queue)
handler_loki = logging_loki.LokiHandler(
url="https://my-loki-instance/loki/api/v1/push",
tags={"application": "my-app"},
auth=("username", "password"),
version="2",
verify_ssl=True
handler = logging_loki.LokiQueueHandler(
Queue(-1),
url="https://loki.example.com/loki/api/v1/push",
tags={"app": "my-application"},
version="2"
)
logging.handlers.QueueListener(queue, handler_loki)

logger = logging.getLogger("my-logger")
logger = logging.getLogger("my-app")
logger.addHandler(handler)
logger.error(...)
logger.info("Non-blocking log message")
```

Or you can use `LokiQueueHandler` shortcut, which will automatically create listener and handler.
---

```python
import logging.handlers
import logging_loki
from multiprocessing import Queue
## ⚙️ Configuration Options

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `url` | `str` | *required* | Loki push endpoint URL |
| `tags` | `dict` | `{}` | Default labels for all logs |
| `auth` | `tuple` | `None` | Basic auth credentials `(username, password)` |
| `headers` | `dict` | `None` | Custom HTTP headers (e.g., for multi-tenancy) |
| `version` | `str` | `"1"` | Loki API version (`"0"`, `"1"`, or `"2"`) |
| `verify_ssl` | `bool` | `True` | Enable/disable SSL certificate verification |

handler = logging_loki.LokiQueueHandler(
Queue(-1),
url="https://my-loki-instance/loki/api/v1/push",
tags={"application": "my-app"},
auth=("username", "password"),
version="2",
verify_ssl=True
---

## 🏷️ Labels

Logs are automatically labeled with:
- **severity** - Log level (INFO, ERROR, etc.)
- **logger** - Logger name
- **Custom tags** - From handler and `extra={"tags": {...}}`

```python
logger.error(
"Database connection failed",
extra={"tags": {"service": "api", "region": "us-east"}}
)
```

logger = logging.getLogger("my-logger")
logger.addHandler(handler)
logger.error(...)
---

## 🔐 Multi-tenant Setup

```python
handler = logging_loki.LokiHandler(
url="https://loki.example.com/loki/api/v1/push",
headers={"X-Scope-OrgID": "tenant-1"},
tags={"app": "my-app"}
)
```

---
Based on [python-logging-loki](https://github.com/GreyZmeem/python-logging-loki) by GreyZmeem.

### Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

---
3 changes: 1 addition & 2 deletions logging_loki/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# -*- coding: utf-8 -*-

from logging_loki.handlers import LokiHandler
from logging_loki.handlers import LokiQueueHandler
from logging_loki.handlers import LokiHandler, LokiQueueHandler

__all__ = ["LokiHandler", "LokiQueueHandler"]
__version__ = "0.3.1"
Expand Down
15 changes: 5 additions & 10 deletions logging_loki/emitter.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,8 @@
import functools
import logging
import time

from logging.config import ConvertingDict
from typing import Any
from typing import Dict
from typing import List
from typing import Optional
from typing import Tuple
from typing import Any, Dict, List, Optional, Tuple

import requests
import rfc3339
Expand All @@ -31,7 +26,7 @@ class LokiEmitter(abc.ABC):
label_replace_with = const.label_replace_with
session_class = requests.Session

def __init__(self, url: str, tags: Optional[dict] = None, auth: BasicAuth = None, headers: Optional[dict] = None, verify_ssl: bool = True):
def __init__(self, url: str, tags: dict | None = None, auth: BasicAuth = None, headers: dict | None = None, verify_ssl: bool = True):
"""
Create new Loki emitter.

Expand All @@ -52,7 +47,7 @@ def __init__(self, url: str, tags: Optional[dict] = None, auth: BasicAuth = None
#: Verfify the host's ssl certificate
self.verify_ssl = verify_ssl

self._session: Optional[requests.Session] = None
self._session: requests.Session | None = None

def __call__(self, record: logging.LogRecord, line: str):
"""Send log record to Loki."""
Expand Down Expand Up @@ -118,7 +113,7 @@ def build_payload(self, record: logging.LogRecord, line) -> dict:
labels = self.build_labels(record)
ts = rfc3339.format_microsecond(record.created)
stream = {
"labels" : labels,
"labels": labels,
"entries": [{"ts": ts, "line": line}],
}
return {"streams": [stream]}
Expand Down Expand Up @@ -154,7 +149,7 @@ class LokiEmitterV2(LokiEmitterV1):
Enables passing additional headers to requests
"""

def __init__(self, url: str, tags: Optional[dict] = None, auth: BasicAuth = None, headers: dict = None):
def __init__(self, url: str, tags: dict | None = None, auth: BasicAuth = None, headers: dict = None):
super().__init__(url, tags, auth, headers)

def __call__(self, record: logging.LogRecord, line: str):
Expand Down
30 changes: 7 additions & 23 deletions logging_loki/handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,11 @@

import logging
import warnings
from logging.handlers import QueueHandler
from logging.handlers import QueueListener
from logging.handlers import QueueHandler, QueueListener
from queue import Queue
from typing import Dict
from typing import Optional
from typing import Type
from typing import Dict, Type

from logging_loki import const
from logging_loki import emitter
from logging_loki import const, emitter


class LokiQueueHandler(QueueHandler):
Expand All @@ -19,7 +15,7 @@ class LokiQueueHandler(QueueHandler):
def __init__(self, queue: Queue, **kwargs):
"""Create new logger handler with the specified queue and kwargs for the `LokiHandler`."""
super().__init__(queue)
self.handler = LokiHandler(**kwargs) # noqa: WPS110
self.handler = LokiHandler(**kwargs)
self.listener = QueueListener(self.queue, self.handler)
self.listener.start()

Expand All @@ -31,21 +27,9 @@ class LokiHandler(logging.Handler):
`Loki API <https://github.com/grafana/loki/blob/master/docs/api.md>`_
"""

emitters: Dict[str, Type[emitter.LokiEmitter]] = {
"0": emitter.LokiEmitterV0,
"1": emitter.LokiEmitterV1,
"2": emitter.LokiEmitterV2
}

def __init__(
self,
url: str,
tags: Optional[dict] = None,
auth: Optional[emitter.BasicAuth] = None,
version: Optional[str] = None,
headers: Optional[dict] = None,
verify_ssl: bool = True
):
emitters: Dict[str, Type[emitter.LokiEmitter]] = {"0": emitter.LokiEmitterV0, "1": emitter.LokiEmitterV1, "2": emitter.LokiEmitterV2}

def __init__(self, url: str, tags: dict | None = None, auth: emitter.BasicAuth | None = None, version: str | None = None, headers: dict | None = None, verify_ssl: bool = True):
"""
Create new Loki logging handler.

Expand Down
36 changes: 36 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,39 @@
[build-system]
requires = ["setuptools >= 40.0.0"]
build-backend = "setuptools.build_meta"


[tool.ruff]
line-length = 200

[tool.ruff.lint]
# Enable flake8-style rules and more
# You can customize this based on wemake-python-styleguide rules you want
select = [
"E", # pycodestyle errors
"W", # pycodestyle warnings
"F", # pyflakes
"I", # isort
"N", # pep8-naming
"UP", # pyupgrade
"B", # flake8-bugbear
"C4", # flake8-comprehensions
"SIM", # flake8-simplify
]

# Ignore specific rules if needed
ignore = [
"UP009", # UTF-8 encoding declaration is unnecessary
"UP035", # typing.X is deprecated, use x instead
"UP006", # Use x instead of X for type annotation
"UP030", # Use implicit references for positional format fields
"UP032", # Use f-string instead of format call
"UP015", # Unnecessary mode argument
"UP045", # New union syntax
"B019", # Use of functools.lru_cache on methods can lead to memory leaks
"B028", # No explicit stacklevel keyword argument found
]

[tool.ruff.format]
# Use double quotes (ruff default, similar to black)
quote-style = "double"
8 changes: 4 additions & 4 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,21 @@

import setuptools

with open("README.md", "r") as fh:
with open("README.md", encoding="utf-8") as fh:
long_description = fh.read()

setuptools.setup(
name="python-logging-loki-v2",
version="0.4.4",
description="Python logging handler for Grafana Loki, with support to headers.",
version="1.0.0",
description="Python logging handler for Grafana Loki",
long_description=long_description,
long_description_content_type="text/markdown",
license="MIT",
author="Roman Rapoport",
author_email="cryos10@gmail.com",
url="https://github.com/RomanR-dev/python-logging-loki",
packages=setuptools.find_packages(exclude=("tests",)),
python_requires=">=3.6",
python_requires=">=3.11",
install_requires=["rfc3339>=6.1", "requests"],
classifiers=[
"Development Status :: 4 - Beta",
Expand Down
2 changes: 1 addition & 1 deletion tests/test_emitter_v0.py
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ def test_session_is_closed(emitter_v0):
emitter(create_record(), "")
emitter.close()
session().close.assert_called_once()
assert emitter._session is None # noqa: WPS437
assert emitter._session is None


def test_can_build_tags_from_converting_dict(emitter_v0):
Expand Down
Loading