Feat/enable logs for passed tests

README.md

@@ -186,17 +186,18 @@ You can use environment variable to control certain features of testomat.io
 
 
 #### Test Run configuration
-| Env variable             | What it does                                                                                                               | Examples                                                           |
-|--------------------------|----------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|
-| TESTOMATIO_TITLE         | Name of a test run to create on testomat.io                                                                                | TESTOMATIO_TITLE="Nightly Smoke Tests" pytest --testomatio report  |
-| TESTOMATIO_RUN_ID        | Id of existing test run to use for sending test results to                                                                 | TESTOMATIO_RUN_ID=98dfas0 pytest --testomatio report               |
-| TESTOMATIO_RUNGROUP_TITLE | Create a group (folder) for a test run. If group already exists, attach test run to it                                     | TESTOMATIO_RUNGROUP_TITLE="Release 2.0" pytest --testomatio report |
-| TESTOMATIO_ENV           | Assign environment to a test run, env variant of **testRunEnv** option. Has a lower precedence than **testRunEnv** option. | TESTOMATIO_ENV="linux,chrome,1920x1080" pytest --testomatio report |
-| TESTOMATIO_LABEL         | Assign labels to a test run. Labels must exist in project and their scope must be enabled for runs                         | TESTOMATIO_LABEL="smoke,regression" pytest --testomatio report     |
-| TESTOMATIO_UPDATE_CODE         | Send code of your test to Testomat.io on each run. If not enabled(default) assumes the code is pushed using **sync** command | TESTOMATIO_UPDATE_CODE=True pytest --testomatio report             |
-| TESTOMATIO_EXCLUDE_SKIPPED         | Exclude skipped tests from the report                                                                                      | TESTOMATIO_EXCLUDE_SKIPPED=1 pytest --testomatio report            |
+| Env variable                  | What it does                                                                                                                                                                     | Examples                                                           |
+|-------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|
+| TESTOMATIO_TITLE              | Name of a test run to create on testomat.io                                                                                                                                      | TESTOMATIO_TITLE="Nightly Smoke Tests" pytest --testomatio report  |
+| TESTOMATIO_RUN_ID             | Id of existing test run to use for sending test results to                                                                                                                       | TESTOMATIO_RUN_ID=98dfas0 pytest --testomatio report               |
+| TESTOMATIO_RUNGROUP_TITLE     | Create a group (folder) for a test run. If group already exists, attach test run to it                                                                                           | TESTOMATIO_RUNGROUP_TITLE="Release 2.0" pytest --testomatio report |
+| TESTOMATIO_ENV                | Assign environment to a test run, env variant of **testRunEnv** option. Has a lower precedence than **testRunEnv** option.                                                       | TESTOMATIO_ENV="linux,chrome,1920x1080" pytest --testomatio report |
+| TESTOMATIO_LABEL              | Assign labels to a test run. Labels must exist in project and their scope must be enabled for runs                                                                               | TESTOMATIO_LABEL="smoke,regression" pytest --testomatio report     |
+| TESTOMATIO_UPDATE_CODE        | Send code of your test to Testomat.io on each run. If not enabled(default) assumes the code is pushed using **sync** command                                                     | TESTOMATIO_UPDATE_CODE=True pytest --testomatio report             |
+| TESTOMATIO_EXCLUDE_SKIPPED    | Exclude skipped tests from the report                                                                                                                                            | TESTOMATIO_EXCLUDE_SKIPPED=1 pytest --testomatio report            |
 | TESTOMATIO_PUBLISH            | Publish run after reporting and provide a public URL                                                                                                                             | TESTOMATIO_PUBLISH=true pytest --testomatio report                                                          |
 | TESTOMATIO_PROCEED            | Do not finalize the run                                                                                                                                                          | TESTOMATIO_PROCEED=1 pytest --testomatio report                                                             |
+| TESTOMATIO_STACK_PASSED       | Enables logs for passed tests. Disabled by default.                                                                                                                              | TESTOMATIO_STACK_PASSED=true pytest --testomatio report                                                          |
 | TESTOMATIO_SHARED_RUN         | Report parallel execution to the same run matching it by title. If the run was created more than 20 minutes ago, a new run will be created instead.                              | TESTOMATIO_TITLE="Run1" TESTOMATIO_SHARED_RUN=1 pytest --testomatio report                                  |
 | TESTOMATIO_SHARED_RUN_TIMEOUT | Changes timeout of shared run. After timeout, shared run won`t accept other runs with same name, and new runs will be created. Timeout is set in minutes, default is 20 minutes. | TESTOMATIO_TITLE="Run1" TESTOMATIO_SHARED_RUN=1 TESTOMATIO_SHARED_RUN_TIMEOUT=10 pytest --testomatio report |
 
@@ -302,6 +303,23 @@ Executing these commands will include the tests in the same run, but as separate
 
 **Note**: Only key:value envs will be passed into tests metadata
 
+### Attach log to test
+The plugin supports manual addition of logs from the test source code. If a test has attached logs, they will be shown in Testomat.io.
+
+To attach a log, you need to use **add_log** function from pytestomatio.utils.logging module
+
+**Note**: By default logs are only displayed for failed tests. You can enable logs for passed tests using TESTOMATIO_STACK_PASSED env variable
+
+Example:
+```python
+from pytestomatio.utils.logging import add_log
+
+def test_addition():
+    add_log(message='test started', level='DEBUG')
+    value = 2+2
+    assert value == 4
+```
+
 ## Contributing
 Use python 3.12
 

pyproject.toml

@@ -13,7 +13,7 @@ version_provider = "pep621"
 update_changelog_on_bump = false
 [project]
 name = "pytestomatio"
-version = "2.10.2"
+version = "2.10.3b2"
 
 
 dependencies = [

pytestomatio/main.py

@@ -9,6 +9,7 @@
 
 from pytestomatio.utils.helper import add_and_enrich_tests, get_test_mapping, collect_tests, read_env_s3_keys
 from pytestomatio.utils.parser_setup import parser_options
+from pytestomatio.utils.logging import get_test_logs, clear_test_logs
 from pytestomatio.utils import validations
 
 from pytestomatio.testomatio.testRunConfig import TestRunConfig
@@ -28,6 +29,15 @@ def pytest_addoption(parser: Parser) -> None:
     parser_options(parser, testomatio)
 
 
+def pytest_runtest_setup(item):
+    pytest._current_item = item
+
+
+def pytest_runtest_teardown(item, nextitem):
+    pytest._current_item = None
+    clear_test_logs(item.nodeid)
+
+
 def pytest_collection(session):
     """Capture original collected items before any filters are applied."""
     # This hook is called after initial test collection, before other filters.
@@ -192,16 +202,21 @@ def pytest_runtest_makereport(item: Item, call: CallInfo):
 
     # TODO: refactor it and use TestItem setter to upate those attributes
     if call.when in ['setup', 'call']:
+        logs = get_test_logs(item.nodeid, True)
+
         if call.excinfo is not None:
             if call.excinfo.typename == 'Skipped':
                 request['status'] = 'skipped'
                 request['message'] = str(call.excinfo.value)
             else:
+                stack = '\n'.join((str(tb) for tb in call.excinfo.traceback))
                 request['message'] = str(call.excinfo.value)
-                request['stack'] = '\n'.join((str(tb) for tb in call.excinfo.traceback))
+                request['stack'] = logs + '\n' + stack
                 request['status'] = 'failed'
         else:
             request['status'] = 'passed' if call.when == 'call' else request['status']
+            if pytest.testomatio.test_run_config.stack_passed and call.when == 'call':
+                request['stack'] = logs if logs else None
 
         if hasattr(item, 'callspec'):
             request['example'] = test_item.safe_params(item.callspec.params)

pytestomatio/testomatio/testRunConfig.py

@@ -13,6 +13,7 @@ def __init__(self):
         shared_run = os.environ.get('TESTOMATIO_SHARED_RUN') in ['True', 'true', '1']
         update_code = os.environ.get('TESTOMATIO_UPDATE_CODE', False) in ['True', 'true', '1']
         exclude_skipped = os.environ.get('TESTOMATIO_EXCLUDE_SKIPPED', False) in ['True', 'true', '1']
+        stack_passed = os.environ.get('TESTOMATIO_STACK_PASSED', False) in ['True', 'true', '1']
         shared_run_timeout = os.environ.get('TESTOMATIO_SHARED_RUN_TIMEOUT', '')
         self.access_event = 'publish' if os.environ.get("TESTOMATIO_PUBLISH") else None
         self.test_run_id = run_id
@@ -26,6 +27,7 @@ def __init__(self):
         # This allows using test run title to group tests under a single test run. This is needed when running tests in different processes or servers.
         self.shared_run = shared_run
         self.shared_run_timeout = shared_run_timeout if shared_run_timeout.isdigit() else None
+        self.stack_passed = stack_passed
         self.proceed = os.getenv('TESTOMATIO_PROCEED', False)
         self.status_request = {}
         self.update_code = update_code

pytestomatio/utils/logging.py

@@ -0,0 +1,35 @@
+import pytest
+from typing import List
+
+_TEST_LOGS = {}
+
+
+def add_log(message: str, level: str = 'INFO'):
+    """
+    Attach log to the test
+    :param message: message text
+    :param level: log level
+    """
+    if hasattr(pytest, '_current_item'):
+        test_id = pytest._current_item.nodeid
+    else:
+        test_id = 'unknown'
+
+    if test_id not in _TEST_LOGS:
+        _TEST_LOGS[test_id] = []
+    _TEST_LOGS[test_id].append(f'[{level.upper()}] {message}')
+
+
+def get_test_logs(test_id: str, as_string=False) -> List[dict] or str:
+    """Receive logs attached to given test"""
+    logs = _TEST_LOGS.get(test_id, [])
+    if as_string:
+        logs = '\n'.join(logs)
+    return logs
+
+
+def clear_test_logs(test_id: str):
+    """Clear logs for given test"""
+    _TEST_LOGS.pop(test_id, None)
+
+

tests/test_testomatio/test_testRunConfig.py

@@ -26,6 +26,7 @@ def test_init_default_values(self):
                 assert config.parallel is True
                 assert config.shared_run is False
                 assert config.shared_run_timeout is None
+                assert config.stack_passed is False
                 assert config.status_request == {}
                 assert config.update_code is False
                 assert config.meta is None
@@ -40,7 +41,8 @@ def test_init_with_env_variables(self):
             'TESTOMATIO_RUNGROUP_TITLE': 'Release 2.0',
             'TESTOMATIO_UPDATE_CODE': '1',
             'TESTOMATIO_PUBLISH': '1',
-            'TESTOMATIO_EXCLUDE_SKIPPED': '1'
+            'TESTOMATIO_EXCLUDE_SKIPPED': '1',
+            'TESTOMATIO_STACK_PASSED': '1'
         }
 
         with patch.dict(os.environ, env_vars, clear=True):
@@ -55,6 +57,7 @@ def test_init_with_env_variables(self):
             assert config.group_title == 'Release 2.0'
             assert config.parallel is True
             assert config.shared_run is False
+            assert config.stack_passed is True
             assert config.update_code is True
             assert config.meta == {'linux': None, 'browser': 'chrome', '1920x1080': None}
 
@@ -78,6 +81,22 @@ def test_init_shared_run_false_variations(self, value):
             assert config.shared_run_timeout is None
             assert config.parallel is True
 
+    @pytest.mark.parametrize('value', ['True', 'true', '1'])
+    def test_init_stack_passed_true_variations(self, value):
+        """Test different true values for TESTOMATIO_STACK_PASSED"""
+        with patch.dict(os.environ, {'TESTOMATIO_STACK_PASSED': value}, clear=True):
+            config = TestRunConfig()
+
+            assert config.stack_passed is True
+
+    @pytest.mark.parametrize('value', ['False', 'false', '0', 'anything'])
+    def test_init_stack_passed_false_variations(self, value):
+        """Test different false values TESTOMATIO_STACK_PASSED"""
+        with patch.dict(os.environ, {'TESTOMATIO_STACK_PASSED': value}, clear=True):
+            config = TestRunConfig()
+
+            assert config.stack_passed is False
+
     @pytest.mark.parametrize('value', ['True', 'true', '1'])
     def test_init_update_code_true_variations(self, value):
         """Test different true values for TESTOMATIO_UPDATE_CODE"""

tests/test_utils/test_logging.py

@@ -0,0 +1,130 @@
+import pytest
+from unittest.mock import Mock, patch
+
+from pytestomatio.utils.logging import _TEST_LOGS, add_log, get_test_logs, clear_test_logs
+
+
+@pytest.fixture(autouse=True)
+def clean_logs():
+    _TEST_LOGS.clear()
+    yield
+    _TEST_LOGS.clear()
+
+
+class TestAddLog:
+    """Tests for add_log function"""
+
+    def test_add_log_with_current_item(self):
+        """Test log added when pytest._current_item is set"""
+        mock_item = Mock()
+        mock_item.nodeid = 'test_module.py::test_function'
+
+        with patch.object(pytest, '_current_item', mock_item, create=True):
+            add_log('Test message')
+
+        assert 'test_module.py::test_function' in _TEST_LOGS
+        assert _TEST_LOGS['test_module.py::test_function'] == ['[INFO] Test message']
+
+    def test_add_log_custom_level(self):
+        """Test log add with custom level"""
+        mock_item = Mock()
+        mock_item.nodeid = 'test_id'
+
+        with patch.object(pytest, '_current_item', mock_item, create=True):
+            add_log('Error message', 'ERROR')
+
+        assert _TEST_LOGS['test_id'] == ['[ERROR] Error message']
+
+    def test_add_multiple_logs(self):
+        """Test attach multiple logs to one test"""
+        mock_item = Mock()
+        mock_item.nodeid = 'test_id'
+
+        with patch.object(pytest, '_current_item', mock_item, create=True):
+            add_log('First message', 'INFO')
+            add_log('Second message', 'WARNING')
+            add_log('Third message', 'ERROR')
+
+        assert len(_TEST_LOGS['test_id']) == 3
+        assert _TEST_LOGS['test_id'] == [
+            '[INFO] First message',
+            '[WARNING] Second message',
+            '[ERROR] Third message'
+        ]
+
+    def test_add_log_to_different_tests(self):
+        """Test log attach to different tests"""
+        mock_item1 = Mock()
+        mock_item1.nodeid = 'test_1'
+
+        with patch.object(pytest, '_current_item', mock_item1, create=True):
+            add_log('Message for test 1')
+
+        mock_item2 = Mock()
+        mock_item2.nodeid = 'test_2'
+
+        with patch.object(pytest, '_current_item', mock_item2, create=True):
+            add_log('Message for test 2')
+
+        assert len(_TEST_LOGS) == 2
+        assert _TEST_LOGS['test_1'] == ['[INFO] Message for test 1']
+        assert _TEST_LOGS['test_2'] == ['[INFO] Message for test 2']
+
+
+class TestGetTestLogs:
+    """Tests for get_test_logs function"""
+
+    def test_get_logs_as_list(self):
+        """Test logs can be received as list"""
+        _TEST_LOGS['test_id'] = ['[INFO] Log 1', '[ERROR] Log 2']
+
+        logs = get_test_logs('test_id')
+
+        assert isinstance(logs, list)
+        assert logs == ['[INFO] Log 1', '[ERROR] Log 2']
+
+    def test_get_logs_as_string(self):
+        """Test logs can be received as string"""
+        _TEST_LOGS['test_id'] = ['[INFO] Log 1', '[ERROR] Log 2']
+
+        logs = get_test_logs('test_id', as_string=True)
+
+        assert isinstance(logs, str)
+        assert logs == '[INFO] Log 1\n[ERROR] Log 2'
+
+    def test_get_logs_single_log(self):
+        """Test log can be received"""
+        _TEST_LOGS['test_id'] = ['[INFO] Single log']
+
+        logs = get_test_logs('test_id', as_string=True)
+
+        assert logs == '[INFO] Single log'
+
+
+class TestClearTestLogs:
+    """Tests for clear_test_logs function"""
+
+    def test_clear_existing_logs(self):
+        """Test clearing existing logs"""
+        _TEST_LOGS['test_id'] = ['[INFO] Log 1', '[ERROR] Log 2']
+
+        clear_test_logs('test_id')
+
+        assert 'test_id' not in _TEST_LOGS
+
+    def test_clear_nonexistent_logs(self):
+        """Test clearing nonexistent logs not raise error"""
+        clear_test_logs('nonexistent_test')
+
+        assert 'nonexistent_test' not in _TEST_LOGS
+
+    def test_clear_does_not_affect_other_tests(self):
+        """Test clearing does not affect other tests"""
+        _TEST_LOGS['test_1'] = ['[INFO] Log 1']
+        _TEST_LOGS['test_2'] = ['[INFO] Log 2']
+
+        clear_test_logs('test_1')
+
+        assert 'test_1' not in _TEST_LOGS
+        assert 'test_2' in _TEST_LOGS
+        assert _TEST_LOGS['test_2'] == ['[INFO] Log 2']