Skip to content
Merged
2 changes: 1 addition & 1 deletion .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14", "3.14t"]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14", "3.14t", "3.15.0-alpha.1"]

steps:
- uses: actions/checkout@v4
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tests_and_coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
strategy:
matrix:
os: [macos-latest, ubuntu-latest, windows-latest]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14", "3.14t"]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14", "3.14t", "3.15.0-alpha.1"]

steps:
- uses: actions/checkout@v4
Expand Down
34 changes: 17 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

![logo](https://raw.githubusercontent.com/mutating/transfunctions/develop/docs/assets/logo_2.svg)

This library is designed to solve one of the most important problems in python programming - dividing all written code into 2 camps: sync and async. We get rid of code duplication by using templates.
This library is designed to solve one of the most important problems in Python programming — splitting code into two categories: sync and async. It reduces code duplication by using templates.


## Table of contents
Expand Down Expand Up @@ -79,13 +79,13 @@ You can also quickly try out this and other packages without having to install u

## The problem

Since the `asyncio` module appeared in Python more than 10 years ago, many well-known libraries have received their asynchronous alternates. A lot of the code in the Python ecosystem has been [duplicated](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself), and you probably know many such examples.
Since the `asyncio` module appeared in Python more than 10 years ago, many well-known libraries have gained asynchronous counterparts. A lot of the code in the Python ecosystem has been [duplicated](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself), and you probably know many such examples.

The reason for this problem is that the Python community has chosen a way to implement asynchrony expressed through syntax. There are new keywords in the language, such as `async` and `await`. Their use makes the code so-called "[multicolored](https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/)": all the functions in it can be red or blue, and depending on the color, the rules for calling them are different. You can only call blue functions from red ones, but not vice versa.
The reason for this problem is that the Python community has chosen a syntax-based approach to asynchrony. There are new keywords in the language, such as `async` and `await`. Their use makes the code so-called "[multicolored](https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/)": functions become “red or blue, and depending on the color, the rules for calling them are different. You can only call blue functions from red ones, but not vice versa.

I must say that implementing asynchronous calls using a special syntax is not the only solution. There are languages like Go where runtime can independently determine "under the hood" where a function should be asynchronous and where not, and choose the correct way to call it. A programmer does not need to manually "colorize" their functions there. Personally, I think that choosing a different path is the mistake of the Python community, but that's not what we're discussing here.
I must say that implementing asynchronous calls using a special syntax is not the only solution. There are languages like Go where the runtime can determine under the hood where a function should be asynchronous and where not, and choose how to execute them. A programmer does not need to manually "colorize" their functions there. Personally, I think that choosing a different path is the mistake of the Python community, but that's not what we're discussing here.

The solution offered by this library is based on templating. You can take a certain function as a template and generate several others based on it: regular, asynchronous, or generator. This allows you to avoid duplicating code where it was previously impossible. And all this without major changes in Python syntax or in the internal structure of the interpreter. We're just "sweeping under the carpet" syntax differences. Combined with the idea of context-aware functions, this makes for an even more powerful tool: `superfunctions`. This allows you to create a single function object that can be handled as you like: as a regular function, as an asynchronous function, or as a generator. The function will behave the way you use it. Thus, this library solves the problem of code duplication caused by the syntactic approach to marking asynchronous execution sections.
The solution offered by this library is based on templating. You can take a certain function as a template and generate several others based on it: regular, asynchronous, or generator. This allows you to avoid duplicating code where it was previously impossible. And all this without major changes in Python syntax or in the internal structure of the interpreter. We are essentially hiding the syntax differences. Combined with the idea of context-aware functions, this makes for an even more powerful tool: `superfunctions`. This allows you to create a single function object that can be handled as you like: as a regular function, as an asynchronous function, or as a generator. The function will behave the way you use it. Thus, this library solves the problem of code duplication caused by the syntactic approach to marking asynchronous execution sections.


## Code generation
Expand Down Expand Up @@ -163,7 +163,7 @@ async def template():
print("it's an async function!")
```

Finally, method `get_generator_function` will return a generator function that looks like this:
Finally, the method `get_generator_function` will return a generator function that looks like this:

```python
def template():
Expand All @@ -174,8 +174,8 @@ def template():

All generated functions:

- Inherit the access to global variables and closures that the original template function had.
- Сan be either ordinary stand-alone functions or bound methods. In the latter case, they will be linked to the same object.
- Inherit access to global variables and closures that the original template function had.
- Can be either ordinary standalone functions or bound methods. In the latter case, they will be linked to the same object.

There is only one known limitation: you cannot use any third-party decorators on the template using the decorator syntax, because in some situations this can lead to ambiguous behavior. If you still really need to use a third-party decorator, just generate any of the functions from the template, and then apply your decorator to the result of the generation.

Expand All @@ -202,7 +202,7 @@ async def template():
await sleep(5)
```

All markers do not need to be imported in order for the generated code to be functional: they are destroyed during the [code generation](#code-generation). However, you can do this if your linter or syntax checker in your IDE requires it:
None of the markers need to be imported in order for the generated code to be functional: they are destroyed during the [code generation](#code-generation). However, you can do this if your linter or syntax checker in your IDE requires it:

```python
from transfunctions import (
Expand Down Expand Up @@ -257,7 +257,7 @@ With the `@superfunction` decorator, you no longer need to call special methods

If you use it as a regular function, a regular function will be created "under the hood" based on the template and then called:

To call a superfunction like a regular function, you need to use a special tilde syntax:
To call a superfunction like a regular function, you need to use a special tilde-based call syntax:

```python
~my_superfunction()
Expand All @@ -282,7 +282,7 @@ list(my_superfunction())
#> so, it's a generator function!
```

How does it work? In fact, `my_superfunction` returns some kind of intermediate object that can be both a coroutine and a generator and an ordinary function. Depending on how it is handled, it lazily code-generates the desired version of the function from a given template and uses it.
How does it work? In fact, `my_superfunction` returns some kind of intermediate object that can behave as a coroutine, a generator, or a regular callable. Depending on how it is handled, it lazily code-generates the desired version of the function from a given template and uses it.

By default, a superfunction is called as a regular function using tilde syntax, but there is another mode. To enable it, use the appropriate flag in the decorator:

Expand All @@ -297,9 +297,9 @@ my_superfunction()
#> so, it's just usual function!
```

However, it is not completely free. The fact is that this mode uses a special trick with a reference counter, a special mechanism inside the interpreter that cleans up memory. When there is no reference to an object, the interpreter deletes it, and you can link your callback to this process. It is inside such a callback that the contents of your function are actually executed. This imposes some restrictions on you:
However, it comes with trade-offs. The fact is that this mode uses a special trick with a reference counter, a special mechanism inside the interpreter that cleans up memory. When there is no reference to an object, the interpreter deletes it, and you can link your callback to this process. It is inside such a callback that the contents of your function are actually executed. This imposes some restrictions on you:

- You cannot use the return values from this function in any way. If you try to save the result of a function call to a variable, the reference counter to the returned object will not reset while this variable exists, and accordingly the function will not actually be called.
- You cannot use the return values from this function in any way. If you try to save the result of a function call to a variable, the reference counter to the returned object will not drop to zero while this variable exists, and accordingly the function will not actually be called.
- Exceptions will not work normally inside this function. Rather, they can be picked up and intercepted in [`sys.unraisablehook`](https://docs.python.org/3/library/sys.html#sys.unraisablehook), but they will not go up the stack above this function. This is due to a feature of CPython: exceptions that occur inside callbacks for finalizing objects are completely escaped.

This mode is well suited for functions such as logging or sending statistics from your code: simple functions from which no exceptions or return values are expected. In all other cases, I recommend using the tilde syntax.
Expand All @@ -311,15 +311,15 @@ Typing is the most difficult problem we faced when developing this library. In m

There are 2 main difficulties in developing typing here:

- Code generation creates code in runtime that is not in the source files of your project. Whereas most type analyzers look at your code statically, at what is actually present in your files.
- Code generation creates code at runtime that is not in the source files of your project. Whereas most type analyzers look at your code statically, at what is actually present in your files.
- We mix several types of syntax in a single template function, but the static analyzer does not know that this is a template and part of the code will be deleted from here. In its opinion, this is the final function that will continue to be used in your project.

As you can see, typing in Python is not well suited for metaprogramming. However, in this project, almost all the problems with typing turned out to be solved in one way or another. The main reason why this is so is that we mostly *remove* code from functions, but hardly *add* it there during code generation. In other words, we almost never encounter the problem of how to type the *added* code. This makes the solution to most typing problems accessible. However! Unfortunately, we were not able to completely hide all the typing problems under the hood, but you should still be aware of some of them if you use `mypy` or another analyzer.
As you can see, typing in Python is not well suited for metaprogramming. However, in this project, almost all the problems with typing turned out to be solved in one way or another. The main reason why this is so is that we mostly *remove* code from functions, but hardly *add* it there during code generation. In other words, we almost never encounter the problem of how to type the *added* code. This makes the solution to most typing problems accessible. However, we were not able to completely hide all the typing problems under the hood, but you should still be aware of some of them if you use `mypy` or another analyzer.

If you use the keyword `yield from`, you need to call the function `yield_from_it` instead:

```python
from transfunctions import yield_it
from transfunctions import yield_from_it

@superfunction
def my_superfunction():
Expand All @@ -333,4 +333,4 @@ def my_superfunction():
yield_from_it([1, 2, 3])
```

The keywords `yield` or `yield from` are available to you and work perfectly, but from the point of view of a static type checker, they turn the function into a generator, which should also mean a special type annotation. By replacing this fragment with a function call, we hack it.
The keywords `yield` or `yield from` are available to you and work perfectly, but from the point of view of a static type checker, they turn the function into a generator, which should also mean a special type annotation. Replacing the syntax with a function call avoids confusing the type checker.
8 changes: 4 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,16 @@ build-backend = "setuptools.build_meta"

[project]
name = "transfunctions"
version = "0.0.10"
version = "0.0.11"
authors = [{ name = "Evgeniy Blinov", email = "zheni-b@yandex.ru" }]
description = 'Say NO to Python fragmentation on sync and async'
readme = "README.md"
requires-python = ">=3.8"
dependencies = [
'displayhooks>=0.0.5',
'getsources>=0.0.2',
'displayhooks>=0.0.6',
'getsources>=0.0.3',
'typing_extensions ; python_version <= "3.10"',
]

classifiers = [
"Operating System :: OS Independent",
'Operating System :: MacOS :: MacOS X',
Expand All @@ -29,6 +28,7 @@ classifiers = [
'Programming Language :: Python :: 3.12',
'Programming Language :: Python :: 3.13',
'Programming Language :: Python :: 3.14',
'Programming Language :: Python :: 3.15',
'Programming Language :: Python :: Free Threading',
'Programming Language :: Python :: Free Threading :: 3 - Stable',
'License :: OSI Approved :: MIT License',
Expand Down
4 changes: 2 additions & 2 deletions requirements_dev.txt
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
pytest==8.0.2
pytest==8.3.5
coverage==7.6.1
build==1.2.2.post1
twine==6.1.0
mypy==1.14.1
pytest-mypy-testing==0.1.3
ruff==0.9.9
ruff==0.14.6
mutmut==3.2.3
full_match==0.0.3
1 change: 0 additions & 1 deletion tests/typing/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +0,0 @@
# noqa: A005
24 changes: 12 additions & 12 deletions transfunctions/__init__.py
Original file line number Diff line number Diff line change
@@ -1,36 +1,36 @@
from transfunctions.decorators.superfunction import (
superfunction as superfunction, # noqa: PLC0414
superfunction as superfunction,
)
from transfunctions.decorators.transfunction import (
transfunction as transfunction, # noqa: PLC0414
transfunction as transfunction,
)
from transfunctions.errors import (
CallTransfunctionDirectlyError as CallTransfunctionDirectlyError, # noqa: PLC0414
CallTransfunctionDirectlyError as CallTransfunctionDirectlyError,
)
from transfunctions.errors import (
DualUseOfDecoratorError as DualUseOfDecoratorError, # noqa: PLC0414
DualUseOfDecoratorError as DualUseOfDecoratorError,
)
from transfunctions.errors import (
WrongDecoratorSyntaxError as WrongDecoratorSyntaxError, # noqa: PLC0414
WrongDecoratorSyntaxError as WrongDecoratorSyntaxError,
)
from transfunctions.errors import (
WrongMarkerSyntaxError as WrongMarkerSyntaxError, # noqa: PLC0414
WrongMarkerSyntaxError as WrongMarkerSyntaxError,
)
from transfunctions.errors import (
WrongTransfunctionSyntaxError as WrongTransfunctionSyntaxError, # noqa: PLC0414
WrongTransfunctionSyntaxError as WrongTransfunctionSyntaxError,
)
from transfunctions.markers import (
async_context as async_context, # noqa: PLC0414
async_context as async_context,
)
from transfunctions.markers import (
await_it as await_it, # noqa: PLC0414
await_it as await_it,
)
from transfunctions.markers import (
generator_context as generator_context, # noqa: PLC0414
generator_context as generator_context,
)
from transfunctions.markers import (
sync_context as sync_context, # noqa: PLC0414
sync_context as sync_context,
)
from transfunctions.markers import (
yield_from_it as yield_from_it, # noqa: PLC0414
yield_from_it as yield_from_it,
)
4 changes: 2 additions & 2 deletions transfunctions/decorators/superfunction.py
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ def decorator(function: Callable[FunctionParams, ReturnType]) -> Callable[Functi

if not tilde_syntax:
class NoReturns(NodeTransformer):
def visit_Return(self, node: Return) -> Optional[Union[AST, List[AST]]]: # noqa: ARG002, N802
def visit_Return(self, node: Return) -> Optional[Union[AST, List[AST]]]: # noqa: ARG002
raise WrongTransfunctionSyntaxError('A superfunction cannot contain a return statement.')
transformer.get_usual_function(addictional_transformers=[NoReturns()])

Expand All @@ -137,7 +137,7 @@ def wrapper(*args: FunctionParams.args, **kwargs: FunctionParams.kwargs) -> Usag

return wrapper

if len(args):
if args:
return decorator(args[0])

return decorator
Loading
Loading