|
| 1 | +****************************** |
| 2 | +Where to put your custom code? |
| 3 | +****************************** |
| 4 | + |
| 5 | +A suggestion for how to manage your personal library of python functions you might use for scripting, data analysis, etc. |
| 6 | + |
| 7 | +TL; DR |
| 8 | +====== |
| 9 | + |
| 10 | +If you have a collection of your own code you want access to for various projects: |
| 11 | + |
| 12 | +Make a "package" out of it so you can manage it in one place, and use it in other places. |
| 13 | + |
| 14 | +Introduction |
| 15 | +------------ |
| 16 | + |
| 17 | +Many folks find they have a collection of little scripts and utilities that they want to be able to use and re-use for various projects. It is really NOT a good idea to simply copy and paste these around for use with each project -- you will regret that! |
| 18 | + |
| 19 | +It is also not a good idea to use the ``PYTHONPATH`` environment variable to set up a directory in which to dump stuff. (Google a bit if you want to know why). |
| 20 | + |
| 21 | +The best way to do this with Python is to use the python package mechanism. |
| 22 | + |
| 23 | +A Python "package" is a collection of modules and scripts -- we usually think of these as something carefully developed for particular purpose and distributed to a wide audience for re-use -- the packages you can install with pip. Indeed that is the case, but the "collection of modules and scripts" part can be used for your own code that no one else is ever going to touch, and the overhead is small if you use it only this way. |
| 24 | + |
| 25 | +Step by step: |
| 26 | +------------- |
| 27 | + |
| 28 | +1) Create a directory in your user (or home, or ... ) dir for your code. Let's call it "my_code". |
| 29 | + |
| 30 | +2) This is going to seem odd, but create another dir with the same name inside that -- this is where the actual code goes. (it's a convention to name the top dir the same thing as the "package" name, but it doesn't matter. But the inner name does -- that is the name of your package. |
| 31 | + |
| 32 | +3) In that dir, put in an empty, for now, file called ``__init__.py``. |
| 33 | + |
| 34 | +4) In the outer dir, put in a file (we'll fill it in later) called ``setup.py``. |
| 35 | + |
| 36 | +So you shoudl have:: |
| 37 | + |
| 38 | + my_code |
| 39 | + my_code |
| 40 | + __init__.py |
| 41 | + setup.py |
| 42 | + |
| 43 | +The inner my_code dir is now a python "package" -- any directory with a __init__.py file is a package. But how to use it? |
| 44 | + |
| 45 | +The setup.py file is where you specify for python how this package is setup. You can do a lot in there, and if you ever want to share your code with anyone else, you should follow: |
| 46 | + |
| 47 | +https://packaging.python.org/tutorials/distributing-packages/. |
| 48 | + |
| 49 | +But for now, we are going to make it as *simple* as possible:: |
| 50 | + |
| 51 | + from setuptools import setup |
| 52 | + |
| 53 | + setup(name='my_code', |
| 54 | + packages=['my_code'], |
| 55 | + ) |
| 56 | + |
| 57 | +That's it -- really! There is a lot you can do here to support multiple packages, scripts, etc, but this is enough to get you started. |
| 58 | + |
| 59 | +Putting in your code |
| 60 | +-------------------- |
| 61 | + |
| 62 | +Now put some code in there! |
| 63 | + |
| 64 | +Create a file for your code, and put it in the inner my_code dir: |
| 65 | + |
| 66 | +``some_code.py``:: |
| 67 | + |
| 68 | + #!/usr/bin/env python |
| 69 | + |
| 70 | + """ |
| 71 | + Just an example, but this could be a collection of utility functions, etc. |
| 72 | + |
| 73 | + Here would be documentation of what's in this file |
| 74 | + |
| 75 | + In this case, just one function to make sure it works. |
| 76 | + """ |
| 77 | + |
| 78 | + def test_fun(): |
| 79 | + print("yup -- this worked!!") |
| 80 | + |
| 81 | +OK -- now you have a (useless) package with some code in it - how to use it? |
| 82 | + |
| 83 | +To use this package, you need to "install" it into the python environment that you want to use. Some of us have a single python install -- maybe Anaconda's root environment, or the python.org python installer, or.... Some of us use virtualenv or conda environments. In any case, get yourself into that environment with a command line in the outer my_code dir, and type:: |
| 84 | + |
| 85 | + python setup.py develop |
| 86 | + |
| 87 | +"develop" is like install, but instead of copying the code into the python environment, it adds it to the python search path (only that particular python instances search path!) so you can import it, but it will always be importing the current version of the files if you change things. |
| 88 | + |
| 89 | +Now you can fire up python (or ipython, or a Jupyter notebook, or write code in a script, or...) and do:: |
| 90 | + |
| 91 | + from my_code import some_code |
| 92 | + |
| 93 | + In [2]: from my_code import some_code |
| 94 | + |
| 95 | + In [3]: some_code.test_fun() |
| 96 | + |
| 97 | + yup -- this worked!! |
| 98 | + |
| 99 | +And you are good to go! |
| 100 | + |
| 101 | +Here is a zip file of my simple example package: :download:`my_code.zip <../code/my_code.zip>` |
| 102 | + |
| 103 | + |
| 104 | +NOTES: |
| 105 | +------ |
| 106 | + |
| 107 | +If you have only a little bit of code, you can do all this with a single module, rather than a package, and have an easier import. But I think most folks have enough stuff that it's better to have multiple modules with related stuff in them. |
| 108 | + |
| 109 | +If you have more than a few modules, it would probably make sense to keep them in separate packages, organized by functionality. |
| 110 | + |
| 111 | +This is only the very simplest way to do it what you really SHOULD do is be more formal about the process: |
| 112 | + - Do some versioning of the package |
| 113 | + - Keep it in source code version control system (like git, etc) |
| 114 | + |
| 115 | +and others... |
| 116 | + |
| 117 | +Look up "Software Carpentry" for many more ideas how better to manage your software for Science. |
| 118 | + |
| 119 | + |
0 commit comments