Using virtualenv
virtualenv is a tool to create isolated
Python environments. Please refer to the virtualenv package
documentation to understand
what it does and how to install it. Virtual environments are not just
for Bots; you can use them for any other python development. That is why
I started investigating this methodology.
This guide is based on information from the following links.
https://zignar.net/2012/06/17/install-python-on-windows/
http://www.stuartellis.eu/articles/python-development-windows/
Installation
-
Install python: You have probably done this already! Make sure you have the paths of your
python27
andpython27\scripts
included in the system path variable. -
Install distribute: Download distribute_setup.py and invoke it using python.
>> python C:\Path\to\distribute_setup.py
-
Install Pip: Download get-pip.py (right click, save as) and invoke it using python.
>> python C:\Path\to\get-pip.py
-
Install Virtualenv: Once Pip is installed, installing any other package (that is available in the Python Package Index) is easy.
>> pip install virtualenv
-
Install C and C++ compilers (optional): Not all python libraries are pure python, some may contain C code that must be compiled to install successfully. Download MinGW, run it, and select the C and C++ compilers to install. When complete, edit the file
Python27\Lib\distutils\cygwincompiler.py
and remove-mno-cygwin
from lines 322-326. This flag is no longer supported. You also need to add a setting to each virtual environment that needs to use the compilers (described in the next section). Make sure you have the path of yourMinGW\bin
included in the system path variable.
Create your environments
Technically your virtual environments can be stored and scattered anywhere, but it makes sense to keep them all grouped together. I suggest you create a "root" directory for environments, each environment will be a subdirectory, eg.
D:\> mkdir PythonEnv
D:\> cd PythonEnv
D:\PythonEnv>
Create as many environments as you need (for ease of use, keep environment names short but meaningful and without spaces), eg. bots310
D:\PythonEnv> virtualenv bots310
New python executable in bots310\Scripts\python.exe
Installing Setuptools...........................................................
............................done.
Installing Pip..................................................................
.......................done.
D:\PythonEnv>
If you installed compilers in the previous section and want to use them
for this environment, then edit Lib\distutils\distutils.cfg
within the
environment folder, and add
[build]
compiler=mingw32
Activate and deactivate environments
To activate an environment, use the activate command in it's script directory. Notice your command prompt changes to show the active environment in brackets. Only one environment can be "activated" at a time, in order to install modules etc.
D:\PythonEnv> bots310\scripts\activate
(bots310) D:\PythonEnv>
To deactivate the current environment, use the deactivate command. Notice your command prompt changes back to show no active environment in brackets.
(bots310) D:\PythonEnv> deactivate
D:\PythonEnv>
Optional; create an activate.bat file in your environment root directory. This gives you a shortcut to activate environments.
REM activate.bat gives you a shortcut to activate python environments
REM eg. activate bots310
call "%1\scripts\activate"
Install Bots in a virtual environment
First, activate the required environment.
Install Bots and dependencies using pip (don't use the Bots Windows
installer, because it installs to the default python folder!)
Install Bots from local downloaded .tar.gz file
(bots310) D:\PythonEnv> pip install .\bots-3.1.0.tar.gz
Unpacking d:\pythonenv\bots-3.1.0.tar.gz
Running setup.py egg_info for package from file:///d7C%5Cpythonenv%5Cbots-3.1.0.tar.gz
Installing collected packages: bots
Running setup.py install for bots
Successfully installed bots
Cleaning up...
(bots310) D:\PythonEnv>
Install Django (version 1.4.x)
(bots310) D:\PythonEnv>pip install Django==1.4.6
Downloading/unpacking Django==1.4.6
Downloading Django-1.4.6.tar.gz (7.7MB): 7.7MB downloaded
Running setup.py egg_info for package Django
Installing collected packages: Django
Running setup.py install for Django
Successfully installed Django
Cleaning up...
(bots310) D:\PythonEnv>
Install cherrypy (latest)
(bots310) D:\PythonEnv> pip install cherrypy
Downloading/unpacking cherrypy
Downloading CherryPy-3.2.4.tar.gz (424kB): 424kB downloaded
Running setup.py egg_info for package cherrypy
Installing collected packages: cherrypy
Running setup.py install for cherrypy
Successfully installed cherrypy
Cleaning up...
(bots310) D:\PythonEnv>
Install Genshi (optional, required for template-html output)
(bots310) D:\PythonEnv> pip install Genshi
Downloading/unpacking Genshi
You are installing a potentially insecure and unverifiable file. Future versions of pip will default to disallowing insecure files.
Downloading Genshi-0.7.tar.gz (491kB): 491kB downloaded
Running setup.py egg_info for package Genshi
warning: no files found matching 'COPYING' under directory 'doc'
warning: no previously-included files matching '*' found under directory 'doc\logo.lineform'
warning: no previously-included files found matching 'doc\2000ft.graffle'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
Installing collected packages: Genshi
Running setup.py install for Genshi
building 'genshi._speedups' extension
C:\MinGW\bin\gcc.exe -mdll -O -Wall -ID:\Python27\include -ID:\PythonEnv\bots310\PC -c genshi/_speedups.c -o build\temp.win32-2.7\Release\genshi\_speedups.o
C:\MinGW\bin\gcc.exe -shared -s build\temp.win32-2.7\Release\genshi\_speedups.o build\temp.win32-2.7\Release\genshi\_speedups.def -LD:\Python27\Libs -LD:\PythonEnv\bots310\libs -LD:\PythonEnv\bots310\PCbuild -lpython27 -lmsvcr90 -obuild\lib.win32-2.7\genshi\_speedups.pyd
warning: no files found matching 'COPYING' under directory 'doc'
warning: no previously-included files matching '*' found under directory 'doc\logo.lineform'
warning: no previously-included files found matching 'doc\2000ft.graffle'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
Successfully installed Genshi
Cleaning up...
(bots310) D:\PythonEnv>
Install cdecimal (optional, improves performance)
This will not install/compile correctly on Windows using pip, and the
installer is an msi (not exe) so easy_install won't work either. You
can install it manually though; the two files needed are cdedimal.pyd
and cdecimal-2.3-py2.7.egg-info
and they go in your virtual
environment's site-packages
directory. There are two ways to get these
files.
- Install cdecimal in the default python folder (eg.
C:\python27\lib\site-packages
) using the windows msi installer, then copy the two files to your virtual environment. - Extract the files from the windows msi installer using a tool such as universal extractor.
Install pycrypto and paramiko (optional, required for sftp channels)
These will not install/compile correctly on Windows using pip. Instead, I used easy_install with a downloaded Windows installer exe.
(bots310) C:\PythonEnv>easy_install pycrypto-2.1.0.win32-py2.7.exe
Processing pycrypto-2.1.0.win32-py2.7.exe
creating 'c:\docume~1\adadmi~3\locals~1\temp\1\easy_install-wy9qt4\pycrypto-2.1.
0-py2.7-win32.egg' and adding 'c:\docume~1\adadmi~3\locals~1\temp\1\easy_install
-wy9qt4\pycrypto-2.1.0-py2.7-win32.egg.tmp' to it
Moving pycrypto-2.1.0-py2.7-win32.egg to c:\pythonenv\bots310\lib\site-packages
Adding pycrypto 2.1.0 to easy-install.pth file
Installed c:\pythonenv\bots310\lib\site-packages\pycrypto-2.1.0-py2.7-win32.egg
Processing dependencies for pycrypto==2.1.0
Finished processing dependencies for pycrypto==2.1.0
(bots310) C:\PythonEnv>easy_install paramiko-1.7.6.win32.exe
Processing paramiko-1.7.6.win32.exe
creating 'c:\docume~1\adadmi~3\locals~1\temp\1\easy_install-mwtlnu\paramiko-1.7.
6-py2.7-win32.egg' and adding 'c:\docume~1\adadmi~3\locals~1\temp\1\easy_install
-mwtlnu\paramiko-1.7.6-py2.7-win32.egg.tmp' to it
Moving paramiko-1.7.6-py2.7-win32.egg to c:\pythonenv\bots310\lib\site-packages
Adding paramiko 1.7.6 to easy-install.pth file
Installed c:\pythonenv\bots310\lib\site-packages\paramiko-1.7.6-py2.7-win32.egg
Processing dependencies for paramiko==1.7.6
Finished processing dependencies for paramiko==1.7.6
Start Bots Webserver in the virtual environment
You can simply start the webserver manually from commandline. The variable %VIRTUAL_ENV% contains the path to the activated environment. Using the start command causes a new console window to be opened.
(bots310) D:\PythonEnv> start python %VIRTUAL_ENV%\scripts\bots-webserver.py
Alternatively you can modify the environment's activate command (it is a
small batch file of about 25 lines). Edit
%VIRTUAL_ENV%\scripts\activate.bat
and add the above command to the
end. Then every time you activate the environment, the webserver is
started too. Same for jobqueueserver (if required). I also add a window
title. eg.
start "webserver (bots310)" python %VIRTUAL_ENV%\scripts\bots-webserver.py
Running multiple Bots environments concurrently
Yes, this is possible!
Although only a single environment can be "activated", once they are
created you can run multiple bots webservers and engines simultaneously
from different environments, if configured with different ports.
bots.ini - use different ports for each environment. eg.
[settings]
#port used to assure only one instance of bots-engine is running. default: 28081
port = 28091
[webserver]
#port at which at bots-gui is server. default is 8080
port = 8090
[jobqueue]
# Port to use for the job queue xmlrpc server (on localhost). Default: 28082
port = 28092
settings.py - add new setting for the session cookie name, and use a different name for each environment. This allows simultaneous login to each environment from the same browser. eg. use the environment name (default is 'sessionid')
#*********sessions, cookies, log out time*************************
SESSION_COOKIE_NAME = 'bots310'