Python is a versatile programming language you can use to create web applications, desktop applications, scripts, and many other types of programs. It is also a popular language for scientific computing, data analysis, artificial intelligence, and more. Basically, you can do almost anything with it.

AlmaLinux is a production-grade enterprise operating system that is binary-compatible with Red Hat Enterprise Linux. It was initially released on March 2021 by CloudLinux to provide a community-supported spiritual successor to CentOS Linux.

In this tutorial, you'll set up a programming environment for Python 3 on your AlmaLinux server.

Installing Python on AlmaLinux 9 - Prerequisites

Before you install Python on AlmaLinux, you need:

• Basic knowledge of the Linux command line.
• An AlmaLinux 9 server with a non-root user with sudo privileges. You can get affordable, and powerful AlmaLinux servers from our website, and you can check out our How to access your server using SSH guide to learn how to access your server and create a sudo user.

Step 1 - Updating your System

Before installing Python on Alamalinux, you first need to update the packages to the latest available versions using the following command:

sudo dnf update -y

Once the update is done, you should see the following output:

Step 2 - Installing Python on AlmaLinux 9

To install Python 3.12 on your AlmaLinux server, first install the packages necessary to build Python 3.12 using the following command:

sudo dnf install gcc openssl-devel bzip2-devel libffi-devel zlib-devel wget make tar -y

You should see the following output:

Next, download the Python 3.12 archive package from the official Python website with wget:

wget https://www.python.org/ftp/python/3.12.3/Python-3.12.3.tar.xz

Once the download finishes, extract the archive package using the tar command:

tar -xf Python-3.12.3.tar.xz

A new Python-3.12.3 directory will appear, navigate to it using cd:

cd Python-3.12.3

Run the configuration file to check that all dependencies are available on your system:

./configure --enable-optimizations

This check will take a few minutes. Once it finishes, use the make command to start building Python:

make -j 2

Note: the -j flag sets the number of CPU cores the make command will use, it is recommended to use all the cores that are available in your server to speed up the build process, use the nproc command to get the number of cores you have on your server, and use that number in the command above.

Once the build process finishes, run the following command to install the Python package on your system:

sudo make install

Now that Python 3.12 is installed on your AlmaLinux server, you can check your Python version by running the following command:

python3 -V

You'll receive your Python version:

With this, you have Python installed on your AlmaLinux 9 server and you can now create a virtual environment.

Step 3 - Creating a Python Project Under a Virtual Environment

Python virtual environments allow you to have multiple isolated Python environments on one computer. This is useful for projects that require different versions of Python, or for projects that you don't want to contaminate with code from other projects. This ensures that each Python project is isolated from your other projects, so that conflict between dependencies does not occur.

To create a Python programming environment, create a project directory somewhere in your system:

mkdir myproject

Navigate to your directory:

cd myproject

Then use the python -m venv command line to create a new virtual environment inside your project folder:

python3 -m venv env

This will create a new directory called env, which contains the dependencies you install while your virtual environment is activated. Here, we call this virtual environment directory env, which is a name commonly used for Python virtual environment directories, but you can change it to another name if you wish.

Note: You can create as many virtual environments as you want, and it is highly recommended to create a virtual environment for each new Python project.

To use this new environment and isolate your code and dependencies from the main AlmaLinux system's Python installation, you need to activate it with the following command. Change env with the name of your virtual environment folder:

source env/bin/activate

Your command line should now have an (env) prefix indicating that your virtual environment is activated.

To install a Python package using pip, you can use the following command:

pip install package

With package being the name of the Python package you wish to install.

For example, to install the Django web framework, you can use the following command:

pip install django

With this, you now have a Python programming environment ready, and you can now manage your Python packages inside isolated virtual environments.

Step 4 - Creating a Small Python Program

With your virtual environment activated inside your project folder, you can now write Python code and install dependencies, and run them in an isolated box without interfering with the Python system installation on your AlmaLinux server.

To edit files, you will need to first install the nano editor:

sudo dnf install nano -y

Once nano is installed on your system. You can now use it to edit Python files.

Let's create a small Python program called hi.py that displays a Hello, World! text:

nano hi.py

Type the following code into it:

print("Hello, World!")

Save and close the file by pressing CTRL + X, then Y and ENTER.

Now, run the program using the python command:

python hi.py

You should receive the following output:

Hello, World!

Note: To deactivate your Python programming environment, use the following command:

deactivate

With this, you now have a working programming environment for your Python projects.

Congrats!

In this tutorial, you've learned how to install Python on AlmaLinux, how to create a programming environment to isolate your Python projects, and how to install Python packages in it. You can learn more about Python by going through the official Python tutorial.

Without virtual environments, working with Python is a nightmare! In case of Ubuntu, the base operating system comes with Python preinstalled, and you might think that that\'s a nice thing. Except, that version of Python is old and pip is not even installed. So you install pip only to find out that it is detecting and trying to update Python programs that are part of the base operating system and are supposed to be managed by your operating system's package manager.

This same pattern of unavailability and conflicts gets worse over time as you start working on multiple Python projects. Dependencies of one might conflict with the other. You may require a specific version of request package for one project and a different version for another, and on and on. This frustration is well-depicted in this XKCD Comic:

This post tries to address all these problems, and then some more. Obviously, most of this is aimmed at Linux users, and is tested on a Ubuntu 20.04 LTS server, but the principles can be generalized for macOS and Windows 10 too.

Goals

The goal is to achieve the following with regards to our Python 3 environment:

• Isolating it from the broader Operating system.
• Making it easy to configure and backup
• Making it easily upgradable
• Using methods recommended by the Official Documentation

We will be using venv module which is the recommended way to create virtual environments

Python venv to the rescue

In Python 3.6 the problem of managing multiple dependencies has received the cleanest resolution -- Virtual Environments or venv. In my opinion, this has rendered a lot of third party options like conda, pyenv, etc less relevant, if not completely redundant.

venv is an official Python module that is meant for running an isolated Python environment. This isolated environment can include Python interpreter, pip, and setuptools. All these dependencies are packaged into a user-specified directory.

Creating a virtual environment or venv

To get started, let\'s install venv on our system first:

$ sudo apt install python3-venv

Let's create a virtual environment next. I will name this virtual environment, my-venv and it will be created in the user\'s home directory for this specific case.

$ python3 -m venv $HOME/my-venv

For Ubuntu 20.04, we had to use python3 in the command instead of just python because python word is reserved for Python version 2. Same is true for macOS, but if you\'ve installed Python on Windows then you can just use python to invoke Python 3 interpreter.

Activating and using a virtual environment or venv

To start using the virtual environment, we need to import a script located at $HOME/my-venv/bin into our SHELL environment. This script will be add the virtual environment\'s directory to your shell\'s PATH environment such that when python or pip or other packages are searched for the SHELL will search the virtual environment\'s directory first search the my-venv directory to look for appropriate python program.

Depending on your SHELL the script you need to invoke will be different. For bash, and zsh interpreters the script is named activate:

root@node-00:~# source ~/my-venv/bin/activate
(my-venv) root@node-00:~#

For fish shell the script is called activate.fish and there is a similar script for tsch in the same directory. For Windows 10, the scripts will have a .bat and .ps1 extensions for command prompt and powershell respectively. Use the command echo $SHELL to know what shell your system is using.

Notice in the above shell, once the venv is activated the prompt changed. It also has a copy of pip install by default. You can verify this and even update it:

$ pip --version
pip 20.0.2 from /root/my-venv/lib/python3.8/site-packages/pip (python 3.8)

To list all the packages installed, and to update them just do the following:

$ pip list
Package       Version
------------- -------
pip           20.0.2
pkg-resources 0.0.0
setuptools    44.0.0

$ pip install --upgrade pip pkg-resources setuptools
$ pip list
Package       Version
------------- -------
pip           21.0.1
pkg-resources 0.0.0
setuptools    56.0.0

This won\'t update your system\'s copy of pip or any other pip package you have installed system-wide. The version of Python being used is the same as the Python command used to create your virtual environment.

You can install any number of pip packages inside this environment and use it for your Python projects. As long as the environment is active, the Python interpreter will prioritize these packages when your scripts call for a specific module, regardless of where you are in your filesystem. Once you are done working on the project, and want to switch gears simply type in deactivate and the virtual environment will be disabled:

(my-venv) root@node-00:~# deactivate
root@node-00:~#

Notice that the prompt is back to being normal. You can always reactivate the venv by running the source command along with the activate script. If your environment is broken simply remove the directory to get rid of it. Make sure that your Python project's file don't live in the same directory as the virtual environment.

Note on Updating Packages and Portability

Once you have installed necessary packages, it is always important to keep a snapshot of all the packages and their version numbers. This allows you to maintain and upgrade things gradually without the risk of things breaking down.

$ pip freeze > requirements.txt

By convention the requirements.txt file is used to store all the packages. Here's a typical output:

click==7.1.2
cycler==0.10.0
joblib==1.0.1
kiwisolver==1.3.1
matplotlib==3.4.1
nltk==3.6.1
numpy==1.20.2
Pillow==8.2.0
pyparsing==2.4.7
python-dateutil==2.8.1
regex==2021.4.4
six==1.15.0
textblob==0.15.3
tqdm==4.60.0

It also allows you to:

• Recreate the environment quickly and exactly, even if the underlying packages update in the future. Reinstall the same packages using the following command:

  $ pip install -r requirements.txt

• Keep track of the dependencies with your version control like git or GitHub.
• Reuse the file for other parts of your CI/CD like when creating a Docker image for your file.

That\'s it for Python environments! Happy computing!

Before you get started with your journey of Machine Learning, it is important to understand the underlying technology that you will be using. In most cases, this will involve using TensorFlow with Python.

ML workloads are quite compute intensive, this is why TensorFlow is written in C++. The Python interface is meant for you to use TensorFlow more effectively and easily without getting bogged down by C++ implementations details. Similar interfaces exist for JavaScript and a few other languages.

We will be using Python for our setup. Also, TensorFlow comes with a GPU specific implementation meant to take advantage of Nvidia GPUs, to keep things simple we will ignore that as well.

SSDNodes TensorFlow Template!

If you are already comfortable with it, or want to jump straight into running Python snippets, try out one of our TensorFlow templates at SSDNodes which will get you started with a Ubuntu 18.04 environment preconfigured with Jupyter Labs and TensorFlow. There are a lot of advantages in offloading your ML workload to our cloud, a few include:  

1. Blazing fast internet connection to download huge sets of training data.
2. Tonnes of memory and compute at low low prices! It will burn through the data before you are done with your coffee break.
3. Freedom to break things. If you screwup, just click reinstall and you have a clean slate to start over again!

Python and its packaging problem

Python is one of the easiest languages to pickup, with readable syntax and projects like TensorFlow that have real world use cases you would be hard pressed to find a better language to begin your journey into computing.

However, some of its success has resulted in a lot of similar projects and confusion surrounding them. Some unintended consequences. Below are a few:

1. Incompatible versions like Python 3 and Python 2
2. Multiple revisions of Python 3 and the differences between them.
3. Setups having multiple installations of Python. For example, from Anaconda, miniconda, base OS, etc.
4. Differently named package managers, e.g, pip and pip3.
5. Obsolete versions of pip being shipped by OS vendors.

In this post, we would explore the optimum balance between stability and having the most cutting edge version of Python/Pip possible. 

This is tested on Ubuntu 18.04 & Debian 10 but similar principles can be applied to other Linux distros.  

Virtual Environments

To begin with, we will stick to the version Python3 that gets shipped with your OS. In case of Ubuntu 18.04, this is going to be version 3.6 at the time of this writing. Next we need to minimize our footprint on system-wide installations. We use Virtual Environments for this.

Virtual environments help us install Pip, and other Python packages inside a single local directory, and easily  use it for our scripts, instead of cluttering your entire system with unknown/conflicting packages.  

Like everything in Python community there are a myriad of implementations of this idea like virtualenv, pyenv, venv. But we will just use venv, which is officially supported and very easy to use.

To achieve this we will need only one extra package. On Debian-based systems this would be python3-venv and we won’t need any version of system-wide pip, TensorFlow or any other Python specific package.

$ sudo apt-install python3-venv

Using Virtual Environments

1. To create a virtual environment, pick a name for your virtual environment, like newenv, and simply use the command:
   $ python3 -m venv newenv
   This command will create a directory newenv inside your current directory. I usually create one inside the directory of my project, but you can create it anywhere, and use it anywhere.
2. To activate it, simply run:
   $ source ./newenv/bin/source
For Windows and other Operating systems the command might be slightly different, but the effect will be still the same. The prompt will change to show you are in a different environment, i.e, (newenv) $
Once that is done, you can start using it.
3. Now you can use it, you will notice that now you have pip for managing Python packages, even if you don’t have it installed system-wide:
   (newenv)$ pip3 --version
pip 18.1 from /home/r/newenv/lib/python3.7/site-packages/pip (python 3.7)

4. You can upgrade pip, install TensorFlow and install TensorFlow as:
   (newenv)$ python3 -m pip --upgrade install pip
(newenv)$ pip install tensorflow jupyterlab 
   With the virtual environment activated you can Python scripts using the simple python yourscript.py and they will make use of packages inside the newenv folder.
5. And once you are done with it, you can simply deactivate the environment:
   (newenv)$ deactivate
Delete the newenv directory to get rid of all the libraries, pacakges and other cruft that was installed.

Tip

Before deleting a virtual environment, activate it and run $ pip freeze > requirements.txt and grab the list of all the packages (with proper version number) into requirements.txt file.

Later can install the packages into a new environment by simply running:
(newenv)$ pip install -r requirements.txt
This would prevent your application from breaking because of any significant changes in the packages or their API.

Conclusion

The name of the game is to type less, get organized, and get more done. Everything from requirements.txt to virtual environments helps you get rid of unnecessary setup. Learn about your setup only once and rinse and repeat!