{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c85f9f25",
   "metadata": {},
   "source": [
    "# 04 · Environments, Packages, Containers and Editors\n",
    "\n",
    "Code that only runs on your machine is not finished. This module covers the practices that make Python\n",
    "projects reproducible and deployable: isolated environments, package managers, lock files, containers,\n",
    "and the editors professionals write code in.\n",
    "\n",
    "**You will learn**\n",
    "1. Why every project needs its own environment, and how `venv` works (demonstrated live)\n",
    "2. Package managers compared: `pip`, `uv`, `pipx`, `conda`/`mamba`, and the system ones (`apt`, `snap`, `brew`, `winget`)\n",
    "3. Pinning dependencies: requirements files, lock files and `pyproject.toml`\n",
    "4. Docker: packaging an application with its whole environment, and deploying it\n",
    "5. Editors and IDEs: VS Code, PyCharm, Eclipse with PyDev, Vim/Neovim, JupyterLab, Spyder\n",
    "6. A project layout and habits worth adopting from day one"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba93589f",
   "metadata": {},
   "source": [
    "## 1. Which Python am I running?\n",
    "There is rarely just one Python on a machine: the operating system's, one from a package manager, one\n",
    "per project. Always know which one runs your code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "286df74d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-17T23:23:01.761838Z",
     "iopub.status.busy": "2026-10-17T23:23:01.761503Z",
     "iopub.status.idle": "2026-10-17T23:23:01.773293Z",
     "shell.execute_reply": "2026-10-17T23:23:01.771334Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "interpreter : /home/patito/Developments/academy/python/.venv/bin/python3\n",
      "version     : 3.12.3\n",
      "prefix      : /home/patito/Developments/academy/python/.venv\n",
      "base prefix : /usr\n",
      "in a venv   : True\n"
     ]
    }
   ],
   "source": [
    "import sys, platform, site\n",
    "print(\"interpreter :\", sys.executable)\n",
    "print(\"version     :\", platform.python_version())\n",
    "print(\"prefix      :\", sys.prefix)               # the environment this interpreter belongs to\n",
    "print(\"base prefix :\", sys.base_prefix)          # the installation it was created from\n",
    "print(\"in a venv   :\", sys.prefix != sys.base_prefix)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74ab8674",
   "metadata": {},
   "source": [
    "## 2. Isolated environments with `venv`\n",
    "Project A needs `pandas 1.5`, project B needs `pandas 2.2`: installed globally, one of them breaks. A\n",
    "**virtual environment** is a folder with its own `site-packages`, so each project gets exactly the\n",
    "versions it needs and nothing leaks between them. Created live below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7cb246cb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-17T23:23:01.778232Z",
     "iopub.status.busy": "2026-10-17T23:23:01.777801Z",
     "iopub.status.idle": "2026-10-17T23:23:09.616920Z",
     "shell.execute_reply": "2026-10-17T23:23:09.614935Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "new environment's prefix: /tmp/tmpwkkc06m_/demo-env\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "packages it contains    : pip==24.0\n",
      "can it import pandas?   : False\n",
      "this kernel has pandas  : True\n"
     ]
    }
   ],
   "source": [
    "import subprocess, tempfile, os, venv\n",
    "\n",
    "env_dir = os.path.join(tempfile.mkdtemp(), \"demo-env\")\n",
    "venv.create(env_dir, with_pip=True)                         # same as: python -m venv demo-env\n",
    "py = os.path.join(env_dir, \"bin\", \"python\") if os.name != \"nt\" else os.path.join(env_dir, \"Scripts\", \"python.exe\")\n",
    "\n",
    "def in_env(*args):\n",
    "    return subprocess.run([py, *args], capture_output=True, text=True).stdout.strip()\n",
    "\n",
    "print(\"new environment's prefix:\", in_env(\"-c\", \"import sys; print(sys.prefix)\"))\n",
    "print(\"packages it contains    :\", in_env(\"-m\", \"pip\", \"list\", \"--format=freeze\") or \"(only pip)\")\n",
    "print(\"can it import pandas?   :\", in_env(\"-c\", \"import importlib.util as u; print(u.find_spec('pandas') is not None)\"))\n",
    "print(\"this kernel has pandas  :\", __import__(\"importlib.util\").util.find_spec(\"pandas\") is not None)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e33e56c9",
   "metadata": {},
   "source": [
    "The new environment is empty: it cannot see the libraries installed for this notebook. That is the point.\n",
    "Day to day, from a terminal:\n",
    "\n",
    "```bash\n",
    "python3 -m venv .venv              # create (once per project)\n",
    "source .venv/bin/activate          # activate (Windows: .venv\\Scripts\\activate)\n",
    "pip install -r requirements.txt    # install the project's dependencies\n",
    "deactivate                         # leave\n",
    "```\n",
    "\n",
    "Add `.venv/` to `.gitignore`: environments are rebuilt from the dependency list, never committed."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bec45d1",
   "metadata": {},
   "source": [
    "## 3. Package managers\n",
    "| Tool | Installs | Use it for |\n",
    "|---|---|---|\n",
    "| **pip** | Python packages from PyPI | the standard; installs into the active environment |\n",
    "| **uv** | Python packages and Python itself | a much faster drop-in for pip/venv/pip-tools, with lock files (`uv lock`, `uv sync`) |\n",
    "| **pipx** | command-line tools, each in its own hidden venv | installing tools like `ruff`, `black`, `httpie` without polluting projects |\n",
    "| **conda / mamba** | Python *and non-Python* packages (C/Fortran libraries, CUDA, R) from conda-forge | scientific stacks with heavy native dependencies, GPU work |\n",
    "| **Poetry / Hatch / PDM** | project managers built on pyproject.toml | dependency resolution, lock files, building and publishing in one tool |\n",
    "| **apt / dnf** (Linux) | system packages | the operating system's own software, compilers, system libraries |\n",
    "| **snap / flatpak** (Linux) | sandboxed applications | desktop apps and tools that bundle their own dependencies |\n",
    "| **brew** (macOS/Linux), **winget / choco** (Windows) | system packages | the same role as apt on other platforms |\n",
    "\n",
    "Good habits:\n",
    "- Never `sudo pip install`: it can break tools your operating system depends on. Use a venv or pipx.\n",
    "- Do not mix pip and conda in the same environment unless you know why; prefer one.\n",
    "- Install Python itself with your system's package manager, `pyenv`, `uv python install`, or the official installer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "aa816fbc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-17T23:23:09.622292Z",
     "iopub.status.busy": "2026-10-17T23:23:09.621779Z",
     "iopub.status.idle": "2026-10-17T23:23:10.890760Z",
     "shell.execute_reply": "2026-10-17T23:23:10.888517Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "175 packages in this kernel's environment, for example:\n",
      "  numpy      2.5.4\n",
      "  pandas     2.3.3\n",
      "  polars     2.0.0\n",
      "  duckdb     1.5.6\n",
      "  torch      2.14.1+cpu\n",
      "  tensorflow 2.21.0\n"
     ]
    }
   ],
   "source": [
    "r = subprocess.run([sys.executable, \"-m\", \"pip\", \"list\", \"--format=json\"], capture_output=True, text=True)\n",
    "import json\n",
    "pkgs = {p[\"name\"].lower(): p[\"version\"] for p in json.loads(r.stdout)}\n",
    "print(f\"{len(pkgs)} packages in this kernel's environment, for example:\")\n",
    "for name in [\"numpy\", \"pandas\", \"polars\", \"duckdb\", \"torch\", \"tensorflow\"]:\n",
    "    print(f\"  {name:<11}{pkgs.get(name, 'not installed')}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7cf93a5e",
   "metadata": {},
   "source": [
    "## 4. Pinning dependencies\n",
    "A project that worked last year may not install today: a dependency released a breaking version. Record\n",
    "what you rely on.\n",
    "\n",
    "| File | What it pins | Example |\n",
    "|---|---|---|\n",
    "| `requirements.txt` with ranges | compatible versions | `pandas>=2.2,<3` |\n",
    "| `requirements.lock` / `pip freeze` | the **exact** versions that were tested | `pandas==2.2.3` |\n",
    "| `pyproject.toml` | a package's own dependencies (module 07) | `dependencies = [\"pandas>=2.2\"]` |\n",
    "| `uv.lock`, `poetry.lock` | exact versions plus hashes for every platform | generated by the tool |\n",
    "\n",
    "Libraries declare ranges (so they combine with other libraries); applications deploy from exact locks\n",
    "(so production runs what was tested)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "64ac4ef5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-17T23:23:10.895336Z",
     "iopub.status.busy": "2026-10-17T23:23:10.894813Z",
     "iopub.status.idle": "2026-10-17T23:23:12.174810Z",
     "shell.execute_reply": "2026-10-17T23:23:12.172369Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "absl-py==2.5.1\n",
      "anyio==4.15.1\n",
      "argon2-cffi==25.1.0\n",
      "argon2-cffi-bindings==26.1.0\n",
      "arrow==1.4.0\n",
      "astropy==8.0.2\n",
      "astropy-iers-data==0.2026.10.12.1.3.27\n",
      "asttokens==3.0.2 \n",
      "... 174 lines in total\n"
     ]
    }
   ],
   "source": [
    "frozen = subprocess.run([sys.executable, \"-m\", \"pip\", \"freeze\"], capture_output=True, text=True).stdout.splitlines()\n",
    "print(\"\\n\".join(frozen[:8]), f\"\\n... {len(frozen)} lines in total\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73b08c35",
   "metadata": {},
   "source": [
    "## 5. Containers: shipping the environment, not just the code\n",
    "A virtual environment isolates Python packages; a **container** isolates everything: the operating system\n",
    "libraries, the Python version, your code and its configuration. The same image runs on your laptop, a\n",
    "colleague's machine and a cloud server, which is why most web services are deployed this way.\n",
    "\n",
    "A Dockerfile for a small web API:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6c341a26",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-17T23:23:12.179816Z",
     "iopub.status.busy": "2026-10-17T23:23:12.179350Z",
     "iopub.status.idle": "2026-10-17T23:23:12.188841Z",
     "shell.execute_reply": "2026-10-17T23:23:12.186436Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# small official base image with Python\n",
      "FROM python:3.12-slim\n",
      "\n",
      "WORKDIR /app\n",
      "# dependencies first: this layer is cached between builds while only the code changes\n",
      "COPY requirements.lock .\n",
      "RUN pip install --no-cache-dir -r requirements.lock\n",
      "\n",
      "# then the code, which changes often\n",
      "COPY . .\n",
      "# do not run as root\n",
      "USER 1000\n",
      "EXPOSE 8000\n",
      "CMD [\"uvicorn\", \"api:app\", \"--host\", \"0.0.0.0\", \"--port\", \"8000\"]\n",
      "\n",
      "services:\n",
      "  api:\n",
      "    build: .\n",
      "    ports: [\"127.0.0.1:8000:8000\"]    # only reachable from this machine (a web server proxies to it)\n",
      "    env_file: .env                     # secrets stay out of the image\n",
      "    restart: unless-stopped\n",
      "    mem_limit: 512m                    # a runaway process cannot take the whole server\n",
      "\n"
     ]
    }
   ],
   "source": [
    "dockerfile = \"\"\"\\\n",
    "# small official base image with Python\n",
    "FROM python:3.12-slim\n",
    "\n",
    "WORKDIR /app\n",
    "# dependencies first: this layer is cached between builds while only the code changes\n",
    "COPY requirements.lock .\n",
    "RUN pip install --no-cache-dir -r requirements.lock\n",
    "\n",
    "# then the code, which changes often\n",
    "COPY . .\n",
    "# do not run as root\n",
    "USER 1000\n",
    "EXPOSE 8000\n",
    "CMD [\"uvicorn\", \"api:app\", \"--host\", \"0.0.0.0\", \"--port\", \"8000\"]\n",
    "\"\"\"\n",
    "compose = \"\"\"\\\n",
    "services:\n",
    "  api:\n",
    "    build: .\n",
    "    ports: [\"127.0.0.1:8000:8000\"]    # only reachable from this machine (a web server proxies to it)\n",
    "    env_file: .env                     # secrets stay out of the image\n",
    "    restart: unless-stopped\n",
    "    mem_limit: 512m                    # a runaway process cannot take the whole server\n",
    "\"\"\"\n",
    "print(dockerfile); print(compose)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "3a690bce",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-17T23:23:12.193238Z",
     "iopub.status.busy": "2026-10-17T23:23:12.192801Z",
     "iopub.status.idle": "2026-10-17T23:23:12.374466Z",
     "shell.execute_reply": "2026-10-17T23:23:12.372827Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Docker version 29.8.1, build 4a63305\n",
      "\n",
      "docker build -t my-api .          # build the image from the Dockerfile\n",
      "docker run -p 8000:8000 my-api    # run one container\n",
      "docker compose up -d              # run the services described in compose.yaml, in the background\n",
      "docker compose logs -f api        # follow the logs\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import shutil\n",
    "docker = shutil.which(\"docker\")\n",
    "if docker:\n",
    "    print(subprocess.run([docker, \"--version\"], capture_output=True, text=True).stdout.strip())\n",
    "else:\n",
    "    print(\"docker is not installed here (normal on Colab). The commands to try on your own machine:\")\n",
    "print('''\n",
    "docker build -t my-api .          # build the image from the Dockerfile\n",
    "docker run -p 8000:8000 my-api    # run one container\n",
    "docker compose up -d              # run the services described in compose.yaml, in the background\n",
    "docker compose logs -f api        # follow the logs\n",
    "''')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf26ebba",
   "metadata": {},
   "source": [
    "**Container good practices:** start from a `slim` image; copy and install dependencies before the code so\n",
    "rebuilds are fast; run as a non-root user; keep secrets in environment variables or files mounted at run\n",
    "time, never baked into the image; set memory limits; one process per container; store data in volumes,\n",
    "not inside the container.\n",
    "\n",
    "| | virtual environment | container |\n",
    "|---|---|---|\n",
    "| isolates | Python packages | OS libraries, Python, packages, config |\n",
    "| size | tens to hundreds of MB | hundreds of MB to a few GB |\n",
    "| best for | development, scripts | deployment, services, reproducing a full stack |\n",
    "\n",
    "## 6. Editors and IDEs\n",
    "| Tool | Type | Strong points |\n",
    "|---|---|---|\n",
    "| **Visual Studio Code** | free editor with extensions | Python and Jupyter extensions, debugger, remote development over SSH and in containers, Git integration. The most used today. |\n",
    "| **PyCharm** | full IDE (free Community edition) | deep code analysis and refactoring, test runner, database tools |\n",
    "| **Eclipse + PyDev** | IDE (free) | familiar to Java teams; debugger, refactoring, Django support |\n",
    "| **Vim / Neovim** | terminal editor | available on every server, extremely fast once learned; Neovim plus a language server gives IDE features |\n",
    "| **JupyterLab** | notebook environment in the browser | exploration, data analysis, teaching (this course) |\n",
    "| **Spyder** | scientific IDE | MATLAB-like variable explorer, popular in science and engineering |\n",
    "| **Visual Studio** (Windows) | full IDE | Python workload for teams already on Microsoft tooling |\n",
    "\n",
    "Whatever you choose, learn three things in it: **run the debugger** with breakpoints, **jump to definition**,\n",
    "and **rename a symbol** across the project. Knowing basic Vim is worth it too: it is often the only editor on\n",
    "a remote server.\n",
    "\n",
    "## 7. A project layout to start from\n",
    "```\n",
    "my-project/\n",
    "├── .venv/                 # environment (not committed)\n",
    "├── src/my_project/        # the code, importable as a package\n",
    "│   ├── __init__.py\n",
    "│   └── core.py\n",
    "├── tests/                 # pytest tests (module 06)\n",
    "├── notebooks/             # exploration, not production code\n",
    "├── pyproject.toml         # metadata, dependencies, tool settings (module 07)\n",
    "├── requirements.lock      # exact versions for deployment\n",
    "├── Dockerfile             # if it is deployed as a service\n",
    "├── .gitignore             # .venv/, __pycache__/, .env, data files\n",
    "├── .env.example           # documents the settings; the real .env is never committed\n",
    "└── README.md\n",
    "```\n",
    "\n",
    "Habits that pay off: a formatter and linter (`ruff format`, `ruff check`) run automatically with\n",
    "`pre-commit`; type hints checked by `mypy` or `pyright`; tests run on every push by CI (GitHub Actions);\n",
    "secrets only in environment variables.\n",
    "\n",
    "## Exercises\n",
    "1. Create a venv for a small project, install `requests`, and write a `requirements.lock` with `pip freeze`. Delete the venv and rebuild it from the lock.\n",
    "2. Install `ruff` with `pipx` and run `ruff check` on one of your scripts.\n",
    "3. Write a Dockerfile for a script that prints the current date; build and run it."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}