20210419_docker_python.rst (5967B)
1 Local python repository 2 ####################### 3 4 :date: 2021-04-26 07:55 5 :category: Offlining 6 :author: Louis Holbrook 7 :tags: docker,python,devops 8 :slug: docker-offline-2-python 9 :summary: Feeding python packages to your offline Docker setup 10 :series: Offline Docker 11 :seriesprefix: docker-offline 12 :seriespart: 2 13 :lang: en 14 :status: published 15 16 17 .. 18 .. CAUTION:: 19 20 This is a purely technical article. You should probably be `this geeky <https://g33k.holbrook.no/c18beedd>`_ to continue reading. 21 22 In the previous part of this series we were able to connect a Docker network to a virtual interface on our host, neither of which have access to the internet. That means we are ready to host content for the container locally. And we will start out with creating a local Python repository. 23 24 Packaging the packages 25 ====================== 26 27 I'll be so bold as to assume that you are using ``pip`` to manage your packages. It gives you not only the option to *install* packages, but merely *download* them to storage aswell. So let's do that and try to serve the packages. 28 29 30 .. code-block:: bash 31 32 $ pip download faker 33 [...] 34 Successfully downloaded faker python-dateutil six text-unidecode 35 $ ls 36 Faker-8.1.0-py3-none-any.whl python_dateutil-2.8.1-py2.py3-none-any.whl six-1.15.0-py2.py3-none-any.whl text_unidecode-1.3-py2.py3-none-any.whl 37 $ python -m RangeHTTPServer 38 39 .. code-block:: bash 40 41 $ python -m venv .venv 42 $ . .venv/bin/activate 43 (.venv) $ pip install --index http://localhost:8000 faker 44 Looking in indexes: http://localhost:8000 45 ERROR: Could not find a version that satisfies the requirement faker (from versions: none) 46 ERROR: No matching distribution found for faker 47 48 Dag nabbit. Apparently not that simple. And indeed, if we read up on the spec for `PEP 503 spec for Simple Repository API`_, we learn that we are going to stick those package files under directories named after the packages. 49 50 Well, nothing that a bit of bash scripting can't sort out: 51 52 .. include:: code/docker-offline-python/pep503.sh 53 :code: bash 54 55 56 Armed with this, let's try again: 57 58 .. code-block:: bash 59 60 $ sh /home/lash/bin/shell/pep503.sh . packages 61 $ mkdir packages 62 $ bash /home/lash/bin/shell/pep503.sh . packages 63 mkdir: created directory 'packages/text-unidecode' 64 './text_unidecode-1.3-py2.py3-none-any.whl' -> 'packages/text-unidecode/text_unidecode-1.3-py2.py3-none-any.whl' 65 mkdir: created directory 'packages/six' 66 './six-1.15.0-py2.py3-none-any.whl' -> 'packages/six/six-1.15.0-py2.py3-none-any.whl' 67 mkdir: created directory 'packages/python-dateutil' 68 './python_dateutil-2.8.1-py2.py3-none-any.whl' -> 'packages/python-dateutil/python_dateutil-2.8.1-py2.py3-none-any.whl' 69 mkdir: created directory 'packages/faker' 70 './Faker-8.1.0-py3-none-any.whl' -> 'packages/faker/Faker-8.1.0-py3-none-any.whl' 71 $ find packages/ -type f 72 packages/faker/Faker-8.1.0-py3-none-any.whl 73 packages/python-dateutil/python_dateutil-2.8.1-py2.py3-none-any.whl 74 packages/six/six-1.15.0-py2.py3-none-any.whl 75 packages/text-unidecode/text_unidecode-1.3-py2.py3-none-any.whl 76 k $ python -m RangeHTTPServer 77 78 .. code-block:: bash 79 80 $ pip install --index http://localhost:8000/packages faker 81 Looking in indexes: http://localhost:8000/packages 82 Collecting faker 83 [...] 84 Successfully installed faker-8.1.0 python-dateutil-2.8.1 six-1.15.0 text-unidecode-1.3 85 86 87 Extra prepping 88 ============== 89 90 There are some basic packages you will most always need, and which ``pip`` often will expect to find in at least one of its available repositories, even regardless of whether its installed or not. If you don't have this on your local offline repository when the internet goes *poof*, then that will block any build you are trying to do. 91 92 Let's make sure we have the packages around all the time: 93 94 .. code-block:: bash 95 96 $ pip download pip setuptools setuptools-markdown wheel 97 [...] 98 $ bash /home/lash/bin/shell/pep503.sh . packages 99 [...] 100 101 102 Choosing a server 103 ================= 104 105 As I tend to favor the classics, I still use ``Apache Web Server`` to host stuff to my local environment. One practical (if not all too safe) thing about it is that it will automatically bind to all interfaces. So to make the repository available, we simply link or add the ``packages`` directory to the virtual root, and restart the server e.g. with 106 107 .. code-block:: bash 108 109 systemctl restart httpd 110 111 Of course you can use any HTTP server you like, as long as you know how to bind it to the virtual interface. 112 113 To verify that the service is providing what's needed, simply point your web browser to the location, e.g. 114 115 .. code-block:: bash 116 117 lynx http://10.1.2.1/packages/ 118 119 Serving the packages 120 ==================== 121 122 Now that we are all prepped, the next step is to do install packages from within a Docker container. 123 124 Let's start with an Archlinux base image with basic python provisions. 125 126 .. include:: code/docker-offline-python/Dockerfile.pythonarch 127 :code: docker 128 129 Build and tag it with ``pythonbase``, and then let's add the Dockerfile to test the repository: 130 131 .. code-block:: docker 132 133 [...] 134 135 RUN pip install --index http://10.1.2.1/packages --trusted-host 10.1.2.1 faker 136 137 138 Now, turn internet off (and lights, too, if you'd like some extra suspense), and build the second file. 139 140 .. code-block:: bash 141 142 $ docker build . 143 Sending build context to Docker daemon 3.072kB 144 [...] 145 Successfully installed faker-8.1.0 python-dateutil-2.8.1 six-1.15.0 text-unidecode-1.3 146 Removing intermediate container 2c10ffcdf3ad 147 ---> 1ba83bb8e111 148 Successfully built 1ba83bb8e111 149 150 151 .. 152 153 .. _PEP 503 spec for Simple Repository API: https://www.python.org/dev/peps/pep-0503