介绍

Jupyter Notebook 是一个开源的交互式计算环境，最初是作为IPython项目的一部分开发的。它提供了一个基于网页的界面，允许用户创建和共享包含代码、文本、图像、公式和可视化内容的文档。

Anaconda直接安装与Docker安装

安装环境

centos 7.9
docker 24.0.7

Anaconda安装

Anaconda下载地址

1
2
3
wget https://repo.anaconda.com/archive/Anaconda3-2023.09-0-Linux-x86_64.sh
chmod +x Anaconda3-2023.09-0-Linux-x86_64.sh
./Anaconda3-2023.09-0-Linux-x86_64.sh

配置环境变量

1
2
3
4
5
6
7
8
vim ~/.bash_profile
# 在文件末尾追加
# /opt/anaconda3是你安装Anaconda的路径，可替换
export ANACONDA=/opt/anaconda3
export PATH=$ANACONDA/bin:$PATH

# 使用:wq保存文件后,刷新配置
source ~/.bash_profile

启动Jupyter Notebook

生成jupyter 配置文件

1
2
3
4
5
jupyter notebook --generate-config
# 修改jupyter的工作路径
vim ~/.jupyter/jupyter_notebook_config.py
# 将c.NotebookApp.notebook_dir的值修改为你的工作路径
c.NotebookApp.notebook_dir = '/opt/jupyter'
1
2
# 监听所有可用的网络接口（0.0.0.0），并指定端口号为 8888，同时关闭默认的 Web 浏览器打开选项（--no-browser）
jupyter notebook --port=8888 --ip=0.0.0.0 --no-browser

获取到jupyter的web

1
2
3
4
[I 15:21:48.611 NotebookApp] Jupyter Notebook 6.5.4 is running at:
[I 15:21:48.611 NotebookApp] http://localhost:8888/?token=xxxxxxxxxxxxxxx
[I 15:21:48.611 NotebookApp]  or http://127.0.0.1:8888/?token=xxxxxxxxxxxxxxx
[I 15:21:48.611 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

Docker构建jupyterlab

镜像可指定python的版本，tensorflow的版本。cuda与cudnn使用本地已经安装好的库，请确保已经安装了显卡相关驱动，cuda与cudnn等，并安装nvidia-container-toolkit，配置daemon.json文件相关runtime设置。

编译python脚本

setup.python.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
#!/usr/bin/env bash
set -xe

source ~/.bashrc
VERSION=$1
PYTHON=$2

# Install Python packages for this container's version
yum -y update
yum -y install epel-release
yum -y install gcc openssl-devel bzip2-devel libffi-devel wget make sqlite-devel
yum clean all

wget https://www.python.org/ftp/python/$PYTHON/Python-$PYTHON.tgz
tar xzf Python-$PYTHON.tgz
cd Python-$PYTHON
./configure --enable-optimizations
make altinstall
cd ..
rm -rf Python-$PYTHON Python-$PYTHON.tgz

# Setup links for TensorFlow to compile.
# Referenced in devel.usertools/*.bazelrc
ln -sf /usr/local/bin/$VERSION /usr/bin/python3
# ln -sf /usr/local/bin/$VERSION /usr/bin/python

# Python 3.10 include headers fix:
# sysconfig.get_path('include') incorrectly points to /usr/local/include/python
# map /usr/include/python3.10 to /usr/local/include/python3.10
if [[ ! -f "/usr/local/include/$VERSION" ]]; then
  ln -sf /usr/include/$VERSION /usr/local/include/$VERSION
fi

# Install pip
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python3 get-pip.py
python3 -m pip install --no-cache-dir --upgrade pip
rm -f get-pip.py

安装jupyterlab脚本

setup.jupyter.sh

1
2
3
4
5
6
7
8
9
10
11
#!/bin/bash
pip uninstall urllib3
pip install urllib3==1.26.7
jupyter serverextension enable --py jupyter_http_over_ws

mkdir /.local
chmod a+rwx /.local

yum remove -y wget

python3 -m ipykernel.kernelspec

安装jupyterlab所需

jupyter.requirements.txt

1
2
3
4
jupyterlab
matplotlib
jupyter_http_over_ws
pysqlite3

构建image

jupyterlab.Dockerfile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
FROM centos:centos7 as base
ENV LANG C.UTF-8
ENV TZ "Asia/Shanghai"

ARG PYTHON_VERSION=python3.7
ARG P_VERSION=3.7.10
ARG TENSORFLOW_PACKAGE=tensorflow==2.6.5
COPY setup.python.sh /setup.python.sh
RUN /setup.python.sh $PYTHON_VERSION $P_VERSION
RUN pip install --no-cache-dir ${TENSORFLOW_PACKAGE} -i https://pypi.tuna.tsinghua.edu.cn/simple
RUN echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64' >> ~/.bashrc
RUN source ~/.bashrc
WORKDIR /workdir
EXPOSE 8888

FROM base as jupyterlab
COPY jupyter.requirements.txt /jupyter.requirements.txt
COPY setup.jupyter.sh /setup.jupyter.sh
RUN python3 -m pip install --no-cache-dir -r /jupyter.requirements.txt -U
RUN /setup.jupyter.sh

CMD ["bash", "-c", "source ~/.bashrc && jupyter lab --notebook-dir=/workdir --ip 0.0.0.0 --no-browser --allow-root"]

生成image

1
2
3
4
5
6
7
chmod +x setup.python.sh setup.jupyter.sh
docker build --target=jupyterlab --build-arg TENSORFLOW_PACKAGE=tensorflow==2.6.5 --build-arg PYTHON_VERSION=python3.7 --build-arg P_VERSION=3.7.10 -t jupyterlab:latest -f jupyterlab.Dockerfile .
# --target 选择FROM构建镜像
# --build-arg dockerfile中参数
# -t 生成镜像名
# -f 镜像文件名
# . 当前目录

运行Container

1
2
3
docker run -it --gpus all --rm -v /usr/local/cuda-11.2/:/usr/local/cuda -p 8888:8888 --name jupyterlab jupyterlab:latest
# --gpus 需要对docker的daemon.json文件中配置"default-runtime": "nvidia"
# -v 指定当前系统中的cuda目录

参考

build/tensorflow_runtime_dockerfiles at master · tensorflow/build (github.com)

kubernetes - Jetson Orin NX nvidia-container-runtime not utilizing GPU - Stack Overflow