Notebook / Extensions / Q & A
===
> #ipython, jupyter notebook, python notebook, jupyterlab, elyra
###### tags: `Jupyter`
###### tags: `Jupyter`, `JupyterLab`, `Notebook`, `Elyra`, `Extension`

<br>

[TOC]

<br>

## 透過 jupyter 指令，顯示當前安裝的 extension
> 資料來源：[Installing Extensions](https://jupyterlab.readthedocs.io/en/1.2.x/user/extensions.html#installing-extensions)

$ jupyter labextension list
JupyterLab v3.2.4
/home/tj/miniconda3/envs/jupyterlab-ext/share/jupyter/labextensions
        myextension v0.1.0 enabled OK
        myextension2 v0.1.0 enabled OK

<br>

## 透過 jupyter 指令，安裝本地端的 extension
> 資料來源：[Installing Extensions](https://jupyterlab.readthedocs.io/en/1.2.x/user/extensions.html#installing-extensions)

# $ jupyter labextension install myextension
myextension$ jupyter labextension install .
Building jupyterlab assets (development)

myextension$ jupyter labextension list
JupyterLab v3.2.4
/home/tj/miniconda3/envs/jupyterlab-ext/share/jupyter/labextensions
        myextension v0.1.0 enabled OK
        myextension2 v0.1.0 enabled OK

Other labextensions (built into JupyterLab)
   app dir: /home/tj/miniconda3/envs/jupyterlab-ext/share/jupyter/lab


   local extensions:
        myextension: /home/tj/Asus/workplace/jupyter/myextension

<br>

## 透過 jlpm 指令，匯入相依的本地端 extension

myextension2 $ jlpm add ../myextension
yarn add v1.21.1
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
warning "@jupyterlab/application > @jupyterlab/ui-components@3.2.5" has unmet peer dependency "react@^17.0.1".
warning "@jupyterlab/application > @lumino/coreutils@1.11.1" has unmet peer dependency "crypto@1.0.1".
warning "@jupyterlab/builder > @jupyterlab/buildutils > verdaccio > clipanion@3.1.0" has unmet peer dependency "typanion@*".
warning Workspaces can only be enabled in private projects.
warning Workspaces can only be enabled in private projects.
[4/4] Building fresh packages...
success Saved lockfile.
warning Workspaces can only be enabled in private projects.
warning Workspaces can only be enabled in private projects.
success Saved 2 new dependencies.
info Direct dependencies
└─ myextension@0.1.0
info All dependencies
├─ @jupyterlab/launcher@3.2.5
└─ myextension@0.1.0
Done in 94.56s.

- `$ jlpm add`
> Installs a package and any packages that it depends on.
- `$ jlpm run`
> Runs a defined package script.
- `$ jlpm run build`
> jlpm run build:lib && jlpm run build:labextension:dev
- `$ jlpm run watch`
> run-p watch:src watch:labextension

<br>

## [佈署方式] pip 套件安裝後的形式
> 底下以 `pip install jupyterlab-drawio` 為例
- `<your_virtualenv>/lib/python3.9/site-packages/jupyterlab-drawio`
> 包含 `labextension` 資料夾

    ├── __init__.py
    ├── labextension
    │   ├── package.json
    │   ├── schemas
    │   │   └── jupyterlab-drawio
    │   │       ├── package.json.orig
    │   │       └── settings.json
    │   └── static
    │       ├── *.js
    │       ├── *.gif
    │       ├── *.png
    │       ├── *.js.LICENSE.txt
    │       ├── remoteEntry.*.js
    │       └── style.js
    └── _version.py

    5 directories, 22 files

- `share/jupyter/labextensions/jupyterlab-drawio`
> 多了 `install.json`

    ├── install.json
    ├── package.json
    ├── schemas
    │   └── jupyterlab-drawio
    │       ├── package.json.orig
    │       └── settings.json
    └── static
        ├── *.js
        ├── *.gif
        ├── *.png
        ├── *.js.LICENSE.txt
        ├── remoteEntry.*.js
        └── style.js

    3 directories, 19 files

<br>

## [佈署方式] jupyter labextension install xxx.tgz
- `<your_virtualenv>`

    ├── lib/python3.9/site-packages/xxx.egg-link
    └── share/jupyter/lab
        ├── static/node_modules_xxx_lib_index_js.<hash>.js
        ├── static/vendors-node_modules_myextension_style_index_js-node_modules_xxx_style_index_js-node-<hash>.<hash>.js
        ├── staging/node_modules/xxx/ (見下面清單)
        ├── staging/build/node_modules_xxx_lib_index_js.<hash>.js
        ├── staging/build/vendors-node_modules_myextension_style_index_js-node_modules_xxx_style_index_js-node-<hash>.<hash>.js
        └── extensions/xxx-0.1.0.tgz

- `xxx.egg-link`

        $ cat my_extension.egg-link
        /home/tj/Asus/workplace/jupyter/my_extension

- `<your_virtualenv>share/jupyter/lab/staging/node_modules/xxx`

    ├── lib
    │   ├── index.d.ts
    │   └── index.js
    ├── LICENSE
    ├── package.json
    ├── README.md
    └── style
        ├── base.css
        ├── index.css
        └── index.js

    2 directories, 8 files

<br>

## [佈署方式] jupyter labextension install # 當前目錄
> 將當前開發目錄打包成 .tgz 並安裝
> 效果同：[`jupyter labextension install xxx.tgz`](#佈署方式-jupyter-labextension-install-xxxtgz)

<br>

## 更新 nodejs 到最新
- [How can I update my nodeJS to the latest version?](https://askubuntu.com/questions/426750/)

    $ sudo npm cache clean -f
    $ sudo npm install -g n
    $ sudo n stable

- 檢查版本(2021/12/15)

    $ node -v
    v17.1.0

- 檔案位置
- /home/tj/.nvm/versions/node/v14.16.1/bin/node
- /home/tj/.nvm/versions/node/v14.16.1/bin/npm

### 第 2 種解法(可work)

# - nodejs & npm: used to build elyra
#   @see https://nodejs.org/en/download/
#
#   - Troubleshooting
#     Can't upgrade nodejs to latest, it's still 8.10
ENV PATH="/usr/local/node-v16.15.0-linux-x64/bin:${PATH}"
RUN apt remove -y nodejs \
  && wget -q https://nodejs.org/dist/v16.15.0/node-v16.15.0-linux-x64.tar.xz \
  && tar -xf node-v16.15.0-linux-x64.tar.xz -C /usr/local \
  && rm node-v16.15.0-linux-x64.tar.xz \
  && node -v \
  && npm -v \
  && jupyter lab build

- user: root
- Linux Binaries (x64) (64bit) 下載來源
https://nodejs.org/en/download/
- :warning: **cannot upgrade**

    # node -v
    v8.10.0

    # apt install nodejs --upgrade
    Reading package lists... Done
    Building dependency tree       
    Reading state information... Done
    nodejs is already the newest version (8.10.0~dfsg-2ubuntu0.4).
    0 upgraded, 0 newly installed, 0 to remove and 52 not upgraded.

    # node -v
    v8.10.0

### 第 3 種解法(可work) :+1: :100:

RUN apt install curl \
  && curl -sL https://deb.nodesource.com/setup_16.x | bash - \
  && apt install -y nodejs \
  && node -v \
  && npm -v \
  && jupyter lab build

- [安裝最新版Node.js & NPM於Ubuntu 20.04 LTS(PPA安裝)](https://blog.impochun.com/how-to-install-latest-nodejs-on-ubuntu/)
- [Installing Node.js and npm from NodeSource](https://linuxize.com/post/how-to-install-node-js-on-ubuntu-20-04/#installing-nodejs-and-npm-from-nodesource)

<br>

## 更新 npm 到最新
- [How to update npm](https://stackoverflow.com/questions/23393707/)

    $ sudo npm install npm -g

- 檢查版本(2021/12/15)

    $ npm -v
    8.1.2

- 檔案位置
- /home/tj/.nvm/versions/node/v14.16.1/bin/node
- /home/tj/.nvm/versions/node/v14.16.1/bin/npm

<br>

## 如何在 Windows 啟用 jlpm watch ?
[](https://i.imgur.com/L2t9f6v.png)
- 參考資料
- [enable Windows user to create symbolic links](https://www.scivision.dev/windows-symbolic-link-permission-enable/)
> Put Windows into Developer Mode: Settings → Updates & Security → For Developers → Use developer features: Select “Developer mode”

<br>

## build failed due to an out-of-memory error

$ jupyter lab build
[LabBuildApp] JupyterLab 3.2.3
[LabBuildApp] Building in /opt/conda/share/jupyter/lab
[LabBuildApp] Building jupyterlab assets (production, minimized)
Build failed.
Troubleshooting: If the build failed due to an 
out-of-memory error, you may be able to fix it by
disabling the `dev_build` and/or `minimize` options.

If you are building via the `jupyter lab build` command, 
you can disable these options like so:

jupyter lab build --dev-build=False --minimize=False

You can also disable these options for all JupyterLab 
builds by adding these lines to a Jupyter config file 
named `jupyter_config.py`:

c.LabBuildApp.minimize = False
c.LabBuildApp.dev_build = False

If you don't already have a `jupyter_config.py` file, 
you can create one by adding a blank file of that name 
to any of the Jupyter config directories.
The config directories can be listed by running:

jupyter --paths

Explanation:

- `dev-build`: This option controls whether a `dev` or a more streamlined
`production` build is used. This option will default to `False` (i.e., the
`production` build) for most users. However, if you have any labextensions
installed from local files, this option will instead default to `True`.
Explicitly setting `dev-build` to `False` will ensure that the `production`
build is used in all circumstances.

- `minimize`: This option controls whether your JS bundle is minified
during the Webpack build, which helps to improve JupyterLab's overall
performance. However, the minifier plugin used by Webpack is very memory
intensive, so turning it off may help the build finish successfully in
low-memory environments.

An error occurred.
RuntimeError: JupyterLab failed to build
See the log file for details:  /tmp/jupyterlab-debug-9hv6cjng.log

<br>

## build failed
### ModuleNotFoundError

An error occurred.
ModuleNotFoundError: There is no labextension at 
/home/tj/Asus/workplace/jupyter/demo/jupyterlab-extension/my_test1.
Errors encountered: [
  ModuleNotFoundError(
    "No module named '/home/tj/Asus/workplace/jupyter/demo/jupyterlab-extension/my_test1'")]

### ERROR: "watch:labextension"

Building extension in .
node:internal/modules/cjs/loader:353
      const err = new Error(
                  ^

Error: Cannot find module 'my_test1/lib/index.js'. 
Please verify that the package.json has a valid "main" entry
...
An error occurred.
subprocess.CalledProcessError: Command 
'['node', 
'my_test1/node_modules/@jupyterlab/builder/lib/build-labextension.js', 
'--core-path', 
'lib/python3.8/site-packages/jupyterlab/staging', 
'--watch', 'my_test1', '--development']' returned non-zero exit status 1.
ERROR: "watch:labextension" exited with 1.

1. 補上底下檔案
- `{package_name}/`
- `__init__.py`
- `_version.py`
2. 再重新 build
3. 再執行 watch, 不能略過 build 直接跳到 watch(?)

<br>

## 關於 package.json 資訊
- ### [[wiki] Packages/1.1](http://wiki.commonjs.org/wiki/Packages/1.1)
- **Package Directory Layout**
A CommonJS package will observe the following:
- A package.json file must be in the top level directory
- Binary files should be in the "bin" directory,
- Javascript code should be under the "lib" directory
- ==**Documentation**== should be under the "doc" directory
- Unit tests should be under the "test" directory
- ### [package.json | Yarn](https://classic.yarnpkg.com/en/docs/package-json)
- ### [[Day-5] 用Yarn取代npm加速開發](https://ithelp.ithome.com.tw/articles/10191745)
1. 何謂 Yarn
2. Yarn 的運做模式
3. npm 與 Yarn 指令比較
- ### [Yarn - 解決 npm 痛點](https://ithelp.ithome.com.tw/articles/10206917)
- Yarn 是一個新的套件管理工具

<br>

## 關於 yarn.lock 資訊
用於詳細定義套件的資訊
i.e. [套件相依的低階描述](https://ithelp.ithome.com.tw/articles/10206917)，以確保安裝在每台機器上的 node_modules 結果是一致的

執行 `$ jlpm add @jupyterlab/filebrowser` 後，yarn.lock 的變化：

+"@jupyterlab/filebrowser@^3.2.5":
+  version "3.2.5"
+  resolved "https://registry.yarnpkg.com/@jupyterlab/filebrowser/-/filebrowser-3.2.5.tgz#d17395e443930eabbe85954482deabb0134e3b01"
+  integrity sha512-wAZI1JmX9PklEQoK4aqHYnhzV59mqEXdcfK+/fNQBizhn6gTzTEMBePyn/zyhzz0fsZ7h0KK8dc1Tcjb+pbYEQ==
+  dependencies:
+    "@jupyterlab/apputils" "^3.2.5"
+    "@jupyterlab/coreutils" "^5.2.5"
+    "@jupyterlab/docmanager" "^3.2.5"
+    "@jupyterlab/docregistry" "^3.2.5"
+    "@jupyterlab/services" "^6.2.5"
+    "@jupyterlab/statedb" "^3.2.5"
+    "@jupyterlab/statusbar" "^3.2.5"
+    "@jupyterlab/translation" "^3.2.5"
+    "@jupyterlab/ui-components" "^3.2.5"
+    "@lumino/algorithm" "^1.3.3"
+    "@lumino/coreutils" "^1.5.3"
+    "@lumino/disposable" "^1.4.3"
+    "@lumino/domutils" "^1.2.3"
+    "@lumino/dragdrop" "^1.7.1"
+    "@lumino/messaging" "^1.4.3"
+    "@lumino/polling" "^1.3.3"
+    "@lumino/signaling" "^1.4.3"
+    "@lumino/virtualdom" "^1.8.0"
+    "@lumino/widgets" "^1.19.0"
+    react "^17.0.1"
+

[](https://i.imgur.com/dq5xtcA.png)

<br>

## 如何查看 tsc 的版本

$ jlpm tsc -v
yarn run v1.21.1
$ /home/tj/Asus/workplace/jupyter/demo/jupyterlab-extension/my_main_area_widget/node_modules/.bin/tsc -v
Version 4.1.6
Done in 0.29s.

$ node_modules/.bin/tsc -v
Version 4.1.6

- ### 參考資料
- [How to know which typescript version running on angular 4 project](https://stackoverflow.com/questions/46783931)

        tsc -v

- 不 work

<br>

## 如何顯示左側欄的 指令面板(Command Palette)？

- 啟用左側欄選項：
1. 進入 Menu: Settings > Command Palette
2. 將 model 設為 ==**false**==
- 說明：
JupyterLab 3.0 預設不顯示左側欄；
可透過快捷鍵 Ctrl + Shift + C 來顯示小視窗

<br>

## 如何在 code 裡引入 json 檔
### 相關問題
- ### Q1: tsconfig 如何解讀
- https://www.typescriptlang.org/tsconfig

        "compilerOptions": {
          "allowJs": true  <--- allow js ?
        }

- ### Q2: [jlpm watch] Requests that should resolve in the current directory need to start with './'.
> 應在當前目錄中解析的請求需要以“./”開頭。

A: `index.ts`
變更前：`import * as data from 'package.json';`
變更後：`import * as data from './package.json';`
- ### Q3: error TS6307: File 'src/package.json' is not listed within the file list of project 'tsconfig.json'. Projects must list all files or use an 'include' pattern.
A: `tsconfig.json`
變更前：`"include": ["src/*"]`
變更後：`"include": ["src/*", "src/package.json"]`

### 可以 work 的方案：import from

//src/index.js
import * as data1 from './package.json';
console.log('[data1]:', data1);
console.log('[data1.author.name]:', data1.author.name);

import data2 from './package.json';
console.log('[data2]:', data2);
console.log('[data2.author.name]:', data2.author.name);

- ### tsconfig.json
`"include": ["src/*", "src/package.json"]`
- ### src/index.ts
- ### 執行結果
- ### 參考資料
- [How to import a JSON file in ECMAScript 6?](https://stackoverflow.com/questions/34944099/)
- [Project must list all files or use an 'include' pattern](https://stackoverflow.com/questions/60029058)
- [[github] webpack-contrib / file-loader](https://github.com/webpack-contrib/file-loader/blob/master/src/index.js)

        import schema from './options.json';

<br>

### 可以 work 的方案：require

//src/index.js
var data3 = require('./package.json');
console.log('[data3]:', data3);

var data4 = require('./package');
console.log('[data4]:', data4);

// 使用 require ，可以讀更上層資料 require('../package')
// 不受到 tsconfig 的限制

- ### error
> src/index.ts:15:13 - error TS2591: Cannot find name 'require'. Do you need to install type definitions for node? Try `npm i --save-dev @types/node` and then add `node` to the types field in your tsconfig.
1. 執行 `npm i --save-dev @types/node`
2. 修改 `tsconfig.json`
- 變更前：`"types": []`
- 變更後：`"types": ["node"]`
- 說明文件：[tsconfig:Types:types](https://www.typescriptlang.org/tsconfig#types)
原先 `./node_modules/@types/` 底下的套件都會引入
現在只引入 `./node_modules/@types/node`
(那為何全部引入不會 work?)
- ### src/index.ts
- ### 執行結果
- ### 參考資料
- [TypeScript getting error TS2304: cannot find name ' require'](https://stackoverflow.com/questions/31173738/)
- [Is there a way to get version from package.json in nodejs code?](https://stackoverflow.com/questions/9153571)

<br>

## 如何在 code 裡引入其他檔案
### 單純紀錄，尚未驗證
- [Is it possible to import HTML using JavaScript?](https://stackoverflow.com/questions/4818854)

    var ajax = new XMLHttpRequest();
    ajax.open("GET", "footer.htm", false);
    ajax.send();
    document.body.innerHTML += ajax.responseText;

- [Webpack ts-loader : change tsconfig filename](https://stackoverflow.com/questions/40426378)

<br>

## ES6 import/export styles
- [Difference between 'export' and 'export default' in JavaScript? [duplicate]](https://stackoverflow.com/questions/42478661)

    // Three different export styles
    export foo;
    export default foo;
    export = foo;

    // The three matching import styles
    import {foo} from 'blah';
    import foo from 'blah';
    import * as foo from 'blah';

Roughly compiles to:

    exports.foo = foo;
    exports['default'] = foo;
    module.exports = foo;

    var foo = require('blah').foo;
    var foo = require('blah')['default'];
    var foo = require('blah');

<br>

## frontend vs backend

### src/index.ts

<br>

### {package_name}/__init__.py

<br>

### MANIFEST.in

<br>

### README.md

<br>

### package.json

<br>

### setup.py

<br>

## [Backend] 如何重啟 tornado or Jupyterlab?
> #backend, reload, restart, tornado, jupyterlab, extension
- ### 底下指令都無效 (2022/01/19)

    jupyter lab build
    jupyter lab --autoreload
    jupyter lab --autoreload --debug --watch
    %reload_ext get_token_via_backend

- **參考資料**
- [tornado.autoreload — Automatically detect code changes in development](https://www.tornadoweb.org/en/stable/autoreload.html)
- `$ jupyter lab --help`

            --autoreload
                Autoreload the webapp
                    Enable reloading of the tornado webapp and all imported Python packages
                    when any changes are made to any Python src files in server or
                    extensions.
                Equivalent to: [--ServerApp.autoreload=True]

- [Reload jupyter serverextension](https://discourse.jupyter.org/t/reload-jupyter-serverextension/2200) (可能有用)
[full notebook](https://gist.github.com/bollwyvl/bd56b58ba0a078534272043327c52bd1)
- ### `$ jupyter lab --autoreload` 其實有效
好像不需要 `--debug`
- 首次安裝, url 依舊是 404
- 後續更新, url 回傳的資料可以更新
- 後續更新, 若有新的 url，則依舊是 404

- ### 如何在 notebook 上測試

    # Dockerfile
    ARG TAG=lab-3.2.1
    FROM jupyter/datascience-notebook:${TAG}

    # Fix "mkdir: cannot create directory ‘/opt/ASUSCloudInfra’: Permission denied"
    USER root

    RUN mkdir -p /opt/ASUSCloudInfra
    RUN echo "success" > /opt/ASUSCloudInfra/.requirements.installed

<br>

## [Backend] `jupyter server extension list` vs `jupyter serverextension list`
- [link text](https:// "title")
> Actually, these are different commands and an unfortunate consequence of history and naming.
>
> `jupyter server extension list` produces a list of extensions relative to [Jupyter Server](https://jupyter-server.readthedocs.io/en/latest/operators/multiple-extensions.html#managing-multiple-extensions).
> `jupyter serverextension list` produces a list of server extensions relative to the [Notebook server](https://jupyter-notebook.readthedocs.io/en/latest/examples/Notebook/Distributing%20Jupyter%20Extensions%20as%20Python%20Packages.html#Installation-of-Jupyter-Extensions).
>
> 

<br>

## 如何上傳 extension
> 資料來源：RELEASE.md

### conda

<br>

## SageMaker 架構

[](https://i.imgur.com/rIQee7H.png)
- CPU / GPU / memory, container, notebook, kernel
- PI 在 Notebook 頻道分享

<br>

## 入門＆值得研究的套件
> 難度由低到高排列

### app
1. [jtpio / jupyterlab-python-file](https://github.com/jtpio/jupyterlab-python-file)
> 有很多概念值得研究
2. [jupyterlab / jupyterlab-latex](https://github.com/jupyterlab/jupyterlab-latex)
> 有很多概念值得研究
- [simple.tex](http://www.maths.liv.ac.uk/~rakow/LatexGuide/very_simple.html)
3. [jupyterlab / extension-examples](https://github.com/jupyterlab/extension-examples)
4. PrimeHub Job 議題
- ### [[PrimeHub] Submit a Notebook as a Job](https://docs.primehub.io/docs/notebook-as-job)
- [InfuseAI / primehub-job](https://github.com/InfuseAI/primehub-job)
- [primehub-job / jupyterlab_primehub](https://github.com/InfuseAI/primehub-job/tree/master/jupyterlab_primehub)

### mime
1. [[mime] jupyterlab / jupyterlab-mp4](https://github.com/jupyterlab/jupyterlab-mp4)
2. [jupyterlab / packages / pdf-extension/](https://github.com/jupyterlab/jupyterlab/tree/3.2.x/packages/pdf-extension)