Skip to content

Commit

Permalink
[docs] Move jupyter notebooks to DJL Demo (#2854)
Browse files Browse the repository at this point in the history
* [docs] Move jupyter notebooks to DJL Demo

This moves the notebooks to the demo repositories. It makes it more clearly
follow the trend where the things in the main repo are master build while those
in demos track nightly snapshots or releases.
  • Loading branch information
zachgk authored Nov 22, 2023
1 parent c846786 commit 969f84e
Show file tree
Hide file tree
Showing 41 changed files with 54 additions and 5,583 deletions.
9 changes: 4 additions & 5 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ name: Docs
on:
pull_request:
paths:
- "**.ipynb"
- "docs/mkdocs.yml"
# Publish docs weekly
schedule:
Expand Down Expand Up @@ -38,10 +37,6 @@ jobs:
- name: add mybinder link
run: |
python3 tools/scripts/add_online_runner.py
- name: run Notebooks
run: |
cd jupyter
bash test_notebook.sh
- name: clone demos
run: |
cd docs
Expand All @@ -50,6 +45,10 @@ jobs:
run: |
cd docs
git clone https://github.com/deepjavalibrary/djl-serving.git serving
- name: run Notebooks
run: |
cd docs/demos/jupyter
bash test_notebook.sh
- name: build docs
run: |
cd docs
Expand Down
9 changes: 4 additions & 5 deletions api/src/main/java/ai/djl/inference/Predictor.java
Original file line number Diff line number Diff line change
Expand Up @@ -60,14 +60,13 @@
*
* <ul>
* <li><a
* href="https://github.com/deepjavalibrary/djl/blob/master/jupyter/tutorial/03_image_classification_with_your_model.ipynb">Inference
* href="http://docs.djl.ai/docs/demos/jupyter/tutorial/03_image_classification_with_your_model.html">Inference
* with a custom trained model</a>
* <li><a
* href="https://github.com/deepjavalibrary/djl/blob/master/jupyter/object_detection_with_model_zoo.ipynb">Inference
* href="http://docs.djl.ai/docs/demos/jupyter/object_detection_with_model_zoo.html">Inference
* with a model zoo model</a>
* <li><a
* href="https://github.com/deepjavalibrary/djl/blob/master/jupyter/load_mxnet_model.ipynb">Inference
* with an MXNet model</a>
* <li><a href="http://docs.djl.ai/docs/demos/jupyter/load_mxnet_model.html">Inference with an
* MXNet model</a>
* </ul>
*
* @param <I> the input type
Expand Down
2 changes: 1 addition & 1 deletion api/src/main/java/ai/djl/nn/Block.java
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@
* further refine these elements, use {@link Block#freezeParameters(boolean)} to unfreeze them.
*
* @see <a
* href="https://github.com/deepjavalibrary/djl/blob/master/jupyter/tutorial/01_create_your_first_network.ipynb">this
* href="http://docs.djl.ai/docs/demos/jupyter/tutorial/01_create_your_first_network.html">this
* tutorial on creating your first network</a>
* @see <a href="https://d2l.djl.ai/chapter_deep-learning-computation/model-construction.html">The
* D2L chapter on blocks</a> and <a
Expand Down
10 changes: 4 additions & 6 deletions api/src/main/java/ai/djl/training/Trainer.java
Original file line number Diff line number Diff line change
Expand Up @@ -52,14 +52,12 @@
*
* <ul>
* <li><a
* href="https://github.com/deepjavalibrary/djl/blob/master/jupyter/tutorial/02_train_your_first_model.ipynb">Training
* href="http://docs.djl.ai/docs/demos/jupyter/tutorial/02_train_your_first_model.html">Training
* your first model</a>
* <li><a
* href="https://github.com/deepjavalibrary/djl/blob/master/jupyter/transfer_learning_on_cifar10.ipynb">Training
* <li><a href="http://docs.djl.ai/docs/demos/jupyter/transfer_learning_on_cifar10.html">Training
* using transfer learning</a>
* <li><a
* href="https://github.com/deepjavalibrary/djl/blob/master/jupyter/load_mxnet_model.ipynb">Inference
* with an MXNet model</a>
* <li><a href="http://docs.djl.ai/docs/demos/jupyter/load_mxnet_model.html">Inference with an
* MXNet model</a>
* </ul>
*
* @see <a href="http://docs.djl.ai/docs/development/memory_management.html">The guide on memory
Expand Down
16 changes: 8 additions & 8 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,14 @@ Note: when searching in JavaDoc, if your access is denied, please try removing t
- [Troubleshooting](development/troubleshooting.md)
- [Inference Optimization](development/inference_performance_optimization.md)

## [Jupyter notebook tutorials](../jupyter/README.md)

- **[Beginner Jupyter Tutorial](../jupyter/tutorial/README.md)**
- [Run object detection with model zoo](../jupyter/object_detection_with_model_zoo.ipynb)
- [Load pre-trained PyTorch model](../jupyter/load_pytorch_model.ipynb)
- [Load pre-trained Apache MXNet model](../jupyter/load_mxnet_model.ipynb)
- [Transfer learning example](../jupyter/transfer_learning_on_cifar10.ipynb)
- [Question answering example](../jupyter/BERTQA.ipynb)
## [Jupyter notebook tutorials](http://docs.djl.ai/docs/demos/jupyter/index.html)

- **[Beginner Jupyter Tutorial](http://docs.djl.ai/docs/demos/jupyter/tutorial/index.html)**
- [Run object detection with model zoo](http://docs.djl.ai/docs/demos/jupyter/object_detection_with_model_zoo.html)
- [Load pre-trained PyTorch model](http://docs.djl.ai/docs/demos/jupyter/load_pytorch_model.html)
- [Load pre-trained Apache MXNet model](http://docs.djl.ai/docs/demos/jupyter/load_mxnet_model.html)
- [Transfer learning example](http://docs.djl.ai/docs/demos/jupyter/transfer_learning_on_cifar10.html)
- [Question answering example](http://docs.djl.ai/docs/demos/jupyter/BERTQA.html)

## [API Examples](../examples/README.md)

Expand Down
6 changes: 3 additions & 3 deletions docs/interactive_tool.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,17 +63,17 @@ After that, click `run` and you should see the following result:

Finally, you can get the running project setup by clicking `Get Template`. This will bring you a gradle project that can be used in your local machine.

## [Java Jupyter Notebook](../jupyter/README.md)
## [Java Jupyter Notebook](http://docs.djl.ai/docs/demos/jupyter/index.html)

Wait a second, are we talking about hosting Jupyter Notebook in python?
No, it’s Java 11, only.

![jupyter](https://djl-ai.s3.amazonaws.com/web-data/images/jupyter.gif)

Inspired by Spencer Park’s [IJava project](https://github.com/SpencerPark/IJava), we integrated DJL with Jupyter Notebooks.
For more information on the simple setup, follow the instructions in [DJL Jupyter notebooks](../jupyter/README.md#setup).
For more information on the simple setup, follow the instructions in [DJL Jupyter notebooks](http://docs.djl.ai/docs/demos/jupyter/index.html#setup).
After that, use the Jupyter Notebook freely in your hosted server. You can do all kinds of work, like block building and plotting a graph.
There are [tutorials and instructions](../jupyter/README.md#djl---jupyter-notebooks) to guide you how you can run training and/or inference with Java.
There are [tutorials and instructions](http://docs.djl.ai/docs/demos/jupyter/index.html#djl---jupyter-notebooks) to guide you how you can run training and/or inference with Java.

## About Future Lab

Expand Down
30 changes: 15 additions & 15 deletions docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,15 +61,15 @@ nav:
- 'docs/faq.md'
- Tutorials:
- Beginner Tutorial:
- 'jupyter/tutorial/01_create_your_first_network.ipynb'
- 'jupyter/tutorial/02_train_your_first_model.ipynb'
- 'jupyter/tutorial/03_image_classification_with_your_model.ipynb'
- 'docs/demos/jupyter/tutorial/01_create_your_first_network.ipynb'
- 'docs/demos/jupyter/tutorial/02_train_your_first_model.ipynb'
- 'docs/demos/jupyter/tutorial/03_image_classification_with_your_model.ipynb'
- 'docs/d2l.md'
- 'jupyter/rank_classification_using_BERT_on_Amazon_Review.ipynb'
- 'jupyter/transfer_learning_on_cifar10.ipynb'
- 'docs/demos/jupyter/rank_classification_using_BERT_on_Amazon_Review.ipynb'
- 'docs/demos/jupyter/transfer_learning_on_cifar10.ipynb'
- Load your own BERT:
- BERT with MXNet: 'jupyter/mxnet/load_your_own_mxnet_bert.ipynb'
- BERT with PyTorch: 'jupyter/pytorch/load_your_own_pytorch_bert.ipynb'
- BERT with MXNet: 'docs/demos/jupyter/mxnet/load_your_own_mxnet_bert.ipynb'
- BERT with PyTorch: 'docs/demos/jupyter/pytorch/load_your_own_pytorch_bert.ipynb'
- Guides:
- Models:
- 'docs/load_model.md'
Expand Down Expand Up @@ -97,25 +97,25 @@ nav:
- PyTorch NDArray Operators: 'docs/pytorch/pytorch-djl-ndarray-cheatsheet.md'
- PyTorch Model Zoo: 'engines/pytorch/pytorch-model-zoo/README.md'
- Import PyTorch Model: 'docs/pytorch/how_to_convert_your_model_to_torchscript.md'
- Load a PyTorch Model: 'jupyter/load_pytorch_model.ipynb'
- Load a PyTorch Model: 'docs/demos/jupyter/load_pytorch_model.ipynb'
- TensorFlow:
- Overview: 'engines/tensorflow/README.md'
- TensorFlow Engine: 'engines/tensorflow/tensorflow-engine/README.md'
- TensorFlow Model Zoo: 'engines/tensorflow/tensorflow-model-zoo/README.md'
- Import TensorFlow Model: 'docs/tensorflow/how_to_import_tensorflow_models_in_DJL.md'
- Load a TensorFlow Model: 'jupyter/tensorflow/pneumonia_detection.ipynb'
- Load a TensorFlow Model: 'docs/demos/jupyter/tensorflow/pneumonia_detection.ipynb'
- Apache MXNet:
- Overview: 'engines/mxnet/README.md'
- MXNet Engine: 'engines/mxnet/mxnet-engine/README.md'
- MXNet Model Zoo: 'engines/mxnet/mxnet-model-zoo/README.md'
- Import Gluon Model: 'docs/mxnet/how_to_convert_your_model_to_symbol.md'
- Load a MXNet Model: 'jupyter/load_mxnet_model.ipynb'
- Load a MXNet Model: 'docs/demos/jupyter/load_mxnet_model.ipynb'
- Backend Optimizer for MXNet: 'docs/mxnet/mxnet_backend_optimizer.md'
- Hybrid engines:
- Hybrid engine overview: 'docs/hybrid_engine.md'
- ONNX Runtime:
- Overview: 'engines/onnxruntime/onnxruntime-engine/README.md'
- Load a ONNX Model: 'jupyter/onnxruntime/machine_learning_with_ONNXRuntime.ipynb'
- Load a ONNX Model: 'docs/demos/jupyter/onnxruntime/machine_learning_with_ONNXRuntime.ipynb'
- PaddlePaddle:
- Overview: 'engines/paddlepaddle/README.md'
- PaddlePaddle Engine: 'engines/paddlepaddle/paddlepaddle-engine/README.md'
Expand All @@ -124,11 +124,11 @@ nav:
- English: 'docs/paddlepaddle/how_to_create_paddlepaddle_model.md'
- 中文: 'docs/paddlepaddle/how_to_create_paddlepaddle_model_zh.md'
- Facemask detection using PaddlePaddle:
- English: 'jupyter/paddlepaddle/face_mask_detection_paddlepaddle.ipynb'
- 中文: 'jupyter/paddlepaddle/face_mask_detection_paddlepaddle_zh.ipynb'
- English: 'docs/demos/jupyter/paddlepaddle/face_mask_detection_paddlepaddle.ipynb'
- 中文: 'docs/demos/jupyter/paddlepaddle/face_mask_detection_paddlepaddle_zh.ipynb'
- PaddleOCR example:
- English: 'jupyter/paddlepaddle/paddle_ocr_java.ipynb'
- 中文: 'jupyter/paddlepaddle/paddle_ocr_java_zh.ipynb'
- English: 'docs/demos/jupyter/paddlepaddle/paddle_ocr_java.ipynb'
- 中文: 'docs/demos/jupyter/paddlepaddle/paddle_ocr_java_zh.ipynb'
- XGBoost: 'engines/ml/xgboost/README.md'
- LightGBM: 'engines/ml/lightgbm/README.md'
- TensorRT: 'engines/tensorrt/README.md'
Expand Down
2 changes: 1 addition & 1 deletion docs/paddlepaddle/how_to_create_paddlepaddle_model.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,5 +157,5 @@ predictor.predict(list);

As mentioned, you need to find out what is the input for the model, like images usually interpret as NCHW (batch_size, channel, height, width).

However, usage like this is really basic, you can write a `Translator` in DJL for it. You can find some code examples [here](../../jupyter/paddlepaddle/face_mask_detection_paddlepaddle.ipynb).
However, usage like this is really basic, you can write a `Translator` in DJL for it. You can find some code examples [here](http://docs.djl.ai/docs/demos/jupyter/paddlepaddle/face_mask_detection_paddlepaddle.html).

2 changes: 1 addition & 1 deletion docs/paddlepaddle/how_to_create_paddlepaddle_model_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,4 +156,4 @@ predictor.predict(list);

在这里,你需要知道模型的输入输出格式, 比如图片经常表达成 NCHW (批大小, RGB通道, 高度, 宽度)的多维矩阵。

虽然这样可以让模型跑起来, 但是最好还是结合 DJL 的 `Translator` class 使用。你可以在 [这里](../../jupyter/paddlepaddle/face_mask_detection_paddlepaddle.ipynb) 找到一些示例代码。
虽然这样可以让模型跑起来, 但是最好还是结合 DJL 的 `Translator` class 使用。你可以在 [这里](http://docs.djl.ai/docs/demos/jupyter/paddlepaddle/face_mask_detection_paddlepaddle.html) 找到一些示例代码。
6 changes: 3 additions & 3 deletions docs/quick_start.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Quick start

Deep Java Library (DJL) is designed to be easy to get started with and simple to use.
The easiest way to learn DJL is to read the [beginner tutorial](../jupyter/tutorial/README.md) or
The easiest way to learn DJL is to read the [beginner tutorial](http://docs.djl.ai/docs/demos/jupyter/tutorial/README.md) or
our [examples](../examples/README.md).

You can also view our 1.5 hour long (in 8 x ~10 minute segments) DJL 101 tutorial video series:
Expand All @@ -22,7 +22,7 @@ See [DJL Future Labs](interactive_tool.md)

## Beginner tutorial

To get started, we recommend that you follow our short [beginner tutorial](../jupyter/tutorial/README.md). It takes you through some of the basics of deep learning to create a model, train your model, and run inference using your trained model.
To get started, we recommend that you follow our short [beginner tutorial](http://docs.djl.ai/docs/demos/jupyter/tutorial/index.html). It takes you through some of the basics of deep learning to create a model, train your model, and run inference using your trained model.

## Run examples

Expand All @@ -33,7 +33,7 @@ All of our examples are executed by a simple command. For detailed command line
- [Train your first model](../examples/docs/train_mnist_mlp.md)
- [Single-shot Object Detection inference example](../examples/docs/object_detection.md)
- [More examples](https://github.com/deepjavalibrary/djl/tree/master/examples)
- [Jupyter examples](../jupyter/README.md)
- [Jupyter examples](http://docs.djl.ai/docs/demos/jupyter/index.html)

## Other resources

Expand Down
2 changes: 1 addition & 1 deletion examples/docs/image_classification.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ In this example, you learn how to implement inference code with Deep Java Librar

The image classification example code can be found at [ImageClassification.java](https://github.com/deepjavalibrary/djl/blob/master/examples/src/main/java/ai/djl/examples/inference/ImageClassification.java).

You can also use the [Jupyter notebook tutorial](../../jupyter/tutorial/03_image_classification_with_your_model.ipynb).
You can also use the [Jupyter notebook tutorial](http://docs.djl.ai/docs/demos/jupyter/tutorial/03_image_classification_with_your_model.html).
The Jupyter notebook explains the key concepts in detail.

## Setup Guide
Expand Down
2 changes: 1 addition & 1 deletion examples/docs/object_detection.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ In this example, you learn how to implement inference code with a [ModelZoo mode

The source code can be found at [ObjectDetection.java](https://github.com/deepjavalibrary/djl/blob/master/examples/src/main/java/ai/djl/examples/inference/ObjectDetection.java).

You can also use the [Jupyter notebook tutorial](../../jupyter/object_detection_with_model_zoo.ipynb).
You can also use the [Jupyter notebook tutorial](http://docs.djl.ai/docs/demos/jupyter/object_detection_with_model_zoo.html).
The Jupyter notebook explains the key concepts in detail.

## Setup guide
Expand Down
2 changes: 1 addition & 1 deletion examples/docs/train_cifar10_resnet.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ In this example, you learn how to train the [CIFAR-10](https://www.cs.toronto.ed

You can find the example source code in: [TrainResnetWithCifar10.java](https://github.com/deepjavalibrary/djl/blob/master/examples/src/main/java/ai/djl/examples/training/transferlearning/TrainResnetWithCifar10.java).

You can also find the Jupyter notebook tutorial [here](../../jupyter/transfer_learning_on_cifar10.ipynb).
You can also find the Jupyter notebook tutorial [here](http://docs.djl.ai/docs/demos/jupyter/transfer_learning_on_cifar10.html).
The Jupyter notebook explains the key concepts in detail.

## Setup guide
Expand Down
2 changes: 1 addition & 1 deletion examples/docs/train_mnist_mlp.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ In this example, you learn how to train the MNIST dataset with Deep Java Library

The source code for this example can be found at [TrainMnist.java](https://github.com/deepjavalibrary/djl/blob/master/examples/src/main/java/ai/djl/examples/training/TrainMnist.java).

You can also use the [Jupyter notebook tutorial](../../jupyter/tutorial/02_train_your_first_model.ipynb).
You can also use the [Jupyter notebook tutorial](http://docs.djl.ai/docs/demos/jupyter/tutorial/02_train_your_first_model.html).
The Jupyter notebook explains the key concepts in detail.

## Setup guide
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,8 @@
* <p>See:
*
* <ul>
* <li>the <a
* href="https://github.com/deepjavalibrary/djl/blob/master/jupyter/BERTQA.ipynb">jupyter
* demo</a> with more information about BERT.
* <li>the <a href="http://docs.djl.ai/docs/demos/jupyter/BERTQA.html">jupyter demo</a> with more
* information about BERT.
* <li>the <a
* href="https://github.com/deepjavalibrary/djl/blob/master/examples/docs/BERT_question_and_answer.md">docs</a>
* for information about running this example.
Expand Down
4 changes: 2 additions & 2 deletions index1.0.html
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@
<li><a href="website/javadoc.html">JavaDoc</a></li>
<li><a href="website/demo.html">Demos</a></li>
<li><a href="website/blog.html">Blogs</a></li>
<li><a href="https://docs.djl.ai/jupyter/tutorial/index.html">Tutorial</a></li>
<li><a href="https://docs.djl.ai/docs/demos/jupyter/tutorial/index.html">Tutorial</a></li>
<li><a href="https://github.com/deepjavalibrary/djl/tree/master/examples">Examples</a></li>
<li><a
href="https://deepjavalibrary.slack.com/join/shared_invite/zt-ar91gjkz-qbXhr1l~LFGEIEeGBibT7w#/">Slack</a>
Expand All @@ -73,7 +73,7 @@
<li><a href="website/javadoc.html">JavaDoc</a></li>
<li><a href="website/demo.html">Demos</a></li>
<li><a href="website/blog.html">Blogs</a></li>
<li><a href="https://docs.djl.ai/jupyter/tutorial/index.html">Tutorial</a></li>
<li><a href="https://docs.djl.ai/docs/demos/jupyter/tutorial/index.html">Tutorial</a></li>
<li><a href="https://github.com/deepjavalibrary/djl/tree/master/examples">Examples</a></li>
<li><a
href="https://deepjavalibrary.slack.com/join/shared_invite/zt-ar91gjkz-qbXhr1l~LFGEIEeGBibT7w#/">Slack</a>
Expand Down
Loading

0 comments on commit 969f84e

Please sign in to comment.