Skip to content

Commit 5ca3207

Browse files
authored
Arm backend: Update examples/arm/README.md (pytorch#19756)
Make the README concise for setup, run.sh usage, example notebooks, applications, and helper scripts. Move broader backend documentation links to the backend README. Signed-off-by: Martin Lindström <Martin.Lindstroem@arm.com>
1 parent 94f9719 commit 5ca3207

2 files changed

Lines changed: 67 additions & 145 deletions

File tree

backends/arm/README.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -136,8 +136,10 @@ The delegated Python API flow is:
136136
For complete examples of that flow, including quantization and target-specific
137137
compile specs, see:
138138

139-
- `docs/source/backends/arm-ethos-u/tutorials/ethos-u-getting-started.md`
140-
- `docs/source/backends/arm-vgf/tutorials/vgf-getting-started.md`
139+
- [Arm Ethos-U tutorial](../../docs/source/backends/arm-ethos-u/tutorials/ethos-u-getting-started.md)
140+
- [Arm VGF tutorial](../../docs/source/backends/arm-vgf/tutorials/vgf-getting-started.md)
141+
- [Arm Cortex-M backend overview](../../docs/source/backends/arm-cortex-m/arm-cortex-m-overview.md)
142+
- [Ethos-U porting guide](../../examples/arm/ethos-u-porting-guide.md)
141143

142144
Additional examples are available in `examples/arm`.
143145

examples/arm/README.md

Lines changed: 63 additions & 143 deletions
Original file line numberDiff line numberDiff line change
@@ -5,175 +5,95 @@ This source code is licensed under the BSD-style license found in the
55
LICENSE file in the root directory of this source tree.
66
-->
77

8-
## ExecuTorch for Arm backends Ethos-U, VGF and Cortex-M
8+
# Examples for Arm backends Ethos-U, VGF and Cortex-M
99

10-
This project contains scripts to help you setup and run a PyTorch
11-
model on a Arm backend via ExecuTorch. This backend supports Ethos-U and VGF as
12-
targets (using TOSA) but you can also use the Ethos-U example runner as an example
13-
on Cortex-M if you do not delegate the model.
10+
This directory contains documentation and scripts to
11+
help you setup and run a PyTorch model on the Arm backend
12+
via ExecuTorch.
1413

15-
The main scripts are `setup.sh`, `run.sh` and
16-
`backends/arm/scripts/aot_arm_compiler.py`.
14+
## setup.sh
1715

18-
`setup.sh` will install the needed tools and with --root-dir <FOLDER>
19-
you can change the path to a scratch folder where it will download and generate build
20-
artifacts. If supplied, you must also supply the same folder to run.sh with
21-
--scratch-dir=<FOLDER> If not supplied both scripts will use examples/arm/arm-scratch.
16+
`setup.sh` downloads the Arm cross-compilation toolchain and Corstone FVP
17+
simulators, installs the Python dependencies for TOSA, Ethos-U Vela, and
18+
Cortex-M/CMSIS-NN, and generates `setup_path.sh` scripts for adding those tools
19+
to your environment. Optional flags also install VGF/MLSDK and Vulkan
20+
dependencies.
2221

23-
`run.sh` can be used to build, run and test a model in an easy way and it will call cmake for you
24-
and in cases you want to run a simulator it will start it also. The script will call `aot_arm_compiler.py`
25-
to convert a model and include it in the build/run.
26-
27-
For bare-metal Ethos-U builds `run.sh` configures the standalone
28-
`examples/arm/executor_runner/standalone` CMake entry point automatically. If
29-
`--build-dir` is omitted, the script creates and owns a build tree under
30-
`arm_test/<target>_<build_type>`. Supplying `--build-dir` reuses an existing tree
31-
(for example a VGF host build or out-of-tree configuration) and `run.sh`
32-
verifies it exposes the runner options it needs before compiling.
33-
34-
Build and test artifacts are by default placed under the folder arm_test folder
35-
this can be changed with --et_build_root=<FOLDER>
36-
37-
`aot_arm_compiler.py` is used to convert a Python model or a saved .pt model to a PTE file and is used by `run.sh`
38-
and other test script but can also be used directly.
39-
40-
41-
## Create a PTE file for Arm backends
42-
43-
There is an easy to use example flow to compile your PyTorch model to a PTE file for the Arm backend called `aot_arm_compiler.py`
44-
that you can use to generate PTE files, it can generate PTE files for the supported targets `-t` or even non delegated (Cortex-M)
45-
using different memory modes and can both use a python file as input or just use the models from examples/models with `--model_name`.
46-
It also supports generating Devtools artifacts like BundleIO BPTE files, and ETRecords. Run it with `--help` to check its capabilities.
47-
48-
You point out the model to convert with `--model_name=<MODELNAME/FILE>` It supports running a model from examples/models or models
49-
from a python file if you just specify `ModelUnderTest` and `ModelInputs` in it.
50-
51-
```
52-
$ python3 -m backends.arm.scripts.aot_arm_compiler --help
53-
```
54-
55-
This is how you generate a BundleIO BPTE of a simple add example
22+
Example to install the default Arm backend dependencies and add them to your current shell:
5623

24+
```bash
25+
./examples/arm/setup.sh --i-agree-to-the-contained-eula
26+
source examples/arm/arm-scratch/setup_path.sh
5727
```
58-
$ python3 -m backends.arm.scripts.aot_arm_compiler --model_name=examples/arm/example_modules/add.py --target=ethos-u55-128 --bundleio
59-
```
60-
61-
The example model used has added two extra variables that is picked up to make this work.
62-
63-
`ModelUnderTest` should be a `torch.nn.module` instance.
64-
65-
`ModelInputs` should be a tuple of inputs to the forward function.
66-
67-
68-
You can also use the models from example/models directly by just using the short name e.g.
69-
70-
```
71-
$ python3 -m backends.arm.scripts.aot_arm_compiler --model_name=mv2 --target=ethos-u55-64
72-
```
73-
74-
75-
`aot_arm_compiler.py` is called from the scripts below so you don't need to, but it can be useful to do by hand in some cases.
7628

77-
## Host VGF example applications
29+
## run.sh
7830

79-
The Arm examples directory also contains host-side VGF reference flows for
80-
specific tasks:
31+
`run.sh` is an end-to-end helper for building and executing an Arm backend
32+
example. It sources the `setup_path.sh` script generated by `setup.sh`, runs
33+
`aot_arm_compiler.py` to convert the selected model to a `.pte` or `.bpte`,
34+
builds the matching runner with CMake, and starts the simulator or runtime for
35+
the selected target when `--build_only` is not set.
8136

82-
- `examples/arm/image_classification_example_vgf` for DEiT image
83-
classification.
84-
- `examples/arm/super_resolution_example_vgf` for Swin2SR image
85-
super-resolution.
86-
87-
88-
## ExecuTorch on Arm Ethos-U55/U65 and U85
89-
90-
This example code will help you get going with the Corstone&trade;-300/320 platforms and
91-
run on the FVP and can be used a starting guide in your porting to your board/HW
92-
93-
We will start from a PyTorch model in python, export it, convert it to a `.pte`
94-
file - A binary format adopted by ExecuTorch. Then we will take the `.pte`
95-
model file and embed that with a baremetal application executor_runner. We will
96-
then take the executor_runner file, which contains not only the `.pte` binary but
97-
also necessary software components to run standalone on a baremetal system.
98-
The build flow will pick up the non delegated ops from the generated PTE file and
99-
add CPU implementation of them.
100-
Lastly, we will run the executor_runner binary on a Corstone&trade;-300/320 FVP Simulator platform.
101-
102-
103-
### Example workflow
104-
105-
Below is example workflow to build an application for Ethos-U55/85. The script below requires an internet connection:
106-
107-
```
108-
# Step [1] - setup necessary tools
109-
$ cd <EXECUTORCH-ROOT-FOLDER>
110-
$ ./examples/arm/setup.sh --i-agree-to-the-contained-eula
111-
112-
# Step [2] - Setup path to tools, The `setup.sh` script has generated a script that you need to source every time you restart you shell.
113-
$ source examples/arm/arm-scratch/setup_path.sh
37+
Build and test artifacts are written to `arm_test` by default. Use
38+
`--et_build_root=<FOLDER>` to choose another build root.
11439

115-
# Step [3] - build and run ExecuTorch and executor_runner baremetal example application
116-
# on a Corstone(TM)-320 FVP to run a simple PyTorch model from a file.
117-
$ ./examples/arm/run.sh --model_name=examples/arm/example_modules/add.py --target=ethos-u85-128
118-
```
119-
120-
The argument `--model_name=<MODEL>` is passed to `aot_arm_compiler.py` so you can use it in the same way
121-
e.g. you can also use the models from example/models directly in the same way as above.
40+
For example, after running `setup.sh` and sourcing the generated
41+
`setup_path.sh`, build and run a model on an Ethos-U85 target with:
12242

123-
```
124-
$ ./examples/arm/run.sh --model_name=mv2 --target=ethos-u55-64
43+
```bash
44+
./examples/arm/run.sh --model_name=examples/arm/example_modules/add.py --target=ethos-u85-128
12545
```
12646

127-
The runner will by default set all inputs to "1" and you are supposed to add/change the code
128-
handling the input for your hardware target to give the model proper input, maybe from your camera
129-
or mic hardware.
47+
For bundled input/output and ETDump testing:
13048

131-
While testing you can use the --bundleio flag to use the input from the python model file and
132-
generate a .bpte instead of a .pte file. This will embed the input example data and reference output
133-
in the bpte file/data, which is used to verify the model's output. You can also use --etdump to generate
134-
an ETRecord and a ETDump trace files from your target (they are printed as base64 strings in the serial log).
135-
136-
Just keep in mind that CPU cycles are NOT accurate on the FVP simulator and it can not be used for
137-
performance measurements, so you need to run on FPGA or actual ASIC to get good results from --etdump.
138-
As a note the printed NPU cycle numbers are still usable and closer to real values if the timing
139-
adaptor is setup correctly.
140-
141-
```
142-
# Build + run with BundleIO and ETDump
143-
$ ./examples/arm/run.sh --model_name=lstm --target=ethos-u85-128 --bundleio --etdump
49+
```bash
50+
./examples/arm/run.sh --model_name=lstm --target=ethos-u85-128 --bundleio --etdump
14451
```
14552

53+
For Cortex-M testing, use a Cortex-M target and bundled I/O:
14654

147-
### Ethos-U minimal example
148-
149-
See the jupyter notebook `ethos_u_minimal_example.ipynb` for an explained minimal example of the full flow for running a
150-
PyTorch module on the EthosUDelegate. The notebook runs directly in some IDE:s s.a. VS Code, otherwise it can be run in
151-
your browser using
152-
```
153-
pip install jupyter
154-
jupyter notebook ethos_u_minimal_example.ipynb
55+
```bash
56+
./examples/arm/run.sh --model_name=mv2 --target=cortex-m55 --bundleio
15557
```
15658

157-
## ExecuTorch on ARM Cortex-M
59+
## Example Contents
15860

159-
For Cortex-M you run the script without delegating e.g `--no_delegate` as the build flow already supports picking up
160-
the non delegated ops from the generated PTE file and add CPU implementation of them this will work out of the box in
161-
most cases.
61+
### Notebook examples
16262

163-
To run mobilenet_v2 on the Cortex-M55 only, without using the Ethos-U try this:
63+
- [ethos_u_minimal_example.ipynb](ethos_u_minimal_example.ipynb) - Minimal
64+
Ethos-U AOT, runtime build, and FVP execution flow.
65+
- [vgf_minimal_example.ipynb](vgf_minimal_example.ipynb) - Minimal VGF
66+
lowering and host execution flow.
67+
- [cortex_m_mv2_example.ipynb](cortex_m_mv2_example.ipynb) - Cortex-M
68+
MobileNetV2 export, quantization, runtime build, and FVP execution flow.
69+
- [pruning_minimal_example.ipynb](pruning_minimal_example.ipynb) - Model
70+
conditioning and pruning flow for Ethos-U85.
71+
- [quantizer_tutorial.ipynb](quantizer_tutorial.ipynb) - Quantizer tutorial
72+
for TOSA, Ethos-U, and VGF quantizers.
16473

165-
```
166-
$ ./examples/arm/run.sh --model_name=mv2 --target=ethos-u55-128 --no_delegate
167-
```
74+
### Application examples
16875

76+
- [image_classification_example_ethos_u](image_classification_example_ethos_u/)
77+
- End-to-end DEiT-Tiny image classification flow for Ethos-U, including
78+
model fine-tuning, export, bare-metal runtime build, and Corstone-320 FVP
79+
execution.
80+
- [image_classification_example_vgf](image_classification_example_vgf/) -
81+
DEiT-Tiny image classification flow for VGF host execution.
82+
- [super_resolution_example_vgf](super_resolution_example_vgf) - Swin2SR image
83+
super-resolution.
84+
- [example_modules/add.py](example_modules/add.py) - Small external model file
85+
usable with `run.sh --model_name=examples/arm/example_modules/add.py`.
16986

170-
### Online Tutorial
87+
### Utility examples and guides
17188

172-
We also have a [tutorial](https://pytorch.org/executorch/stable/backends-arm-ethos-u) explaining the steps performed in these
173-
scripts, expected results, possible problems and more. It is a step-by-step guide
174-
you can follow to better understand this delegate.
89+
- [ethos-u-porting-guide.md](ethos-u-porting-guide.md) - Notes for adapting
90+
the example Ethos-U runtime integration to another target.
91+
- [export_standalone_tosa_graph.py](export_standalone_tosa_graph.py) -
92+
Example of exporting a standalone TOSA graph with multiple outputs.
93+
- [visualize.py](visualize.py) - Helper used by `run.sh --model_explorer` to
94+
visualize TOSA or PTE graphs.
17595

176-
### Project Templates
96+
## Project Templates
17797

17898
These project templates provide alternative starting points with different toolchains and build systems:
17999

0 commit comments

Comments
 (0)