@@ -5,175 +5,95 @@ This source code is licensed under the BSD-style license found in the
55LICENSE file in the root directory of this source tree.
66-->
77
8- ## ExecuTorch for Arm backends Ethos-U, VGF and Cortex-M
8+ # Examples for Arm backends Ethos-U, VGF and Cortex-M
99
10- This project contains scripts to help you setup and run a PyTorch
11- model on a Arm backend via ExecuTorch. This backend supports Ethos-U and VGF as
12- targets (using TOSA) but you can also use the Ethos-U example runner as an example
13- on Cortex-M if you do not delegate the model.
10+ This directory contains documentation and scripts to
11+ help you setup and run a PyTorch model on the Arm backend
12+ via ExecuTorch.
1413
15- The main scripts are ` setup.sh ` , ` run.sh ` and
16- ` backends/arm/scripts/aot_arm_compiler.py ` .
14+ ## setup.sh
1715
18- ` setup.sh ` will install the needed tools and with --root-dir <FOLDER >
19- you can change the path to a scratch folder where it will download and generate build
20- artifacts. If supplied, you must also supply the same folder to run.sh with
21- --scratch-dir=<FOLDER > If not supplied both scripts will use examples/arm/arm-scratch.
16+ ` setup.sh ` downloads the Arm cross-compilation toolchain and Corstone FVP
17+ simulators, installs the Python dependencies for TOSA, Ethos-U Vela, and
18+ Cortex-M/CMSIS-NN, and generates ` setup_path.sh ` scripts for adding those tools
19+ to your environment. Optional flags also install VGF/MLSDK and Vulkan
20+ dependencies.
2221
23- ` run.sh ` can be used to build, run and test a model in an easy way and it will call cmake for you
24- and in cases you want to run a simulator it will start it also. The script will call ` aot_arm_compiler.py `
25- to convert a model and include it in the build/run.
26-
27- For bare-metal Ethos-U builds ` run.sh ` configures the standalone
28- ` examples/arm/executor_runner/standalone ` CMake entry point automatically. If
29- ` --build-dir ` is omitted, the script creates and owns a build tree under
30- ` arm_test/<target>_<build_type> ` . Supplying ` --build-dir ` reuses an existing tree
31- (for example a VGF host build or out-of-tree configuration) and ` run.sh `
32- verifies it exposes the runner options it needs before compiling.
33-
34- Build and test artifacts are by default placed under the folder arm_test folder
35- this can be changed with --et_build_root=<FOLDER >
36-
37- ` aot_arm_compiler.py ` is used to convert a Python model or a saved .pt model to a PTE file and is used by ` run.sh `
38- and other test script but can also be used directly.
39-
40-
41- ## Create a PTE file for Arm backends
42-
43- There is an easy to use example flow to compile your PyTorch model to a PTE file for the Arm backend called ` aot_arm_compiler.py `
44- that you can use to generate PTE files, it can generate PTE files for the supported targets ` -t ` or even non delegated (Cortex-M)
45- using different memory modes and can both use a python file as input or just use the models from examples/models with ` --model_name ` .
46- It also supports generating Devtools artifacts like BundleIO BPTE files, and ETRecords. Run it with ` --help ` to check its capabilities.
47-
48- You point out the model to convert with ` --model_name=<MODELNAME/FILE> ` It supports running a model from examples/models or models
49- from a python file if you just specify ` ModelUnderTest ` and ` ModelInputs ` in it.
50-
51- ```
52- $ python3 -m backends.arm.scripts.aot_arm_compiler --help
53- ```
54-
55- This is how you generate a BundleIO BPTE of a simple add example
22+ Example to install the default Arm backend dependencies and add them to your current shell:
5623
24+ ``` bash
25+ ./examples/arm/setup.sh --i-agree-to-the-contained-eula
26+ source examples/arm/arm-scratch/setup_path.sh
5727```
58- $ python3 -m backends.arm.scripts.aot_arm_compiler --model_name=examples/arm/example_modules/add.py --target=ethos-u55-128 --bundleio
59- ```
60-
61- The example model used has added two extra variables that is picked up to make this work.
62-
63- ` ModelUnderTest ` should be a ` torch.nn.module ` instance.
64-
65- ` ModelInputs ` should be a tuple of inputs to the forward function.
66-
67-
68- You can also use the models from example/models directly by just using the short name e.g.
69-
70- ```
71- $ python3 -m backends.arm.scripts.aot_arm_compiler --model_name=mv2 --target=ethos-u55-64
72- ```
73-
74-
75- ` aot_arm_compiler.py ` is called from the scripts below so you don't need to, but it can be useful to do by hand in some cases.
7628
77- ## Host VGF example applications
29+ ## run.sh
7830
79- The Arm examples directory also contains host-side VGF reference flows for
80- specific tasks:
31+ ` run.sh ` is an end-to-end helper for building and executing an Arm backend
32+ example. It sources the ` setup_path.sh ` script generated by ` setup.sh ` , runs
33+ ` aot_arm_compiler.py ` to convert the selected model to a ` .pte ` or ` .bpte ` ,
34+ builds the matching runner with CMake, and starts the simulator or runtime for
35+ the selected target when ` --build_only ` is not set.
8136
82- - ` examples/arm/image_classification_example_vgf ` for DEiT image
83- classification.
84- - ` examples/arm/super_resolution_example_vgf ` for Swin2SR image
85- super-resolution.
86-
87-
88- ## ExecuTorch on Arm Ethos-U55/U65 and U85
89-
90- This example code will help you get going with the Corstone&trade ; -300/320 platforms and
91- run on the FVP and can be used a starting guide in your porting to your board/HW
92-
93- We will start from a PyTorch model in python, export it, convert it to a ` .pte `
94- file - A binary format adopted by ExecuTorch. Then we will take the ` .pte `
95- model file and embed that with a baremetal application executor_runner. We will
96- then take the executor_runner file, which contains not only the ` .pte ` binary but
97- also necessary software components to run standalone on a baremetal system.
98- The build flow will pick up the non delegated ops from the generated PTE file and
99- add CPU implementation of them.
100- Lastly, we will run the executor_runner binary on a Corstone&trade ; -300/320 FVP Simulator platform.
101-
102-
103- ### Example workflow
104-
105- Below is example workflow to build an application for Ethos-U55/85. The script below requires an internet connection:
106-
107- ```
108- # Step [1] - setup necessary tools
109- $ cd <EXECUTORCH-ROOT-FOLDER>
110- $ ./examples/arm/setup.sh --i-agree-to-the-contained-eula
111-
112- # Step [2] - Setup path to tools, The `setup.sh` script has generated a script that you need to source every time you restart you shell.
113- $ source examples/arm/arm-scratch/setup_path.sh
37+ Build and test artifacts are written to ` arm_test ` by default. Use
38+ ` --et_build_root=<FOLDER> ` to choose another build root.
11439
115- # Step [3] - build and run ExecuTorch and executor_runner baremetal example application
116- # on a Corstone(TM)-320 FVP to run a simple PyTorch model from a file.
117- $ ./examples/arm/run.sh --model_name=examples/arm/example_modules/add.py --target=ethos-u85-128
118- ```
119-
120- The argument ` --model_name=<MODEL> ` is passed to ` aot_arm_compiler.py ` so you can use it in the same way
121- e.g. you can also use the models from example/models directly in the same way as above.
40+ For example, after running ` setup.sh ` and sourcing the generated
41+ ` setup_path.sh ` , build and run a model on an Ethos-U85 target with:
12242
123- ```
124- $ ./examples/arm/run.sh --model_name=mv2 --target=ethos-u55-64
43+ ``` bash
44+ ./examples/arm/run.sh --model_name=examples/arm/example_modules/add.py --target=ethos-u85-128
12545```
12646
127- The runner will by default set all inputs to "1" and you are supposed to add/change the code
128- handling the input for your hardware target to give the model proper input, maybe from your camera
129- or mic hardware.
47+ For bundled input/output and ETDump testing:
13048
131- While testing you can use the --bundleio flag to use the input from the python model file and
132- generate a .bpte instead of a .pte file. This will embed the input example data and reference output
133- in the bpte file/data, which is used to verify the model's output. You can also use --etdump to generate
134- an ETRecord and a ETDump trace files from your target (they are printed as base64 strings in the serial log).
135-
136- Just keep in mind that CPU cycles are NOT accurate on the FVP simulator and it can not be used for
137- performance measurements, so you need to run on FPGA or actual ASIC to get good results from --etdump.
138- As a note the printed NPU cycle numbers are still usable and closer to real values if the timing
139- adaptor is setup correctly.
140-
141- ```
142- # Build + run with BundleIO and ETDump
143- $ ./examples/arm/run.sh --model_name=lstm --target=ethos-u85-128 --bundleio --etdump
49+ ``` bash
50+ ./examples/arm/run.sh --model_name=lstm --target=ethos-u85-128 --bundleio --etdump
14451```
14552
53+ For Cortex-M testing, use a Cortex-M target and bundled I/O:
14654
147- ### Ethos-U minimal example
148-
149- See the jupyter notebook ` ethos_u_minimal_example.ipynb ` for an explained minimal example of the full flow for running a
150- PyTorch module on the EthosUDelegate. The notebook runs directly in some IDE: s s.a. VS Code, otherwise it can be run in
151- your browser using
152- ```
153- pip install jupyter
154- jupyter notebook ethos_u_minimal_example.ipynb
55+ ``` bash
56+ ./examples/arm/run.sh --model_name=mv2 --target=cortex-m55 --bundleio
15557```
15658
157- ## ExecuTorch on ARM Cortex-M
59+ ## Example Contents
15860
159- For Cortex-M you run the script without delegating e.g ` --no_delegate ` as the build flow already supports picking up
160- the non delegated ops from the generated PTE file and add CPU implementation of them this will work out of the box in
161- most cases.
61+ ### Notebook examples
16262
163- To run mobilenet_v2 on the Cortex-M55 only, without using the Ethos-U try this:
63+ - [ ethos_u_minimal_example.ipynb] ( ethos_u_minimal_example.ipynb ) - Minimal
64+ Ethos-U AOT, runtime build, and FVP execution flow.
65+ - [ vgf_minimal_example.ipynb] ( vgf_minimal_example.ipynb ) - Minimal VGF
66+ lowering and host execution flow.
67+ - [ cortex_m_mv2_example.ipynb] ( cortex_m_mv2_example.ipynb ) - Cortex-M
68+ MobileNetV2 export, quantization, runtime build, and FVP execution flow.
69+ - [ pruning_minimal_example.ipynb] ( pruning_minimal_example.ipynb ) - Model
70+ conditioning and pruning flow for Ethos-U85.
71+ - [ quantizer_tutorial.ipynb] ( quantizer_tutorial.ipynb ) - Quantizer tutorial
72+ for TOSA, Ethos-U, and VGF quantizers.
16473
165- ```
166- $ ./examples/arm/run.sh --model_name=mv2 --target=ethos-u55-128 --no_delegate
167- ```
74+ ### Application examples
16875
76+ - [ image_classification_example_ethos_u] ( image_classification_example_ethos_u/ )
77+ - End-to-end DEiT-Tiny image classification flow for Ethos-U, including
78+ model fine-tuning, export, bare-metal runtime build, and Corstone-320 FVP
79+ execution.
80+ - [ image_classification_example_vgf] ( image_classification_example_vgf/ ) -
81+ DEiT-Tiny image classification flow for VGF host execution.
82+ - [ super_resolution_example_vgf] ( super_resolution_example_vgf ) - Swin2SR image
83+ super-resolution.
84+ - [ example_modules/add.py] ( example_modules/add.py ) - Small external model file
85+ usable with ` run.sh --model_name=examples/arm/example_modules/add.py ` .
16986
170- ### Online Tutorial
87+ ### Utility examples and guides
17188
172- We also have a [ tutorial] ( https://pytorch.org/executorch/stable/backends-arm-ethos-u ) explaining the steps performed in these
173- scripts, expected results, possible problems and more. It is a step-by-step guide
174- you can follow to better understand this delegate.
89+ - [ ethos-u-porting-guide.md] ( ethos-u-porting-guide.md ) - Notes for adapting
90+ the example Ethos-U runtime integration to another target.
91+ - [ export_standalone_tosa_graph.py] ( export_standalone_tosa_graph.py ) -
92+ Example of exporting a standalone TOSA graph with multiple outputs.
93+ - [ visualize.py] ( visualize.py ) - Helper used by ` run.sh --model_explorer ` to
94+ visualize TOSA or PTE graphs.
17595
176- ### Project Templates
96+ ## Project Templates
17797
17898These project templates provide alternative starting points with different toolchains and build systems:
17999
0 commit comments