Executorch LLM tutorial

00-Executorch-LLM-Tutorial.md

Executorch LLM tutorial

Converting LLMs to Executorch format

• Conversion of stories110M (Llama-2 architecture) to Executorch on Colab

Building Executorch for edge devices

Using Executorch on edge devices

Android with the Llama Demo App

• Usage tutorial

01-AndroidDemo-Usage.md

Usage of the Android Demo App

You can download the v0.2.0 APK from GitHub Actions or build it yourself from commit 6a1703e (v0.2.0 with buildfixes for Android).

1. Download and install the APK.

On-device instructions using Shizuku and aShell

2. Download and install Shizuku and aShell
3. Open Shizuku, pair it with your device via wireless debugging, and start the Shizuku service.
4. Download an Executorch model such as stories110M (you need the tokenizer.bin and the .pte file) to your download folder (which is usually /sdcard/Download)
5. Open aShell and type:

mkdir -p /data/local/tmp/llama
cp /sdcard/Download/tokenizer.bin /data/local/tmp/llama/tokenizer.bin
cp /sdcard/Download/xnnpack_llama2.pte /data/local/tmp/llama/xnnpack_llama2.pte

(make sure to change the file paths if they're different for your device or the model you're using!) 6. Open ExecuTorchLlamaDemo

Right now I'm getting the error "Load failed: 34".

I posted about this error at https://discuss.pytorch.org/t/android-demo-app-load-failed-34/203796.