If you are like me and use AI often to help with complicated tasks and own a Flipper Zero, you might want a Super Tiny Language Model (STLM).
Why this is useful
Until now, there have been no language models small enough to run on a small device, such as a Flipper Zero. Flippermind relies on lightweight Python3 installs and a local connection to Qwen2.5-0.5B. Although this is configured for Flipper Zero, you can run it on other small machines like a Raspberry Pi.
Challenges
Throughout development, I found errors within the language model not loading. PyTorch and Python3 transformers are used to run the Python3 dependencies, and they would not run on Debian Linux distros. I fixed the install.sh to run on any OS.
How it works
There are a couple main components that make this function. The main function is the qwen_2_5_ask.py script located in tools/. This is script runs the language model when asked a question.
Example:
python3 qwen_2_5_ask.py "Hello world"
The install.sh file located in main works as the installer for the model.
You run it using:
bash install.sh
You can download it on the GitHub repo linked below.
How to contribute
To help me continue to work on this, check out our Github Repo
If you have questions or just want to talk, talk to me in the comments or email me at [email protected]
Suggestion:
"What do you guys think? Any ideas? I'm curious if anyone has successfully run other quantized models on hardware with this little RAM—let me know in the comments!"
SOCIAL SHARE CARD GENERATOR