What is the meaning of life? — As answered by a tiny edge AI . . . all locally.

Michael McAnally
3 min readNov 11, 2024

--

Image of my SBC with quarter for scale, 2 Wi-Fi antenna taped on
Using LLamafile running on Ubuntu one of a few answers given from different AI LLMs

Using LLamafile running on Ubuntu, here’s one of the answers given from different AI LLMs. Running various open-source LLMs on an SBC (Single Board Computer) like my Orange Pi 5 Plus can be challenging for such a small but powerful computer.

LLamafile, by Mozilla, ran the AIs on CPU only, and in some of the CPU graphs shown below, the usage is maxed out at 100%. Although it sometimes took several minutes to answer the question of life’s meaning, it’s still amazing to me that a small computer, running a condensed version of humanity’s trained, generalized knowledge, could even attempt to answer such a question. We are definitely in a new age of AI.

CPU graphics shown pegged out at 100%

Although the answer to lifes meaning took it in some cases several minutes. It’s still amazing to me that a little computer running a condensed version of all of humanities trained generalized knowledge could even answer such a question. We are definitely in a new age of AI, no doubt.

Note: All of the answers where inferenced locally on the SBC’s CPUs only. The ethernet cable was connected to download the needed software and allow for headless access from another computer screen.

I plan to experiment more with small edge AI devices and projects. I have a really exciting one coming up after figuring a few things out about portability, packaging, and power.

So stay tuned, and give a clap if you enjoyed this!

My favorite uncensored open source model is Dolphin 2.9.4 Llama 3.1

Other recent articles I have written about AI on computers.

--

--

Michael McAnally
Michael McAnally

Written by Michael McAnally

Temporary gathering of sentient stardust. Free thinker, evolving human, writer, coder, artist, humanitarian. Semi-retired, doctorate level researcher.

No responses yet