

Sure! I mostly followed this random youtuber’s video for getting Wyoming protocols offloaded (Whisper/Piper), but he didn’t get Ollama to use his GPU: https://youtu.be/XvbVePuP7NY.
For getting the Nvidia/Docker passthrough, I used this guide: https://www.bittenbypython.com/en/posts/install_ollama_openwebui_ubuntu_nvidia/.
It’s working fairly great at this point!
Hosting on the public web isn’t too crazy - start with port forwarding on standard ports (443 for sale/web) and add in a dynamic DNS address.
More than likely your residential ISP doesn’t change your IP that often, but Dynamic DNS solves that problem before it hits. I use Cloudflare, but mostly because I’m lazy and haven’t moved off of them after their most recent sketch behavior.