tl-dr

-Can someone give me step by step instructions (ELI5) on how to get access to my LLM’s on my rig from my phone?

Jan seems the easiest but I’ve tried with Ollama, librechat, etc.

…

I’ve taken steps to secure my data and now I’m going the selfhosting route. I don’t care to become a savant with the technical aspects of this stuff but even the basics are hard to grasp! I’ve been able to install a LLM provider on my rig (Ollama, Librechat, Jan, all of em) and I can successfully get models running on them. BUT what I would LOVE to do is access the LLM’s on my rig from my phone while I’m within proximity. I’ve read that I can do that via wifi or LAN or something like that but I have had absolutely no luck. Jan seems the easiest because all you have to do is something with an API key but I can’t even figure that out.

Any help?

  • BlackSnack@lemmy.zipOPEnglish
    2·
    3 hours ago

    There are 3 lines with the :11434 in them. No brackets or anything like that. -1 has 0.0.0.0 in front -2 has 10.#.#.# in front and has a foreign address that is something other than 0.0.0 -3 is like the 2nd but a slightly different foreign address

    The iPhone does have a 10.#.#.# ip number that is slightly different than the PCs.

    The subnet mask is 255.255.255.0

    Oh yes. I’m on windows 10 as well.

    I have taken a pause here while we trouble shoot the subnet mask. We’re getting close!!

    • tal@lemmy.todayEnglish
      1·
      2 hours ago

      There are 3 lines with the :11434 in them. No brackets or anything like that. -1 has 0.0.0.0 in front -2 has 10.#.#.# in front and has a foreign address that is something other than 0.0.0 -3 is like the 2nd but a slightly different foreign address

      Okay, that…should be okay. As long as all of the addresses that it’s listening on are IPv4 — of the format “x.x.x.x”. No colons in them (other than the colon preceeding “11434”). Not IPv6.

      The subnet mask is 255.255.255.0

      Okay, gotcha. In that case, go ahead with the instructions above, just instead of “/8”, do “/24”. So:

      For “local IP addresses”, you want “These IP Addresses”, and enter 10.0.0.0/24. That’ll be every IPv4 address on your Windows LLM that has “10” as its first number and the first following two numbers the same as yours.

      For “remote IP addresses”, you want “These IP Addresses”, and enter 10.0.0.0/24. Same thing all addresses that start with a “10.” followed by the same following two numbers, which should include your iOS device.

      Oh yes. I’m on windows 10 as well.

      Okay. I think that the interface to add the firewall rule there looks the same as the one I Iinked to above. I went searching for screenshots of adding a hole for a port on Windows 10, and the control panel looks identical to me.

      So, yeah, should be good to go ahead with the above instructions, just using “/24” instead of “/8” in the two places where I mention “/8”. Hopefully after that it’ll be working; if not, then we’ll need to troubleshoot.