𞋴𝛂𝛋𝛆

  • 4 Posts
  • 44 Comments
Joined 3 years ago
cake
Cake day: June 9th, 2023

help-circle

  • Just be aware that W11 is secure boot only.

    There is a lot of ambiguous nonsense about this subject by people that lack a fundamental understanding of secure boot. Secure Boot, is not supported by Linux at all. It is part of systems distros build outside of the kernel. These are different for various distros. Fedora does it best IMO, but Ubuntu has an advanced system too. Gentoo has tutorial information about how to setup the system properly yourself.

    The US government also has a handy PDF about setting up secure boot properly. This subject is somewhat complicated by the fact the UEFI bootloader graphical interface standard is only a reference implementation, with no guarantee that it is fully implemented, (especially the case in consumer grade hardware). Last I checked, Gentoo has the only tutorial guide about how to use an application called Keytool to boot directly into the UEFI system, bypassing the GUI implemented on your hardware, and where you are able to set your own keys manually.

    If you choose to try this, some guides will suggest using a better encryption key than the default. The worst that can happen is that the new keys will get rejected and a default will be refreshed. It may seem like your system does not support custom keys. Be sure to try again with the default for UEFI in your bootloader GUI implementation. If it still does not work, you must use Keytool.

    The TPM module is a small physical hardware chip. Inside there is a register that has a secret hardware encryption key hard coded. This secret key is never accessible in software. Instead, this key is used to encrypt new keys, and hash against those keys to verify that whatever software package is untampered with, and to decrypt information outside of the rest of the system using Direct Memory Access (DMA), as in DRAM/system memory. This effectively means some piece of software is able to create secure connections to the outside world using encrypted communications that cannot be read by anything else running on your system.

    As a more tangible example, Google Pixel phones are the only ones with a TPM chip. This TPM chip is how and why Graphene OS exists. They leverage the TPM chip to encrypt the device operating system that can be verified, and they create the secure encrypted communication path to manage Over The Air software updates automatically.

    There are multiple Keys in your UEFI bootloader on your computer. The main key is by the hardware manufacturer. Anyone with this key is able to change all software from UEFI down in your device. These occasionally get leaked or compromised too, and often the issue is never resolved. It is up to you to monitor and update… - as insane as it sounds.

    The next level key below, is the package key for an operating system. It cannot alter UEFI software, but does control anything that boots after. This is typically where the Microsoft key is the default. It means they effectively control what operating system boots. Microsoft has issued what are called shim keys to Ubuntu and Fedora. Last I heard, these keys expired in October 2025 and had to be refreshed or may not have been reissued by M$. This shim was like a pass for these two distros to work under the M$ PKey. In other words, vanilla Ubuntu and Fedora Workstation could just work with Secure Boot enabled.

    All issues in this space have nothing to do with where you put the operating systems on your drives. Stating nonsense about dual booting a partition is the stupid ambiguous misinformation that causes all of the problems. It is irrelevant where the operating systems are placed. Your specific bootloader implementation may be optimised to boot faster by jumping into the first one it finds. That is not the correct way for secure boot to work. It is supposed to check for any bootable code and deplete anything without a signed encryption key. People that do not understand this system, are playing a game of Russian Roulette. There one drive may get registered first in UEFI 99% of the time due to physical hardware PCB design and layout. That one time some random power quality issue shows up due to a power transient or whatnot, suddenly their OS boot entry is deleted.

    The main key, and package keys are the encryption key owners of your hardware. People can literally use these to log into your machine if they have access to these keys. They can install or remove software from this interface. You have the right to take ownership of your machine by setting these yourself. You can set the main key, then you can use the Microsoft system online to get a new package key to run W10 w/SB or W11. You can sign any distro or other bootable code with your main key. Other than the issue of one of the default keys from the manufacturer or Microsoft getting compromised, I think the only vulnerabilities that secure boot protects against are physical access based attacks in terms of 3rd party issues. The system places a lot of trust in the manufacturer and Microsoft, and they are the owners of the hardware that are able to lock you out of, surveil, or theoretically exploit you with stalkerware. In practice, these connections are still using DNS on your network. If you have not disabled or blocked ECH like cloudflare-ech.com, I believe it is possible for a server to make an ECH connection and then create a side channel connection that would not show up on your network at all. Theoretically, I believe Microsoft could use their PKey on your hardware to connect to your hardware through ECH after your machine connects to any of their infrastructure.

    Then the TMP chip becomes insidious and has the potential to create a surveillance state, as it can be used to further encrypt communications. The underlying hardware in all modern computers has another secret operating system too, so it does not need to cross your machine. For Intel, this system is call the Management Engine. In AMD it is the Platform Security Processor. In ARM it is called TrustZone.

    Anyways, all of that is why it is why the Linux kernel does not directly support secure boot, the broader machinery, and the abstracted broader implications of why it matters.

    I have a dual boot w11 partition on the same drive with secure boot and have had this for the last 2 years without ever having an issue. It is practically required to do this if you want to run CUDA stuff. I recommend owning your own hardware whenever possible.



  • The UEFI boot system is tricky and you need to get along with Secure Boot to do this. Secure Boot is outside of the Linux kernel. Both Fedora and Ubuntu have systems for this. Fedora uses the Anaconda system and I believe they do it best. I have had a W11 partition for 2 years and never used it once. It can’t even get on the internet with my firewall setup, but it is there and never had any issues the 3 times I logged into it.

    I think all of the Fedora systems support the shim key and secure boot but I know Workstation does. For Ubuntu I think it is just the regular vanilla Ubuntu desktop that the shim supports. This may be somewhat sketchy with Nvidia or maybe not. Nvidia “”““open sourced””“” their kernel code but the actual nvcc compiler required to build the binaries is still proprietary crap.

    I have a 3080Ti gaming laptop. It isn’t half bad with 16 GB of video RAM from all the way back in 2021. Nvidia is artificially holding back the vram because of monopoly nonsense. The new stuff has very little real consumer value as a result, at least with AI stuff I run. The hardware is a little faster, but more vram is absolutely critical and new stuff that is the same or worse than what I have from 3 generations and nearly 5 years ago is ridiculous.

    The battery life blows and the GPU likely won’t even work on battery. It will get donkey balls hot with AI workloads, especially any kind of image gen. This results in lots of thermal throttling. All AI packages run as servers on your network. If you are thinking along these lines if running your own models, get a tower and run the thing remotely.

    I manage, and need the ergonomics for physical disability reasons, but I still would prefer to have a separate tower to run models from.

    Anyways, you can sign your own UEFI keys to use any distro, but this can be daunting for some people. The US defense department has a good PDF guide on setting your own keys. The UEFI bootloader for the machine may not have all key signing features implemented. There is a way to boot into UEFI directly and set the keys manually but this is not easy to find great guides on how to do it step by step. Gentoo has a tutorial on this, but it assumes a high level of competency.

    Other than signing your own keys, the shim keys mentioned are special keys signed by Microsoft for the principal maintainer of the distro. These slide under the Microsoft key to keep secure boot enabled.

    If you boot any secure boot enabled OS, the bootloader is required to delete any bootable unsigned code it finds. It does not matter if it is a shimmed Fedora or W11. If you have any other OS present in the boot list, it should be deleted. W11 is SB only, and this is where the real issues arise.


  • 𞋴𝛂𝛋𝛆@lemmy.worldtoLinux@lemmy.mlSecurity Focused Daily Driving Distros?
    link
    fedilink
    English
    arrow-up
    4
    arrow-down
    3
    ·
    edit-2
    6 months ago

    Are you insane? Debian is a base distro like any other and runs more hardware than any other. It has all of the bootstrapping tools to get hardware working.

    Canonical is a server company and Ubuntu server is literally the product.

    Arch is absolute garbage for most users unless you have a CS degree or you have entirely too much time on your hands and don’t mind an OS as your life project. Arch abhors tutorial content in all documentation and therefore dumps users into a rabbit hole regularly. Pacman is the worst package manager as it will actively break a system and present the user with the dumbest of choices at random because the maintainers are ultimately sadistic and lackadaisical. Arch is nearly identical to Gentoo with Arch binaries often based on Gentoo builds, yet Gentoo provides relevant instruction and documentation with any changes that require user intervention and does so at a responsible and ethical level that shows kindness, respect, and consideration completely absent from Arch. Arch is a troll by trolls for trolls. I’m more than capable of running it now, but I would never bother with such inconsiderate behavior.







  • Why are you confrontational? I’m just casually tossing out ideas and learning. Of course I understand what you are saying. However, busybox covers the core of a POSIX system and with the size constraints, it is likely standardising something like this. On Gentoo, such a change might be more straight forward instead of some sloppy hack with a wrapper.

    I imagine you must be good at memorizing a lot of information. I am not. I am good at abstraction and must explore in abstraction to understand heuristically. I understand heuristic connections better than most people. Neither method is better or worse. Being toxic about interchanges of information is useless nonsense. I know far more than I let on, but I’m well aware that I am a jack of all trades and expert of none. All the projects don’t matter relative to those that are used the most. If most projects can be colorized, it will motivate others to fall in line or prompt rewrites assuming such a change was popular. Colorized manpages and help pages should be standard and should have been a decade ago. No one is using an IDE without syntax highlighting. The terminal is an extension of the abstracted language of Linux. Without universal syntax highlighting for new users in these spaces, Linux is presenting an outdated language format ripe for deprecation. These details have long term consequences.










  • I haven’t looked into the issue of PCIe lanes and the GPU.

    I don’t think it should matter with a smaller PCIe bus, in theory, if I understand correctly (unlikely). The only time a lot of data is transferred is when the model layers are initially loaded. Like with Oobabooga when I load a model, most of the time my desktop RAM monitor widget does not even have the time to refresh and tell me how much memory was used on the CPU side. What is loaded in the GPU is around 90% static. I have a script that monitors this so that I can tune the maximum number of layers. I leave overhead room for the context to build up over time but there are no major changes happening aside from initial loading. One just sets the number of layers to offload on the GPU and loads the model. However many seconds that takes is irrelevant startup delay that only happens once when initiating the server.

    So assuming the kernel modules and hardware support the more narrow bandwidth, it should work… I think. There are laptops that have options for an external FireWire GPU too, so I don’t think the PCIe bus is too baked in.


  • 𞋴𝛂𝛋𝛆@lemmy.worldtoSelfhosted@lemmy.worldConsumer GPUs to run LLMs
    link
    fedilink
    English
    arrow-up
    9
    arrow-down
    1
    ·
    edit-2
    9 months ago
    Anything under 16 is a no go. Your number of CPU cores are important. Use Oobabooga Textgen for an advanced llama.cpp setup that splits between the CPU and GPU. You'll need at least 64 GB of RAM or be willing to offload layers using the NVME with deepspeed. I can run up to a 72b model with 4 bit quantization in GGUF with a 12700 laptop with a mobile 3080Ti which has 16GB of VRAM (mobile is like that).

    I prefer to run a 8×7b mixture of experts model because only 2 of the 8 are ever running at the same time. I am running that in 4 bit quantized GGUF and it takes 56 GB total to load. Once loaded it is about like a 13b model for speed but is ~90% of the capabilities of a 70b. The streaming speed is faster than my fastest reading pace.

    A 70b model streams at my slowest tenable reading pace.

    Both of these options are exponentially more capable than any of the smaller model sizes even if you screw around with training. Unfortunately, this streaming speed is still pretty slow for most advanced agentic stuff. Maybe if I had 24 to 48gb it would be different, I cannot say. If I was building now, I would be looking at what hardware options have the largest L1 cache, the most cores that include the most advanced AVX instructions. Generally, anything with efficiency cores are removing AVX and because the CPU schedulers in kernels are usually unable to handle this asymmetry consumer junk has poor AVX support. It is quite likely that all the problems Intel has had in recent years has been due to how they tried to block consumer stuff from accessing the advanced P-core instructions that were only blocked in microcode. It requires disabling the e-cores or setting up a CPU set isolation in Linux or BSD distros.

    You need good Linux support even if you run windows. Most good and advanced stuff with AI will be done with WSL if you haven’t ditched doz for whatever reason. Use https://linux-hardware.org/ to see support for devices.

    The reason I mentioned avoid consumer e-cores is because there have been some articles popping up lately about all p-core hardware.

    The main constraint for the CPU is the L2 to L1 cache bus width. Researching this deeply may be beneficial.

    Splitting the load between multiple GPUs may be an option too. As of a year ago, the cheapest option for a 16 GB GPU in a machine was a second hand 12th gen Intel laptop with a 3080Ti by a considerable margin when all of it is added up. It is noisy, gets hot, and I hate it many times, wishing I had gotten a server like setup for AI, but I have something and that is what matters.