• 0 Posts
  • 14 Comments
Joined 1 year ago
cake
Cake day: June 12th, 2023

help-circle



  • Taking ollama for instance, either the whole model runs in vram and compute is done on the gpu, or it runs in system ram and compute is done on the cpu. Running models on CPU is horribly slow. You won’t want to do it for large models

    LM studio and others allow you to run part of the model on GPU and part on CPU, splitting memory requirements but still pretty slow.

    Even the smaller 7B parameter models run pretty slow in CPU and the huge models are orders of magnitude slower

    So technically more system ram will let you run some larger models but you will quickly figure out you just don’t want to do it.






  • Just a note, the orange pi drivers are not in great shape. It’s getting better but I have a cluster of raspberry pi’s for development, bought an orange pi without first checking out much about them and it’s rough. Rockchip CPUs are great, and the driver / firmware situation is getting better, but something I’d read up on before buying one.

    I’d still look at the N100, it’s about 2.5x the performance of raspberry pi 5, and being x86 you have more options than arm.



  • TPM & secure boot. Look into sbctl for secure boot if you’re not on something that uses the signed shim like ubuntu. I know some hate secure boot but storing the unlock key in tpm is at least much more secure than having the key sitting on a usb drive

    Tang - network based unlock. If you have a separate raspberry pi or something you can set it up as a tang server. You’ll want that thing encrypted too, can set that up to require manual unlock so if someone boosts your servers the tang server never comes up, storage server won’t either

    Or just manually unlock the server with a password every boot?

    That’s roughly my prioritized/preferred list




  • It depends how they clone it. I’m assuming now your 250gb drive is c, 1tb drive is d. After the cloning if you want the 1tb to be c and 2TB to be d, just tell them what you want and they should be able to make that happen.

    For a bit more technical info, you also have a small EFI partition (unless this pc is very old), probably on your 250gb drive. This partition is what your computer boots from initially. When windows is installed it writes information to that partition, both initial boot binaries but also information about where your windows partition is. When they clone the 250gb drive they’ll also clone that partition, and depending on the method they use to clone, that pointer from efi will either not need to be modified, or they’ll fix it with tools called bcdboot and bcdedit. Bcedit has some read-only commands but I wouldn’t suggest messing with either, just mentioning them if you’re curious to read about them and understand the process a bit more