• 2 Posts
  • 98 Comments
Joined 8 months ago
cake
Cake day: March 22nd, 2024

help-circle











  • Colonization doesn’t make sense in light of what’s likely to come first. Artifical intelligence, mind uploading, extensive genetic engineering, programmable nanotech for fabrication, take your pick… All these are infinitely more reachable and cheaper than dedicating tons of resources to sustaining a squishy, fragile human bodies in space while the vast majority are still stuck on Earth due to economic constraints.

    It’s just not economical until humans are so different that it doesn’t really resemble are Star Trek-ish visions of humans on space boats (eg they’re flying around in computers, AI are sent ahead to construct habitation, bodies are genetically engineered for survival in space, that sort of thing).

    Again, I am not talking about research or the glory of stepping foot somewhere, but I just don’t see the point of trying to emulate a traditional human living in an environment where it’s so impractical.


  • Two things:

    • That was kinda the dream after WWII, no?

    • Exploring space should be a uniting purpose of humanity, but colonizing space, as humans live now, is just wildly, hilariously impractical. It would be orders of magnitude cheaper and easier to live at the bottom of the ocean, or under the antarctic ice sheet. And this is speaking as someone really into exotic rocketry and transcendental sci-fi.

    I’d recommend reading through Project Rho, if you’re interested: https://projectrho.com/public_html/rocket/

    As well as “farther future” but grounded Sci-Fi like Orion’s Arm, where humanity doesn’t really resemble its current form. And play KSP! The more you read and see, the more you realize “wow, sending humans through space is hard, and living there kinda doesn’t make sense right now.”


  • If Ehud Barak had gone back to the Israeli people with “You have to give them back their houses and stop encircling/blockading their settlements”, he’d have been assassinated by the Israelis.

    Isn’t that the nature of a “winner takes all” knife’s edge political system, though? If the opposition were in power, they would have done something like this, and Israel would hate it, but they’d have to take it just like they took what they didn’t like over the past decades. Maybe they’d lose the next election (and get assassinated), but the deed would already be done.

    …Or maybe I’m totally wrong.






  • To go into more detail:

    • Exllama is faster than llama.cpp with all other things being equal.

    • exllama’s quantized KV cache implementation is also far superior, and nearly lossless at Q4 while llama.cpp is nearly unusable at Q4 (and needs to be turned up to Q5_1/Q4_0 or Q8_0/Q4_1 for good quality)

    • With ollama specifically, you get locked out of a lot of knobs like this enhanced llama.cpp KV cache quantization, more advanced quantization (like iMatrix IQ quantizations or the ARM/AVX optimized Q4_0_4_4/Q4_0_8_8 quantizations), advanced sampling like DRY, batched inference and such.

    It’s not evidence or options… it’s missing features, thats my big issue with ollama. I simply get far worse, and far slower, LLM responses out of ollama than tabbyAPI/EXUI on the same hardware, and there’s no way around it.

    Also, I’ve been frustrated with implementation bugs in llama.cpp specifically, like how llama 3.1 (for instance) was bugged past 8K at launch because it doesn’t properly support its rope scaling. Ollama inherits all these quirks.

    I don’t want to go into the issues I have with the ollama devs behavior though, as that’s way more subjective.


  • It’s less optimal.

    On a 3090, I simply can’t run Command-R or Qwen 2.5 34B well at 64K-80K context with ollama. Its slow even at lower context, the lack of DRY sampling and some other things majorly hit quality.

    Ollama is meant to be turnkey, and thats fine, but LLMs are extremely resource intense. Sometimes the manual setup/configuration is worth it to squeeze out every ounce of extra performance and quantization quality.

    Even on CPU-only setups, you are missing out on (for instance) the CPU-optimized quantizations llama.cpp offers now, or the more advanced sampling kobold.cpp offers, or more fine grained tuning of flash attention configs, or batched inference, just to start.

    And as I hinted at, I don’t like some other aspects of ollama, like how they “leech” off llama.cpp and kinda hide the association without contributing upstream, some hype and controversies in the past, and hints that they may be cooking up something commercial.