• 0 Posts
  • 5 Comments
Joined 6 months ago
cake
Cake day: July 29th, 2024

help-circle
  • Name…absolutely does not check out.

    Uhh, oh, fair enough (゚∀゚)

    Saliently enough, have you managed to try DeepSeek, or even get it set up locally?

    Yeah, I’ve successfully run the cut down version of deepseek-r1 through Ollama. The model itself is the 7b (I’m VRAM limited to 8GB). I used an M1 Mac Mini to run it, in terms of performance, is fast and the quality of the generated content is okay.

    Depending on your hardware and SO, you will or not be able to get to run a LLM locally with reasonable speed. You might want to check the GPU support for Ollama. You don’t need a GPU as it can run on the CPU, but it’ll certainly be slower.