{"id":2459,"date":"2025-11-18T10:26:19","date_gmt":"2025-11-18T09:26:19","guid":{"rendered":"https:\/\/yellotab.se\/x056\/?p=2459"},"modified":"2025-11-18T18:20:11","modified_gmt":"2025-11-18T17:20:11","slug":"decentraliserad-llm-pa-proxmox","status":"publish","type":"post","link":"https:\/\/yellotab.se\/x056\/2025\/11\/18\/decentraliserad-llm-pa-proxmox\/","title":{"rendered":"Decentraliserad LLM p\u00e5 Proxmox"},"content":{"rendered":"\n

Setup f\u00f6r LLM’s p\u00e5 Proxmox Virtualizer p\u00e5 Bee-link GTi15 med docka och ext. GPU<\/em>
Intel Core Ultra 9 258H (Meteor Lake)<\/strong> \u2192 stark CPU + Intel Arc iGPU (Xe LPG)<\/p>\n\n\n\n

Extern docka med Sapphire NITRO+ 8GB (AMD GPU)<\/strong> \u2192 via eGPU-docka<\/p>\n\n\n\n

Ollama<\/strong>\u00a0\u00e4r ett program (runtime) som l\u00e5ter dig\u00a0k\u00f6ra stora spr\u00e5kmodeller (LLM)<\/strong>\u00a0lokalt p\u00e5 din egen dator eller server, t.ex. p\u00e5 en Proxmox-VM eller LXC. Det fungerar ungef\u00e4r som\u00a0Docker, men f\u00f6r AI-modeller<\/em>.
Llama<\/strong>\u00a0(Large Language Model Meta AI) \u00e4r en serie\u00a0spr\u00e5kmodeller utvecklade av Meta<\/strong>. De \u00e4r \u00f6ppna och effektiva, vilket g\u00f6r dem v\u00e4ldigt popul\u00e4ra att k\u00f6ra lokalt via just Ollama.
Hugging Face<\/strong> \u00e4r en stor plattform d\u00e4r AI-modeller delas,Det fungerar som\u00a0GitHub f\u00f6r AI-modeller<\/strong>.<\/p>\n\n\n\n

Vad Ollama g\u00f6r:<\/strong><\/h3>\n\n\n\n
    \n
  • H\u00e4mtar modeller (t.ex. Llama, Mistral, Phi m.fl.).<\/li>\n\n\n\n
  • Optimerar dem f\u00f6r din h\u00e5rdvara (CPU\/GPU).<\/li>\n\n\n\n
  • Startar\/stoppar modeller via kommandon som:Ger ett API du kan anv\u00e4nda i appar eller script.<\/li>\n\n\n\n
  • Du kan allts\u00e5 k\u00f6ra avancerade AI-modeller offline, gratis och utan moln<\/strong>.<\/li>\n<\/ul>\n\n\n\n

    Hardware Spec<\/strong>: Intel Core Ultra9 285H , 64GB, 1TB Saphire Nitro 8GB AMD
    Plattform<\/strong>: Proxmox, Debian 9<\/p>\n\n\n\n

    Host system:<\/strong> LXC Ubuntu 24.04 LTS, Privileged, GPU Passthrough,
    Application<\/strong>: , Ollama, LLaMA 3.1 8B<\/strong><\/p>\n\n\n\n

    <VMID> = 102 | Id f\u00f6r LXC container LLM<\/p>\n\n\n\n

    Funktion<\/th>LXC container<\/th>VM<\/th><\/tr><\/thead>
    Viktiga skillnader<\/td>Privileged containers:<\/strong>
    \u2705 B\u00e4ttre GPU\/device support
    \u2705 Enklare att k\u00f6ra Docker
    \u2705 B\u00e4ttre prestanda f\u00f6r vissa arbetsbelastningar
    \u274c Mindre s\u00e4kra (k\u00f6r som root p\u00e5 host)
    \u274c Kan komma \u00e5t host systemet mer
    Unprivileged containers:<\/strong>
    \u2705 Mycket s\u00e4krare
    \u2705 Isolerade UID\/GID mappningar
    \u274c Begr\u00e4nsad hardware access
    \u274c Kr\u00e5ngligare med GPU passthrough<\/td>    		F\u00f6r GPU + AI-arbete<\/strong> skulle jag rekommendera L\u00f6sning 4 (VM)<\/strong> eftersom:
    \u2705 B\u00e4ttre GPU passthrough support
    \u2705 Enklare drivrutinsinstallation
    \u2705 B\u00e4ttre prestanda f\u00f6r AI-modeller
    \u2705 Mindre kr\u00e5ngel med privilegier<\/td><\/tr>
    <\/td>\ud83e\udde9 Viktigt: Vilken GPU kan anv\u00e4ndas i LXC?<\/strong>

    \u2714 CPU fungerar alltid (Intel Core Ultra 9)<\/strong>
    LLaMA 3.1 8B fungerar bra p\u00e5 CPU-only med 16\u201320 GB RAM.
    \u2714 Intel iGPU (Xe)<\/strong>
    Intel har st\u00f6d i llama.cpp<\/strong> via Intel oneAPI<\/strong>.
    \u2192 Kan k\u00f6ras i LXC.

    \u2714 AMD Sapphire Nitro 8GB (eGPU)<\/strong>
    AMD GPU fungerar inte direkt i LXC<\/em> utan specialkonfig, men fungerar i KVM\/VM utan problem<\/strong>.

    AMD ROCm i LXC fungerar endast p\u00e5 vissa k\u00e4rnor och kr\u00e4ver privileged + device passthrough.<\/strong>
    Vi kan testa, men CPU eller Intel iGPU \u00e4r enklare.<\/td>    		<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

    I Proxmox host<\/h2>\n\n\n\n
    <\/th><\/th><\/tr><\/thead>
    pct set 102 -unprivileged 0<br>pct set 102 -features nesting=1<br>pct set 102 -features "keyctl=1,nesting=1,fuse=1"<\/code><\/td><\/td><\/tr>
    <\/td><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

    I LXC container<\/h2>\n\n\n\n
    <\/th>Gemensamt<\/th><\/tr><\/thead>
    AI Container: Ollama<\/span><\/td>Ollama beh\u00f6ver Python 3.11+ och pip.<\/span><\/td><\/tr>
    Spr\u00e5kmodell: LLaMa<\/td><\/td><\/tr><\/tbody><\/table><\/figure>\n","protected":false},"excerpt":{"rendered":"

    Setup f\u00f6r LLM’s p\u00e5 Proxmox Virtualizer p\u00e5 Bee-link GTi15 med docka och ext. GPUIntel Core Ultra 9 258H (Meteor Lake) \u2192 stark CPU + Intel Arc iGPU (Xe LPG) Extern docka med Sapphire NITRO+ 8GB (AMD GPU) \u2192 via eGPU-docka Ollama\u00a0\u00e4r ett program (runtime) som l\u00e5ter dig\u00a0k\u00f6ra stora spr\u00e5kmodeller (LLM)\u00a0lokalt p\u00e5 din egen dator eller server, t.ex. […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-2459","post","type-post","status-publish","format-standard","hentry","category-news"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/yellotab.se\/x056\/wp-json\/wp\/v2\/posts\/2459","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/yellotab.se\/x056\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/yellotab.se\/x056\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/yellotab.se\/x056\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/yellotab.se\/x056\/wp-json\/wp\/v2\/comments?post=2459"}],"version-history":[{"count":13,"href":"https:\/\/yellotab.se\/x056\/wp-json\/wp\/v2\/posts\/2459\/revisions"}],"predecessor-version":[{"id":2480,"href":"https:\/\/yellotab.se\/x056\/wp-json\/wp\/v2\/posts\/2459\/revisions\/2480"}],"wp:attachment":[{"href":"https:\/\/yellotab.se\/x056\/wp-json\/wp\/v2\/media?parent=2459"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/yellotab.se\/x056\/wp-json\/wp\/v2\/categories?post=2459"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/yellotab.se\/x056\/wp-json\/wp\/v2\/tags?post=2459"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}