> ## Documentation Index
> Fetch the complete documentation index at: https://usefulai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Best Local LLM Tools in 2026

> We compared the best local LLM tools for running open-weight models on your own hardware, compared on setup speed, privacy, APIs, and hardware support.

<div className="uai-updated-row">Updated June 1, 2026</div>

Local LLM tools let you run open-weight models on your own hardware - no API keys, no per-token billing, no data leaving the machine. The category is not one thing anymore: runtimes, desktop apps, shared web UIs, and document workspaces solve different jobs. We compared more than 15 options and selected seven picks that cover the main local AI workflows.

## Best Local LLM Tools

<div className="uai-overview-table uai-overview-table--ranked">
  | # | Tool | Best for | Type |
  | -: | - | - | - |
  | 1 | <a href="https://ollama.com/" target="_blank" rel="noreferrer"><img src={"/images/icons/48/ollama.com.png"} alt="" noZoom />Ollama</a> | Default local backend and API | <span className="uai-badge uai-badge--sky">Runtime</span> |
  | 2 | <a href="https://lmstudio.ai/" target="_blank" rel="noreferrer"><img src={"/images/icons/48/lmstudio.ai.png"} alt="" noZoom />LM Studio</a> | All-in-one desktop app | <span className="uai-badge uai-badge--violet">Desktop</span> |
  | 3 | <a href="https://www.jan.ai/" target="_blank" rel="noreferrer"><img src={"/images/icons/48/jan.ai.png"} alt="" noZoom />Jan</a> | Open-source desktop assistant | <span className="uai-badge uai-badge--violet">Desktop</span> |
  | 4 | <a href="https://openwebui.com/" target="_blank" rel="noreferrer"><img src={"/images/icons/48/openwebui.com.png"} alt="" noZoom />Open WebUI</a> | Self-hosted shared web UI | <span className="uai-badge uai-badge--amber">Portal</span> |
  | 5 | <a href="https://github.com/ggml-org/llama.cpp" target="_blank" rel="noreferrer"><img src={"/images/icons/48/github.com.png"} alt="" noZoom />llama.cpp</a> | Low-level runtime control | <span className="uai-badge uai-badge--sky">Runtime</span> |
  | 6 | <a href="https://anythingllm.com/" target="_blank" rel="noreferrer"><img src={"/images/icons/48/anythingllm.com.png"} alt="" noZoom />AnythingLLM</a> | Local document and RAG workspace | <span className="uai-badge uai-badge--zinc">Workspace</span> |
  | 7 | <a href="https://github.com/oobabooga/textgen" target="_blank" rel="noreferrer"><img src={"/images/icons/48/github.com.png"} alt="" noZoom />TextGen</a> | Power-user local workbench | <span className="uai-badge uai-badge--zinc">Workspace</span> |
</div>

<label className="uai-overview-more">
  <input type="checkbox" className="uai-overview-toggle" />

  <span className="uai-overview-more-open"><span className="uai-overview-more-count">Show more</span><Icon icon="chevron-down" size={13} /></span>
  <span className="uai-overview-more-close"><span>Show less</span><Icon icon="chevron-up" size={13} /></span>
</label>

***

<div className="uai-itemcard" role="article">
  <div className="uai-itemcard-head">
    <span className="uai-itemcard-icon">
      <img src="https://mintcdn.com/usefulai/Te6KzZ86-OxPuEC2/images/icons/144/ollama.com.png?fit=max&auto=format&n=Te6KzZ86-OxPuEC2&q=85&s=dd9820be3d7debedc48768a402c1988f" alt="" noZoom loading="lazy" width="144" height="144" data-path="images/icons/144/ollama.com.png" />
    </span>

    <div className="uai-itemcard-identity">
      <div className="uai-itemcard-row uai-itemcard-row--title">
        ## [Ollama](https://ollama.com/)
      </div>

      <div className="uai-itemcard-row">
        <span className="uai-itemcard-note uai-itemcard-note--green">Best for default local backend and API</span>
      </div>
    </div>

    <div className="uai-itemcard-end">
      <a href="https://ollama.com/" target="_blank" rel="noreferrer" aria-label="Visit Ollama" className="uai-itemcard-cta no-underline">Visit Ollama</a>
    </div>
  </div>

  <div className="uai-itemcard-body">
    <div className="uai-itemcard-summary">
      Ollama is what most local LLM tutorials and apps assume you have running in the background. Pull a model, run it, and point any OpenAI-compatible client at the local API. It is not a polished chat app or a document workspace, but it is the shortest path from zero to a working local model that other tools can use. Treat it as the backend, not the whole product.
    </div>

    <div className="uai-itemcard-facts" aria-label="Ollama facts">
      <span>Platforms <span className="uai-itemcard-platforms" aria-label="Mac, Windows, Linux, API"><span title="Mac" className="uai-itemcard-pficon uai-itemcard-pficon--apple" /><span title="Windows" className="uai-itemcard-pficon uai-itemcard-pficon--windows" /><span title="Linux" className="uai-itemcard-pficon uai-itemcard-pficon--linux" /><span title="API" className="uai-itemcard-pficon uai-itemcard-pficon--code" /></span></span>
      <span>Type <strong>Runtime</strong></span>
    </div>

    <div className="uai-itemcard-details-group">
      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-ollama-strengths" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-ollama-strengths"><span>Strengths</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          <ul>
            <li>Fastest path to a usable local API - pull a model and serve it, then point Open WebUI, AnythingLLM, a coding agent, or your own script at localhost.</li>
            <li>Reusable model configurations - Modelfiles bake a system prompt, parameters, and base model into a named variant that behaves the same across scripts and teammates.</li>
            <li>Clear local-vs-cloud boundary - cloud tiers exist but don't gate local use; local hardware inference stays free and unlimited, and you can disable cloud entirely.</li>
          </ul>
        </div>
      </div>

      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-ollama-weaknesses" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-ollama-weaknesses"><span>Weaknesses</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          <ul>
            <li>Not a complete workspace - it runs models but gives no polished chat UI, document workspace, or team portal; pair it with Open WebUI or AnythingLLM.</li>
            <li>Agent tools need another layer - MCP and tool use usually depend on a client, bridge, or separate UI, so setup friction arrives fast from the runtime alone.</li>
          </ul>
        </div>
      </div>

      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-ollama-who-it-s-for" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-ollama-who-it-s-for"><span>Who it's for</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          Best if you are wiring local models into other apps, scripts, or APIs, or running a home lab. Skip for a polished GUI - LM Studio handles that. Skip if your real workflow is documents - AnythingLLM packages that better.
        </div>
      </div>
    </div>
  </div>
</div>

<div className="uai-itemcard" role="article">
  <div className="uai-itemcard-head">
    <span className="uai-itemcard-icon">
      <img src="https://mintcdn.com/usefulai/52KaILwddzz5TNZ_/images/icons/144/lmstudio.ai.png?fit=max&auto=format&n=52KaILwddzz5TNZ_&q=85&s=ff7dbc670a9a10510028aa6c8908663f" alt="" noZoom loading="lazy" width="144" height="144" data-path="images/icons/144/lmstudio.ai.png" />
    </span>

    <div className="uai-itemcard-identity">
      <div className="uai-itemcard-row uai-itemcard-row--title">
        ## [LM Studio](https://lmstudio.ai/)
      </div>

      <div className="uai-itemcard-row">
        <span className="uai-itemcard-note uai-itemcard-note--green">Best for all-in-one desktop app</span>
      </div>
    </div>

    <div className="uai-itemcard-end">
      <a href="https://lmstudio.ai/" target="_blank" rel="noreferrer" aria-label="Visit LM Studio" className="uai-itemcard-cta no-underline">Visit LM Studio</a>
    </div>
  </div>

  <div className="uai-itemcard-body">
    <div className="uai-itemcard-summary">
      LM Studio is the easiest way to see, download, chat with, and serve local models without touching a terminal. Browse the catalog in-app, watch your VRAM as inference runs, then flip on an OpenAI-compatible server when other clients need to connect. Most of the first-month friction goes away.
    </div>

    <div className="uai-itemcard-facts" aria-label="LM Studio facts">
      <span>Platforms <span className="uai-itemcard-platforms" aria-label="Mac, Windows, Linux, API"><span title="Mac" className="uai-itemcard-pficon uai-itemcard-pficon--apple" /><span title="Windows" className="uai-itemcard-pficon uai-itemcard-pficon--windows" /><span title="Linux" className="uai-itemcard-pficon uai-itemcard-pficon--linux" /><span title="API" className="uai-itemcard-pficon uai-itemcard-pficon--code" /></span></span>
      <span>Type <strong>Desktop</strong></span>
    </div>

    <div className="uai-itemcard-details-group">
      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-lm-studio-strengths" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-lm-studio-strengths"><span>Strengths</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          <ul>
            <li>Best desktop model browsing - search, download, compare model sizes, and watch hardware use in one app, so you know what fits your GPU before downloading weights.</li>
            <li>Flips from GUI to local server - start in the chat window, then turn on an OpenAI- or Anthropic-compatible API for any local client, skipping the CLI runtime step.</li>
            <li>Strong Apple Silicon path - MLX updates and MTP speculative decoding sit behind GUI toggles, the fastest way to feel a speed difference on M-series Macs.</li>
          </ul>
        </div>
      </div>

      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-lm-studio-weaknesses" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-lm-studio-weaknesses"><span>Weaknesses</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          <ul>
            <li>Heavier than a minimal server - for a small always-on local model service the full desktop app feels like overhead; Ollama or llama.cpp's server are leaner.</li>
            <li>Advanced runtime details are abstracted - friendly defaults hide enough that you hit limits tuning unusual models or backends; llama.cpp or TextGen give more knobs.</li>
          </ul>
        </div>
      </div>

      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-lm-studio-who-it-s-for" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-lm-studio-who-it-s-for"><span>Who it's for</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          Best if you want a friendly desktop app with a local API on tap, especially on a Mac. Skip if you need a minimal always-on server - Ollama or llama.cpp are leaner. Skip if open source is a hard requirement - try Jan.
        </div>
      </div>
    </div>
  </div>
</div>

<div className="uai-itemcard" role="article">
  <div className="uai-itemcard-head">
    <span className="uai-itemcard-icon">
      <img src="https://mintcdn.com/usefulai/52KaILwddzz5TNZ_/images/icons/144/jan.ai.png?fit=max&auto=format&n=52KaILwddzz5TNZ_&q=85&s=241fd625576d4442cd4076257649e7a2" alt="" noZoom loading="lazy" width="144" height="144" data-path="images/icons/144/jan.ai.png" />
    </span>

    <div className="uai-itemcard-identity">
      <div className="uai-itemcard-row uai-itemcard-row--title">
        ## [Jan](https://www.jan.ai/)
      </div>

      <div className="uai-itemcard-row">
        <span className="uai-itemcard-note uai-itemcard-note--green">Best for open-source desktop assistant</span>
      </div>
    </div>

    <div className="uai-itemcard-end">
      <a href="https://www.jan.ai/" target="_blank" rel="noreferrer" aria-label="Visit Jan" className="uai-itemcard-cta no-underline">Visit Jan</a>
    </div>
  </div>

  <div className="uai-itemcard-body">
    <div className="uai-itemcard-summary">
      Jan is what LM Studio would look like if you started from open-source-first principles and wanted a desktop assistant rather than a model browser. The app handles local chat, hands its models off to a CLI and OpenAI-compatible server, and is pushing toward local agent launches for coding and tool workflows. The product shape is current; some agent and router pieces are still maturing.
    </div>

    <Frame>
      <img src="https://mintcdn.com/usefulai/FsW1srcineeRePXa/images/tools/local-llm-tools/jan.png?fit=max&auto=format&n=FsW1srcineeRePXa&q=85&s=4591f8aebaa7e20f7f85b86d38aba9a4" alt="Jan local AI desktop assistant" width="1440" height="900" data-path="images/tools/local-llm-tools/jan.png" />
    </Frame>

    <div className="uai-itemcard-facts" aria-label="Jan facts">
      <span>Platforms <span className="uai-itemcard-platforms" aria-label="Mac, Windows, Linux, API"><span title="Mac" className="uai-itemcard-pficon uai-itemcard-pficon--apple" /><span title="Windows" className="uai-itemcard-pficon uai-itemcard-pficon--windows" /><span title="Linux" className="uai-itemcard-pficon uai-itemcard-pficon--linux" /><span title="API" className="uai-itemcard-pficon uai-itemcard-pficon--code" /></span></span>
      <span>Type <strong>Desktop</strong></span>
    </div>

    <div className="uai-itemcard-details-group">
      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-jan-strengths" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-jan-strengths"><span>Strengths</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          <ul>
            <li>Open-source desktop assistant - Apache 2.0, no account, familiar chat shape, useful when you want LM Studio's feel in a tool you can inspect or fork.</li>
            <li>Desktop models carry into the CLI - models from the GUI are available to Jan's CLI and local server, removing duplicate setup when you wire a model into another app.</li>
            <li>Local agent launch is built in - it pushes beyond desktop chat into local model launch for coding and agent clients pointed at local hardware.</li>
          </ul>
        </div>
      </div>

      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-jan-weaknesses" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-jan-weaknesses"><span>Weaknesses</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          <ul>
            <li>Router controls are still settling - the CLI accepts some inference flags but ignores others, pushing tuning back through GUI presets; llama.cpp or TextGen for flag-level control.</li>
            <li>Thinner recipe library - third-party setup and troubleshooting writeups are less plentiful than for Ollama or LM Studio, so odd failures leave you on your own more.</li>
          </ul>
        </div>
      </div>

      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-jan-who-it-s-for" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-jan-who-it-s-for"><span>Who it's for</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          Best if you want LM Studio's experience but need open source, full local control, or a CLI/API alongside the chat window. Skip if you need a team portal - Open WebUI fits that. Skip for the most battle-tested setup recipes - start with Ollama.
        </div>
      </div>
    </div>
  </div>
</div>

<div className="uai-itemcard" role="article">
  <div className="uai-itemcard-head">
    <span className="uai-itemcard-icon">
      <img src="https://mintcdn.com/usefulai/Te6KzZ86-OxPuEC2/images/icons/144/openwebui.com.png?fit=max&auto=format&n=Te6KzZ86-OxPuEC2&q=85&s=da235210a1e679496f88c8a6adf7e91a" alt="" noZoom loading="lazy" width="144" height="144" data-path="images/icons/144/openwebui.com.png" />
    </span>

    <div className="uai-itemcard-identity">
      <div className="uai-itemcard-row uai-itemcard-row--title">
        ## [Open WebUI](https://openwebui.com/)
      </div>

      <div className="uai-itemcard-row">
        <span className="uai-itemcard-note uai-itemcard-note--green">Best for self-hosted shared web UI</span>
      </div>
    </div>

    <div className="uai-itemcard-end">
      <a href="https://openwebui.com/" target="_blank" rel="noreferrer" aria-label="Visit Open WebUI" className="uai-itemcard-cta no-underline">Visit Open WebUI</a>
    </div>
  </div>

  <div className="uai-itemcard-body">
    <div className="uai-itemcard-summary">
      Open WebUI gives you the ChatGPT-style browser experience over local backends like Ollama, llama.cpp, or any OpenAI-compatible provider - without putting your prompts through a third party. You run it, usually with Docker, point it at your runtime, and end up with a multi-user web app with RBAC, SSO, and admin controls. It is a UI and platform layer, not a model runtime, so you still need something underneath it.
    </div>

    <div className="uai-itemcard-facts" aria-label="Open WebUI facts">
      <span>Platforms <span className="uai-itemcard-platforms" aria-label="Web, Mac, Windows, Linux"><span title="Web" className="uai-itemcard-pficon uai-itemcard-pficon--globe" /><span title="Mac" className="uai-itemcard-pficon uai-itemcard-pficon--apple" /><span title="Windows" className="uai-itemcard-pficon uai-itemcard-pficon--windows" /><span title="Linux" className="uai-itemcard-pficon uai-itemcard-pficon--linux" /></span></span>
      <span>Type <strong>Portal</strong></span>
    </div>

    <div className="uai-itemcard-details-group">
      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-open-webui-strengths" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-open-webui-strengths"><span>Strengths</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          <ul>
            <li>Strongest shared browser UI for local AI - it makes a single Ollama or LM Studio install feel like a team product, with multiple users, conversations, and a model picker.</li>
            <li>Real admin controls - RBAC, groups, SSO/OIDC/LDAP, SCIM, API keys, and analytics make it the only pick here that fits an org chart and a security review.</li>
            <li>Backend-agnostic - it sits over Ollama, OpenAI-compatible providers, and multiple model sources at once, so you swap runtimes without changing the portal.</li>
          </ul>
        </div>
      </div>

      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-open-webui-weaknesses" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-open-webui-weaknesses"><span>Weaknesses</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          <ul>
            <li>Still need a backend underneath - it doesn't run models itself, so pair it with Ollama, llama.cpp, LM Studio, or an OpenAI-compatible endpoint.</li>
            <li>Ops burden grows with it - Docker, upgrades, security patches, database migrations, and auth become your problem as it scales.</li>
          </ul>
        </div>
      </div>

      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-open-webui-who-it-s-for" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-open-webui-who-it-s-for"><span>Who it's for</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          Best if you need a shared browser portal over local AI with real admin controls - team, lab, classroom, or home server. Skip if you are solo and want one app to install and chat - try LM Studio. Skip for document workflows - AnythingLLM is more direct.
        </div>
      </div>
    </div>
  </div>
</div>

<div className="uai-itemcard" role="article">
  <div className="uai-itemcard-head">
    <span className="uai-itemcard-icon">
      <img src="https://mintcdn.com/usefulai/Ez-pJDkPpztLE7Cr/images/icons/144/github.com.png?fit=max&auto=format&n=Ez-pJDkPpztLE7Cr&q=85&s=ae26fba344d6ff6a16b305e4452caac9" alt="" noZoom loading="lazy" width="144" height="144" data-path="images/icons/144/github.com.png" />
    </span>

    <div className="uai-itemcard-identity">
      <div className="uai-itemcard-row uai-itemcard-row--title">
        ## [llama.cpp](https://github.com/ggml-org/llama.cpp)
      </div>

      <div className="uai-itemcard-row">
        <span className="uai-itemcard-note uai-itemcard-note--green">Best for low-level runtime control</span>
      </div>
    </div>

    <div className="uai-itemcard-end">
      <a href="https://github.com/ggml-org/llama.cpp" target="_blank" rel="noreferrer" aria-label="Visit llama.cpp" className="uai-itemcard-cta no-underline">Visit llama.cpp</a>
    </div>
  </div>

  <div className="uai-itemcard-body">
    <div className="uai-itemcard-summary">
      llama.cpp is the engine most other tools wrap, exposed for you to drive directly. C/C++ inference, GGUF support, and a long list of hardware backends (Metal, CUDA, HIP, Vulkan, SYCL, OpenVINO, WebGPU), plus `llama-server` for an OpenAI-compatible local API. It is not a chat app and not friendly if you are new to local models. Reach for it when a wrapper starts hiding the knob you actually need to turn.
    </div>

    <div className="uai-itemcard-facts" aria-label="llama.cpp facts">
      <span>Platforms <span className="uai-itemcard-platforms" aria-label="Mac, Windows, Linux, API"><span title="Mac" className="uai-itemcard-pficon uai-itemcard-pficon--apple" /><span title="Windows" className="uai-itemcard-pficon uai-itemcard-pficon--windows" /><span title="Linux" className="uai-itemcard-pficon uai-itemcard-pficon--linux" /><span title="API" className="uai-itemcard-pficon uai-itemcard-pficon--code" /></span></span>
      <span>Type <strong>Runtime</strong></span>
    </div>

    <div className="uai-itemcard-details-group">
      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-llama-cpp-strengths" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-llama-cpp-strengths"><span>Strengths</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          <ul>
            <li>Most direct runtime control - flags, files, server behavior, context, backends, and quantization are all yours with no abstraction rounding off the decisions.</li>
            <li>Best unusual-hardware path - AMD/Vulkan, older machines, and experimental setups are where direct build choices pay off, off the CUDA happy path.</li>
            <li>Small scriptable server - llama-server gives a local API without a full desktop app, useful as a minimal service behind Open WebUI, a notebook, or your own app.</li>
          </ul>
        </div>
      </div>

      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-llama-cpp-weaknesses" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-llama-cpp-weaknesses"><span>Weaknesses</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          <ul>
            <li>Requires runtime literacy - model files, flags, quantization, context, ports, backends, and sometimes build steps; if make or cmake aren't familiar, start with Ollama.</li>
            <li>No polished workspace - no model browser, chat app, document workspace, users, or admin, so you'll bolt on a front end like Open WebUI for anything beyond an API.</li>
          </ul>
        </div>
      </div>

      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-llama-cpp-who-it-s-for" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-llama-cpp-who-it-s-for"><span>Who it's for</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          Best if you are tuning quantization, picking backends, or running unusual hardware. Skip if you want a chat UI out of the box - try LM Studio. Skip if you want documents indexed and queried - AnythingLLM saves a lot of wiring.
        </div>
      </div>
    </div>
  </div>
</div>

<div className="uai-itemcard" role="article">
  <div className="uai-itemcard-head">
    <span className="uai-itemcard-icon">
      <img src="https://mintcdn.com/usefulai/tO2qspLJNjFc61Zv/images/icons/144/anythingllm.com.png?fit=max&auto=format&n=tO2qspLJNjFc61Zv&q=85&s=b3f8a09146b6cdcc62e88da925373aef" alt="" noZoom loading="lazy" width="144" height="144" data-path="images/icons/144/anythingllm.com.png" />
    </span>

    <div className="uai-itemcard-identity">
      <div className="uai-itemcard-row uai-itemcard-row--title">
        ## [AnythingLLM](https://anythingllm.com/)
      </div>

      <div className="uai-itemcard-row">
        <span className="uai-itemcard-note uai-itemcard-note--green">Best for local document and RAG workspace</span>
      </div>
    </div>

    <div className="uai-itemcard-end">
      <a href="https://anythingllm.com/" target="_blank" rel="noreferrer" aria-label="Visit AnythingLLM" className="uai-itemcard-cta no-underline">Visit AnythingLLM</a>
    </div>
  </div>

  <div className="uai-itemcard-body">
    <div className="uai-itemcard-summary">
      AnythingLLM is the direct answer to "I want local AI over my own files." It packages workspaces, ingestion, embeddings, citations, and provider choice around a chat window - the real product is the document workflow, not the model. It connects to a runtime (Ollama, LM Studio, an OpenAI-compatible provider, or its bundled option); you still pick the model.
    </div>

    <div className="uai-itemcard-facts" aria-label="AnythingLLM facts">
      <span>Platforms <span className="uai-itemcard-platforms" aria-label="Mac, Windows, Linux, Web, API"><span title="Mac" className="uai-itemcard-pficon uai-itemcard-pficon--apple" /><span title="Windows" className="uai-itemcard-pficon uai-itemcard-pficon--windows" /><span title="Linux" className="uai-itemcard-pficon uai-itemcard-pficon--linux" /><span title="Web" className="uai-itemcard-pficon uai-itemcard-pficon--globe" /><span title="API" className="uai-itemcard-pficon uai-itemcard-pficon--code" /></span></span>
      <span>Type <strong>Workspace</strong></span>
    </div>

    <div className="uai-itemcard-details-group">
      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-anythingllm-strengths" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-anythingllm-strengths"><span>Strengths</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          <ul>
            <li>Best packaged document workspace - uploads, workspaces, embeddings, citations, and chat wired together, stronger than gluing a runtime, vector DB, and UI yourself.</li>
            <li>Clear desktop vs Docker split - desktop is single-user, no account, fully local; Docker and hosted modes add multi-user, browser access, and admin, so pick upfront.</li>
            <li>Broad ingestion and provider support - many file types, several embedding backends, multiple vector DBs, and most major LLM providers, so you swap layers without rebuilding.</li>
          </ul>
        </div>
      </div>

      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-anythingllm-weaknesses" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-anythingllm-weaknesses"><span>Weaknesses</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          <ul>
            <li>Messy files still need prep - uploading .docx, spreadsheets, or scanned PDFs isn't reliable retrieval, so expect to convert, structure folders, and tune chunking.</li>
          </ul>
        </div>
      </div>

      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-anythingllm-who-it-s-for" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-anythingllm-who-it-s-for"><span>Who it's for</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          Best if your real workflow is documents or citations - solo or shared. Skip for a raw runtime - Ollama or llama.cpp. Skip for a generic team chat portal - Open WebUI is more direct.
        </div>
      </div>
    </div>
  </div>
</div>

<div className="uai-itemcard" role="article">
  <div className="uai-itemcard-head">
    <span className="uai-itemcard-icon">
      <img src="https://mintcdn.com/usefulai/Ez-pJDkPpztLE7Cr/images/icons/144/github.com.png?fit=max&auto=format&n=Ez-pJDkPpztLE7Cr&q=85&s=ae26fba344d6ff6a16b305e4452caac9" alt="" noZoom loading="lazy" width="144" height="144" data-path="images/icons/144/github.com.png" />
    </span>

    <div className="uai-itemcard-identity">
      <div className="uai-itemcard-row uai-itemcard-row--title">
        ## [TextGen](https://github.com/oobabooga/textgen)
      </div>

      <div className="uai-itemcard-row">
        <span className="uai-itemcard-note uai-itemcard-note--green">Best for power-user local workbench</span>
      </div>
    </div>

    <div className="uai-itemcard-end">
      <a href="https://github.com/oobabooga/textgen" target="_blank" rel="noreferrer" aria-label="Visit TextGen" className="uai-itemcard-cta no-underline">Visit TextGen</a>
    </div>
  </div>

  <div className="uai-itemcard-body">
    <div className="uai-itemcard-summary">
      TextGen, formerly oobabooga/text-generation-webui, is the local AI workbench you reach for after outgrowing LM Studio. Portable desktop builds, OpenAI- and Anthropic-compatible APIs, MCP tool calls, multiple backends including ik\_llama.cpp and ExLlama variants, web search, and PDF extraction all live in one place. It is busier and noisier than LM Studio, which is the point if you want a local lab rather than a calm app.
    </div>

    <Frame>
      <img src="https://mintcdn.com/usefulai/FsW1srcineeRePXa/images/tools/local-llm-tools/textgen.png?fit=max&auto=format&n=FsW1srcineeRePXa&q=85&s=787fdd846877160e35f1ad15fa247271" alt="TextGen local LLM workbench" width="1686" height="998" data-path="images/tools/local-llm-tools/textgen.png" />
    </Frame>

    <div className="uai-itemcard-facts" aria-label="TextGen facts">
      <span>Platforms <span className="uai-itemcard-platforms" aria-label="Mac, Windows, Linux, API"><span title="Mac" className="uai-itemcard-pficon uai-itemcard-pficon--apple" /><span title="Windows" className="uai-itemcard-pficon uai-itemcard-pficon--windows" /><span title="Linux" className="uai-itemcard-pficon uai-itemcard-pficon--linux" /><span title="API" className="uai-itemcard-pficon uai-itemcard-pficon--code" /></span></span>
      <span>Type <strong>Workspace</strong></span>
    </div>

    <div className="uai-itemcard-details-group">
      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-textgen-strengths" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-textgen-strengths"><span>Strengths</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          <ul>
            <li>Broad power-user workbench - chat, multiple backends, tools, files, vision, APIs, and local workflow helpers in one app for comparing backends or running a coding agent.</li>
            <li>Portable desktop packaging - builds unzip and run as a native Electron window with all data inside the folder, useful for portability or an external drive.</li>
            <li>Strong local API and tool story - OpenAI- and Anthropic-compatible endpoints, MCP server support, tool-call confirmation, and Python tool hooks for agent experimentation.</li>
          </ul>
        </div>
      </div>

      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-textgen-weaknesses" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-textgen-weaknesses"><span>Weaknesses</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          <ul>
            <li>Too much if you just want to chat - the backends, MCP options, and tool flags that make it useful are exactly what get in the way of a simple chat; LM Studio or Jan instead.</li>
          </ul>
        </div>
      </div>

      <div className="uai-itemcard-details">
        <input type="checkbox" id="local-llm-tools-textgen-who-it-s-for" className="uai-itemcard-details-toggle" />

        <label htmlFor="local-llm-tools-textgen-who-it-s-for"><span>Who it's for</span><span className="uai-itemcard-details-chevron" /></label>

        <div className="uai-itemcard-details-body">
          Best if you want more control than LM Studio gives - swapping backends, running local agents with tool loops, or comparing quantizations. Skip if you are new to local models - LM Studio or Jan are calmer entry points. Skip if AGPL-3.0 is a problem for your commercial use case.
        </div>
      </div>
    </div>
  </div>
</div>

***

## Selection Guide

<div className="not-prose my-4 flex flex-col gap-1.5 uai-article-prose text-zinc-700 dark:text-zinc-300">
  <span className="flex items-baseline gap-2.5"><span aria-hidden="true" className="relative -top-0.5 inline-block h-1.5 w-1.5 shrink-0 rounded-full bg-zinc-300 dark:bg-zinc-600" /><span>If you need a local model API for other tools, choose <img src="https://mintcdn.com/usefulai/AV2yy8lLL0_-iTHA/images/icons/48/ollama.com.png?fit=max&auto=format&n=AV2yy8lLL0_-iTHA&q=85&s=bf359aad922c930fd293273e79073a06" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/ollama.com.png" /><a href="https://ollama.com/" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">Ollama</a></span></span>
  <span className="flex items-baseline gap-2.5"><span aria-hidden="true" className="relative -top-0.5 inline-block h-1.5 w-1.5 shrink-0 rounded-full bg-zinc-300 dark:bg-zinc-600" /><span>If you want a polished desktop app to explore models, choose <img src="https://mintcdn.com/usefulai/AV2yy8lLL0_-iTHA/images/icons/48/lmstudio.ai.png?fit=max&auto=format&n=AV2yy8lLL0_-iTHA&q=85&s=7cb39665aa95997c6b8675910bcdd898" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/lmstudio.ai.png" /><a href="https://lmstudio.ai/" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">LM Studio</a></span></span>
  <span className="flex items-baseline gap-2.5"><span aria-hidden="true" className="relative -top-0.5 inline-block h-1.5 w-1.5 shrink-0 rounded-full bg-zinc-300 dark:bg-zinc-600" /><span>If you want an open-source desktop assistant, choose <img src="https://mintcdn.com/usefulai/AV2yy8lLL0_-iTHA/images/icons/48/jan.ai.png?fit=max&auto=format&n=AV2yy8lLL0_-iTHA&q=85&s=58029fe9f33a1a92a72cdd7571a9dc1c" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/jan.ai.png" /><a href="https://www.jan.ai/" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">Jan</a></span></span>
  <span className="flex items-baseline gap-2.5"><span aria-hidden="true" className="relative -top-0.5 inline-block h-1.5 w-1.5 shrink-0 rounded-full bg-zinc-300 dark:bg-zinc-600" /><span>If a team needs a shared browser portal, choose <img src="https://mintcdn.com/usefulai/AV2yy8lLL0_-iTHA/images/icons/48/openwebui.com.png?fit=max&auto=format&n=AV2yy8lLL0_-iTHA&q=85&s=4ed454b2407ff2d05e8251713b6c6905" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/openwebui.com.png" /><a href="https://openwebui.com/" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">Open WebUI</a></span></span>
  <span className="flex items-baseline gap-2.5"><span aria-hidden="true" className="relative -top-0.5 inline-block h-1.5 w-1.5 shrink-0 rounded-full bg-zinc-300 dark:bg-zinc-600" /><span>If you are tuning quantization, backends, or unusual hardware, choose <img src="https://mintcdn.com/usefulai/82PG1Up2qz4DPkMj/images/icons/48/github.com.png?fit=max&auto=format&n=82PG1Up2qz4DPkMj&q=85&s=428c02db7ed3f5dc78be2e665f3142c1" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/github.com.png" /><a href="https://github.com/ggml-org/llama.cpp" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">llama.cpp</a></span></span>
  <span className="flex items-baseline gap-2.5"><span aria-hidden="true" className="relative -top-0.5 inline-block h-1.5 w-1.5 shrink-0 rounded-full bg-zinc-300 dark:bg-zinc-600" /><span>If your workflow is private documents and citations, choose <img src="https://mintcdn.com/usefulai/AV2yy8lLL0_-iTHA/images/icons/48/anythingllm.com.png?fit=max&auto=format&n=AV2yy8lLL0_-iTHA&q=85&s=600e9afaf97a3431e54fdd08a5863864" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/anythingllm.com.png" /><a href="https://anythingllm.com/" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">AnythingLLM</a></span></span>
  <span className="flex items-baseline gap-2.5"><span aria-hidden="true" className="relative -top-0.5 inline-block h-1.5 w-1.5 shrink-0 rounded-full bg-zinc-300 dark:bg-zinc-600" /><span>If you want a power-user local lab with MCP and tools, choose <img src="https://mintcdn.com/usefulai/82PG1Up2qz4DPkMj/images/icons/48/github.com.png?fit=max&auto=format&n=82PG1Up2qz4DPkMj&q=85&s=428c02db7ed3f5dc78be2e665f3142c1" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/github.com.png" /><a href="https://github.com/oobabooga/textgen" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">TextGen</a></span></span>
</div>

***

## How We Evaluated

We evaluated more than 15 local LLM tools and selected seven for this guide. We do not use affiliate links, accept sponsorships, or take payment from tool makers. Pricing, platform support, licenses, and recent product changes were checked against official sources before inclusion.

### Selection Criteria

* **Hands-on usability:** How fast you get from install to a useful answer on local hardware.
* **Runtime and serving fit:** Whether the tool actually covers the job you came for (runtime, desktop, UI, RAG, workbench) without overpromising.
* **Privacy posture:** How clearly the tool keeps prompts and files on the local machine, and whether cloud paths are optional and visible.
* **Current compatibility:** Whether the tool keeps up with new model formats, local APIs, document handling, and agent workflows.

### How We Compared

We compared each tool across model setup, first useful chat, API and server behavior, hardware support, and where relevant, document ingestion and tool/agent calls. We focused on friction patterns we saw repeatedly - install pain on Windows or AMD, retrieval breaking on `.docx` and spreadsheets, and agent loops that work on cloud models but stall locally - rather than isolated one-off failures.

***

## What You Need to Know Before Using Local LLM Tools

Local LLMs solve some privacy and cost problems and create new ones. A few things are worth checking before you commit a workflow to local hardware.

### Model License vs. Tool License

Each of these tools sits on a model you separately download. The tool license (MIT, Apache 2.0, AGPL-3.0) governs the app; the model license (Llama community, Gemma terms, Qwen license, custom non-commercial) governs the weights. Commercial use, redistribution, and hosted services need both checked. If your company has a license review process, run it once - before standardizing on a model family - rather than per-project.

### Data That Leaves the Machine Even When You Don't Mean It To

Local does not automatically mean offline. Cloud-tier features, provider API keys you wire in, web search and web fetch plugins, document ingestion that pings a remote embedder, and update checks can all send data outward. Before you assume a workflow is private, audit which features are on, set `OLLAMA_NO_CLOUD=1` or its equivalent, and test with the network detached if confidentiality matters.

### Self-Hosted Web UI Security

Anything that exposes a chat UI over the network has the security profile of a small web app - your problem, not the model's. If you run Open WebUI or AnythingLLM Docker for other users, treat auth, HTTPS, upgrades, backups, and provider keys as part of the deployment. The model is local; the attack surface is not.

***

## Alternatives to Consider

### Other Tools Worth Considering

<ul className="not-prose m-0 my-4 flex list-none flex-col gap-2 p-0">
  * <img src="https://mintcdn.com/usefulai/-BdB1T4cTAq9nNj4/images/icons/48/nomic.ai.png?fit=max&auto=format&n=-BdB1T4cTAq9nNj4&q=85&s=566e5557c12967b6fe8a1e62eb0f622e" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/nomic.ai.png" /><a href="https://www.nomic.ai/gpt4all" target="_blank" rel="noreferrer">GPT4All</a>: simple private desktop chat with LocalDocs and light hardware needs
  * <img src="https://mintcdn.com/usefulai/-BdB1T4cTAq9nNj4/images/icons/48/msty.ai.png?fit=max&auto=format&n=-BdB1T4cTAq9nNj4&q=85&s=6541b46d558f5d269ae14d25e9d478c8" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/msty.ai.png" /><a href="https://msty.ai/" target="_blank" rel="noreferrer">Msty Studio</a>: polished workspace blending local and online models
  * <img src="https://mintcdn.com/usefulai/-BdB1T4cTAq9nNj4/images/icons/48/mozilla.ai.png?fit=max&auto=format&n=-BdB1T4cTAq9nNj4&q=85&s=8f9cef8d019e46e1fd76877cb3f192ac" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/mozilla.ai.png" /><a href="https://www.mozilla.ai/open-tools/llamafile" target="_blank" rel="noreferrer">llamafile</a>: single-file portable executable bundling model and runtime together
  * <img src="https://mintcdn.com/usefulai/-BdB1T4cTAq9nNj4/images/icons/48/koboldcpp.com.png?fit=max&auto=format&n=-BdB1T4cTAq9nNj4&q=85&s=3ef3eb03395bde6bd1f02404b7a041f5" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/koboldcpp.com.png" /><a href="https://koboldcpp.com/" target="_blank" rel="noreferrer">KoboldCpp</a>: standalone GGUF runner popular in roleplay/storytelling stacks
  * <img src="https://mintcdn.com/usefulai/-BdB1T4cTAq9nNj4/images/icons/48/localai.io.png?fit=max&auto=format&n=-BdB1T4cTAq9nNj4&q=85&s=a7d5e689473a1d8814f4164eea54932c" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/localai.io.png" /><a href="https://localai.io/" target="_blank" rel="noreferrer">LocalAI</a>: self-hosted OpenAI-compatible API for text, image, audio, embeddings
  * <img src="https://mintcdn.com/usefulai/XcFilG3Fp3z_o28-/images/icons/48/docker.com.png?fit=max&auto=format&n=XcFilG3Fp3z_o28-&q=85&s=390e61bdeca97376e700947cda555866" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/docker.com.png" /><a href="https://docs.docker.com/ai/model-runner/" target="_blank" rel="noreferrer">Docker Model Runner</a>: Docker-native local model workflow inside Docker Desktop
  * <img src="https://mintcdn.com/usefulai/82PG1Up2qz4DPkMj/images/icons/48/github.com.png?fit=max&auto=format&n=82PG1Up2qz4DPkMj&q=85&s=428c02db7ed3f5dc78be2e665f3142c1" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/github.com.png" /><a href="https://github.com/zylon-ai/private-gpt" target="_blank" rel="noreferrer">PrivateGPT</a>: private document chat project, narrower than AnythingLLM
</ul>

### Adjacent Categories

<ul className="not-prose m-0 my-4 flex list-none flex-col gap-2 p-0">
  <li className="flex items-baseline gap-2.5 uai-article-prose text-zinc-700 dark:text-zinc-300"><span aria-hidden="true" className="relative -top-0.5 inline-block h-1.5 w-1.5 shrink-0 rounded-full bg-zinc-300 dark:bg-zinc-600" /><span><span className="font-semibold">Production inference servers</span> (<img src="https://mintcdn.com/usefulai/ivp-heC-RDy_VFnc/images/icons/48/vllm.ai.png?fit=max&auto=format&n=ivp-heC-RDy_VFnc&q=85&s=bd6ff0d1e1ba7ae312461af145371915" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/vllm.ai.png" /><a href="https://docs.vllm.ai/en/latest/" target="_blank" rel="noreferrer" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">vLLM</a>, <img src="https://mintcdn.com/usefulai/-BdB1T4cTAq9nNj4/images/icons/48/sglang.io.png?fit=max&auto=format&n=-BdB1T4cTAq9nNj4&q=85&s=944488f9b02dd1965e18c175afff8e25" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/sglang.io.png" /><a href="https://www.sglang.io/" target="_blank" rel="noreferrer" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">SGLang</a>, <img src="https://mintcdn.com/usefulai/82PG1Up2qz4DPkMj/images/icons/48/github.com.png?fit=max&auto=format&n=82PG1Up2qz4DPkMj&q=85&s=428c02db7ed3f5dc78be2e665f3142c1" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/github.com.png" /><a href="https://github.com/NVIDIA/TensorRT-LLM" target="_blank" rel="noreferrer" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">TensorRT-LLM</a>): Throughput, batching, and dedicated GPU serving - not personal local chat. Choose these when you are serving many people from real GPU infrastructure.</span></li>
  <li className="flex items-baseline gap-2.5 uai-article-prose text-zinc-700 dark:text-zinc-300"><span aria-hidden="true" className="relative -top-0.5 inline-block h-1.5 w-1.5 shrink-0 rounded-full bg-zinc-300 dark:bg-zinc-600" /><span><span className="font-semibold">Local coding assistants and agents</span> (<img src="https://mintcdn.com/usefulai/XcFilG3Fp3z_o28-/images/icons/48/continue.dev.png?fit=max&auto=format&n=XcFilG3Fp3z_o28-&q=85&s=0d5e7b3a9c859d7e78dcb1dea119e8e4" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/continue.dev.png" /><a href="https://www.continue.dev/" target="_blank" rel="noreferrer" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">Continue</a>, <img src="https://mintcdn.com/usefulai/VO-39uGyUn-FtcaD/images/icons/48/cline.bot.png?fit=max&auto=format&n=VO-39uGyUn-FtcaD&q=85&s=78f17a2beeff4e44f34bfdd3ca6d9b91" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/cline.bot.png" /><a href="https://cline.bot/" target="_blank" rel="noreferrer" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">Cline</a>, <img src="https://mintcdn.com/usefulai/XcFilG3Fp3z_o28-/images/icons/48/aider.chat.png?fit=max&auto=format&n=XcFilG3Fp3z_o28-&q=85&s=38da47743fe66ffd175e36efc715e1d8" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/aider.chat.png" /><a href="https://aider.chat/" target="_blank" rel="noreferrer" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">Aider</a>, <img src="https://mintcdn.com/usefulai/VO-39uGyUn-FtcaD/images/icons/48/opencode.ai.png?fit=max&auto=format&n=VO-39uGyUn-FtcaD&q=85&s=fc20ccacb77a5b77b43b4d6d5f037b79" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/opencode.ai.png" /><a href="https://opencode.ai/" target="_blank" rel="noreferrer" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">OpenCode</a>): These consume a local model endpoint rather than run the model themselves. Choose these if your real job is repo Q\&A, editing files, or running terminal agents on top of a local backend.</span></li>
  <li className="flex items-baseline gap-2.5 uai-article-prose text-zinc-700 dark:text-zinc-300"><span aria-hidden="true" className="relative -top-0.5 inline-block h-1.5 w-1.5 shrink-0 rounded-full bg-zinc-300 dark:bg-zinc-600" /><span><span className="font-semibold">Mobile and framework runtimes</span> (<img src="https://mintcdn.com/usefulai/82PG1Up2qz4DPkMj/images/icons/48/github.com.png?fit=max&auto=format&n=82PG1Up2qz4DPkMj&q=85&s=428c02db7ed3f5dc78be2e665f3142c1" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/github.com.png" /><a href="https://github.com/ml-explore/mlx-lm" target="_blank" rel="noreferrer" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">MLX-LM</a>, <img src="https://mintcdn.com/usefulai/-BdB1T4cTAq9nNj4/images/icons/48/mlc.ai.png?fit=max&auto=format&n=-BdB1T4cTAq9nNj4&q=85&s=fd559b26920f885999eb8699e941a472" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/mlc.ai.png" /><a href="https://llm.mlc.ai/" target="_blank" rel="noreferrer" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">MLC LLM</a>, <img src="https://mintcdn.com/usefulai/-BdB1T4cTAq9nNj4/images/icons/48/mlc.ai.png?fit=max&auto=format&n=-BdB1T4cTAq9nNj4&q=85&s=fd559b26920f885999eb8699e941a472" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/mlc.ai.png" /><a href="https://webllm.mlc.ai/" target="_blank" rel="noreferrer" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">WebLLM</a>, <img src="https://mintcdn.com/usefulai/82PG1Up2qz4DPkMj/images/icons/48/github.com.png?fit=max&auto=format&n=82PG1Up2qz4DPkMj&q=85&s=428c02db7ed3f5dc78be2e665f3142c1" alt="" noZoom className="relative -top-px mr-1 inline h-4 w-4 rounded-sm" width="48" height="48" data-path="images/icons/48/github.com.png" /><a href="https://github.com/a-ghorbani/pocketpal-ai" target="_blank" rel="noreferrer" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">PocketPal AI</a>): Platform-specific stacks for Apple Silicon, phones, browsers, or embedded targets. Choose these if you are optimizing for a specific device class.</span></li>
</ul>

## Frequently Asked Questions

<AccordionGroup>
  <Accordion title="What's the difference between a local LLM runtime and a local LLM app?">
    A runtime like Ollama or llama.cpp loads weights and serves inference through a CLI and local API. An app like LM Studio or Jan wraps a runtime with a chat UI. Document workspaces and shared web UIs sit on top of either.
  </Accordion>

  <Accordion title="How much hardware do I really need?">
    Modern laptops handle 4B-8B models at 4-bit quantization. 12B-30B models comfortably need a recent GPU with 12-24GB of VRAM. 70B+ wants workstation hardware or aggressive quantization. If you are unsure, start with an 8B model in LM Studio - it shows live VRAM use.
  </Accordion>

  <Accordion title="Can I use these tools commercially?">
    The tool license is usually permissive (MIT, Apache 2.0). The exception is TextGen, which is AGPL-3.0 and needs review before commercial redistribution or hosted-service use. The model license is separate and varies: Llama's community license has acceptable-use rules, Gemma has its own terms, several Qwen and DeepSeek variants are Apache 2.0. Check both before shipping.
  </Accordion>

  <Accordion title="Can a local model replace a cloud coding agent like Claude Code?">
    Sometimes, but rarely on the first try. Local coding agents depend on the tool harness, the model's tool-calling reliability, the prompt format, and the hardware. A 30B-class coding model on a strong GPU handles many edits; a 7B model rarely can. Test on real tasks before switching from cloud.
  </Accordion>

  <Accordion title="Can I run multiple tools side by side?">
    Yes, and it is common. Ollama as the backend, Open WebUI in front, AnythingLLM pointed at Ollama for documents - a normal stack. Watch for port conflicts (11434, 1234, 7860, 8080) and shared model-file directories.
  </Accordion>

  <Accordion title="What happens to my chat history if I uninstall?">
    For desktop apps (LM Studio, Jan, AnythingLLM Desktop, TextGen), chats sit in the app's local data folder - usually preserved across upgrades, removed on full uninstall. Ollama and llama.cpp do not store chats; whatever client you used does. Back up the data folder before reinstalling, and check whether the app has an export option first.
  </Accordion>
</AccordionGroup>

<div className="not-prose mt-10 rounded-lg bg-zinc-50/60 px-4 py-3 text-[13px] leading-5 text-zinc-600 dark:bg-white/[0.02] dark:text-zinc-300">We update this guide as new tools launch and existing ones change shape. If you are still unsure, <a href="https://ollama.com/" target="_blank" rel="noreferrer" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">Ollama</a> is the safest starting point - install it, then point <a href="https://openwebui.com/" target="_blank" rel="noreferrer" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">Open WebUI</a> or <a href="https://anythingllm.com/" target="_blank" rel="noreferrer" className="font-medium text-zinc-950 underline underline-offset-2 dark:text-white">AnythingLLM</a> at it later if you need more. Questions or suggestions? Let us know.</div>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.