How do I make this accessible to other computers on my network?

Launch with the `--listen` flag. By default, it binds to `0.0.0.0` and serves on port 7860. Use `--listen-host 192.168.x.x` to bind to a specific local IP.

How do I enable the OpenAI-compatible API?

Start the server with the `--api` flag. You can also add this to `user_data/CMD_FLAGS.txt` to make it persistent. The API will be available at `http://127.0.0.1:5000/v1/`.

What GPU memory is required for a model?

It depends on the model size and quantization. Use the linked GGUF Memory Calculator for estimates. Generally, a 7B-parameter model in 4-bit quant uses ~4-5GB VRAM.

How do I use a model with vision/multimodal support?

Load a model with a vision projection file (mmproj). Use the `--mmproj` flag or select it in the UI. Then attach images in the chat interface. The underlying loader (typically llama.cpp) handles the processing.

How do I write a custom tool for the model to call?

Create a single `.py` file in the `extensions/` directory following the template. It must define a `params()` dict and a `tool()` function. The model will be instructed to output JSON matching your schema to trigger it.

Directory/Coding/AI Assistants/TextGen

TextGen

TextGen is a desktop application for running large language models (LLMs) locally on your machine. It provides a web-based UI and API for chat, instruction following, vision tasks, and tool-calling, all with complete privacy and zero telemetry. The tool supports multiple backends including llama.cpp, ExLlamaV3, Transformers, and TensorRT-LLM, allowing users to switch between models and backends without restarting. It offers a simple one-click installer for portable builds on Windows, Linux, and macOS, with no external dependencies required. Key capabilities include multimodal vision, file attachments (PDFs, DOCX), conversation branching, and a dedicated notebook for free-form text generation. It features a training tab for fine-tuning LoRAs on chat datasets and an image generation tab for diffusers models. The API is compatible with OpenAI and Anthropic endpoints, making it a local drop-in replacement for cloud services. It also supports extensions for TTS, voice input, and translatio

Coding

Visit website

Added on

Sep 22, 2026

Local LLM runnerDesktop appNo telemetryMulti-backendVision supportTool-callingOpenAI APILoRA trainingImage generationExtensions

Product information

Everything worth knowing about TextGen

The complete picture — from everyday features to the technical detail builders and IT folks dig for.

Core features

What it actually does

Desktop App for Local LLMs

Run language models entirely on your machine with a dedicated desktop application. Portable builds available for Linux, Windows, and macOS with various GPU backends.

Vision and File Attachments

Attach images for multimodal understanding and upload text/PDF/.docx files to interact with their contents in chat.

Multiple Chat and Generation Modes

Switch between instruct, chat-instruct, and chat modes for instruction-following or character conversations. Includes a notebook for free-form text generation.

Tool-Calling and Extensions

Enable models to call custom functions (web search, math, etc.) and integrate community extensions for TTS, voice input, and translation.

Technical capabilities

OpenAI/Anthropic-Compatible API

Provides a local API server compatible with OpenAI and Anthropic endpoints, including chat, completions, and tool-calling support.

Multiple Backend Support

Switch between inference backends like llama.cpp, Transformers, ExLlamaV3, and TensorRT-LLM without restarting the application.

Must watch videos

Chat With AI Characters Privately On Your PC! (OobaBooga WebUI Quick Install)

12:05

Tutorial12:05

Chat With AI Characters Privately On Your PC! (OobaBooga WebUI Quick Install)

No filter, works on 8 GB VRAM GPUs. Using the OobaBooga WebUI you can chat and roleplay with pretty much an unlimited number of AI characters including your custom made ones! When it comes to character.ai alternatives, this is my go-to choice. Free, open-source, and 100% configurable and private. And you can use uncensored unfiltered LLM models with it! Give it a quick install and thank me later! Here are the links used in the video: OobaBooga WebUI Github repository: https://github.com/oobabooga/text-generation-webui The Dolphin 2.6 Mistral 7B AWQ Model: https://huggingface.co/TheBloke/dolphin-2.6-mistral-7B-AWQ

The ULTIMATE Local AI Interface: Text Generation WebUI (Oobabooga) MAX Customization EXPLAINED

5:39

Tutorial5:39

The ULTIMATE Local AI Interface: Text Generation WebUI (Oobabooga) MAX Customization EXPLAINED

What is Text Generation WebUI (Oobabooga)? This is the advanced interface built for power users, researchers, and customization enthusiasts who demand maximum control over their local LLMs. If you've hit the limitations of simpler tools like LM Studio or GPT4All, Text Generation WebUI is the answer. Think of simpler tools as automatic cameras, while Oobabooga is the professional camera with manual controls and RAW mode, allowing for professional results. Inside this guide, we break down Text Gen WebUI's professional-grade features: • Maximum Customization: Access and adjust every single AI parameter, including temperature, top-p, and repetition penalty, enabling precise fine-tuning of model behavior. • Extensions & Capabilities: Add-ons for new capabilities, such as long-term memory, API integration, and web search functionality. • Model Flexibility: Load virtually any model format (GGUF, GPTQ, AWQ) and perform advanced actions like model merging and quantization. Learn when this complex but powerful tool is the perfect fit for your research or experimentation need

How To Install TextGen WebUI - Use ANY MODEL Locally!

9:46

Tutorial9:46

How To Install TextGen WebUI - Use ANY MODEL Locally!

In this video, I show you how to install TextGen WebUI on a Windows machine and get models installed and running. TextGen WebUI is like Automatic1111 for LLMs. Easily run any open source model locally on your computer. Enjoy :) Join My Newsletter for Regular AI Updates 👇🏼 https://forwardfuture.ai/ My Links 🔗 👉🏻 Subscribe: https://www.youtube.com/@matthew_berman 👉🏻 Twitter: https://twitter.com/matthewberman 👉🏻 Discord: https://discord.gg/xxysSXBxFW 👉🏻 Patreon: https://patreon.com/MatthewBerman Media/Sponsorship Inquiries 📈 https://bit.ly/44TC45V Links: Github Repo - https://github.com/oobabooga/text-generation-webui Install Commands - https://gist.github.com/mberman84/f092a28e4151dd5cecebfc58ac1cbc0e The Bloke HF - https://huggingface.co/TheBloke

Who uses it

Real use cases, no hype

Local AI Development & Experimentation

Test, compare, and run various open-source LLMs locally without cloud dependencies or costs.

Private Sensitive Data Processing

Analyze confidential documents, legal text, or personal data with a model that never phones home.

On-Premise API Deployment

Provide a local OpenAI-compatible API endpoint for internal tools, fine-tuned for specific business tasks.

Multimodal Prototyping

Rapidly prototype chatbots with vision capabilities and tool-calling for real-world application demos.

Custom Model Fine-Tuning

Train and apply LoRA adapters on custom datasets directly within the same unified interface.

What's great

True 100% offline & private: zero telemetry or external calls.
Extremely easy one-click setup with portable builds for all platforms.
Massive community & active development (47k+ stars, 5k+ commits).

Technical strengths

Multi-backend support: Switch between llama.cpp, ExLlamaV3, Transformers, TensorRT-LLM.
OpenAI/Anthropic-compatible API with native tool-calling and MCP server support.
Advanced features: Speculative decoding, LoRA training, image generation, and multimodal support.

Where it falls short

Can be resource-heavy: Full install requires ~10GB disk and downloads PyTorch.
UI can feel cluttered with its many tabs and options.
Docker setup requires manual file linking and configuration.

Technical limitations

Non-portable features (ExLlamaV3, Transformers) require a separate, complex Conda installation.
Continuous rapid development can lead to breaking changes or instability between updates.

For technical folks

The deep-dive specs

FAQ

Includes technical Q&A

The portable build is a self-contained package for quick GGUF model use via llama.cpp. The full install uses Conda, downloads PyTorch, and unlocks additional backends (ExLlamaV3, Transformers), training, image generation, and extensions.

Traffic Insights

Monthly Visits

—

October 2026

Growth Rate

vs last month

Dominance

in Coding

Top Country

—

Top Countries Breakdown

—

Growth Trend

Stable

Traffic has remained stable.

Pricing overview

Unknown Pricing

TextGen Pricing

TextGen uses a unknown pricing structure. Real-time rates, custom enterprise quotas, and plan details are managed directly on their official platform.

Check Pricing on TextGen

Reviews