> ## Documentation Index
> Fetch the complete documentation index at: https://docs.typingmind.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Local AI 

## Overview

Typing Mind allows you to connect the app with any local model you want.

<Frame>
  <img src="https://mintcdn.com/typingmind/pm53BB_D69NXvWmb/images/local-ai.webp?fit=max&auto=format&n=pm53BB_D69NXvWmb&q=85&s=2d51e6e597b0b3514eae1d14cbb57354" alt="Local Ai" width="2566" height="1672" data-path="images/local-ai.webp" />
</Frame>

**Requirement:**

* The model must be served via an OpenAI-compatible API endpoint.
* You must have some relevant technical skills to setup a custom model on your own server/endpoint.

Below is a short instruction on how to setup Typing Mind with a popular local AI setup using the open-source project [LocalAI](https://github.com/go-skynet/LocalAI).

## Setup LocalAI on your device

<Note>
  👉 If you already have another setup for the local AI model endpoint, you can skip this step.
</Note>

Go to [https://github.com/go-skynet/LocalAI](https://github.com/go-skynet/LocalAI) and follow their instruction to run a model on your device.

For example, here is the command to setup LocalAI with Docker:

```bash theme={null}
docker run -p 8080:8080 -ti --rm -v /Users/tonydinh/Desktop/models:/app/models quay.io/go-skynet/local-ai:latest --models-path /app/models --context-size 700 --threads 4 --cors true
```

<Note>
  💡 Note that we added the `--cors true` parameter to the command to make sure the local server is accessible from the browser. Typing Mind will send requests to the local model directly from the browser.
</Note>

If you’re doing it correctly, you should see a message like this:

<img src="https://mintcdn.com/typingmind/AJebv9GNiO42w_6y/images/Untitled-1-1.webp?fit=max&auto=format&n=AJebv9GNiO42w_6y&q=85&s=0dfde5d08a65e3fdf1e34a79eb999476" alt="Untitled 1 1" width="1734" height="926" data-path="images/Untitled-1-1.webp" />

Now you can go to [http://localhost:8080/v1/models](http://localhost:8080/v1/models) and confirm that the model is ready:

<img src="https://mintcdn.com/typingmind/AJebv9GNiO42w_6y/images/Untitled-2-2.webp?fit=max&auto=format&n=AJebv9GNiO42w_6y&q=85&s=e3ad6bec228b519b47901d618951e0a2" alt="Untitled 2 2" width="1302" height="1108" data-path="images/Untitled-2-2.webp" />

## Setup Custom Model on TypingMind

Open TypingMind → Models → Add Custom Model:

<Frame>
  <img src="https://mintcdn.com/typingmind/AJebv9GNiO42w_6y/images/Untitled-4-1.webp?fit=max&auto=format&n=AJebv9GNiO42w_6y&q=85&s=381e37b7d1678199419d36a05ad44b4a" alt="Local Ai" width="1162" height="1050" data-path="images/Untitled-4-1.webp" />
</Frame>

Enter all the details, then click “Test” to check if your custom model is working correctly.

### Popular problems at this step

| Issue                   | Resolution                                                                                                                                                  |
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **CORS related issues** | Make sure your server configuration allows the endpoint to be accessible from the browser. Open the Network tab in the browser console to see more details. |
| **Long waiting time**   | In the first request, your model can take a long time to respond. Check the terminal log of the Docker process to see if anything goes wrong.               |
| **API Key Missing**     | Typing Mind does not support API key authentication for custom model yet. Please reconfigurate your custom model to remove API key requirement.             |

## Chat with the new Custom Model

Once the model is tested and added successfully, you can select the custom model and chat with it normally.

<img src="https://mintcdn.com/typingmind/AJebv9GNiO42w_6y/images/Untitled-6.webp?fit=max&auto=format&n=AJebv9GNiO42w_6y&q=85&s=62c00d291d8530fec1c5d7ddfad460af" alt="Untitled 6" width="1668" height="1178" data-path="images/Untitled-6.webp" />
