GrepAI Embeddings with LM Studio

This skill covers using LM Studio as the embedding provider for GrepAI, offering a user-friendly GUI for managing local models.

When to Use This Skill

• Want local embeddings with a graphical interface
• Already using LM Studio for other AI tasks
• Prefer visual model management over CLI
• Need to easily switch between models

What is LM Studio?

LM Studio is a desktop application for running local LLMs with:

• 🖥️ Graphical user interface
• 📦 Easy model downloading
• 🔌 OpenAI-compatible API
• 🔒 100% private, local processing

Prerequisites

1. Download LM Studio from lmstudio.ai
2. Install and launch the application
3. Download an embedding model

Installation

Step 1: Download LM Studio

Visit lmstudio.ai and download for your platform:

• macOS (Intel or Apple Silicon)
• Windows
• Linux

Step 2: Launch and Download a Model

1. Open LM Studio
2. Go to the Search tab
3. Search for an embedding model:
   • nomic-embed-text-v1.5
   • bge-small-en-v1.5
   • bge-large-en-v1.5
4. Click Download

Step 3: Start the Local Server

1. Go to the Local Server tab
2. Select your embedding model
3. Click Start Server
4. Note the endpoint (default: http://localhost:1234)

Configuration

Basic Configuration

# .grepai/config.yaml
embedder:
  provider: lmstudio
  model: nomic-embed-text-v1.5
  endpoint: http://localhost:1234

With Custom Port

embedder:
  provider: lmstudio
  model: nomic-embed-text-v1.5
  endpoint: http://localhost:8080

With Explicit Dimensions

embedder:
  provider: lmstudio
  model: nomic-embed-text-v1.5
  endpoint: http://localhost:1234
  dimensions: 768

Available Models

nomic-embed-text-v1.5 (Recommended)

embedder:
  provider: lmstudio
  model: nomic-embed-text-v1.5

bge-small-en-v1.5

Best for: Smaller codebases, faster indexing.

embedder:
  provider: lmstudio
  model: bge-small-en-v1.5
  dimensions: 384

bge-large-en-v1.5

Best for: Maximum accuracy.

embedder:
  provider: lmstudio
  model: bge-large-en-v1.5
  dimensions: 1024

Model Comparison

LM Studio Server Setup

Starting the Server

1. Open LM Studio
2. Navigate to Local Server tab (left sidebar)
3. Select an embedding model from the dropdown
4. Configure settings:
   • Port: 1234 (default)
   • Enable Embedding Endpoint
5. Click Start Server

Server Status

Look for the green indicator showing the server is running.

Verifying the Server

# Check server is responding
curl http://localhost:1234/v1/models

# Test embedding
curl http://localhost:1234/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nomic-embed-text-v1.5",
    "input": "function authenticate(user)"
  }'

LM Studio Settings

Recommended Settings

In LM Studio's Local Server tab:

GPU Acceleration

LM Studio automatically uses:

• macOS: Metal (Apple Silicon)
• Windows/Linux: CUDA (NVIDIA)

Adjust GPU layers in settings for memory/speed balance.

Running LM Studio Headless

For server environments, LM Studio supports CLI mode:

# Start server without GUI (check LM Studio docs for exact syntax)
lmstudio server start --model nomic-embed-text-v1.5 --port 1234

Common Issues

❌ Problem: Connection refused ✅ Solution: Ensure LM Studio server is running:

1. Open LM Studio
2. Go to Local Server tab
3. Click Start Server

❌ Problem: Model not found ✅ Solution:

1. Download the model in LM Studio's Search tab
2. Select it in the Local Server dropdown

❌ Problem: Slow embedding generation ✅ Solutions:

• Enable GPU acceleration in LM Studio settings
• Use a smaller model (bge-small-en-v1.5)
• Close other GPU-intensive applications

❌ Problem: Port already in use ✅ Solution: Change port in LM Studio settings:

embedder:
  endpoint: http://localhost:8080  # Different port

❌ Problem: LM Studio closes and server stops ✅ Solution: Keep LM Studio running in the background, or consider using Ollama which runs as a system service

LM Studio vs Ollama

Recommendation: Use LM Studio if you prefer a GUI, Ollama for always-on background service.

Migrating from LM Studio to Ollama

If you need a more reliable background service:

1. Install Ollama:

brew install ollama
ollama serve &
ollama pull nomic-embed-text

2. Update config:

embedder:
  provider: ollama
  model: nomic-embed-text
  endpoint: http://localhost:11434

3. Re-index:

rm .grepai/index.gob
grepai watch

Best Practices

1. Keep LM Studio running: Server stops when app closes
2. Use recommended model: nomic-embed-text-v1.5 for best balance
3. Enable GPU: Faster embeddings with hardware acceleration
4. Check server before indexing: Ensure green status indicator
5. Consider Ollama for production: More reliable as background service

Output Format

Successful LM Studio configuration:

✅ LM Studio Embedding Provider Configured

   Provider: LM Studio
   Model: nomic-embed-text-v1.5
   Endpoint: http://localhost:1234
   Dimensions: 768 (auto-detected)
   Status: Connected

   Note: Keep LM Studio running for embeddings to work.