Description
Local LLM applications can expose an OpenAI-compatible API with GPU acceleration through CUDA, Vulkan, or OpenCL. This server is useful for developers testing AI tools against local models while using available graphics hardware. Prompts, model files, GPU memory pressure, and network binding should be managed deliberately.