Getting Started with RESTai: Unified LLM Integration

Before installing RESTai, ensure your development environment is prepared. You will need Docker and Docker Compose installed on your machine, as RESTai is primarily distributed as a containerized service. Additionally, if you plan to use local models, install Ollama separately and pull at least one model (e.g., `llama3` or `mistral`) to test against. Ensure you have API keys for any public providers you intend to support, such as OpenAI or Anthropic. This setup allows you to compare local inference costs against commercial APIs using a single interface.

First, clone the RESTai repository from GitHub. Open your terminal and run: `git clone https://github.com/apocas/restai.git && cd restai`. Inside the directory, you will find a `docker-compose.yml` file and a `.env.example` file. Copy the example environment file to `.env` using `cp .env.example .env`. Open the `.env` file in your text editor. This file is crucial for configuration. You will need to add your API keys here. For example, uncomment and fill in `OPENAI_API_KEY` if you want to support OpenAI models. If you are using Ollama, ensure the environment variable pointing to the Ollama host is set correctly, usually `OLLAMA_HOST=http://host.docker.internal:11434` on macOS/Windows or `http://localhost:11434` on Linux.

With your configuration ready, start the service using Docker Compose. Run the command: `docker compose up -d`. This command pulls the necessary images and starts the RESTai container in detached mode. You can monitor the logs to ensure the service starts correctly by running `docker compose logs -f`. Look for a message indicating that the server is listening on port 8000 (or the port you configured in the compose file). Once the logs show the service is ready, you can stop the log stream with `Ctrl+C`. The service is now running locally and ready to accept API requests on `http://localhost:8000`.

RESTai automatically generates interactive API documentation. Open your web browser and navigate to `http://localhost:8000/docs`. This Swagger UI interface provides a visual representation of all available endpoints, including `POST /v1/chat/completions`, `POST /v1/embeddings`, and fine-tuning endpoints. You can test the API directly from the browser by clicking on the endpoints and hitting 'Try it out'. This is an excellent way to verify connectivity without writing code. Examine the request bodies to understand the required parameters, such as `model`, `prompt`, and `max_tokens`. The documentation also highlights which providers are currently active based on your `.env` configuration.

Now, let's integrate RESTai into a Python application. Create a file named `test_restai.py`. Use the `requests` library to send a chat completion request. The endpoint mimics the OpenAI API structure, making migration easier. Here is a sample code snippet: ```python import requests url = "http://localhost:8000/v1/chat/completions" payload = { "model": "ollama/llama3", # Prefix with provider "messages": [{"role": "user", "content": "Say hello"}], "max_tokens": 100 } headers = {"Content-Type": "application/json"} response = requests.post(url, json=payload, headers=headers) print(response.json()) ``` Run this script with `python test_restai.py`. You should receive a JSON response containing the generated text. Notice the `model` parameter; RESTai allows you to route requests to different backends by prefixing the model name (e.g., `openai/gpt-4` or `ollama/mistral`).

One of RESTai's core benefits is unified routing. Modify the `model` field in your Python script to test different backends. For example, change `"model": "ollama/llama3"` to `"model": "openai/gpt-4o-mini"` (assuming you have an OpenAI key). Run the script again. RESTai handles the authentication and request formatting for each provider internally. This allows your application code to remain unchanged while you switch between local, cost-effective models and powerful commercial APIs. You can also test embedding generation by changing the endpoint to `/v1/embeddings` and adjusting the payload accordingly, useful for RAG applications.

For production use, consider securing your RESTai instance. Bind the service to `127.0.0.1` only if accessed locally, or place it behind a reverse proxy like Nginx or Caddy with SSL termination. If exposing it to a network, ensure you enable authentication middleware if available in the configuration. Monitor your usage through the logs to track token consumption and latency. Regularly update your Docker image by running `docker compose pull && docker compose up -d` to get the latest features and security patches. This setup provides a robust, scalable foundation for AI-driven applications.