Skip to content

LLM Inference API Guide

The UT HPC Center provides an OpenAI-compatible LLM API with access to chat, transcription, and embedding models.

Base URL: https://llm.hpc.ut.ee/v1

Getting Access

API Key Required

Order the UT HPC LLM inference resource in the minu.etais.ee self-service portal and create an API key for the resource. The full procedure, from ordering the resource to revealing and revoking keys, is described in Ordering an LLM inference resource.

Keys are used for authentication and usage tracking - keep yours confidential and do not share it.

Contact support@hpc.ut.ee if you have questions about access to the service.

Available Models

Chat & Reasoning

Model name Base model Quantization Best for Context Pricing
deepseek-v4.1-flash DeepSeek V4.1 Flash — multimodal MoE (552 B total) FP8 (native), FP8 KV cache Complex reasoning, coding, math, and agentic tasks; text + image input. Reasoning is on by default — tune its depth with reasoning_effort. 1,048,576 (1M) Input: €0.30 / 1M tokens
Output: €1.20 / 1M tokens
Cache: €0.06 / 1M tokens
qwen3.8-27b Qwen3.8 27B NVFP4 — dense, vision-language NVFP4 (W4A4) General chat, coding, research, and summarization; text + image input. Thinking is on by default — tune the depth with reasoning_effort or turn it off per request. 262,144 (256K) Input: €0.17 / 1M tokens
Output: €0.50 / 1M tokens
Cache: €0.005 / 1M tokens
nemotron-3.5-super-vl Nemotron 3.5 Super VL 120B-A12B — MoE (120 B total / 12 B active), vision-language NVFP4 Long-context reasoning and document/video understanding; text + image + video input. Reasoning is on by default — tune its depth with reasoning_effort. 524,288 (512K) Input: €0.17 / 1M tokens
Output: €0.50 / 1M tokens
Cache: €0.005 / 1M tokens

Reasoning & thinking

  • DeepSeek V4.1 Flash — reasoning is enabled by default. reasoning_effort takes a string (low, medium, or high) to tune its depth, or none to turn off reasoning. Reasoning tokens are reported in usage.completion_tokens_details.reasoning_tokens.
  • Qwen3.8 27B — thinking is enabled by default. reasoning_effort selects the depth: xhigh (default), medium, or low. For a direct, non-thinking response, set reasoning_effort to none or pass "chat_template_kwargs": {"enable_thinking": false}.
  • Nemotron 3.5 Super VL — reasoning is enabled by default. reasoning_effort takes low, medium, high, or xhigh to tune its depth, or none to turn off reasoning. Reasoning tokens are reported in usage.completion_tokens_details.reasoning_tokens.

Transcription

Model name Base model Quantization Best for Pricing
whisper-large-v3-turbo-et TalTechNLP whisper-large-v3-turbo-et-verbatim-2604 — Estonian verbatim fine-tune of Whisper large-v3-turbo BF16 Audio transcription, Estonian speech-to-text (verbatim, including filler words) Input: €0.0001 / second

Embeddings

Model name Base model Quantization Dimensions Context Best for Pricing
qwen3-embedding-8B Qwen3-Embedding-8B — dense, decoder-only BF16 4096 32,768 Text retrieval, semantic search, RAG pipelines, multilingual classification Input: €0.01 / 1M tokens
bge-m3 BGE-M3 — dense, multilingual (100+ languages) BF16 1024 8,192 Cheaper multilingual retrieval and RAG over long documents Input: €0.002 / 1M tokens

Examples

Chat Completion

curl https://llm.hpc.ut.ee/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.8-27b",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Explain the CAP theorem in two sentences."}
    ],
    "temperature": 0.7,
    "top_p": 0.8,
    "max_tokens": 1024
  }'

Install the SDK if you haven't already: 

pip install openai


from openai import OpenAI

client = OpenAI(
    base_url="https://llm.hpc.ut.ee/v1",
    api_key="YOUR_API_KEY",
)

response = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain the CAP theorem in two sentences."},
    ],
    temperature=0.7,
    top_p=0.8,
    max_tokens=1024,
)

print(response.choices[0].message.content)

import requests

response = requests.post(
    "https://llm.hpc.ut.ee/v1/chat/completions",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "model": "qwen3.8-27b",
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Explain the CAP theorem in two sentences."},
        ],
        "temperature": 0.7,
        "top_p": 0.8,
        "max_tokens": 1024,
    },
)

print(response.json()["choices"][0]["message"]["content"])

Streaming

Streaming returns tokens incrementally as they are generated, rather than waiting for the full response.

curl https://llm.hpc.ut.ee/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.8-27b",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Write a short poem about distributed systems."}
    ],
    "stream": true,
    "temperature": 0.7,
    "top_p": 0.8,
    "max_tokens": 1024
  }'

import httpx
from openai import OpenAI

client = OpenAI(
    base_url="https://llm.hpc.ut.ee/v1",
    api_key="YOUR_API_KEY",
    timeout=httpx.Timeout(300.0, connect=10.0),  # (1)!
)

stream = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Write a short poem about distributed systems."},
    ],
    stream=True,
    temperature=0.7,
    top_p=0.8,
    max_tokens=1024,
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
print()
  1. A generous read timeout is recommended for streaming — long responses may take several minutes to complete.

Image Understanding

Pass an image URL alongside your text prompt. Supported by the deepseek-v4.1-flash, qwen3.8-27b and nemotron-3.5-super-vl models (the latter also accepts video).

curl https://llm.hpc.ut.ee/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4.1-flash",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "What is in this image?"},
          {
            "type": "image_url",
            "image_url": {
              "url": "https://hpc.ut.ee/assets/images/server_room_3.png"
            }
          }
        ]
      }
    ],
    "temperature": 0.7,
    "top_p": 0.8,
    "max_tokens": 1024
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://llm.hpc.ut.ee/v1",
    api_key="YOUR_API_KEY",
)

response = client.chat.completions.create(
    model="deepseek-v4.1-flash",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is in this image?"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://hpc.ut.ee/assets/images/server_room_3.png"
                    },
                },
            ],
        },
    ],
    temperature=0.7,
    top_p=0.8,
    max_tokens=1024,
)

print(response.choices[0].message.content)

Transcription

Convert audio files to text using whisper-large-v3-turbo-et, an Estonian verbatim fine-tune of Whisper large-v3-turbo. Use the language parameter to improve accuracy for a known language.

curl https://llm.hpc.ut.ee/v1/audio/transcriptions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -F "model=whisper-large-v3-turbo-et" \
  -F "language=et" \
  -F "file=@/path/to/clip.mp3"

from openai import OpenAI

client = OpenAI(
    base_url="https://llm.hpc.ut.ee/v1",
    api_key="YOUR_API_KEY",
)

with open("/path/to/clip.mp3", "rb") as audio_file:
    transcription = client.audio.transcriptions.create(
        model="whisper-large-v3-turbo-et",
        file=audio_file,
        language="et",  # (1)!
    )

print(transcription.text)
  1. Optional. Providing the language (ISO-639-1 code) improves speed and accuracy. Omit to enable automatic language detection.

Embeddings

Generate vector representations of text for use in semantic search, RAG pipelines, or classification.

curl https://llm.hpc.ut.ee/v1/embeddings \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-embedding-8B",
    "input": "The University of Tartu HPC cluster provides GPU compute resources."
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://llm.hpc.ut.ee/v1",
    api_key="YOUR_API_KEY",
)

response = client.embeddings.create(
    model="qwen3-embedding-8B",
    input="The University of Tartu HPC cluster provides GPU compute resources.",
)

vector = response.data[0].embedding  # 4096-dimensional float list
print(f"Embedding dimensions: {len(vector)}")

Tip

Use a lower temperature for factual, analytical, or structured tasks. Use a higher temperature for creative or generative ones.

Recommended values from the official model card.

Parameter Recommended value Description
temperature 1.0 Controls randomness.
top_p 0.95 (or 1.0) Nucleus sampling — sample from the top 95% probability mass.
max_tokens ≥ 262144 Maximum tokens to generate. The served context window is 1,048,576 tokens.
reasoning_effort low / medium / high / none String level; higher values spend more tokens on reasoning, none disables it. Reasoning tokens are reported in usage.completion_tokens_details.reasoning_tokens.

Recommended values from the official model card. They differ by mode — thinking is on by default.

Thinking mode

Parameter Recommended value
temperature 1.0
top_p 0.95
top_k 20
min_p 0.0
presence_penalty 0.0
repetition_penalty 1.0
reasoning_effort xhigh (default)

Non-thinking (instruct) mode

Parameter Recommended value
temperature 0.7
top_p 0.80
top_k 20
min_p 0.0
presence_penalty 1.5
repetition_penalty 1.0
reasoning_effort none

The model card's separate reasoning/output budgets (up to 262,144 tokens for reasoning content and 131,072 for the final response) assume the extended 1M-token context. On this endpoint the combined input + output is capped at 262,144 tokens.

Rate Limits

HTTP 429 — Too Many Requests

Rate limits are enforced per API key. If you receive 429 responses, you have exceeded your quota. Contact support@hpc.ut.ee to review or increase your limits.

Compatibility

The API is fully compatible with the OpenAI API specification. Any OpenAI-compatible library or tool works out of the box — set the base URL to https://llm.hpc.ut.ee/v1 and provide your API key.

Compatible tools & frameworks

  • LangChain — use ChatOpenAI with base_url="https://llm.hpc.ut.ee/v1"
  • LlamaIndex — use the OpenAI LLM class with a custom api_base
  • Continue — configure as an OpenAI-compatible provider in your IDE
  • Any tool supporting the OPENAI_BASE_URL and OPENAI_API_KEY environment variables