Skip to content
ParseHawkParseHawk Developer Docs

Use the bundled vLLM runtime

The bundled runtime is ParseHawk’s zero-configuration provider path. It serves numind/NuExtract3-W4A16 through an OpenAI-compatible API and keeps document content on the local machine. It supports both structured extraction and document-to-Markdown parsing.

Start the default stack

Section titled “Start the default stack”
Terminal window
parsehawk start

ParseHawk chooses the platform implementation automatically:

  • macOS Apple Silicon runs vLLM Metal natively on the host.
  • Linux x86_64 or ARM64 with NVIDIA runs vLLM in Docker Compose.

Inspect and test the runtime

Section titled “Inspect and test the runtime”
Terminal window
parsehawk runtime info
parsehawk runtime doctor
parsehawk runtime test

The runtime API listens on http://127.0.0.1:8080/v1 by default. The first start downloads model artifacts and warms the engine, so readiness takes longer than on later starts.

Assign an extractor or parser

Section titled “Assign an extractor or parser”

New extractors and parsers default to the openai_compatible_api provider. Leave the model unset to inherit the active bundled model, or set it explicitly:

Terminal window
parsehawk extractors update invoice_v1 \
  --provider openai_compatible_api \
  --model numind/NuExtract3-W4A16


parsehawk parsers put technical-markdown \
  --display-name "Technical document Markdown" \
  --instructions "Preserve section numbers." \
  --provider openai_compatible_api \
  --model numind/NuExtract3-W4A16

The built-in receipt extractor and document-to-markdown parser already use these defaults. Exact NuExtract3 variants use a fine-tuned structured template for extraction and a separate Markdown mode for parsing. Other model names use ParseHawk’s generic workflow-specific prompts.

Change runtime resources

Section titled “Change runtime resources”
Terminal window
PARSEHAWK_VLLM_MAX_MODEL_LEN=16384 \
PARSEHAWK_VLLM_MAX_NUM_SEQS=2 \
parsehawk restart

Context length and concurrency consume more unified memory or VRAM. See deployment and hardware before raising defaults on a constrained machine. Parsing also has a separate per-page output budget through PARSEHAWK_PARSING_MAX_TOKENS.

Return after using another compatible server

Section titled “Return after using another compatible server”

Ollama and other compatible servers reuse the same provider slot. To restore the bundled endpoint, configure the URL visible to the Docker worker, then restart with the runtime enabled:

Terminal window
# macOS Apple Silicon
parsehawk providers configure openai_compatible_api \
  --base-url http://host.docker.internal:8080/v1
parsehawk restart

On Linux Compose, use the service-network URL instead:

Terminal window
parsehawk providers configure openai_compatible_api \
  --base-url http://runtime:8080/v1
parsehawk restart