AI Node 2026

2026

Qwen3.6 35B weights distributed across four Nvidia RTX5060ti 16GB using NVFP4 for maximum performance and capacity.

Supports uo tp 256K context but designed for up to 32 concurrent conversations each with average 32K context.

Average context per conversation Approximate concurrent conversations
32K 32
64K 22
128K 11
256K 5

Some token capacity are wasted due to initial 4 stage pipeline parallel operation, the hybrid pipeline and tensor parallel operation will take over when available token drop below 20% to increase overall token capacity.

Performance

Comparing Qwen3.6 35B with Qwen3.8 27B.
Note Qwen3.8 activates the entire 27B dense network while Qwen3.6 activates only about 3B MoE parameters per token.

Using FP32 for recurrent GDN/Mamba states since we want to support large 256K context.

Parameter Qwen3.6 current Qwen3.8 tested Comparison
--model-path nvidia/Qwen3.6-35B-A3B-NVFP4 RadixArk/Qwen3.8-27B-NVFP4 Different
--host 0.0.0.0 0.0.0.0 Same
--port 8080 8080 Same
--trust-remote-code Enabled Enabled Same
--tp-size 1 1 Same
--pp-size 4 4 Same
PP layer partition 10,10,10,10 16,16,16,16 Different layer counts
--mm-feature-transport cpu cpu Same
--disable-custom-all-reduce Enabled Enabled Same
--context-length 262144 262144 Same
--kv-cache-dtype fp8_e4m3 auto → FP8 E4M3 Different selection method
--attention-backend Not specified — automatic flashinfer Different
--mem-fraction-static 0.88 0.80 Different, required for Qwen3.8 images
--chunked-prefill-size 4096 4096 Same
--max-running-requests 32 32 Same
--max-mamba-cache-size 160 32 Different
--reasoning-parser qwen3 qwen3 Same
--tool-call-parser qwen3_coder qwen3_coder Same
--enable-cache-report Enabled Enabled Same
--enable-metrics Enabled Enabled Same
--moe-runner-backend flashinfer_cutlass Not specified/not applicable Qwen3.6 is MoE; Qwen3.8 is dense
Measurement Qwen3.6 Qwen3.8
Architecture 35B MoE, ~3B active 27B dense
Transformer layers 40 64
PP partition 10/10/10/10 16/16/16/16
Mamba slots 160 32
Reported token pool 1,469,006 468,012
PP0 weight usage 6.53 GB 7.36 GB
PP0 free after graphs 2.59 GB 2.39 GB
KV scaling Falls back to scale 1.0 warning Checkpoint calibration loaded
Text inference Working Working
Image inference Working at 0.88 Working at 0.80

Main Qwen3.8 Differences

SGLANG_PP_LAYER_PARTITION=16,16,16,16
--kv-cache-dtype auto
--attention-backend flashinfer
--mem-fraction-static 0.80
--max-mamba-cache-size 32

It does not need --moe-runner-backend flashinfer_cutlass because Qwen3.8-27B is dense, not MoE.

Per-user storage Qwen3.6 Qwen3.8 Difference
GDN layers 30 48 1.6×
GDN value heads 32 48 1.5×
FP32 state per slot ~64 MB 153.9 MB ~2.4×
Five slots per active request ~320 MB ~770 MB ~2.4×
Full-attention layers 10 16 1.6×
KV heads 2 4
FP8 KV per context token 10.24 KB 32.77 KB 3.2×

Active Conversations

  • Qwen3.6 token pool with 160 state slots: 1,469,006
  • Qwen3.8 token pool with 32 state slots: 468,012
Context Qwen3.6 FP32 state Qwen3.8 FP32 state
32K ~0.65 GB ~1.82 GB
128K ~1.63 GB ~4.96 GB
256K ~2.94 GB ~9.16 GB

Sample Image

Settings

{
  "temperature": 0,
  "max_tokens": 5000,
  "chat_template_kwargs": {
    "enable_thinking": false
  },
  "response_format": {
    "type": "json_object"
  }
}

Prompt

OCR this receipt. Return valid JSON only. Include every visible line in reading order in raw_text_lines and also structure merchant, store, transaction, items, discounts, totals, and uncertain. Preserve printed wording and numbers. Use null for unreadable values and never guess.

Result

{
  "raw_text_lines": [
    "Coles Supermarkets Australia Pty Ltd",
    "Tax Invoice ABN: 45 004 189 708",
    "coles",
    "Store: 853 - CS BONDI WESTFIELD",
    "Store Manager: Justin",
    "Phone: 02 8383 4800",
    "Served By: Pinky",
    "Register: 007",
    "Receipt: 96",
    "Date: 09/03/2025",
    "Time: 12:42",
    "Description",
    "$",
    "% SORBENT TOILET TSS 24PACK 13.00",
    "ANGAS PARK PITTED PR 1KG 14.00",
    "NUDIE ORANGE JUICE 2LITRE 19.00",
    "2 @ $9.50 EACH",
    "A2 DAIRY FULL CREAM 2LITRE 13.80",
    "2 @ $6.90 EACH",
    "% COLES PIZZA SUPREME 600GRAM 10.00",
    "ICEBERG COS SALAD 500GRAM 5.00",
    "BABY LEAF BLEND 300GRAM 5.00",
    "* TWININGS TEA BAGS:EN 100PACK 6.75",
    "% COLES FZ CONVENIENCE 375GRAM 4.00",
    "% COLES FZ CONVENIENCE 375GRAM 4.00",
    "% COLES MLS SATAY CHIC 375GRAM 4.00",
    "COLES MEALS 3 FOR $10.95 -$1.05",
    "% PARKERS PRETZELS ORI 150GRAM 4.50",
    "% COBS SWEET & SALTY 65GRAM 4.00",
    "CHEEZELS JUMPYS 2 FOR $7 -$1.50",
    "* JALNA KIDS ORGANIC Y 100GRAM 6.00",
    "4 @ $1.50 EACH",
    "* JALNA KIDS ORGANIC Y 100GRAM 1.50",
    "* SPC PUREE FRUIT 4PACK 4.00",
    "* GREENS PANCAKE SHAKE 325GRAM 2.80",
    "% YAKULT 10PK DRINK 65ML 7.70",
    "* La Espanola Mild Fla 1LITRE 18.00",
    "COBRAM ESTATE INFUSE 250ML 12.00",
    "BABY BROCCOLI 1BUNCH 6.40",
    "2 @ $3.20 EACH",
    "MAN/VALLEY 12PK FREE 700GRAM 7.80",
    "BANANAS PERKG 3.42",
    "0.761 kg NET @ $4.50/kg",
    "WHITE GRAPES PERKG 6.13",
    "1.115 kg NET @ $5.50/kg",
    "*%MENTOS MULTI MINT 112GRAM 2.75",
    "COLES CHICKEN SAUSAG 550GRAM 5.90",
    "* TASSAL SMOKED SALMON 250GRAM 16.00",
    "Total for 33 items: $204.90",
    "EFT $204.90",
    "GST INCLUDED IN TOTAL $4.67"
  ],
  "merchant": {
    "name": "Coles Supermarkets Australia Pty Ltd",
    "abn": "45 004 189 708"
  },
  "store": {
    "id": "853",
    "name": "CS BONDI WESTFIELD",
    "manager": "Justin",
    "phone": "02 8383 4800"
  },
  "transaction": {
    "register": "007",
    "receipt_number": "96",
    "date": "09/03/2025",
    "time": "12:42",
    "served_by": "Pinky"
  },
  "items": [
    {
      "description": "SORBENT TOILET TSS 24PACK",
      "price": 13.00,
      "type": "discounted"
    },
    {
      "description": "ANGAS PARK PITTED PR 1KG",
      "price": 14.00
    },
    {
      "description": "NUDIE ORANGE JUICE 2LITRE",
      "price": 19.00,
      "quantity": 2,
      "unit_price": 9.50
    },
    {
      "description": "A2 DAIRY FULL CREAM 2LITRE",
      "price": 13.80,
      "quantity": 2,
      "unit_price": 6.90
    },
    {
      "description": "COLES PIZZA SUPREME 600GRAM",
      "price": 10.00,
      "type": "discounted"
    },
    {
      "description": "ICEBERG COS SALAD 500GRAM",
      "price": 5.00
    },
    {
      "description": "BABY LEAF BLEND 300GRAM",
      "price": 5.00
    },
    {
      "description": "TWININGS TEA BAGS:EN 100PACK",
      "price": 6.75,
      "type": "loyalty"
    },
    {
      "description": "COLES FZ CONVENIENCE 375GRAM",
      "price": 4.00,
      "type": "discounted"
    },
    {
      "description": "COLES FZ CONVENIENCE 375GRAM",
      "price": 4.00,
      "type": "discounted"
    },
    {
      "description": "COLES MLS SATAY CHIC 375GRAM",
      "price": 4.00,
      "type": "discounted"
    },
    {
      "description": "COLES MEALS 3 FOR $10.95",
      "discount": 1.05,
      "type": "multi_buy"
    },
    {
      "description": "PARKERS PRETZELS ORI 150GRAM",
      "price": 4.50,
      "type": "discounted"
    },
    {
      "description": "COBS SWEET & SALTY 65GRAM",
      "price": 4.00,
      "type": "discounted"
    },
    {
      "description": "CHEEZELS JUMPYS 2 FOR $7",
      "discount": 1.50,
      "type": "multi_buy"
    },
    {
      "description": "JALNA KIDS ORGANIC Y 100GRAM",
      "price": 6.00,
      "quantity": 4,
      "unit_price": 1.50,
      "type": "loyalty"
    },
    {
      "description": "JALNA KIDS ORGANIC Y 100GRAM",
      "price": 1.50,
      "type": "loyalty"
    },
    {
      "description": "SPC PUREE FRUIT 4PACK",
      "price": 4.00,
      "type": "loyalty"
    },
    {
      "description": "GREENS PANCAKE SHAKE 325GRAM",
      "price": 2.80,
      "type": "loyalty"
    },
    {
      "description": "YAKULT 10PK DRINK 65ML",
      "price": 7.70,
      "type": "discounted"
    },
    {
      "description": "La Espanola Mild Fla 1LITRE",
      "price": 18.00,
      "type": "loyalty"
    },
    {
      "description": "COBRAM ESTATE INFUSE 250ML",
      "price": 12.00
    },
    {
      "description": "BABY BROCCOLI 1BUNCH",
      "price": 6.40,
      "quantity": 2,
      "unit_price": 3.20
    },
    {
      "description": "MAN/VALLEY 12PK FREE 700GRAM",
      "price": 7.80
    },
    {
      "description": "BANANAS PERKG",
      "price": 3.42,
      "weight": "0.761 kg",
      "unit_price": 4.50
    },
    {
      "description": "WHITE GRAPES PERKG",
      "price": 6.13,
      "weight": "1.115 kg",
      "unit_price": 5.50
    },
    {
      "description": "MENTOS MULTI MINT 112GRAM",
      "price": 2.75,
      "type": "discounted_loyalty"
    },
    {
      "description": "COLES CHICKEN SAUSAG 550GRAM",
      "price": 5.90
    },
    {
      "description": "TASSAL SMOKED SALMON 250GRAM",
      "price": 16.00,
      "type": "loyalty"
    }
  ],
  "discounts": [
    {
      "description": "COLES MEALS 3 FOR $10.95",
      "amount": 1.05
    },
    {
      "description": "CHEEZELS JUMPYS 2 FOR $7",
      "amount": 1.50
    }
  ],
  "totals": {
    "total_items": 33,
    "total_amount": 204.90,
    "payment_method": "EFT",
    "gst_included": 4.67
  },
  "uncertain": []
}
Result Qwen3.6 Qwen3.8
End-to-end latency 8.97 s 14.79 s
Image tokens 4,480 4,480
Total input tokens 4,511 4,553
Completion tokens 153 113