Adaptive Model Dispatcher

Interactive Pitch

Project Presentation

Click through the tabs below to explore how we optimized accuracy and token usage for the AMD Hackathon.

Summary

Challenge Overview & Dispatcher Strategy

The AMD Developer Hackathon (Track 1) evaluates agents based on Accuracy Gate first, and then sorts the passing entries by Total Token Consumption (lower is better).

🎯

The Accuracy Gate Challenge

Missing or failing the LLM-Judge's accuracy threshold means immediate disqualification from the leaderboard. High accuracy is non-negotiable.

⚡

Token Optimization Strategy

Instead of sending raw prompts directly to expensive models, we compress, solve deterministically, and direct queries to specialized models.

Key Breakthrough: We achieved near-zero token usage on math, unit conversions, and classification, routing only complex reasoning and coding tasks to premium Fireworks APIs.

Architecture

5-Layer Sequential Processing Pipeline

1. Compression

Compresses repeated spaces, blank lines, and URLs. Decreases prompt token count by up to 15% while preserving code indentation.

2. Tier-0 Deterministic

Intercepts tasks and solves simple arithmetic, percentages, powers, and unit conversions natively in Python at 0 token cost.

3. Triage & Classify

Two-tier classifier analyzes prompt. Fast-path regex rules run first (0 tokens). Lightweight LLM classifies complex cases (~100 tokens).

4. Model Dispatch

Selects the most cost-efficient allowed model on Fireworks AI for the task category. Uses local Qwen 2.5 inside the Docker container to bypass API calls.

5. Validation & Clean

Validates the response structure (free local check). If validation fails, triggers corrective retry. Cleans code fences and intro preambles.

Tier 0

Deterministic Python Solver (0 Tokens)

Sending simple math, percentage, and conversion questions to LLMs wastes tokens and risks arithmetic errors. We solve these natively with 100% accuracy and zero token cost.

🔢 Arithmetic

Evaluates complex equations using a safe Abstract Syntax Tree (AST) parser.

"15.5 * (40 / (2.5 + 1.5)) - 100"

🏷️ Percentages

Processes sales discounts, interest rates, and percentage margins.

"What is 15% of 200?"

⚖️ Unit Conversion

Converts lengths, weights, currencies, and time intervals dynamically.

"How many cm in 5 meters?"

Routing

Adaptive Model Routing & Gemma Circuit-Breaker

We dispatch each category to a custom model role to balance quality, pricing, and bonus payouts.

MiniMax M3 & Kimi K2.7 Code

MiniMax M3 is assigned as the default and reasoning model (high general accuracy). Kimi K2.7 Code handles code generation and debugging tasks, ensuring syntactic correctness.

Google Gemma 4 & Circuit Breaker

To qualify for the Gemma bonus prize, we attempt Gemma 4 first. However, since serverless Gemma isn't guaranteed on Fireworks, we implement a circuit-breaker: if a Gemma call fails or times out once, the dispatcher dynamically bypasses Gemma for the rest of the batch to save time.

Accuracy Guard

Speculative Validation & Corrective Retries

To guarantee we pass the Accuracy Gate, the output of every model call is inspected by local validation rules before finalizing results.

❌

Inspection

Checks for refusals ("I am sorry", "As an AI"), empty strings, truncated code, or invalid JSON arrays (NER).

→

🔄

Corrective Retry

If validation fails, a second speculative attempt is dispatched with a lower temperature and stricter formatting prompt.

→

✓

Output Clean

Strips conversational headers ("Sure!", "Here is...") and markdown fences to deliver exact answers to the LLM-Judge.

8 Supported Task Categories

📚 Factual Knowledge

🔢 Mathematical Reasoning

💬 Sentiment Classification

📝 Text Summarization

🏷️ Named Entity Recognition

🐛 Code Debugging

🧩 Logical Reasoning

💻 Code Generation

Technology Stack

🐍

Python 3.11

🔥

Fireworks AI

🐳

Docker

🧪

133 Tests

📊

Chart.js

🤖

MiniMax M3

💎

Kimi K2P7

✨

Gemma 4

Project Presentation

Executive Summary

5-Layer Pipeline

Tier 0 Engine

Adaptive Routing

Validation Guard