Arabic Speech Recognition in Laravel with Deepgram Nova-3 (Part 2 of 4)

Why Arabic STT Is Hard

Standard Arabic (فصحى) is what Arabic news channels use. But everyday Arabic in the UAE is a mix of Gulf dialect (عامية خليجية), English loanwords, and brand names — sometimes all in one sentence:

"أبي أروح Downtown، كم الفير؟"
("I want to go Downtown, what's the fare?")

Most speech models fail on this. Google Speech-to-Text misses dialects. Azure Cognitive Services is inconsistent on Gulf Arabic. Deepgram Nova-3 (January 2026) was the first model we tested that handled this well, including code-switching between Arabic and English.

What the Flutter App Sends

The Flutter app records audio using the record package — it outputs .webm format (Opus codec). Before sending, it encodes the bytes as base64:

// Flutter side (for reference — not Laravel code)
final audioBytes = await File(audioPath).readAsBytes();
final audioBase64 = base64Encode(audioBytes);

// Sent in the request body:
// { "audio_base64": "...", "module_context": "booking", "session_id": "..." }

On the Laravel side, the controller reads audio_base64 from the request and passes it to the STT service.

The SpeechToTextService

Place this at Modules/Assistant/app/Services/SpeechToTextService.php.

<?php

namespace Modules\Assistant\Services;

use Illuminate\Http\Client\RequestException;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Log;

class SpeechToTextService
{
    private string $apiKey;
    private string $baseUrl;
    private string $model;
    private string $language;
    private array  $options;

    public function __construct()
    {
        $this->apiKey   = (string) (config('assistant.deepgram.api_key') ?? '');
        $this->baseUrl  = (string) (config('assistant.deepgram.base_url') ?? '');
        $this->model    = (string) (config('assistant.deepgram.model') ?? '');
        $this->language = (string) (config('assistant.deepgram.language') ?? '');
        $this->options  = (array)  (config('assistant.deepgram.options') ?? []);
    }

    /**
     * Transcribe base64-encoded audio from Flutter.
     *
     * @return array{
     *   transcript: string,
     *   confidence: float,
     *   language_detected: string,
     *   duration_sec: float,
     *   latency_ms: int,
     *   request_id: string|null
     * }
     * @throws \RuntimeException on API failure
     */
    public function processAudio(string $audioBase64): array
    {
        $audioBytes = base64_decode($audioBase64);

        $queryParams = array_merge([
            'model'    => $this->model,
            'language' => $this->language,
        ], $this->prepareTranscriptionParams());

        $url = $this->baseUrl . '/listen?' . http_build_query($queryParams);

        $startMs = (int) (microtime(true) * 1000);

        try {
            $response = Http::withHeaders([
                'Authorization' => 'Token ' . $this->apiKey,
                'Content-Type'  => 'audio/webm',
            ])
            ->withBody($audioBytes, 'audio/webm')
            ->timeout(30)
            ->post($url);

            $latencyMs = (int) (microtime(true) * 1000) - $startMs;

            if ($response->failed()) {
                Log::error('Deepgram STT error', [
                    'status' => $response->status(),
                    'body'   => $response->body(),
                ]);
                throw new \RuntimeException('STT service error: ' . $response->status());
            }

            $data = $response->json();

            $channel     = $data['results']['channels'][0] ?? [];
            $alternative = $channel['alternatives'][0] ?? [];
            $transcript  = $alternative['transcript'] ?? '';
            $confidence  = $alternative['confidence'] ?? 0.0;
            $durationSec = $data['metadata']['duration'] ?? 0.0;

            // Deepgram returns detected_language per channel when language=multi
            $detectedLang = $channel['detected_language'] ?? 'ar';
            $normalised   = $this->mapToLanguageCode($detectedLang);

            return [
                'transcript'        => $transcript,
                'confidence'        => (float) $confidence,
                'language_detected' => $normalised,
                'duration_sec'      => (float) $durationSec,
                'latency_ms'        => $latencyMs,
                'request_id'        => $data['metadata']['request_id'] ?? null,
            ];
        } catch (RequestException $e) {
            throw new \RuntimeException('STT request failed: ' . $e->getMessage(), 0, $e);
        }
    }

    private function prepareTranscriptionParams(): array
    {
        $params = [];

        if ($this->options['smart_format'] ?? false) {
            $params['smart_format'] = 'true';
        }
        if ($this->options['utterances'] ?? false) {
            $params['utterances'] = 'true';
        }
        if (isset($this->options['endpointing'])) {
            $params['endpointing'] = $this->options['endpointing'];
        }
        // Keyterms are repeated query params: keyterm=حجز&keyterm=مطار
        foreach ($this->options['keyterms'] ?? [] as $term) {
            $params['keyterm'][] = $term;
        }

        return $params;
    }

    private function mapToLanguageCode(string $deepgramLang): string
    {
        if (str_starts_with($deepgramLang, 'ar')) {
            return 'ar';
        }
        if (str_starts_with($deepgramLang, 'en')) {
            return 'en';
        }
        return 'mixed';
    }
}

Three Things Worth Understanding

1. language=multi is the key setting

Most STT APIs make you pick one language and stick with it. Deepgram's multi mode detects Arabic or English per utterance automatically. For UAE users who naturally mix languages, this is not optional — it's essential. The detected language (e.g. ar-AE) comes back in channel.detected_language, which we normalise to ar, en, or mixed.

2. Keyterms boost domain-specific vocabulary

Deepgram's keyterm parameter hints the model toward words common in your domain. Without it, the model might transcribe the Arabic word for "booking" as something phonetically similar but wrong. We boost ride-booking vocabulary: حجز (booking), مطار (airport), تاكسي (taxi), plus English landmarks like Downtown and Marina.

3. Smart format adds punctuation automatically

Without smart_format=true, you get a wall of unpunctuated text. With it, Deepgram adds periods, commas, and question marks — which makes Claude's job easier (it reads more natural sentences) and also improves the TTS rhythm in Part 4.

Language Detection for Text Input

When the user types instead of speaking, there's no Deepgram to detect the language. We detect it from Unicode character ranges — Arabic characters (\p{Arabic}) vs. Latin characters:

private function identifyLanguage(string $text): LanguageEnum
{
    if (empty(trim($text))) {
        return LanguageEnum::Arabic; // UAE default
    }

    preg_match_all('/[\p{Arabic}]/u', $text, $arabicMatches);
    preg_match_all('/[a-zA-Z]/u', $text, $latinMatches);

    $arabicCount = count($arabicMatches[0]);
    $latinCount  = count($latinMatches[0]);

    if ($arabicCount > 0 && $latinCount > 0) {
        return LanguageEnum::Mixed;
    }

    return $arabicCount > 0 ? LanguageEnum::Arabic : LanguageEnum::English;
}

This runs on the controller side and feeds the same LanguageEnum value into the pipeline — so the text path and the voice path are treated identically downstream.

Cost Tracking: STT Pricing

Deepgram charges per second of audio transcribed. Our pricing snapshot: $0.00011944 per second ($4.30 per 1,000 minutes). For a 5-second voice clip, that's about $0.0006 — effectively free per message, but it adds up across thousands of users. The cost tracking system (Part 4) logs every STT call with duration and cost so you can see the real numbers.

What's Next

Part 3 covers the most complex part of the system: sending the transcript to Claude, designing a system prompt that works in both Arabic and English, defining 15+ tool calls for booking and cargo actions, and handling the two-turn tool use flow that Claude requires whenever it needs to execute an action before it can respond.

Pair push with transactional email for full communication coverage. Mailgun's MENA-region routing reliably reaches UAE and KSA inboxes with a generous free tier to start.