FlutterFlow Voice Assistant: Skip the STT + TTS Pipeline

Every FlutterFlow voice assistant tutorial follows the same recipe: add the speech_to_text package as a custom action, send the transcript to an LLM via an API call, then pipe the reply through OpenAI's /v1/audio/speech endpoint and play the MP3. FlutterFlow's own multilingual voice assistant tutorial teaches exactly this — a startListening action, a stopListening action, an OpenAI fetch, and audio playback.

It works in a demo. In a shipped app, it feels like a walkie-talkie: press, speak, wait, listen, repeat. Before you sink a weekend into that pipeline, it's worth understanding why it can never feel like a real conversation — and what the alternative looks like.

What the tutorial pipeline actually requires

To build the classic flutterflow speech to text custom action flow you need all of this:

1. Pubspec dependency: speech_to_text (currently v7.4.0 on pub.dev) declared in your custom action's settings.
2. iOS permissions: NSSpeechRecognitionUsageDescription and NSMicrophoneUsageDescription in Info.plist.
3. Android permissions: RECORD_AUDIO, INTERNET, plus Bluetooth permissions and a <queries> block for the speech service on SDK 30+.
4. Two custom actions to start and stop listening, wired to page state so the transcript renders live.
5. An LLM API call on the transcript.
6. A TTS fetch against OpenAI's /v1/audio/speech and an audio-playback step.

The listening action typically looks like this:

// FlutterFlow custom action: startListening
// Dependency: speech_to_text: ^7.4.0
import 'package:speech_to_text/speech_to_text.dart' as stt;

final stt.SpeechToText speech = stt.SpeechToText();

Future<void> startListening() async {
  final available = await speech.initialize();
  if (!available) return;
  await speech.listen(
    onResult: (result) {
      // Write the transcript into FlutterFlow page state
      FFAppState().update(() {
        FFAppState().transcript = result.recognizedWords;
      });
    },
  );
}

Then a second action calls speech.stop(), and a third posts the reply text to OpenAI for audio:

// FlutterFlow custom action: fetch OpenAI TTS audio
final response = await http.post(
  Uri.parse('https://api.openai.com/v1/audio/speech'),
  headers: {
    'Authorization': 'Bearer $OPENAI_API_KEY', // shipped inside your app!
    'Content-Type': 'application/json',
  },
  body: jsonEncode({
    'model': 'gpt-4o-mini-tts',
    'voice': 'alloy',
    'input': assistantReplyText,
  }),
);
// response.bodyBytes is an MP3 you now have to buffer and play

Six moving parts, three vendors' docs, and native config on both platforms — for the flutterflow text to speech openai approach alone.

Where the turn-based pipeline breaks

It's strictly turn-based — no barge-in

The user speaks, then waits for STT to finalize, then waits for the LLM, then waits for the entire TTS file to generate and download, then listens. If the assistant misunderstood, the user must sit through the whole wrong answer. Interrupting mid-reply (barge-in) is architecturally impossible: your mic action isn't even running while audio plays, and turning it on would make STT transcribe your own TTS output.

speech_to_text isn't built for conversation

The package maintainers are explicit: it's "designed for short intermittent use, like when expecting a response to a question, or issuing a single voice command." On Android, recognition stops after pauses of a few seconds; on iOS, Apple caps a session at roughly one minute. Users who pause mid-sentence get cut off, and you end up writing restart-loop hacks that drain battery.

Latency stacks linearly

Each stage waits for the previous one to fully finish: final transcript → full LLM response → full audio file → playback start. Even with fast models, users routinely wait several seconds in silence between turns. Real-time voice systems avoid this by streaming audio both ways continuously — something you cannot bolt onto this pipeline with more custom actions.

Your OpenAI key ships in the app

Custom actions run on-device, so the Authorization header above means your API key is embedded in the binary. Anyone with a proxy or a decompiler can extract it and run up your bill. Doing it properly means building and hosting your own token-minting backend — a whole second project.

Web support is a minefield

Browser speech recognition support varies by engine, autoplay policies block your TTS playback until a user gesture, and the same custom action code often needs web-specific branches.

The alternative: embedded real-time speech-to-speech

If what you actually want is add voice ai to flutterflow app — a support assistant your users can talk to — you can skip the pipeline entirely. WidgetChat's live voice chat gives you true flutter speech to speech ai inside the same embeddable support widget you'd use for text chat:

• Tap the mic, start a real-time voice call. The assistant listens and replies out loud in a natural voice — no STT/LLM/TTS relay you maintain.
• Barge-in works. Users can interrupt the assistant mid-sentence and it stops and listens, which is the single biggest thing the turn-based pipeline can't do.
• Live captions render during the call, and the assistant can show rich product cards on screen while it speaks — so a voice answer about a product can be accompanied by the actual product on screen.
• Keys stay server-side. Provider API keys are never shipped in your app, which deletes the decompile-and-steal problem above.
• One integration, three platforms: the same widget works in iOS, Android, and web Flutter apps.
• Same conversation, same dashboard as text chat. Voice is configured per project in the dashboard's Voice section — enable/disable it, pick the voice, set max session length and the captions default — and usage is metered by a monthly voice-minute pool on your plan.

For text chat, integration is a plain HTTP call from a custom action — no proprietary SDK. WidgetChat streams answers token-by-token over Server-Sent Events:

// Streaming text chat: works from a FlutterFlow custom action
final request = http.Request(
  'POST',
  Uri.parse('https://api.widgetchat.app/v1/chat/stream'),
);
request.headers['Content-Type'] = 'application/json';
request.body = jsonEncode({'message': userMessage});

final response = await http.Client().send(request);
await for (final chunk
    in response.stream.transform(utf8.decoder)) {
  for (final line in chunk.split('\n')) {
    if (line.startsWith('data: ')) {
      appendToChatUI(line.substring(6)); // token-by-token
    }
  }
}

Voice rides on top of that same widget and conversation — you don't write the audio plumbing at all. Enable voice in the dashboard, and the mic button appears in the embedded widget.

Decision guide

Build the speech_to_text + OpenAI TTS pipeline yourself when you need a single voice command — "search for red shoes," dictating a note into a field. That's what the package is designed for, and one custom action is genuinely the right tool.

Use an embedded speech-to-speech widget when you're building a real-time voice chatbot flutter users converse with — support questions, product guidance, anything multi-turn. Turn-based pipelines structurally can't deliver interruption, continuous listening, or low-latency back-and-forth, and a flutterflow voice assistant that can't be interrupted gets abandoned after the first wrong answer.

Ship a voice assistant this afternoon

Embed the WidgetChat widget in your Flutter or FlutterFlow app, feed it your content, and flip on voice in the dashboard's Voice section. Your users get a support assistant they can genuinely talk to — with barge-in, live captions, and on-screen product cards — and you never touch a pubspec speech dependency, an Info.plist string, or a client-side API key.

Try WidgetChat free — start with text chat on the free tier and enable live voice when you're ready.