@dhermes, to continue the conversation we were having on chat.

What should the goals be for this?
I would love make a shortcut to access the transcript data more easily but I think that some the streaming interim results make that difficult. The other data that's contextual to the responses should be easy to access as well.

Here's my proposal for a strategy.

Make sync and async recognize return results with the transcripts available directly from the result, since there should be only one result (result.transcript, result.confidence). Then alternatives can be on result.alternatives[0].transcript etc.
Streaming recognize returns many different kinds of responses (see example below). I think we could wrap these responses and just yield them to the consumer. Then you could have things like result.audio_started, result.speech_started(this implies tracking some state though). Then you could have something similar to sync and async where you have the top transcript available directly on the result but other data could be there as well but it could be deeper.
If we added back that result container object that I setup, then we could offer the consumer a method they could poll to get the best over all guess for the response so far. Which in the example below, would be appending the top result for each result_index.

Example at this point

results { alternatives { transcript: " that is" } stability: 0.9 } results { alternatives { transcript: " the question" } stability: 0.01 } result_index: 1

you could call results.get_best_text() = 'to be or not to be that is.

Example of accessing data for

results { alternatives { transcript: " that is" } stability: 0.9 } results { alternatives { transcript: " the question" } stability: 0.01 } result_index: 1

>>> print(result.transcript)
 that is
>>> print(result.alternatives[0].transcript)
 the question

The docs example of streaming_recognize set of responses.

endpointer_type: START_OF_SPEECH
results { alternatives { transcript: "tube" } stability: 0.01 } result_index: 0
results { alternatives { transcript: "to be a" } stability: 0.01 } result_index: 0
results { alternatives { transcript: "to be" } stability: 0.9 } results { alternatives { transcript: " or not to be" } stability: 0.01 } result_index: 0
results { alternatives { transcript: "to be or not to be" confidence: 0.92 } alternatives { transcript: "to bee or not to bee" } is_final: true } result_index: 0
results { alternatives { transcript: " that's" } stability: 0.01 } result_index: 1
results { alternatives { transcript: " that is" } stability: 0.9 } results { alternatives { transcript: " the question" } stability: 0.01 } result_index: 1
endpointer_type: END_OF_SPEECH
results { alternatives { transcript: " that is the question" confidence: 0.98 } alternatives { transcript: " that was the question" } is_final: true } result_index: 1
endpointer_type: END_OF_AUDIO

Speech streaming_recognize() surface could be improved? #2709

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions