CTCDecoder

class torchaudio.models.decoder.CTCDecoder

CTC beam search decoder from Flashlight [Kahn et al., 2022].

Note

To build the decoder, please use the factory function ctc_decoder().

Tutorials using CTCDecoder:

ASR Inference with CTC Decoder

ASR Inference with CTC Decoder

Methods

__call__

CTCDecoder.__call__(emissions: FloatTensor, lengths: Optional[Tensor] = None) → List[List[CTCHypothesis]]

Performs batched offline decoding.

Note

This method performs offline decoding in one go. To perform incremental decoding, please refer to decode_step().

Parameters:

• emissions (torch.FloatTensor) – CPU tensor of shape (batch, frame, num_tokens) storing sequences of probability distribution over labels; output of acoustic model.

• lengths (Tensor or None, optional) – CPU tensor of shape (batch, ) storing the valid length of in time axis of the output Tensor in each batch.

Returns:

List of sorted best hypotheses for each audio sequence in the batch.

Return type:

List[List[CTCHypothesis]]

decode_begin

CTCDecoder.decode_begin()

Initialize the internal state of the decoder.

See decode_step() for the usage.

Note

This method is required only when performing online decoding. It is not necessary when performing batch decoding with __call__().

decode_end

CTCDecoder.decode_end()

Finalize the internal state of the decoder.

See decode_step() for the usage.

Note

This method is required only when performing online decoding. It is not necessary when performing batch decoding with __call__().

decode_step

CTCDecoder.decode_step(emissions: FloatTensor)

Perform incremental decoding on top of the curent internal state.

Note

This method is required only when performing online decoding. It is not necessary when performing batch decoding with __call__().

Parameters:

emissions (torch.FloatTensor) – CPU tensor of shape (frame, num_tokens) storing sequences of probability distribution over labels; output of acoustic model.

Example

>>> decoder = torchaudio.models.decoder.ctc_decoder(...)
>>> decoder.decode_begin()
>>> decoder.decode_step(emission1)
>>> decoder.decode_step(emission2)
>>> decoder.decode_end()
>>> result = decoder.get_final_hypothesis()

get_final_hypothesis

CTCDecoder.get_final_hypothesis() → List[CTCHypothesis]

Get the final hypothesis

Returns:

List of sorted best hypotheses.

Return type:

List[CTCHypothesis]

Note

This method is required only when performing online decoding. It is not necessary when performing batch decoding with __call__().

idxs_to_tokens

CTCDecoder.idxs_to_tokens(idxs: LongTensor) → List

Map raw token IDs into corresponding tokens

Parameters:

idxs (LongTensor) – raw token IDs generated from decoder

Returns:

tokens corresponding to the input IDs

Return type:

List

Support Structures

CTCHypothesis

class torchaudio.models.decoder.CTCHypothesis(tokens: torch.LongTensor, words: List[str], score: float, timesteps: torch.IntTensor)

Represents hypothesis generated by CTC beam search decoder CTCDecoder.

Tutorials using CTCHypothesis:

ASR Inference with CTC Decoder

ASR Inference with CTC Decoder

tokens: LongTensor

Predicted sequence of token IDs. Shape (L, ), where L is the length of the output sequence

words: List[str]

List of predicted words.

Note

This attribute is only applicable if a lexicon is provided to the decoder. If decoding without a lexicon, it will be blank. Please refer to tokens and idxs_to_tokens() instead.

score: float

Score corresponding to hypothesis

timesteps: IntTensor

Timesteps corresponding to the tokens. Shape (L, ), where L is the length of the output sequence

CTCDecoderLM

class torchaudio.models.decoder.CTCDecoderLM

Language model base class for creating custom language models to use with the decoder.

Tutorials using CTCDecoderLM:

ASR Inference with CTC Decoder

ASR Inference with CTC Decoder

abstract start(start_with_nothing: bool) → CTCDecoderLMState

Initialize or reset the language model.

Parameters:

start_with_nothing (bool) – whether or not to start sentence with sil token.

Returns:

starting state

Return type:

CTCDecoderLMState

abstract score(state: CTCDecoderLMState, usr_token_idx: int) → Tuple[CTCDecoderLMState, float]

Evaluate the language model based on the current LM state and new word.

Parameters:

• state (CTCDecoderLMState) – current LM state

• usr_token_idx (int) – index of the word

Returns:

(CTCDecoderLMState, float)

CTCDecoderLMState:

new LM state

float:

score

abstract finish(state: CTCDecoderLMState) → Tuple[CTCDecoderLMState, float]

Evaluate end for language model based on current LM state.

Parameters:

state (CTCDecoderLMState) – current LM state

Returns:

(CTCDecoderLMState, float)

CTCDecoderLMState:

new LM state

float:

score

CTCDecoderLMState

class torchaudio.models.decoder.CTCDecoderLMState

Language model state.

Tutorials using CTCDecoderLMState:

ASR Inference with CTC Decoder

ASR Inference with CTC Decoder

property children: Dict[int, CTCDecoderLMState]

Map of indices to LM states

child(usr_index: int) → CTCDecoderLMState

Returns child corresponding to usr_index, or creates and returns a new state if input index is not found.

Parameters:

usr_index (int) – index corresponding to child state

Returns:

child state corresponding to usr_index

Return type:

CTCDecoderLMState

compare(state: CTCDecoderLMState) → CTCDecoderLMState

Compare two language model states.

Parameters:

state (CTCDecoderLMState) – LM state to compare against

Returns:

0 if the states are the same, -1 if self is less, +1 if self is greater.

Return type:

int