Skip to main content

Speech Model

The Speech model can classify audio into "speech", "music", "silence" or "other". The "silence" class indicates audio fragments with very low volume (according to the requested threshold) and the "other" class contains everything which is not speech or music.

Our latest SpeechRT outperforms the current Speech model in terms of speech classification in common use cases. When deciding between the Speech and SpeechRT models, it is advisable to experiment with your own data and review the latest advice to ensure optimal performance for your use case.

The receptive field of this model is 1082ms.

Specification​

Receptive FieldResult Type
1082msresult ∈ ["speech", "music", "other", "silence"]

Time-series​

The time-series result will be an iterable with elements that contain the following information:

{
  "timestamp": 0,
  "results":{
    "speech": {
        "result": "speech",  
        "confidence": 0.92
    }
  }
}

Time-series with raw values​

If raw values were requested, they will be added to the time-series result:

{
  "timestamp": 0,
  "results":{
    "speech": {
        "result": "speech",  
        "confidence": 0.92
    }
  },
  "raw": {
    "speech": {
      "music": 0.0499,
      "other": 0.0301,
      "speech": 0.92
    }
  }
}

Summary​

In case a summary is requested the following will be returned

{
  "speech": {
    "speech_fraction": 0.30,
    "other_fraction": 0.65,
    "music_fraction": 0.05,
    "silence_fraction": 0.0
  }
}

where x_fraction represents the percentage of time that x class was identified for the duration of the input.

Transitions​

In case the transitions are requested a time-series with the following transition elements will be returned:

{
  "timestamp_start": 0,
  "timestamp_end": 1500,
  "result": "other",
  "confidence": 0.96
},
{
  "timestamp_start": 1500,
  "timestamp_end": 6000,
  "result": "music",
  "confidence": 0.86
}

The result above means that the first 1500ms of the audio snippet contained no speech or music, and between 1500ms and 6000ms of the audio - music was detected.