Audio to Text using OpenAI's Whisper

Integrating OpenAI models with PROCESIO is something anyone can do due to the simplicity of using the Call API action. If you don't believe us, we're going to prove how easy it is with this brand-new use case.

Scenario:

Let's say you work for a company that specializes in providing accessibility services for people with hearing impairments. You want to generate transcripts from audio files such as podcasts or radio theater so that anyone who suffers from hearing loss or similar impairments can enjoy this type of content.

We're going to use the Whisper model from OpenAI to generate transcripts from audio files. First, we retrieve the audio file from our Google Drive and then we generate the transcript.

The audio file that will be used in our example is an MP3 recording from Marian.

Retrieve the file from Google Drive

We'll start by creating the Credential to access Google Drive.

We define some variables to be used in our flow:

• FileURL ➜ String containing the audio file location.
  • To create the file URL for your own files, you can follow this tutorial.
• Endpoint ➜ String representing the Google Drive endpoint to download from.
• DownloadStatusCode ➜ Integer representing the status code of the download request.
• AudioFile ➜ File representing the downloaded audio file.

Then we build the actual flow:

• We extract the Endpoint by removing the base URL from FileURL using String Replace
• We download the file using Call API with our Google Drive credential.

For Call API we will have to use the following headers:

• Content-Type ➜ application/force-download
• Content-Disposition ➜ attachment

Generate the transcript

We create the Credential for OpenAI.

We define some variables to be used in our flow:

• FileURL ➜ String containing the audio file location.
• AudioFile ➜ File representing the downloaded audio file.
• queryResponse ➜ Json holding the model's response.
• queryStatusCode ➜ Integer representing the model response status code.
• Transcript ➜ String containing the transcript for the audio file.

Then we build the actual flow:

• We download the audio file by using Call Subprocess with our first process.
• We query the Whisper model using Call API with a form-data body.
• We extract the transcript from the model's response using Json Mapper.

For Call API we will use form-data:

• file [File] ➜ insert AudioFile variable
• model [Text] ➜ whisper-1
• response_format [Text] ➜ json



Transcript

Action Pool

• Call API
• String Replace
• Call Subprocess
• Json Mapper

Import File