This repository provides a tool for transcribing audio (MP3) files on the LEO5 compute cluster at the University of Innsbruck. It uses AI models to perform speech-to-text, speaker diarization (identifying who is speaking), and optional translation. An optional workflow including encryption at rest enhances security.
$SCRATCH space)./scratch/llm/audio_transcription/scripts/stt_leo5.slurm) to your $HOME or $SCRATCH directory.sbatch stt_leo5.slurm.Please refer to the documentation of these models for details on supported languages and other features.
READ permissions.gocryptfs installed on your local machine for working with encrypted data.# For Debian/Ubuntu
sudo apt install gocryptfs
The necessary software is provided in a Singularity container at /scratch/llm/audio_transcription/container. The scripts are located in /scratch/llm/audio_transcription/scripts. All files are in a central file location on LEO5 for ease of use for everyone. No installation is required on LEO5.
The stt.sh script automatically caches Hugging Face and PyTorch models in your $SCRATCH directory to avoid filling up your $HOME directory.
Set up an encrypted workspace on your local machine. See the Working with Encrypted Data section for detailed instructions.
Ensure the audio files are in MP3 format. If that is not the case, convert them
before use.
Create a directory on LEO5 (preferably in $SCRATCH) and place your audio files there.
Copy the SLURM script to a location where you have write permissions (e.g., your $HOME or $SCRATCH directory).
Edit the "User configuration" section of the SLURM script:
| Parameter | Description |
|---|---|
ENCRYPTED |
Set to 1 for yes, otherwise 0. |
WORKSPACE |
Path to the directory containing your audio files. |
FILENAME |
Name of the file (in the workspace) to process. |
NUM_SPEAKERS |
The number of speakers in the audio (can be 1). |
MAX_SPEAKERS |
The maximum number of speakers if the exact number is unknown. |
LANGUAGE |
The primary language spoken in the audio. |
TASK |
transcribe, translate, or translate_transcript. |
TARGET_LANGUAGE |
Target language for translation. |
MAIL_ADDRESS |
Email addresses to send the transcript to (not used for encrypted data). |
HF_TOKEN |
Hugging Face access token (only for the first use of models – delete after!) |
PASSWORD |
Password for the encrypted directory. Delete after every submission! |
Submit the SLURM script using sbatch:
sbatch stt_leo5.slurm
The audio file and the transcript are removed from LEO5 after the job completes.
Error messages are written to a file called errors.txt.
This workflow ensures that the data remains encrypted from your local machine to the LEO5 cluster and back.
Note that there is a script available that guides through all the steps:
workflow_leo5.sh.
Using ssh-keys is highly recommended. It is also recommended to add the LEO5 user name and the UIBK mail address to the script in order not be prompted for them at every use.
Create the workspace:
cd ~/Documents
mkdir -p stt-workspace/cryptdir stt-workspace/mountpoint
cd stt-workspace
Initialize the encrypted directory:
This command sets up the cryptdir as an encrypted filesystem and prompts you to create a password.
gocryptfs -init cryptdir
Important: Back up the master key that is displayed. Without it, you cannot recover your data if you forget the password.
$SCRATCH space on LEO5.mkdir -p $SCRATCH/stt-workspace/cryptdir
a) Mount and Add Data (Local):
cd ~/Documents/stt-workspace
gocryptfs cryptdir mountpoint
cp ~/Downloads/interview.mp3 mountpoint/
fusermount -u mountpoint
b) Sync to LEO5 (Local):
rsync -av cryptdir/ YOUR_USERNAME@leo5:/scratch/YOUR_USERNAME/stt-workspace/cryptdir/
(Replace YOUR_USERNAME with your LEO5 username.)
c) Run the Job (LEO5):
Execute the SLURM job on LEO5: sbatch stt_leo5.slurm
d) Sync Results Back (Local):
cd ~/Documents/stt-workspace
rsync -av YOUR_USERNAME@leo5:/scratch/YOUR_USERNAME/stt-workspace/cryptdir/ cryptdir/
(Do not forget the trailing slashes.)
e) Access the Results (Local):
gocryptfs cryptdir mountpoint
Your results will be inside the mountpoint directory. Remember to unmount when you are done:
fusermount -u mountpoint
Since on other systems the container is not provided, it has to be built with the definition file. The path to the Python environment package list file has to be adapted therein.
The scripts need to be placed in a directory with read and execute
permission.
The SLURM file is very generic and is expected to work on most systems, depending on the node scheduling policy.
The paths to the directory containing the container and to the
scripts have to be adapted.
Singularity or Apptainer must be available; it is currently loaded as a LEO5-specific module.
All necessary directories (for models and caches) have to be bound to the
container; e.g. with the SINGULARITY_BIND variable.
At the srun command, the path to stt.sh has to be adjusted.
It is also an option to run without SLURM by directly executing
stt.sh after updating all settings therein.
This work got support from EuroCC Austria, the Austrian National Competence Centre for High-Performance Computing, High-Performance Data Analytics and Artificial Intelligence, which has received funding from the European High Performance Computing Joint Undertaking (JU) and Germany, Bulgaria, Austria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, Greece, Hungary, Ireland, Italy, Lithuania, Latvia, Poland, Portugal, Romania, Slovenia, Spain, Sweden, France, Netherlands, Belgium, Luxembourg, Slovakia, Norway, Türkiye, Republic of North Macedonia, Iceland, Montenegro, Serbia under grant agreement No 101101903.
This project was developed by Andreas Lindner with support from the HPC team at the University of Innsbruck.
Please note that the transcription of audio recordings of people constitutes the processing of personal data, which may also include sensitive content. Ensure that you comply with the applicable legal requirements when using this software.