Translate .mp4/.mp3 files

Created on 5 September 2024, 8 months ago

Problem/Motivation

Hi all,

I recently was asked if there is already a way to translate video or audio files in the way that Spotify is reported to be doing in order to release their podcasts in various languages but with the original voice. Maybe the voice isn't a strict requirement in this case, but nevertheless, I think it's very interesting if this were possible. It has the potential to drastically increase a podcaster's or Youtuber's reach.

The idea would be to either upload an audio track and have it returned in a different language so it can be loaded into the video manually, or actually have AI work with the video file to better sync the audio to it. The first option would already be very useful.

Thanks!

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Feature request
Status

Active

Version

1.0

Component

AI Automators

Created by

🇹🇭Thailand AlfTheCat

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @AlfTheCat
  • 🇩🇪Germany marcus_johansson

    We could create an operation type for video dubbing and audio dubbing, since vendors like Elevenlabs can do this. This would be easy to integrate into AI Automators or some other place.

    https://elevenlabs.io/docs/api-reference/create-dub

    But please also note, unless you need automation or Drupal as the main GUI, there are products like this: https://elevenlabs.io/dubbing

  • 🇹🇭Thailand AlfTheCat

    Hi Marcus,

    Thanks for the answer! I'll definitely pass on that link to eleven labs. I suppose it's up to you and the team to decide if it would be useful to have Drupal as the GUI. I would think it's a valuable feature to have, basically just with this feature alone Drupal could be turned into a useful app for many podcasters.

    And automation is always a good thing to have :)

  • Status changed to Closed: outdated about 10 hours ago
  • 🇩🇪Germany marcus_johansson

    SpeechToSpeech exists now. So mp3 should be possible if a provider exists.

Production build 0.71.5 2024