2

I'd like to search for videos that contain specific words in a foreign language. I wonder if it's possible to search for videos through keywords in a determined language. I also wonder if it's possible to limit the time frame within which the words in question are spoken.

For instance, I type the words "Verantwortung, Recht, beschützen, Anwalt", then I select the language - German, then I limit the time frame that such words should be found, say 3 minutes (so the words I type should be said within that time frame). Then the AI recommends the videos that matches my prompt.

2
  • 1
    Searching for existing videos, right? (as opposed as generating videos). If yes, better remove the text-to-video tag. Commented Nov 10, 2023 at 8:12
  • That's right! Thanks!
    – Kevin
    Commented Nov 10, 2023 at 14:22

1 Answer 1

0

You can use GPT-4 Vision, Gemini or Claude 3.

To do this you have to make screenshots for the video and send to the models to analyze it with your prompt.

Another not so known model that can possibly do this is Marengo from TwelveLabs but you should check the languages it supports.

For open source it has to be a try and miss thing but it does have some multimodal models that can possibly achieve this like: LLAVA, NextGPT, ImageBind, ClipInterrogator and others but the same rule applies to having to screenshot.

LLAVA has a video version it might exist others I don't know about: https://huggingface.co/LanguageBind/Video-LLaVA-7B

OBS: If I am not mistaken, Gemini allow to send the videos directly if their are from youtube or you upload it to a bucket on gcloud first and give it its path.

Here is a doc in how to analyze videos with GPT-4-V: https://cookbook.openai.com/examples/gpt_with_vision_for_video_understanding

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.