thousandofwind / aivideogenerator Goto Github PK

AI Video Generator

Search news trending, collect information, and generate the news video automatically.
View Generated Video · Report Bug · Request Feature

This project focuses on generating short videos based on news webpages. It begins by parsing the content of the news webpage, after which it utilizes the GPT4 model to generate a news anchor script. Based on this script, appropriate images are collected or generated. All these materials are then compiled to create a news short video. The final generated video cites the news source and each image includes a website watermark for reference.

Support originality, all resources used in video have marked with its source!

This project is under developing, so interface may change anytime.

(back to top)

(back to top)

To get a local copy up and running follow these simple example steps.

• Have an Azure account
• Create Azure Openai Service
• Create Azure Speech Service
• Create Bing Search Service

1. Clone the repo

   git clone https://github.com/ThousandOfWind/AIVideoGenerator.git

2. Make sure you are above python 3.9, and install python packages

   pip install -r requirement.txt

3. Creat a .env file in root path, then enter your endpoint and key

    OPANAI_API_ENDPOINT=https://{}.openai.azure.com/
    OPANAI_API_KEY={}
    SPEECH_HOST=customvoice.api.speech.microsoft.com            
    SPEECH_KEY={}
    SPEECH_REGION={}
    BING_SEARCH_ENDPOINT=https://api.bing.microsoft.com/v7.0/
    BING_SEARCH_KEY={}

(back to top)

• '-r', '%.02f' % fps, must be real number, not NoneType Not sure why it happen, can be fixed only by assign value for fps in moviepy/video/io/ffmpeg_writer.py, currently I assign the 30.0
• an temp audio file will be generate by will not be delete automatically
• the avatar video in webm type have transparent background, it automatically become white, hid the image for news

On going fix for the issue: use FFMPEG instead of moviepy

Take example_gen_video.py as an example that generate a video for 1st sport news in China.

• To get news, use BingSearchAdapter in tools/search_adapter.py. It depends on your bing search service. A free plan is enough for this project.
• Avatar functionality requires a non-free plan in limited region, please refer to Azure Speech Service for details.
• OCR functionality is powered by EasyOCR

load_dotenv()
storage = LocalStorage(os.path.join("output", str(time.time_ns())))
# storage = LocalStorage("output/1708786735037148000")

config = ManagerConfig({
    "director_config": DirectorConfig({
        "use_image_in_webpage": True,
        "search_online_image": True,
        "use_table_in_webpage": True,
        "use_avatar": True,
        "use_ocr":True
    }),
    "information_config": InformationConfig(),
    "ai_config": AIConfig({
        "type": "AzureOpenAI",
        "api_version": "2023-12-01-preview",
        "api_key": os.getenv('OPANAI_API_KEY'), 
        "endpoint": os.getenv('OPANAI_ENDPOINT')
    }),
    "speech_config": SpeechConfig({
        "key": os.getenv('SPEECH_KEY'),
        "region": 'southeastasia'
    }),
    "search_config": SearchConfig({
        "bing_search_key": os.getenv('BING_SEARCH_KEY')
    })
})

manager = Manager(storage, config)
news = manager.search.news_category_trending(ChinaCategory.Auto.value)[0]
webpage_info = manager.information_collector.get_webpage(news['url'])
script = manager.video_director.webpage2script(webpage_info)
draft_video = manager.video_director.direct(script, webpage_info.name)
output_video_info = manager.video_director.export(draft_video)

(back to top)

• Script initialize.
  • news webpage to script
  • any webpage to script
  • any document to script
  • any topic to script
• Collect/Generate multimedia resource for a script
  • Text to speech
  • Search online image for text
  • Draw image if no ideal existing image
  • Speech to avatar
  • Search online video, bgm for text
• Merge all resource to video
• Video improvement
  • Remove white background of avatar / change to another way to add avatar
  • Improve word segmentation, tone and gesture
  • Better turnaround
  • BGM
  • Fix Avatar background issue
  • Different length
  • Any size -> Avatar position and size auto-adjust
  • Image normalize by cut, move camera aperture.
• [Current on going] Go deeper into content
  • Download image/video in webpage
  • Add OCR when review image for news
  • Search related information
  • Draw table / chart if need
  • Download linked webpage
  • RAG on knowledge
• UX
  • UI Design
  • GUI
  • CMD
  • REST
• Infra
  • More log
  • Async methods
  • More Comments
  • Error handling
  • Cost analysis
  • Test
  • Name of variables
• Integrate social media
• Integrate Lang Chain
• Onboard GPT store
• Monitor time and cost for a generation

(back to top)

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

Distributed under the MIT License. See LICENSE.txt for more information.

(back to top)

Julie Zhu - [email protected]

Project Link: https://github.com/ThousandOfWind/AIVideoGenerator

(back to top)

• BGM Neon Lights from: https://www.fiftysounds.com/zh/

(back to top)