• Multi-Provider Support: Choose between OpenAI (GPT-5-mini, GPT-4o) or Google Gemini for scene analysis
• Semantic Analysis Modes:
  • action — Focus on intense combat/action moments
  • funny — Detect fail compilations and humorous moments
  • highlight — Find memorable achievements and clutch plays
  • mixed — Auto-detect the best category for each clip (recommended)

• Speech Mode: Uses OpenAI Whisper to transcribe voice/commentary
• AI Captions Mode: AI-generated contextual captions for gameplay without voice
• Caption Styles: gaming, dramatic, funny, minimal, or auto
• PyCaps Integration: Multiple visual templates including hype, retro-gaming, neo-minimal
• AI Enhancement: Semantic tagging and emoji suggestions (e.g., "HEADSHOT! 💀🔥")

• Local TTS Generation: No cloud API needed for voice synthesis
• Emotion Control: Adjustable emotion/exaggeration levels for English
• Multilingual Support: 20+ languages including Japanese, Korean, Chinese, Spanish, French, and more
• Voice Cloning: Optional reference audio for custom voice styles
• Smart Mixing: Automatic ducking of game audio when voiceover plays

• Scene Detection: Custom implementation using decord + PyTorch on GPU
• Audio Analysis: torchaudio on GPU for fast RMS and spectral flux calculation
• Video Analysis: GPU streaming via decord for stable motion estimation
• Image Processing: cupy (CUDA-accelerated NumPy) for blur and transforms
• Rendering: PyTorch + NVENC hardware encoder for ultra-fast rendering

• Scenes ranked by combined action score (audio 0.6 + video 0.4 weights)
• Configurable aspect ratio (default 9:16 for TikTok/Shorts/Reels)
• Smart cropping with optional blurred background for non-vertical footage
• Retry logic during rendering to avoid spurious failures

AutoShorts is designed to work even when optimal components fail:

• NVIDIA GPU with CUDA support (RTX series recommended for NVENC + TTS)
• NVIDIA Drivers compatible with CUDA 12.x

• Python 3.10
• FFmpeg 4.4.2 (for Decord compatibility)
• CUDA Toolkit with nvcc (for building Decord from source)
• System libraries: libgl1, libglib2.0-0

The Makefile handles everything automatically—environment creation, dependency installation, and building Decord with CUDA support.

git clone https://github.com/divyaprakash0426/autoshorts.git
cd autoshorts

# Run the installer (uses conda/micromamba automatically)
make install

# Activate the environment
overlay use .venv/bin/activate.nu    # For Nushell
# OR
source .venv/bin/activate            # For Bash/Zsh

The Makefile will:

1. Download micromamba if conda/mamba is not found
2. Create a Python 3.10 environment with FFmpeg 4.4.2
3. Install NV Codec Headers for NVENC support
4. Build Decord from source with CUDA enabled
5. Install all pip requirements

Prerequisite: NVIDIA Container Toolkit must be installed.

# Build the image
docker build -t autoshorts .

# Run with GPU access
docker run --rm \
    --gpus all \
    -v $(pwd)/gameplay:/app/gameplay \
    -v $(pwd)/generated:/app/generated \
    --env-file .env \
    autoshorts

Note: The --gpus all flag is essential for NVENC and CUDA acceleration.

Copy .env.example to .env and configure:

See .env.example for the complete list with detailed descriptions.

1. Place source videos in the gameplay/ directory

2. Run the script:

3. Generated clips are saved to generated/

generated/
├── video_name scene-0.mp4          # Rendered short clip
├── video_name scene-0_sub.json     # Subtitle data
├── video_name scene-0.ffmpeg.log   # Render log
├── video_name scene-1.mp4
└── ...

pip install ruff
ruff check .

Tests mock GPU availability and can run in standard CI environments.

For faster iteration during development, you can skip expensive steps using these environment variables in your .env:

Example workflow for testing subtitles only:

# In .env
DEBUG_SKIP_ANALYSIS=1
DEBUG_SKIP_RENDER=1
DEBUG_RENDERED_CLIPS="generated/test_clip.mp4:action"

This project builds upon the excellent work of:

This project is licensed under the MIT License.