diff --git a/README.md b/README.md index a9c4d06..2d07385 100644 --- a/README.md +++ b/README.md @@ -1,38 +1,61 @@ -# ffmpeg home video tools +# FFmpeg Home Video Tools -This is a small colection of useful shell scripts that utilize ffmpeg and assist in creating an order in the usual modern format zoo of the home video world. This came as a result of a personal need and worked well to bring a bit of an order to the chaos of my personal home video collection. For now there is no unified solution - scripts can be run as needed with careful inspection of the results until the desired is reached. In the future might create a more universal and configurable ffmpeg wrapper/tool. +Welcome to the FFmpeg Home Video Tools repository, a curated collection of shell scripts designed to streamline and enhance your home video organization using FFmpeg. These scripts have been developed to address the challenges of managing a diverse range of video formats commonly found in personal collections. -### WARNING - use this at your own risk and carefully. This has so far been only tested to my own needs, so data loss is possible if you are not thinking what you are doing. I am not responsible for any lost data when using this. There might be some edge cases still, where everything goes horribly wrong. Please do tests on small batches before. -Main script: -## batch_convert_to_mp4.sh -Small shell script to identify and batch convert all files in called folder to high quality/low compression mp4 in unified fullHD resolution - useful for batch video standardization. +## Usage Warning -Make sure to create "converted" directory there before running script, as for now the default path assumes videos to be stored relative to the scripts in the video directory and converted ones are created in video/converted. Feel free to change it to fit your needs. +Please exercise caution and use these scripts at your own risk. While they have been tested to meet personal requirements, there is a potential risk of data loss if not used carefully. Ensure thorough testing on small batches before applying these tools to larger datasets. -There are three poossible ffmpeg actions in the main script: +## Getting Started --Vertically rotated video is upscaled to 1080p and black bars are padded to the sides. All encoded to mp4 with high quality/low compression +All Bash scripts are located in the "bash" directory. The primary entry point is the `process_videos.sh` script, which serves as the main interface for handling input and output parameters. --1080p video is reencoded to mp4 with high quality/low compression +```bash +sh ./process_videos.sh input_folder output_folder +``` --Lower res. video ir reencoded to 1080p upscaled mp4 with high quality/low compression. +## Main Script: `batch_convert_to_mp4.sh` -In all cases audio is reencoded to 320k aac. Video frame rate is forced to 25 fps. +This script identifies and batch converts files within a specified folder to high-quality, low-compression MP4 format with a unified Full HD resolution. The script performs the following actions based on the input file characteristics: -This script takes care of the most cases, however some extra cases remain. For now there are two helper scripts to assist: +- Vertically rotated videos are upscaled to 1080p with blurred bars padded to the sides, encoded to MP4 with high quality/low compression. +- 1080p videos are reencoded to MP4 with high quality/low compression. +- Lower resolution videos are reencoded to 1080p upscaled MP4 with high quality/low compression. -## pad_to_fullhd.sh -Goes over the converted files and checks if there are any that are not exactly 1920x1080. These are padded with black bars where needed. On some edge cases it does not work as intended for now. +Audio is consistently reencoded to 320k AAC, and the video frame rate is forced to 25 fps. -## crop_to_fullhd.sh -Does the same as the padding script, however it crops video where the resoltion is more than fullhd. Can take care of most cases where padding won't do. Will create an universal script in the future. +**Note:** Ensure a "converted" directory is created before running the script. -## add_missing_audio_tracks.sh -As not all video contain audio, however for concat all files need an audio track, this script adds 0 audio to such tracks, so that everythin works. +## Helper Scripts -## resample_all_audio.sh -In case you find that normal concat produces out of sync audio and video, try to run this. It resamples audio with aresample=async=1000, that should fix this. Creates resampled_ prefixed files. concat_videos.sh assumes that this step is used. If it is skipped, change file select configuration there where needed. +### `pad_to_fullhd.sh` -Next stage is concatenation: -## concat_videos.sh -This creates a concat_list.txt of all the converted mp4's and joins them together in concatenated mp4. This is ready for encoding in a lower bitrate if needed. +This script checks converted files for resolutions other than 1920x1080 and pads them with black bars as needed. Some edge cases may not work as intended. + +### `crop_to_fullhd.sh` + +Similar to the padding script, this script crops video where the resolution exceeds Full HD. A more universal script is planned for future releases. + +### `add_missing_audio_tracks.sh` + +For videos lacking audio tracks, this script adds a silent audio track to facilitate concatenation. + +### `resample_all_audio.sh` + +If audio and video sync issues arise during concatenation, running this script can help by resampling audio using `aresample=async=1000`. Adjust the file selection configuration in `concat_videos.sh` if this step is skipped. + +## Concatenation +```markdown +Script: `concat_videos.sh` + +This script generates a `concat_list.txt` of all converted MP4 files and merges them into a concatenated MP4. This file is ready for further encoding at a lower bitrate if required. + +## Master Script: `process_videos.sh` + +This master script orchestrates the sequential execution of the above scripts, ensuring a smooth processing flow. Provide input and output paths as parameters: + +```bash +sh ./process_videos.sh input_folder output_folder +``` + +Feel free to explore and adapt these tools to suit your specific needs. Your feedback and contributions are welcome! diff --git a/add_missing_audio_tracks.sh b/add_missing_audio_tracks.sh deleted file mode 100644 index aa9be0f..0000000 --- a/add_missing_audio_tracks.sh +++ /dev/null @@ -1,22 +0,0 @@ -#!/bin/bash -## -#Searches for files with missing audio and adds an empty track for these, so that concatenation does not loose all audio. -## -##CONFIG - START -converted_video_path="videos/converted/" -##CONFIG - END - -for file in ${converted_video_path}*.mp4 -do - -audio_track=$(ffprobe -i "$file" -show_streams 2>&1 | grep 'Stream #0:1') - -if [[ -z "${audio_track// }" ]]; then - -printf "Found file without an audio: ${file}}\n" -base_filename=$(basename $file) -ffmpeg -y -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=48000 -i $file \ - -shortest -c:v copy -c:a aac "${converted_video_path}with_empty_audio_${base_filename}" -hide_banner - -fi -done diff --git a/bash/process_videos.sh b/bash/process_videos.sh new file mode 100644 index 0000000..c1cda72 --- /dev/null +++ b/bash/process_videos.sh @@ -0,0 +1,27 @@ +#!/bin/bash +## +# Master script to process videos using various sub-scripts. +## + +## CONFIG - START +input_video_path="$1" +output_video_path="$2" +## CONFIG - END + +# Ensure input and output paths are provided +if [ -z "$input_video_path" ] || [ -z "$output_video_path" ]; then + echo "Usage: $0 " + exit 1 +fi + +steps_dir="./processing_steps" + +# Run each script sequentially in their respective subdirectories +$steps_dir/batch_convert_to_mp4.sh "$input_video_path" "$output_video_path" +$steps_dir/pad_to_fullhd.sh "$output_video_path/" +$steps_dir/crop_to_fullhd.sh "$output_video_path/" +$steps_dir/add_missing_audio_tracks.sh "$output_video_path/" +$steps_dir/resample_all_audio.sh "$output_video_path/" +$steps_dir/concat_videos.sh "$output_video_path/" + +echo "Processing completed." diff --git a/bash/processing_steps/add_missing_audio_tracks.sh b/bash/processing_steps/add_missing_audio_tracks.sh new file mode 100755 index 0000000..a5bac49 --- /dev/null +++ b/bash/processing_steps/add_missing_audio_tracks.sh @@ -0,0 +1,25 @@ +#!/bin/bash +## +# Adds zero audio tracks to videos without audio, making them ready for concatenation. +## + +## CONFIG - START +output_video_path=$1 +## CONFIG - END + +# Ensure output path is provided +if [ -z "$output_video_path" ]; then + echo "Usage: $0 " + exit 1 +fi + +# Iterate over each file in the specified directory +for file in "${output_video_path}"*.mp4; do + audio_track=$(ffprobe -i "$file" -show_streams 2>&1 | grep 'Stream #0:1') + + if [[ -z "${audio_track// }" ]]; then + printf "Found file without audio: ${file}\n" + base_filename=$(basename "$file") + ffmpeg -y -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=48000 -i "$file" -shortest -c:v copy -c:a aac "${output_video_path}with_empty_audio_${base_filename}" -hide_banner + fi +done diff --git a/bash/processing_steps/batch_convert_to_mp4.sh b/bash/processing_steps/batch_convert_to_mp4.sh new file mode 100755 index 0000000..3ef16f0 --- /dev/null +++ b/bash/processing_steps/batch_convert_to_mp4.sh @@ -0,0 +1,46 @@ +#!/bin/bash +## +# Small shell script to identify and batch convert all files in the called folder +# to high-quality/low-compression mp4 in unified fullHD resolution and 25fps. +# Useful for batch video standardization. +## + +## CONFIG - START +input_video_path=$1 +output_video_path=$2 +converted_video_path="${output_video_path}/preprocessed/" +target_fps=25 +## CONFIG - END + +# Ensure input path is provided +if [ -z "$input_video_path" ] || [ -z "$output_video_path" ]; then + echo "Usage: $0 " + exit 1 +fi + +# Create "preprocessed" directory if it doesn't exist +mkdir -p "$converted_video_path" + +# Iterate over each file in the specified directory +for file in "$input_video_path/"*.*; do + # Check if there are matching files + if [ -e "$file" ]; then + base_filename=$(basename "$file") + + # Get input video frame rate + input_fps=$(ffprobe -v error -select_streams v:0 -show_entries stream=r_frame_rate -of default=nw=1:nk=1 "$file") + input_fps=$(printf "%.0f" $(echo "$input_fps" | bc -l)) # Convert to integer + + # Determine ffmpeg action based on video properties + if ffprobe -v error -select_streams v:0 -show_entries stream_tags=rotate -of default=nw=1:nk=1 "$file" | grep -q '^90'; then + # Vertically rotated video is upscaled to 1080p with black bars padded to the sides + ffmpeg -i "$file" -vf "scale=w=1080:h=1920:force_original_aspect_ratio=1,pad=1080:1920:(ow-iw)/2:(oh-ih)/2" -c:v libx264 -preset veryfast -crf 18 -c:a aac -b:a 320k -r $target_fps "${converted_video_path}${base_filename}.mp4" -hide_banner + elif [ "$input_fps" -eq "$target_fps" ]; then + # Video already at target frame rate, reencode to mp4 with high quality/low compression + ffmpeg -i "$file" -c:v libx264 -preset veryfast -crf 18 -c:a aac -b:a 320k -r $target_fps "${converted_video_path}${base_filename}.mp4" -hide_banner + else + # Lower-res video is reencoded to 1080p upscaled mp4 with high quality/low compression and converted to 25fps + ffmpeg -i "$file" -vf "scale=1920:1080" -c:v libx264 -preset veryfast -crf 18 -c:a aac -b:a 320k -r $target_fps "${converted_video_path}${base_filename}.mp4" -hide_banner + fi + fi +done diff --git a/bash/processing_steps/concat_videos.sh b/bash/processing_steps/concat_videos.sh new file mode 100755 index 0000000..5c5ac84 --- /dev/null +++ b/bash/processing_steps/concat_videos.sh @@ -0,0 +1,44 @@ +#!/bin/bash +## +# Creates a concat_list.txt of all the converted mp4's and joins them together in a concatenated mp4. +# This is ready for encoding in a lower bitrate if needed. +## + +## CONFIG - START +output_video_path=$1 +## CONFIG - END + +# Ensure output path is provided +if [ -z "$output_video_path" ]; then + echo "Usage: $0 " + exit 1 +fi + +# Create a temporary text file containing a list of mp4 files to join +for file in "${output_video_path}"*.mp4; do + base_filename=$(basename "$file") + echo "file '$base_filename'" >> "${output_video_path}concat_list.txt" +done + +# Check if there are files to concatenate +if [ -s "${output_video_path}concat_list.txt" ]; then + # Use ffmpeg to concatenate the list of files and create the final joined video + current_datetime=$(date "+%Y%m%d_%H%M%S") + output_vod_name="processed_vod_${current_datetime}.mp4" + + ffmpeg -f concat -safe 0 -i "${output_video_path}concat_list.txt" -c copy "${output_video_path}${output_vod_name}" + + + # Check if ffmpeg command succeeded + if [ $? -eq 0 ]; then + # Remove files with the correct prefix + rm "${output_video_path}resampled_"*.mp4 + rm -Rf "${output_video_path}preprocessed" + rm "${output_video_path}concat_list.txt" + echo "Processing completed." + else + echo "Error: ffmpeg command failed." + fi +else + echo "No files to concatenate." +fi diff --git a/bash/processing_steps/crop_to_fullhd.sh b/bash/processing_steps/crop_to_fullhd.sh new file mode 100755 index 0000000..dbf5b77 --- /dev/null +++ b/bash/processing_steps/crop_to_fullhd.sh @@ -0,0 +1,29 @@ +#!/bin/bash +## +# Goes over the converted files and checks if there are any that have a resolution +# more than fullHD. These are cropped where needed. +## + +## CONFIG - START +output_video_path=$1 +## CONFIG - END + +# Ensure output path is provided +if [ -z "$output_video_path" ]; then + echo "Usage: $0 " + exit 1 +fi + +# Iterate over each file in the specified directory +for file in "${output_video_path}preprocessed/"*.*; do + eval $(ffprobe -v error -of flat=s=_ -select_streams v:0 -show_entries stream=height,width "$file") + size=${streams_stream_0_width}x${streams_stream_0_height}; + + if [ "${size}" != "1920x1080" ]; then + printf "Found non-standard aspect ratio: ${file}_${size}\n" + base_filename=$(basename "$file") + ffmpeg -i "$file" -vf "crop=min(iw\,ih*(16/9)):ow/(16/9)" -c:v libx264 -preset veryfast -crf 18 -c:a aac -b:a 320k "${output_video_path}preprocessed/cropped_${base_filename}" -hide_banner; + # Delete the original file + rm "$file" + fi +done diff --git a/bash/processing_steps/pad_to_fullhd.sh b/bash/processing_steps/pad_to_fullhd.sh new file mode 100755 index 0000000..0fe0dad --- /dev/null +++ b/bash/processing_steps/pad_to_fullhd.sh @@ -0,0 +1,32 @@ +#!/bin/bash +## +# Goes over the converted files and checks if there are any that are not exactly 1920x1080. +# These are padded with a more blurred background video. +## + +## CONFIG - START +output_video_path=$1 +## CONFIG - END + +# Ensure output path is provided +if [ -z "$output_video_path" ]; then + echo "Usage: $0 " + exit 1 +fi + +# Iterate over each file in the specified directory +for file in "${output_video_path}preprocessed/"*.*; do + eval $(ffprobe -v error -of flat=s=_ -select_streams v:0 -show_entries stream=height,width "$file") + size=${streams_stream_0_width}x${streams_stream_0_height}; + + if [ "${size}" != "1920x1080" ]; then + printf "Found non-standard aspect ratio: ${file}_${size}\n" + base_filename=$(basename "$file") + + # Overlay the original video on top of a more blurred background + ffmpeg -i "$file" -filter_complex "[0]scale=1920*2:1080*2,boxblur=luma_radius=min(h\,w)/20:luma_power=1:chroma_radius=min(cw\,ch)/20:chroma_power=1[bg];[0]scale=-1:1080[ov];[bg][ov]overlay=(W-w)/2:(H-h)/2:format=yuv444,crop=w=1920:h=1080" -c:v libx264 -preset veryfast -crf 23 -c:a aac -b:a 320k -movflags +faststart "${output_video_path}preprocessed/padded_${base_filename}" -hide_banner; + + # Delete the original file + rm "$file" + fi +done diff --git a/bash/processing_steps/resample_all_audio.sh b/bash/processing_steps/resample_all_audio.sh new file mode 100755 index 0000000..274d411 --- /dev/null +++ b/bash/processing_steps/resample_all_audio.sh @@ -0,0 +1,22 @@ +#!/bin/bash +## +# Resamples audio to 48000 stereo and adds compensation where needed. +# Useful if normal concat produces out-of-sync audio and video. +## + +## CONFIG - START +output_video_path=$1 +## CONFIG - END + +# Ensure output path is provided +if [ -z "$output_video_path" ]; then + echo "Usage: $0 " + exit 1 +fi + +# Iterate over each .mp4 file in the specified directory +for file in "${output_video_path}preprocessed/"*.mp4; do + base_filename=$(basename "$file") + ffmpeg -i "$file" -c:v copy -c:a aac -ar 48000 -ac 2 -af "aresample=async=1000" "${output_video_path}resampled_${base_filename}" -hide_banner +done +rm -Rf ${output_video_path}preprocessed \ No newline at end of file diff --git a/batch_convert_to_mp4.sh b/batch_convert_to_mp4.sh deleted file mode 100644 index 8200509..0000000 --- a/batch_convert_to_mp4.sh +++ /dev/null @@ -1,37 +0,0 @@ -#!/bin/bash - -## -# Main transcode script - transforms the zoo to mezzanine mp4's -## -##CONFIG - START -#Path to zoo videos -input_video_path="videos/" -#Path to videos to check -output_video_path="videos/converted/" -##CONFIG - END - -for file in ${input_video_path}*.* -do -currentDate=$(date +%s%3N) - -eval $(ffprobe -v error -of flat=s=_ -select_streams v:0 -show_entries stream=height,width "$file") -size=${streams_stream_0_width}x${streams_stream_0_height}; - -rotate=$(ffprobe -loglevel error -select_streams v:0 -show_entries stream_tags=rotate -of default=nw=1:nk=1 "$file" -) - -#logic for portrait videos - adds black bars. Upscales to fullHD. -if [ "${rotate}" == "90" ]; then - -ffmpeg -i "$file" -r 25 -f mp4 -vf "scale=w=1920:h=1080:force_original_aspect_ratio=1,pad=1920:1080:(ow-iw)/2:(oh-ih)/2" -c:v libx264 -preset veryfast -crf 18 -c:a aac -b:a 320k "${output_video_path}${currentDate}.mp4" -hide_banner; - -#Does no resizing for fullHD, just conversion. -elif [ "${size}" == "1920x1080" ]; then - -ffmpeg -i "$file" -r 25 -f mp4 -c:v libx264 -preset veryfast -crf 18 -c:a aac -b:a 320k "${output_video_path}${currentDate}.mp4" -hide_banner; -#For all other cases upscales before conversion. -else -ffmpeg -i "$file" -r 25 -f mp4 -vf "scale=-1:1080:flags=lanczos" -c:v libx264 -preset veryfast -crf 18 -c:a aac -b:a 320k "${output_video_path}${currentDate}.mp4" -hide_banner; - -fi -done diff --git a/concat_videos.sh b/concat_videos.sh deleted file mode 100644 index 91c1799..0000000 --- a/concat_videos.sh +++ /dev/null @@ -1,16 +0,0 @@ -#!/bin/bash - -## -# Generates a tmp text file of all mp4 in the converted directory and outputs merged video to indicated path. -## -##CONFIG - START -#Base converted video path -converted_video_path="videos/converted/" -#Path where to store generated list of files to join. -concat_file="concat_list.txt" -#Output path for the final joined video. -output_video_path="${converted_video_path}concat_video.mp4" -##CONFIG - END - -for file in ${converted_video_path}resampled_*.mp4; do echo "file '$file'" >> $concat_file; done -ffmpeg -f concat -safe 0 -i $concat_file -c copy $output_video_path diff --git a/crop_to_fullhd.sh b/crop_to_fullhd.sh deleted file mode 100644 index 0cd6559..0000000 --- a/crop_to_fullhd.sh +++ /dev/null @@ -1,23 +0,0 @@ -#!/bin/bash - -## -# Goes over transcoded files to check for any non-standart not quite fullHD aspect ratios. For these files, sides are cropped a new cropped_ file version is created. -## -##CONFIG - START -#Path to videos to check -output_video_path="videos/converted/" -##CONFIG - END - -for file in ${output_video_path}*.* -do - -eval $(ffprobe -v error -of flat=s=_ -select_streams v:0 -show_entries stream=height,width "$file") -size=${streams_stream_0_width}x${streams_stream_0_height}; - -if [ "${size}" != "1920x1080" ]; then -printf "Found non-standart aspect ratio: ${file}_${size}\n" -base_filename=$(basename $file) -ffmpeg -i "$file" -r 25 -f mp4 -vf "scale=iw*sar:ih , crop=min(iw\,ih*(16/9)):ow/(16/9)" -aspect 16:9 -c:v libx264 -preset veryfast -crf 18 -c:a aac -b:a 320k "${output_video_path}cropped_${base_filename}" -hide_banner; -fi - -done diff --git a/pad_to_fullhd.sh b/pad_to_fullhd.sh deleted file mode 100644 index ea2ecc3..0000000 --- a/pad_to_fullhd.sh +++ /dev/null @@ -1,23 +0,0 @@ -#!/bin/bash - -## -# Goes over transcoded files to check for any non-standart not quite fullHD aspect ratios. For these files, padding is added and a new padded_ file version is created. -## -##CONFIG - START -#Path to videos to check -output_video_path="videos/converted/" -##CONFIG - END - -for file in ${output_video_path}*.* -do - -eval $(ffprobe -v error -of flat=s=_ -select_streams v:0 -show_entries stream=height,width "$file") -size=${streams_stream_0_width}x${streams_stream_0_height}; - -if [ "${size}" != "1920x1080" ]; then -printf "Found non-standart aspect ratio: ${file}_${size}\n" -base_filename=$(basename $file) -ffmpeg -i "$file" -r 25 -f mp4 -vf "scale=iw*sar:ih , pad=max(iw\,ih*(16/9)):ow/(16/9):(ow-iw)/2:(oh-ih)/2" -aspect 16:9 -c:v libx264 -preset veryfast -crf 18 -c:a aac -b:a 320k "${output_video_path}padded_${base_filename}" -hide_banner; -fi - -done diff --git a/resample_all_audio.sh b/resample_all_audio.sh deleted file mode 100644 index 2197171..0000000 --- a/resample_all_audio.sh +++ /dev/null @@ -1,16 +0,0 @@ -#!/bin/bash -## -#Resamples audio to 48000 stereo and adds compensation where needed. Useful if on normal concat audio goes out of sync. -## -##CONFIG - START -converted_video_path="videos/converted/" -##CONFIG - END - -for file in ${converted_video_path}*.mp4 -do - -base_filename=$(basename $file) - -ffmpeg -i $file -c:v copy -c:a aac -ar 48000 -ac 2 -af "aresample=async=1000" "${converted_video_path}resampled_${base_filename}" -hide_banner - -done