Skip to content

msxdan/repovault

BranchesTags

Repository files navigation

RepoVault

Automated git repository backup tool

Release Docker Platforms

Backs up git repositories on a schedule. Supports GitHub, GitLab, Bitbucket, and self-hosted servers.

Features

  • Configurable sync intervals (5m, 1h, 30s, etc.)
  • Automatic backups before syncing (preserves history on force pushes)
  • Configurable backup retention (keep last N backups per branch)
  • Catch-up syncs after downtime
  • Wildcard branch support (* syncs all branches)
  • Concurrency control to prevent memory spikes (configurable)
  • SSH, HTTP, and token authentication
  • Works with multiple accounts per provider
  • Environment variable support in config
  • Three log levels: verbose, normal, quiet
  • Docker support
  • Lightweight single binary

Quick Start

Docker (Recommended)

Supported Architectures:

  • linux/amd64 - x86-64 (Intel/AMD processors)
  • linux/arm64 - ARM 64-bit (AWS Graviton, Apple Silicon, Raspberry Pi 4/5)
  • linux/arm/v7 - ARM 32-bit (Raspberry Pi 3 and older)

Run using docker:

# Create your config
mkdir repovault && cd repovault && mkdir backup
nano config.yaml  # Create your config (see Configuration section or check config.example.yaml)

# Run with docker-compose
docker run -d \
  --name repovault \
  --restart unless-stopped \
  -v ./config.yaml:/config/config.yaml:ro \
  -v ./backup:/backup:rw \
  ghcr.io/msxdan/repovault:latest

Run using docker compose: docker-compose.yml:

services:
  repovault:
    image: ghcr.io/msxdan/repovault:latest
    container_name: repovault
    restart: unless-stopped
    environment:
      - GITHUB_TOKEN=${GITHUB_TOKEN}
      - SSH_KEY_PASSWORD=${SSH_KEY_PASSWORD}
    volumes:
      - ./config.yaml:/config/config.yaml:ro
      - ./ssh-keys:/ssh:ro # If using SSH
      - ./backup:/backup:rw
    deploy:
      resources:
        limits:
          memory: 2G  # Increase if syncing many large repos
          cpus: "2"
        reservations:
          memory: 512M

Building from source:

git clone https://github.com/msxdan/repovault.git
cd repovault/src
go build -ldflags "-s -w" -a -o repovault main.go

# create a multiline env file (if using auth)
echo "GITHUB_TOKEN=github_pat_11" > .env
echo "SSH_KEY_PASSWORD=MyUltraSecretPass" >> .env
export $(cat .env | xargs)

./repovault config.yaml /path/to/backup/directory

Configuration

Example config.yaml:

defaults:
  period: "30m"
  backup_retention: 5 # Keep last 5 backups per branch
  log_level: "normal" # Options: "verbose", "normal", "quiet"
  max_concurrent_repos: 3 # Max repos to sync simultaneously (prevents memory spikes)

repositories:
  # GitHub with token auth
  - name: "my-app"
    url: "https://github.com/username/my-app.git"
    branches: ["main", "develop"]
    path: "github/my-app"
    period: "5m" # Override default period
    auth:
      type: "http"
      username: "your-username"
      password: $GITHUB_TOKEN # Use environment variable

  # SSH authentication with all branches
  - name: "api-service"
    url: "git@github.com:username/api-service.git"
    branches: ["*"]
    path: "github/api-service"
    backup_retention: 10 # Override default retention
    log_level: "verbose" # Override log level for this repo
    auth:
      type: "ssh"
      ssh_key_path: "/ssh/id_rsa"
      ssh_key_password: $SSH_KEY_PASSWORD # Use env var if key is encrypted

  # Public repository
  - name: "open-source-lib"
    url: "https://github.com/open-source/library.git"
    branches: ["main"]
    path: "github/library"
    auth:
      type: "none"

Global Defaults

Field Description Default Example
defaults.period Default sync interval for all repos Required "30m", "1h"
defaults.backup_retention Number of backups to keep per branch 0 (disabled) 5
defaults.log_level Default log level "normal" "verbose", "normal", "quiet"
defaults.max_concurrent_repos Max repos to sync simultaneously (prevents OOM) 3 2, 5

Repository Options

Field Description Example
name Repository identifier "my-app"
url Git repository URL "https://github.com/user/repo.git"
branches Branches to sync, or ["*"] for all ["main", "develop"]
path Local path relative to backup directory (optional, defaults to name) "github/my-app"
period Sync interval (optional, overrides default) "5m", "1h", "24h"
backup_retention Backups to keep (optional, overrides default) 10
log_level Log level (optional, overrides default) "verbose", "normal", "quiet"
auth.type Authentication method "ssh", "http", "none"
auth.username Username for HTTP auth "your-username"
auth.password Password/token for HTTP auth "ghp_token123" or $GITHUB_TOKEN
auth.ssh_key_path Path to SSH private key "/ssh/id_rsa"
auth.ssh_key_password SSH key password (if encrypted) "password" or $SSH_PASSWORD

Environment Variables

Config values can reference environment variables by prefixing with $:

auth:
  username: "myuser"
  password: $GITHUB_TOKEN # Reads from GITHUB_TOKEN env var

Warning

Using * for branches can take a long time if there are many branches.

Note

Recommended minimum period is 5m.

Authentication

SSH

auth:
  type: "ssh"
  ssh_key_path: "/path/to/ssh/key"
  ssh_key_password: $SSH_KEY_PASSWORD # Only if key is encrypted

For Docker, mount your key:

volumes:
  - ./ssh-keys/id_rsa:/ssh/id_rsa:ro

HTTP (Tokens)

auth:
  type: "http"
  username: "your-username"
  password: $GITHUB_TOKEN
  • GitHub: Personal Access Token (ghp_...) with repo scope (or Contents: read-only for fine-grained tokens)
  • GitLab: Personal Access Token (glpat_...)
  • Bitbucket: App Password

Public Repos

auth:
  type: "none"

Automatic Backups

RepoVault automatically creates backups before syncing when the local branch differs from remote. This protects against data loss from force pushes.

How It Works

  1. Before syncing, RepoVault compares local and remote commits
  2. If they differ, it creates a backup reference: refs/backups/YYYY-MM-DD-HH-MM-SS/branch-name
  3. Old backups are automatically cleaned up based on backup_retention setting
  4. Backups are stored as git references (no extra disk space for commits already in the repo)

Configuration

defaults:
  backup_retention: 5 # Keep last 5 backups per branch

repositories:
  - name: "critical-repo"
    backup_retention: 20 # Keep more backups for critical repos

Set backup_retention: 0 to disable backups for a repository.

Viewing Backups

List all backups:

git show-ref | grep refs/backups

Restore from a backup:

git checkout refs/backups/2025-10-25-14-30-00/main

Log Levels

Control output verbosity globally or per-repository:

  • verbose: Shows all branch operations, progress updates, and backup details
  • normal: Shows sync progress every 20% or 10 branches (default)
  • quiet: Only shows errors and final summaries
defaults:
  log_level: "normal"

repositories:
  - name: "noisy-repo"
    log_level: "quiet" # Override for specific repo

Memory Usage & Concurrency

RepoVault limits the number of repositories that sync concurrently to prevent memory spikes. By default, only 3 repositories sync at once during initial startup.

Why This Matters

  • Initial clones are memory-intensive (can use 1-2GB per large repo)
  • Concurrent operations multiply memory usage
  • After initial sync, periodic updates use minimal memory

Adjusting Concurrency

defaults:
  max_concurrent_repos: 2  # Lower if you have limited memory
  # max_concurrent_repos: 5  # Higher if you have plenty of RAM

Memory Guidelines:

  • max_concurrent_repos: 2 → ~1.5GB RAM recommended
  • max_concurrent_repos: 3 → ~2GB RAM recommended (default)
  • max_concurrent_repos: 5 → ~3-4GB RAM recommended

Note: After initial sync completes, all repositories enter their periodic sync loops which use much less memory.

Missed Sync Recovery

RepoVault tracks sync times in <backup-dir>/.repovault-state.json. On startup, it checks for missed syncs and catches up automatically.

Example: If your last sync was 7 hours ago and your period is 3 hours, RepoVault will sync immediately on startup, then resume the normal schedule.

2025-01-10 23:40:29 INFO  github.com/username/my-app: Missed sync detected (last sync: 7h ago), performing catch-up sync

Example Output

2025-01-10 09:59:18 INFO  Starting RepoVault with 5 repositories (max 3 concurrent)
2025-01-10 09:59:18 INFO  github.com/msxdan/homelab-private: Cloning repository...
2025-01-10 09:59:18 INFO  github.com/msxdan/dotfiles: Cloning repository...
2025-01-10 09:59:18 INFO  github.com/msxdan/marvincloud.io: Cloning repository...
2025-01-10 09:59:18 INFO  github.com/msxdan/dotfiles_private: Cloning repository...
2025-01-10 09:59:18 INFO  github.com/msxdan/zet: Cloning repository...
2025-01-10 09:59:20 INFO  github.com/msxdan/dotfiles_private: Clone completed
2025-01-10 09:59:20 INFO  github.com/msxdan/dotfiles_private: Syncing 1 branches...
2025-01-10 09:59:20 INFO  github.com/msxdan/dotfiles: Clone completed
2025-01-10 09:59:20 INFO  github.com/msxdan/homelab-private: Clone completed
2025-01-10 09:59:20 INFO  github.com/msxdan/dotfiles_private: 1/1
2025-01-10 09:59:20 INFO  github.com/msxdan/dotfiles_private: Synced 1 branches (1.7s, next: 20:59:20)
2025-01-10 09:59:20 INFO  github.com/msxdan/dotfiles: Syncing 2 branches...
2025-01-10 09:59:20 INFO  github.com/msxdan/homelab-private: Syncing 2 branches...
2025-01-10 09:59:21 INFO  github.com/msxdan/marvincloud.io: Clone completed
2025-01-10 09:59:21 INFO  github.com/msxdan/marvincloud.io: Syncing 1 branches...
2025-01-10 09:59:21 INFO  github.com/msxdan/dotfiles: 2/2
2025-01-10 09:59:21 INFO  github.com/msxdan/dotfiles: Synced 2 branches (2.5s, next: 20:59:21)
2025-01-10 09:59:21 INFO  github.com/msxdan/homelab-private: 2/2
2025-01-10 09:59:21 INFO  github.com/msxdan/marvincloud.io: 1/1
2025-01-10 09:59:21 INFO  github.com/msxdan/marvincloud.io: Synced 1 branches (2.7s, next: 20:59:21)
2025-01-10 09:59:21 INFO  github.com/msxdan/homelab-private: Synced 2 branches (2.7s, next: 20:59:21)
2025-01-10 09:59:33 INFO  github.com/msxdan/zet: Cloning... (15s elapsed)
2025-01-10 09:59:41 INFO  github.com/msxdan/zet: Clone completed
2025-01-10 09:59:41 INFO  github.com/msxdan/zet: Syncing 1 branches...
2025-01-10 09:59:41 INFO  github.com/msxdan/zet: 1/1
2025-01-10 09:59:41 INFO  github.com/msxdan/zet: Synced 1 branches (23.4s, next: 20:59:41)

Troubleshooting

Authentication errors:

chmod 600 ./ssh-keys/id_rsa  # Fix SSH key permissions

Encrypted SSH key error: If you see "bcrypt_pbkdf" error, your SSH key is encrypted. Add ssh_key_password to your config.

"Working directory not clean" warnings: This is normal - RepoVault forces sync to match the remote exactly. Local changes are backed up before being overwritten.

Wildcard branches not working: Check that your token has permission to list and read all branches.

View logs:

docker compose logs -f repovault

License

MIT - see LICENSE file.

About

Simple to use git backup solution

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases 3

Release v0.3.0 Latest
Nov 10, 2025
+ 2 releases

Packages

 
 
 

Contributors

Languages