Git Subdirectory Filter

git-subdir-filter is a command line tool which extracts a given sub directory from a source git repository into a separate repository. In other words, given a source respository, this tool takes a sub/directory/ argument and posts the contents from the root of sub/directory to a secondary repository.

Use Cases

There are two main use cases.

• Moving a snapshot of a subdirectory into it's own git repository
• Keeping a target git repository in sync with a source's subdirectory as a read-only shadow clone

Creating a new git repository allows for sub-project isolation where as tracking upstream changes could allow project owners to distribute independent parts of a larger project.

Motivation

The motivation for developing this tool was to solve a business use case, after not finding and adequite tool for the job. This led to the ideation of a more performant version of git filter-branch --subdirectory-filter and to address some its shortcomings.

Thus, the development goals of this project became:

• Implement a performant subdirectory filter command
• Allow for subsequent commands from the same source directory to resume filtering for tracking active development.

Command Features

• Auto-scaling work threads based on CPU count (configurable with -work-threads)
• Auto-resume filtering based on last-filtered commit
• Uses a single working directory
• Space saving - re-uses blob objects in working directory
• Force-update of target repository

Build Requirements

• libgit.v27

Usage

Requirements

The following is a list of requirements for running git-subdir-filter

• A source git repository branch with a desired sub/directory
• A target repository (can be local)
• rsa keys must be in the standard $USER/.ssh/ location
• SSH_PASSKEY must be set if your private key needs unlocking

Snapshot a sub directory

This is basically run-once. You should run the command and then you have a brand new target git repository to develop on.

Note It may be possible to convert the source sub/directory into a git module, or git external repository such that the source repository loses ownership of the sub/directory and ends up tracking the new target.

Track a sub directory

This mode is the original motivation of the tool. Using this mode assumes the source sub/directory is in active development. The target becomes a "clone-only" repository, and may be distributed.

In this model, the source repository retains ownership of the sub/directory, and any development, from the git source OR a target clone, MUST upstream git commits to the source repository for it to flow down into the new target.

The command should run periodically, ideally scheduled with a chron job, such that the tracking repository (target) does not drift.

Notice

The target repository branch MUST NOT be in active development for continuous filtering, as this tool effectively performs a git push --force to the target branch.

Example command

./git-subdir-filter -source-repo=git@github.com:missionsix/source.git
    -source-branch=master -target-repo=git@gitlab.com:missionsix/subdir.git
    -filter-dir=subdir

In this example subdir in the repository github.com:missionsix/source.git is to be extracted and pushed to the target repository github.com:missionsix/subdir.git.