How to move a folder from one repo to another and keep its commit history

gistfile1.sh

# source: http://st-on-it.blogspot.com/2010/01/how-to-move-folders-between-git.html
# First of all you need to have a clean clone of the source repository so we didn't screw the things up.

git clone git://server.com/my-repo1.git

# After that you need to do some preparations on the source repository, nuking all the entries except the folder you need to move. Use the following command

git filter-branch --subdirectory-filter your_dir -- -- all

# This will nuke all the other entries and their history, creating a clean git repository that contains only data and history from the directory you need. If you need to move several folders, you have to collect them in a single directory using the git mv command.

# You also might need to move all your content into some directory so it didn't conflict with the new repository when you merge it. Use commands like that

mkdir new_directory/
git mv my_stuff new_directory/

# Once you've done commit your changes, but don't push!

git commit -m "Collected the data I need to move"

# This is all about the source repository preparations.
# Now go to your destination repository

cd ../my-repo2/

# And here is the trick. You need to connect your source repository as a remote using a local reference.

git remote add repo1 ../my-repo1/

# After that simply fetch the remote source, create a branch and merge it with the destination repository in usual way

git fetch repo1
git branch repo1 remotes/repo1/master

git merge repo1 --allow-unrelated-histories

# This is pretty much it, all your code and history were moved from one repository to another. All you need is to clean up a bit and push the changes to the server

git remote rm repo1
git branch -d repo1

git push origin master

# That's all. After that you can nuke the temporary source repository.

Thanks!

Very nice, but it fails with signed history's.
After that procedure, all gpg signatures are lost.

As an workaround:
After filter in the destination repo use this:
https://gist.github.com/qdequele/490e413f82ab16e640510c180bb99b5b

Thank you this worked perfectly for me.
I thought git filter-branch --subdirectory-filter your_dir -- -- all with the -- -- was a typo but it is correct (another piece of git magic)

Very useful article. I also want to move the folder to a specific location within the new repository.

Example:
old_repo/your_dir
new_repo/other_folder/your_dir

Any advice on how to move the contents of your_dir to the location in the new repository?

Can someone advice how to do the following
Move all files and directories of repo1 to repo2/test/ with preserving the history of repo1.

@itsmeabhijeet you can skip git filter-branch and create the test directory in repo1 before moving. Your example would look like this:

git clone git://server.com/my-repo1.git
mkdir test/
git mv everything test/
git commit -m "Collected the data I need to move"
// Continue original script

@MaikoKingma : Thanks for the response.
But in the target repo i.e repo2, i want the source repo to be moved inside a sub-directory. for example the end result should be repo2/test/repo1.
In this case should i try to execute the commands from repo2/test ?

when i try at repo2 ( cd repo2), all the files are fetched under repo2. Also when i move inside repo2/test/ , i get git errors.

Could you please help or provide full commands. My requirements is kind of similar like your earlier post. Thanks in Advance

Adding repo1 to a sub directory inside repo2 seems tricky. Please advice.
End result should be something like repo2/dir1/repo1. Thanks in advance.

Take a look at "git filter-repo". Most posts are many years old - this is the latest solution that works great, and is recommended by GIT.

Thanks, worked like a charm.

Can I keep using this procedure continuously? I mean, can I still develop in my-repo1 and periodically merge into my-repo2, or is this a one time only operation? And would that still be possible if other commits are added to my-repo2 between the merges?

@itsmeabhijeet you can skip git filter-branch and create the test directory in repo1 before moving. Your example would look like this:

git clone git://server.com/my-repo1.git
mkdir test/
git mv everything test/
git commit -m "Collected the data I need to move"
// Continue original script

Except that this essentially cuts off the history for the items you want to keep!

Hi,
I've tested this against a test repo and it worked, thank you, however I have a few questions;

1. In the script you create new_directory and move stuff into it. In my case this "stuff" needs to reside in the root, however, git merge repo1 --allow-unrelated-histories commits new_directory/my_stuff1, new_directory/my_stuff2 where it should be /my_stuff1 /my_stuff2. How can I make this happen?
2. I'm a newbie to all this, after completing all the steps in your script, do you delete everything in the folder and restore the cloned folder then delete the folders i've moved to the new repo?
3. I have multiple features, develop and master branches, do I have to repeat the whole process for each branch?

For example repeat your steps but changing the following based on the branch;

git clone git://server.com/my-repo1.git (against develop branch)
git branch repo1 remotes/repo1/master
git push origin develop


git clone git://server.com/my-repo1.git (against master branch)
git branch repo1 remotes/repo1/master
git push origin master

git clone git://server.com/my-repo1.git (against feature1 branch)
git branch repo1 remotes/repo1/master
git push origin features/feature1

feature2
feature3...

thank you very much for this ! you made my day :)

Awesome! Thank you, helped a lot!

works as expected. great stuff.

very helpful <3

Hi,
I've tested this against a test repo and it worked, thank you, however I have a few questions;

1. In the script you create new_directory and move stuff into it. In my case this "stuff" needs to reside in the root, however, git merge repo1 --allow-unrelated-histories commits new_directory/my_stuff1, new_directory/my_stuff2 where it should be /my_stuff1 /my_stuff2. How can I make this happen?
2. I'm a newbie to all this, after completing all the steps in your script, do you delete everything in the folder and restore the cloned folder then delete the folders i've moved to the new repo?
3. I have multiple features, develop and master branches, do I have to repeat the whole process for each branch?

For example repeat your steps but changing the following based on the branch;

git clone git://server.com/my-repo1.git (against develop branch)
git branch repo1 remotes/repo1/master
git push origin develop


git clone git://server.com/my-repo1.git (against master branch)
git branch repo1 remotes/repo1/master
git push origin master

git clone git://server.com/my-repo1.git (against feature1 branch)
git branch repo1 remotes/repo1/master
git push origin features/feature1

feature2
feature3...

Hi,
Would greatly appreciate if you could respond to my query.

Great article. +1 to @keithb95 regarding git filter-repo

You can use git filter-repo --path foo.txt --path foo/ --invert-paths to remove particular files or folders. And remember you may want to remove ones not currently in latest HEAD :-)

Hi,
I've tested this against a test repo and it worked, thank you, however I have a few questions;

1. In the script you create new_directory and move stuff into it. In my case this "stuff" needs to reside in the root, however, git merge repo1 --allow-unrelated-histories commits new_directory/my_stuff1, new_directory/my_stuff2 where it should be /my_stuff1 /my_stuff2. How can I make this happen?
2. I'm a newbie to all this, after completing all the steps in your script, do you delete everything in the folder and restore the cloned folder then delete the folders i've moved to the new repo?
3. I have multiple features, develop and master branches, do I have to repeat the whole process for each branch?

For example repeat your steps but changing the following based on the branch;

git clone git://server.com/my-repo1.git (against develop branch)
git branch repo1 remotes/repo1/master
git push origin develop


git clone git://server.com/my-repo1.git (against master branch)
git branch repo1 remotes/repo1/master
git push origin master

git clone git://server.com/my-repo1.git (against feature1 branch)
git branch repo1 remotes/repo1/master
git push origin features/feature1

feature2
feature3...

Hi,
Would greatly appreciate if you could respond to my query.

Can't you just move things back? With a simple

mv new_directory/my_stuff1 . 
git add
git commit -m "Moved to root"
git push origin master

Hi All,
i am trying to shift contents of a folder from source repo to target repo but when i am trying to execute merge command i am getting an error, it seems the files of the folder got deleted in head but in the new branch which i created it is present, but i have executed all steps in the same way:(, anyone could please help

ERROR on merging command "git merge $source_repo_name --allow-unrelated-histories" :
From ../testproject

• [new branch] master -> testproject/master
  Branch 'testproject' set up to track remote branch 'master' from 'testproject'.
  CONFLICT (rename/delete): testTenant deleted in HEAD and renamed to service/testService in testproject. Version testproject of service/testService left in tree.
  Automatic merge failed; fix conflicts and then commit the result.
  error: The branch 'testproject' is not fully merged.

My sh file:
#!/bin/bash

echo "Enter the source repo url: "
read source_repo_url
echo "Enter the source repo name: "
read source_repo_name
echo "Enter the source branch: "
read source_branch
echo "Enter the source folder: "
read source_folder
echo "Enter the target repo url: "
read target_repo_url
echo "Enter the target repo name: "
read target_repo_name

git clone $source_repo_url
git clone $target_repo_url
cd $source_repo_name
git filter-branch --subdirectory-filter $source_folder -- -- all
git reset --hard
git gc --aggressive
git prune
git clean -fd
mkdir $source_folder
git mv -k * ./$source_folder
git commit -m "Collected the folders We need to move"
cd ../$target_repo_name
git remote add $source_repo_name ../$source_repo_name/
git fetch $source_repo_name
git branch $source_repo_name remotes/$source_repo_name/master
git merge $source_repo_name --allow-unrelated-histories
git remote rm $source_repo_name
git branch -d $source_repo_name
git commit -m "Added source folder to target repo"
git push origin master

This process works perfectly fine if I am moving a folder from repo 1 to the ROOT of repo 2 (& I think this is the default behavior of the methodology) preserving the history. Could you please let me know how can I move the repo 1 folder to a sub directory of repo 2 (NOT ROOT) ===> For example repo1/folder1 has to be moved to repo2/folderABC/folder1.

Hey, works great! But is there a way to keep the tags? And maybe the branches too?

@KastenBrot Tags are assigned to specific commit hash. This solution will change all commit hashes although commit messages are still there. So I don't think you can keep tags.

Regarding keeping the branches, you check out a new non-default branch after switching to my-repo2 and do the merge from there.

@KastenBrot Tags are assigned to specific commit hash. This solution will change all commit hashes although commit messages are still there. So I don't think you can keep tags.

Regarding keeping the branches, you check out a new non-default branch after switching to my-repo2 and do the merge from there.

Yeah, thanks for the Snippet tho :) If anyone is interested, I used this tutorial as a workaround. Clone the repo and all branches you need, manually delete everything you don't want and change the remote. It keeps the branches and tags. Not the cleanest way but it works.

Thank you!

Works perfectly! Thanks.

Hi,

Iam following the same, but in the push getting following error.Kindly help

Total 4203 (delta 1720), reused 4202 (delta 1720), pack-reused 0
remote: Resolving deltas: 100% (1720/1720), done.
remote: Rejected by YACC
remote:
remote: (c).-.(c) (c).-.(c) (c).-.(c) (c).-.(c) (c).-.(c)
remote: / .. \ / .. \ / .. \ / .. \ / ..
remote: ( Y )/ ( Y )/ ( Y )/ ( Y )/ ( Y )/
remote: (.-/'-'-.)(.-/'-'-.)(.-/'-'-.)(.-/'-'-.)(.-/'-'-._)
remote: || E || || R || || R || || O || || R ||
remote: .' -' '._ _.' -' '. .' -' '._ _.' -' '. _.' -' '. remote: (.-./-'.-.)(.-./-.-.)(.-./-.-.)(.-./-'\.-.)(.-./-\.-.) remote: -' -' -' -' -' -' -' -' -' `-'
remote:
remote:
remote: Push rejected.
remote:
remote: refs/heads/master: 1f77a4445ec: expected committer email '[email protected]' but found '[email protected]'
remote:
remote: refs/heads/master: be3b4b47d9a: expected committer email '[email protected]' but found '[email protected]'

noticed an issue when running git merge repo1 --allow-unrelated-histories to a repository with an existing main/master branch.
Scenario: copying over a branch in repo1 to repo2 with full commit history, repo2 already has an existing main and develop branch.
Problem: When running the last step of merging featurebranch from repo1 to repo2 using git merge repo1 --allow-unrelated-histories, it fails to completely copy the code from featurebranch in repo1.

Solution: before creating featurebranch in repo2, I ran git checkout --orphan featurebranch and git rm -rf . in this order. Since there is nothing to reference before running git merge repo1 --allow-unrelated-histories it will completely copy featurebranch from repo1 to repo2 with no conflicts.

If there are no main/master branches then this solution works just fine, Thanks! @trongthanh

Is there any way to move the history under repo2/(target folder) ? because after migration it is available only under repo2 root folder ... The history of repo2/(target folder) shows only 1 commit : "Collected the data I need to move"