Adding a Submodule in Git

After you have created a git repository one step you may want to take is to add a submodule.  This is a way to include one git repository as a subdirectory within another repository. This allows you to combine multiple independent repositories into a single large project, providing you with a way to manage dependencies across several projects.

Adding a submodule

To add a submodule to an existing project, you can use the following `git submodule add` command:

 $ git submodule add <repository_url> <subdirectory>

Run in Warp

Where the <repository_url> is the URL of the repository you want to add and <subdirectory> is the optional target path of the submodule within the main project.

For example:

 $ git submodule add https://github.com/team/vendors.git vendors

Run in Warp

To verify that the submodule has been added to the repository you can use the git status command. This should show that a .gitmodules file has been created within the parent project along with the new directory.

You can also specify which branch of the remote repository you want to add as a submodule by using the -b <branch> flag. This will change the command to:

 $ git submodule add -b <branch_name> <repository_url> <subdirectory>

Run in Warp

Easily retrieve this command using Warp’s AI Command Search

If you’re using Warp as your terminal, you can easily retrieve this command using the Warp AI Command Search feature:

Entering git add submodule branch into the AI command search will prompt a git submodule add command that can then be quickly inserted into your shell by doing CMD+ENTER.

Tracking submodule changes

Tracking changes of certain submodules

In some cases you may want to add several submodules to your repository but only want to track the changes made in a few of them. In this case you can take advantage of the git submodule init command to specify which submodules you want to update when performing the git submodule update –remote command. 

For example, you have a repository that has three submodules: sub1, sub2 and sub3. You may only want to retrieve updates for sub1 and sub2 as you want sub3 to remain stable. For this you would use the command:

 $ git submodule init sub1 sub2

Run in Warp

This would mean that when you perform the git submodule update –remote command, it will only fetch remote changes for the sub1 and sub2[.inline-code] repositories.

Changing the remote URL for a submodule

An alternative use for this command is to change the remote repository URL for a submodule. This is because the git submodule init command creates a .git/config file which contains the remote URL of each submodule to update from. After performing the git submodule init command you can edit the .git/config file to change the URL of the submodule from which to fetch updates when performing the git submodule update –remote command.

Adding a branch to a submodule

A Git submodule works on a “no branch” approach, which means that each submodule uses a specific commit rather than tracking a specific branch. This can make working on the submodule difficult and tricky to manage as any new commits added to the submodule will often be lost when updating it in the future. To get around this you need to add a branch to the submodule to work on. This can be done using the commands:

 $ cd <submodule_directory>
$ git checkout <branch_name>

Run in Warp

When making any changes to this branch and then subsequently updating the submodule, you need to add the --merge or --rebase flag to ensure that your changes are not lost. This turns the update command into either:

 $ git submodule update –rebase
$ git submodule update –merge

Run in Warp

If you forget the --merge or --rebase flags, Git will just update the submodule to whatever is on the server and reset your project to a detached HEAD state. If this happens, you can go back into the directory and check out your branch again and merge or rebase origin/stable (or whatever remote branch you want), manually.

Issues with submodules

Because of their complexity, using submodules is often difficult when working within a git repository. This is complicated by two common operations:

Switching branches

Prior to Git version 2.13, switching branches in a repository containing submodules can be quite difficult. For example, if you create a new branch, add a submodule there, and then switch back to a branch without that submodule, you will still have that submodule as an untracked directory. You will then have to remove that directory and will need to run git submodule update –init to repopulate it when switching back to the branch.

Switching from subdirectory to submodules

If you have been tracking files in your project and want to move them into a submodule, then this can create issues with Git. For example, if you delete a subdirectory and then add a submodule with the same name then Git will not let you. You first need to unstage the original directory and then add the submodule. This can become further complicated when switching branches as you will need to force checkouts which can often lead to missing files in subdirectories or submodules.

Alternatives to submodules

Git submodules are useful as they allow you to use another project from within the project you are working on. However, they can often be difficult to manage. This means that there are several alternatives available to use:

Git subtree

git subtree lets you nest one repository inside another as a subdirectory and is another method for managing project dependencies. The benefits of using git subtree are that:

• The sub-projects code is available right after the clone of the super project is done.
• git subtree does not require new users of your project to know how submodules work.
• git subtree does not add new metadata files as git submodules do.
• Contents of the module can be modified without having a separate copy of the dependency somewhere else.
• The subdirectory created by the subtree can be committed to, branched, and merged with the project in any way you want.

But there are also drawbacks:

• You must learn about a new merge strategy (git subtree)
• Contributing code back to the sub-projects can still be complicated
• The responsibility of not mixing super and sub-project code in commits lies with the user

Git slave

Git slave is a “script for coordinated version control of large projects” which combines code from multiple independent repositories using Git. This means that it provides a wrapper around git that manages a tree of directories with multiple “sub” repositories in the directories so that you can control all “sub” repositories from the “main” repository itself. This is useful when you want to perform operations such as adding tags and branches and performing pushes and pulls to all repositories at the same time. The main drawback of this is it may be difficult to operate on a Windows machine in particular and it requires Perl. 

Nested repositories

If your software has a plugin architecture, with each plugin being its own sub-directory, it can often make sense to git-ignore these plugin directories and make each of them their own git repository. This way ensures that all the files form a single directory tree but are managed by different git repositories. The main downside to this however is if you specify any of the sub-repositories code in the main repository, other developers need to know their file relationship to run the main repository.