Basic Merging

Now you and Sally are working on parallel branches of the project: you're working on a private branch, and Sally is working on the trunk, or main line of development.

For projects that have a large number of contributors, it's common for most people to have working copies of the trunk. Whenever someone needs to make a long-running change that is likely to disrupt the trunk, a standard procedure is to create a private branch and commit changes there until all the work is complete.

So, the good news is that you and Sally aren't interfering with each other. The bad news is that it's very easy to drift too far apart. Remember that one of the problems with the crawl in a hole strategy is that by the time you're finished with your branch, it may be near-impossible to merge your changes back into the trunk without a huge number of conflicts.

Instead, you and Sally might continue to share changes as you work. It's up to you to decide which changes are worth sharing; Subversion gives you the ability to selectively copy changes between branches. And when you're completely finished with your branch, your entire set of branch changes can be copied back into the trunk. In Subversion terminology, the general act of replicating changes from one branch to another is called merging, and it is performed using various invocations of the svn merge command.

In the examples that follow, we're assuming that both your Subversion client and server are running Subversion 1.5 (or later). If either client or server is older than version 1.5, things are more complicated: the system won't track changes automatically, and you'll have to use painful manual methods to achieve similar results. That is, you'll always need to use the detailed merge syntax to specify specific ranges of revisions to replicate (see the section called “Merge Syntax: Full Disclosure” later in this chapter), and take special care to keep track of what's already been merged and what hasn't. For this reason, we strongly recommend that you make sure your client and server are at least at version 1.5.

Changesets

Before we proceed further, we should warn you that there's going to be a lot of discussion of changes in the pages ahead. A lot of people experienced with version control systems use the terms change and changeset interchangeably, and we should clarify what Subversion understands as a changeset.

Everyone seems to have a slightly different definition of changeset, or at least a different expectation of what it means for a version control system to have one. For our purposes, let's say that a changeset is just a collection of changes with a unique name. The changes might include textual edits to file contents, modifications to tree structure, or tweaks to metadata. In more common speak, a changeset is just a patch with a name you can refer to.

In Subversion, a global revision number N names a tree in the repository: it's the way the repository looked after the Nth commit. It's also the name of an implicit changeset: if you compare tree N with tree N−1, you can derive the exact patch that was committed. For this reason, it's easy to think of revision N as not just a tree, but a changeset as well. If you use an issue tracker to manage bugs, you can use the revision numbers to refer to particular patches that fix bugs—for example, this issue was fixed by r9238. Somebody can then run svn log -r 9238 to read about the exact changeset that fixed the bug, and run svn diff -c 9238 to see the patch itself. And (as you'll see shortly) Subversion's svn merge command is able to use revision numbers. You can merge specific changesets from one branch to another by naming them in the merge arguments: passing -c 9238 to svn merge would merge changeset r9238 into your working copy.

Keeping a Branch in Sync

Continuing with our running example, let's suppose that a week has passed since you started working on your private branch. Your new feature isn't finished yet, but at the same time you know that other people on your team have continued to make important changes in the project's /trunk. It's in your best interest to replicate those changes to your own branch, just to make sure they mesh well with your changes. In fact, this is a best practice: frequently keeping your branch in sync with the main development line helps prevent surprise conflicts when it comes time for you to fold your changes back into the trunk.

Subversion is aware of the history of your branch and knows when it divided away from the trunk. To replicate the latest, greatest trunk changes to your branch, first make sure your working copy of the branch is clean—that it has no local modifications reported by svn status. Then simply run:

$ pwd
/home/user/my-calc-branch

$ svn merge ^/trunk
--- Merging r345 through r356 into '.':
U    button.c
U    integer.c

This basic syntax—svn merge URL—tells Subversion to merge all recent changes from the URL to the current working directory (which is typically the root of your working copy). Also notice that we're using the caret (^) syntax[21] to avoid having to type out the entire /trunk URL.

After running the prior example, your branch working copy now contains new local modifications, and these edits are duplications of all of the changes that have happened on the trunk since you first created your branch:

$ svn status
 M     .
M      button.c
M      integer.c

At this point, the wise thing to do is look at the changes carefully with svn diff, and then build and test your branch. Notice that the current working directory (.) has also been modified; the svn diff will show that its svn:mergeinfo property has been either created or modified. This is important merge-related metadata that you should not touch, since it will be needed by future svn merge commands. (We'll learn more about this metadata later in the chapter.)

After performing the merge, you might also need to resolve some conflicts (just as you do with svn update) or possibly make some small edits to get things working properly. (Remember, just because there are no syntactic conflicts doesn't mean there aren't any semantic conflicts!) If you encounter serious problems, you can always abort the local changes by running svn revert . -R (which will undo all local modifications) and start a long what's going on? discussion with your collaborators. If things look good, however, you can submit these changes into the repository:

$ svn commit -m "Merged latest trunk changes to my-calc-branch."
Sending        .
Sending        button.c
Sending        integer.c
Transmitting file data ..
Committed revision 357.

At this point, your private branch is now in sync with the trunk, so you can rest easier knowing that as you continue to work in isolation, you're not drifting too far away from what everyone else is doing.

Suppose that another week has passed. You've committed more changes to your branch, and your comrades have continued to improve the trunk as well. Once again, you'd like to replicate the latest trunk changes to your branch and bring yourself in sync. Just run the same merge command again!

$ svn merge ^/trunk
--- Merging r357 through r380 into '.':
U    integer.c
U    Makefile
A    README

Subversion knows which trunk changes you've already replicated to your branch, so it carefully replicates only those changes you don't yet have. Once again, you'll have to build, test, and svn commit the local modifications to your branch.

Reintegrating a Branch

What happens when you finally finish your work, though? Your new feature is done, and you're ready to merge your branch changes back to the trunk (so your team can enjoy the bounty of your labor). The process is simple. First, bring your branch in sync with the trunk again, just as you've been doing all along:

$ svn merge ^/trunk
--- Merging r381 through r385 into '.':
U    button.c
U    README

$ # build, test, ...

$ svn commit -m "Final merge of trunk changes to my-calc-branch."
Sending        .
Sending        button.c
Sending        README
Transmitting file data ..
Committed revision 390.

Now, you use svn merge to replicate your branch changes back into the trunk. You'll need an up-to-date working copy of /trunk. You can do this by either doing an svn checkout, dredging up an old trunk working copy from somewhere on your disk, or using svn switch (see the section called “Traversing Branches”.) However you get a trunk working copy, remember that it's a best practice to do your merge into a working copy that has no local edits and has been recently updated (i.e., is not a mixture of local revisions). If your working copy isn't clean in these ways, you can run into some unnecessary conflict-related headaches and svn merge will likely return an error.

Once you have a clean working copy of the trunk, you're ready to merge your branch back into it:

$ pwd
/home/user/calc-trunk

$ svn update  # (make sure the working copy is up to date)
At revision 390.

$ svn merge --reintegrate ^/branches/my-calc-branch
--- Merging differences between repository URLs into '.':
U    button.c
U    integer.c
U    Makefile
 U   .

$ # build, test, verify, ...

$ svn commit -m "Merge my-calc-branch back into trunk!"
Sending        .
Sending        button.c
Sending        integer.c
Sending        Makefile
Transmitting file data ..
Committed revision 391.

Congratulations, your branch has now been remerged back into the main line of development. Notice our use of the --reintegrate option this time around. The option is critical for reintegrating changes from a branch back into its original line of development—don't forget it! It's needed because this sort of merge back is a different sort of work than what you've been doing up until now. Previously, we had been asking svn merge to grab the next set of changes from one line of development (the trunk) and duplicate them to another (your branch). This is fairly straightforward, and each time Subversion knows how to pick up where it left off. In our prior examples, you can see that first it merges the ranges 345:356 from trunk to branch; later on, it continues by merging the next contiguously available range, 356:380. When doing the final sync, it merges the range 380:385.

When merging your branch back to the trunk, however, the underlying mathematics is quite different. Your feature branch is now a mishmosh of both duplicated trunk changes and private branch changes, so there's no simple contiguous range of revisions to copy over. By specifying the --reintegrate option, you're asking Subversion to carefully replicate only those changes unique to your branch. (And in fact, it does this by comparing the latest trunk tree with the latest branch tree: the resulting difference is exactly your branch changes!)

Now that your private branch is merged to trunk, you may wish to remove it from the repository:

$ svn delete ^/branches/my-calc-branch \
      -m "Remove my-calc-branch."
Committed revision 392.

But wait! Isn't the history of that branch valuable? What if somebody wants to audit the evolution of your feature someday and look at all of your branch changes? No need to worry. Remember that even though your branch is no longer visible in the /branches directory, its existence is still an immutable part of the repository's history. A simple svn log command on the /branches URL will show the entire history of your branch. Your branch can even be resurrected at some point, should you desire (see the section called “Resurrecting Deleted Items”).

In Subversion 1.5, once a --reintegrate merge is done from branch to trunk, the branch is no longer usable for further work. It's not able to correctly absorb new trunk changes, nor can it be properly reintegrated to trunk again. For this reason, if you want to keep working on your feature branch, we recommend destroying it and then re-creating it from the trunk:

$ svn delete http://svn.example.com/repos/calc/branches/my-calc-branch \
      -m "Remove my-calc-branch."
Committed revision 392.

$ svn copy http://svn.example.com/repos/calc/trunk \
           http://svn.example.com/repos/calc/branches/new-branch
      -m "Create a new branch from trunk."
Committed revision 393.

$ cd my-calc-branch

$ svn switch ^/branches/new-branch
Updated to revision 393.

The final command in the prior example—svn switch—is a way of updating an existing working copy to reflect a different repository directory. We'll discuss this more in the section called “Traversing Branches”.

There is another way of making the branch usable again after reintegration, without deleting the branch. See the section called “Keeping a reintegrated branch alive”.

Mergeinfo and Previews

The basic mechanism Subversion uses to track changesets—that is, which changes have been merged to which branches—is by recording data in properties. Specifically, merge data is tracked in the svn:mergeinfo property attached to files and directories. (If you're not familiar with Subversion properties, now is the time to skim the section called “Properties”.)

You can examine the property, just like any other:

$ cd my-calc-branch
$ svn propget svn:mergeinfo .
/trunk:341-390

It is not recommended that you change the value of this property yourself, unless you really know what you're doing. This property is automatically maintained by Subversion whenever you run svn merge. Its value indicates which changes (at a given path) have been replicated into the directory in question. In this case, the path is /trunk and the directory which has received the specific changes is /branches/my-calc-branch.

There's also a subcommand, svn mergeinfo, which can be helpful in seeing not only which changesets a directory has absorbed, but also which changesets it's still eligible to receive. This gives a sort of preview of the next set of changes that svn merge will replicate to your branch.

$ cd my-calc-branch

# Which changes have already been merged from trunk to branch?
$ svn mergeinfo ^/trunk
r341
r342
r343
…
r388
r389
r390

# Which changes are still eligible to merge from trunk to branch?
$ svn mergeinfo ^/trunk --show-revs eligible
r391
r392
r393
r394
r395

The svn mergeinfo command requires a source URL (where the changes would be coming from), and takes an optional target URL (where the changes would be merged to). If no target URL is given, it assumes that the current working directory is the target. In the prior example, because we're querying our branch working copy, the command assumes we're interested in receiving changes to /branches/mybranch from the specified trunk URL.

Another way to get a more precise preview of a merge operation is to use the --dry-run option:

$ svn merge ^/trunk --dry-run
U    integer.c

$ svn status
#  nothing printed, working copy is still unchanged.

The --dry-run option doesn't actually apply any local changes to the working copy. It shows only status codes that would be printed in a real merge. It's useful for getting a high-level preview of the potential merge, for those times when running svn diff gives too much detail.

[Tip] Tip

After performing a merge operation, but before committing the results of the merge, you can use svn diff --depth=empty /path/to/merge/target to see only the changes to the immediate target of your merge. If your merge target was a directory, only property differences will be displayed. This is a handy way to see the changes to the svn:mergeinfo property recorded by the merge operation, which will remind you about what you've just merged.

Of course, the best way to preview a merge operation is to just do it! Remember, running svn merge isn't an inherently risky thing (unless you've made local modifications to your working copy—but we've already stressed that you shouldn't be merging into such an environment). If you don't like the results of the merge, simply run svn revert . -R to revert the changes from your working copy and retry the command with different options. The merge isn't final until you actually svn commit the results.

[Tip] Tip

While it's perfectly fine to experiment with merges by running svn merge and svn revert over and over, you may run into some annoying (but easily bypassed) roadblocks. For example, if the merge operation adds a new file (i.e., schedules it for addition), svn revert won't actually remove the file; it simply unschedules the addition. You're left with an unversioned file. If you then attempt to run the merge again, you may get conflicts due to the unversioned file being in the way. Solution? After performing a revert, be sure to clean up the working copy and remove unversioned files and directories. The output of svn status should be as clean as possible, ideally showing no output.

Undoing Changes

An extremely common use for svn merge is to roll back a change that has already been committed. Suppose you're working away happily on a working copy of /calc/trunk, and you discover that the change made way back in revision 303, which changed integer.c, is completely wrong. It never should have been committed. You can use svn merge to undo the change in your working copy, and then commit the local modification to the repository. All you need to do is to specify a reverse difference. (You can do this by specifying --revision 303:302, or by an equivalent --change -303.)

$ svn merge -c -303 ^/trunk
--- Reverse-merging r303 into 'integer.c':
U    integer.c

$ svn status
 M     .
M      integer.c

$ svn diff
…
# verify that the change is removed
…

$ svn commit -m "Undoing change committed in r303."
Sending        integer.c
Transmitting file data .
Committed revision 350.

As we mentioned earlier, one way to think about a repository revision is as a specific changeset. By using the -r option, you can ask svn merge to apply a changeset, or a whole range of changesets, to your working copy. In our case of undoing a change, we're asking svn merge to apply changeset #303 to our working copy backward.

Keep in mind that rolling back a change like this is just like any other svn merge operation, so you should use svn status and svn diff to confirm that your work is in the state you want it to be in, and then use svn commit to send the final version to the repository. After committing, this particular changeset is no longer reflected in the HEAD revision.

Again, you may be thinking: well, that really didn't undo the commit, did it? The change still exists in revision 303. If somebody checks out a version of the calc project between revisions 303 and 349, she'll still see the bad change, right?

Yes, that's true. When we talk about removing a change, we're really talking about removing it from the HEAD revision. The original change still exists in the repository's history. For most situations, this is good enough. Most people are only interested in tracking the HEAD of a project anyway. There are special cases, however, where you really might want to destroy all evidence of the commit. (Perhaps somebody accidentally committed a confidential document.) This isn't so easy, it turns out, because Subversion was deliberately designed to never lose information. Revisions are immutable trees that build upon one another. Removing a revision from history would cause a domino effect, creating chaos in all subsequent revisions and possibly invalidating all working copies. [22]

Resurrecting Deleted Items

The great thing about version control systems is that information is never lost. Even when you delete a file or directory, it may be gone from the HEAD revision, but the object still exists in earlier revisions. One of the most common questions new users ask is, How do I get my old file or directory back?

The first step is to define exactly which item you're trying to resurrect. Here's a useful metaphor: you can think of every object in the repository as existing in a sort of two-dimensional coordinate system. The first coordinate is a particular revision tree, and the second coordinate is a path within that tree. So every version of your file or directory can be defined by a specific coordinate pair. (Remember the peg revision syntax—foo.c@224—mentioned back in the section called “Peg and Operative Revisions”.)

First, you might need to use svn log to discover the exact coordinate pair you wish to resurrect. A good strategy is to run svn log --verbose in a directory that used to contain your deleted item. The --verbose (-v) option shows a list of all changed items in each revision; all you need to do is find the revision in which you deleted the file or directory. You can do this visually, or by using another tool to examine the log output (via grep, or perhaps via an incremental search in an editor).

$ cd parent-dir
$ svn log -v
…
------------------------------------------------------------------------
r808 | joe | 2003-12-26 14:29:40 -0600 (Fri, 26 Dec 2003) | 3 lines
Changed paths:
   D /calc/trunk/real.c
   M /calc/trunk/integer.c

Added fast fourier transform functions to integer.c.
Removed real.c because code now in double.c.
…

In the example, we're assuming that you're looking for a deleted file real.c. By looking through the logs of a parent directory, you've spotted that this file was deleted in revision 808. Therefore, the last version of the file to exist was in the revision right before that. Conclusion: you want to resurrect the path /calc/trunk/real.c from revision 807.

That was the hard part—the research. Now that you know what you want to restore, you have two different choices.

One option is to use svn merge to apply revision 808 in reverse. (We already discussed how to undo changes in the section called “Undoing Changes”.) This would have the effect of re-adding real.c as a local modification. The file would be scheduled for addition, and after a commit, the file would again exist in HEAD.

In this particular example, however, this is probably not the best strategy. Reverse-applying revision 808 would not only schedule real.c for addition, but the log message indicates that it would also undo certain changes to integer.c, which you don't want. Certainly, you could reverse-merge revision 808 and then svn revert the local modifications to integer.c, but this technique doesn't scale well. What if 90 files were changed in revision 808?

A second, more targeted strategy is not to use svn merge at all, but rather to use the svn copy command. Simply copy the exact revision and path coordinate pair from the repository to your working copy:

$ svn copy ^/trunk/real.c@807 ./real.c

$ svn status
A  +   real.c

$ svn commit -m "Resurrected real.c from revision 807, /calc/trunk/real.c."
Adding         real.c
Transmitting file data .
Committed revision 1390.

The plus sign in the status output indicates that the item isn't merely scheduled for addition, but scheduled for addition with history. Subversion remembers where it was copied from. In the future, running svn log on this file will traverse back through the file's resurrection and through all the history it had prior to revision 807. In other words, this new real.c isn't really new; it's a direct descendant of the original, deleted file. This is usually considered a good and useful thing. If, however, you wanted to resurrect the file without maintaining a historical link to the old file, this technique works just as well:

$ svn cat ^/trunk/real.c@807 > ./real.c

$ svn add real.c
A         real.c

$ svn commit -m "Re-created real.c from revision 807."
Adding         real.c
Transmitting file data .
Committed revision 1390.

Although our example shows us resurrecting a file, note that these same techniques work just as well for resurrecting deleted directories. Also note that a resurrection doesn't have to happen in your working copy—it can happen entirely in the repository:

$ svn copy ^/trunk/real.c@807 ^/trunk/ \
      -m "Resurrect real.c from revision 807."
Committed revision 1390.

$ svn update
A    real.c
Updated to revision 1390.


[21] This was introduced in svn 1.6.

[22] The Subversion project has plans, however, to someday implement a command that would accomplish the task of permanently deleting information. In the meantime, see the section called “svndumpfilter” for a possible workaround.