| Register | Forget password
Tech - Reading a topic
<<  <  1  >  >>

Organize and version control digital photos with Subversion (basic)

good(0) bad(0) view(23549) comments(8)
Wen Send a message to Wen Send an email to Wen
poster face
Level: ◆◆◆◆◇◇◇

What is Subversion?

Subversion is an open-source cross-platform version control system[1]. Its official website is http://subversion.tigris.org. What is version control system (VCS)? It's a software for managing the whole development progress of a project (software project, documentation project, and so on). It stores all the development history in a place called repository. It manages privileges of developers. For example, who can add stuff to the repository, who can modify stuff in the repository, and who can delete stuff from the repository, are all managed by the VCS. It also deals with the conflicts when two or more developers are trying to modify the same file in the repository simultaneously.

Why use VCS to organize digital photos?

  1. The photos are safe, once they are imported into VCS's repository. Here "safe" means you can do any post-processing to the photos without losing the original copy. Because the repository stores the whole history of the files, and you can get a copy of any historical time.
  2. You can tidy up your photo folders by deleting all bad photos without hesitation. If you later find you deleted a good photo which you thought is bad, you can restore it.
  3. To backup the photos is easy. VCS provides simple commands to backup and recover a repository.

Why use Subversion, not other VCS?

  1. Subversion is open-source, so you can use it free of charge.
  2. Subversion is cross-platform. No matter what operating system you are using, you can install Subversion on it. At least it supports Windows, Mac OS X, Linux, FreeBSD, OpenBSD[2].
  3. CVS is another popular open-source cross-platform VCS. Subversion is a newer generation VCS than CVS. The most important difference in our case, is that digital photos (jpeg, RAW) are binary files and Subversion handles binary files more gracefully than CVS does[1].

How to organize photos with Subversion?

Install Subversion on Windows

  1. Download latest Subversion package for Windows from Subversion official website. There're a lot of packages there. In our case we should download the one described as "Windows installer with the basic win32 binaries" and named svn-x.x.x-setup.exe, with "x.x.x" the latest version number.
  2. Run the file we just downloaded to install Subversion.

Install Subversion on Ubuntu
Open a terminal (Applications -> Accessories -> Terminal) and execute the following command:

sudo apt-get install subversion

Install Subversion on FreeBSD from ports

  1. Make sure you've already installed ports and updated them to the latest version.
  2. Execute the following commands with the root account:

    cd /usr/ports/devel/subversion/
    make install clean

For installing Subversion on other operating systems, please refer to the Subversion documentation and the OS documentation.

On Windows, almost all Subversion operations will be executed under "Command Prompt." When you read "run the following commands" in this article, you are supposed to open a "Command Prompt" and type the commands in it to run. There's a GUI tool called TortoiseSVN, which enables you to operate Subversion on a graphical interface. You can get it and its documentation on its official website http://tortoisesvn.tigris.org. I'm not going to use this tool in this article.

Create a repository

  1. Choose where to put the repository. Suppose originally we store all the photos in D:\photo . Now we are going to create a repository at D:\repos\photo . Please note that the deepest repository folder name should be the same as the one where the photos originally stored. For example, you can choose to put the repository at C:\1\2\3\photo , but you're not supposed to choose a folder like D:\repos\foto .
  2. Run the following command:

    svnadmin create D:\repos\photo
  3. Now we created an empty repository.

Import the photos into the repository

  1. If you don't have any photo yet, you can ignore this importing step.
  2. Run the following commands:

    D:
    cd \
    svn import photo file:///d:/repos/photo -m "Initial import"
  3. If the screen displays "Committed revision 1" at the end, the importing is successful. Now we can delete the folder D:\photo , and we are supposed to do so. If you're afraid of losing the original data, you can rename the folder.

Initial checkout to make a working copy

  1. Make sure the original photo folder D:\photo is deleted or renamed.
  2. Run the following commands:

    D:
    cd \
    svn checkout file:///d:/repos/photo
  3. Now you get the folder D:\photo again. And it's called the working copy. Remember, you are supposed to process your photos on the working copy, not the repository. The repository stores the whole history of the working copy.

How to organize the photos?

I'll give an example to describe all the basic operations of organizing the photos. Suppose you take three photos today, named 01.jpg, 02.jpg, and 03.jpg in your memory card.

Importing the photos

  1. Create a new folder D:\photo\20070901 .
  2. Directly copy the three photos into the new folder D:\photo\20070901 .
  3. Run the following commands:

    D:
    cd \photo
    svn add 20070901
    svn commit -m "Added 20070901"
  4. If "Committed revision X" is displayed with X the repository version number, the new photos have been successfully imported into the repository. Please make sure the photos are successfully imported into the repository before you go on to next step.

Deleting a bad photo

  1. Suppose you took two pictures, say 01.jpg and 02.jpg, of a same scenery. Now you compare the two pictures and decide to keep only 02.jpg and delete 01.jpg .
  2. Run the following command:

    svn delete D:\photo\20070901\01.jpg
  3. This will only affect the working copy. We will submit the deletion to the repository later.
  4. Remeber to use the above command to delete a photo. Don't simply delete it using "delete" or "rm" command or delete it in Windows Explorer.
  5. If you now you change your mind and want to recover 01.jpg, run the following command:

    svn revert D:\photo\20070901\01.jpg

Post-processing a photo

  1. Suppose you want to process 03.jpg with Photoshop. Just open 03.jpg with Photoshop and process it as you nornally do.
  2. When done, save to the same file 03.jpg, overwrite it. Don't worry, the repository already saved the original 03.jpg. So you can feel free to make any modification to the 03.jpg in the working copy.
  3. If you don't like your modification, run the following command:

    svn revert D:\photo\20070901\03.jpg

    Please note that your modification has not been stored in the repository yet. By running the above command, you lose your modification.

Commit all modifications to the repository

Run the following commands:

D:
cd \photo\20070901
svn commit -m "Modified some photos of 20070901"

Now only the most beautiful photos stay in D:\photo\20070901 , with the original photos stored in the repository.

Summary

From the above example, we learned these Subversion commands:
svnadmin create (create a repository)
svn import (import photos into a repository)
svn checkout (checkout a working copy from the repository)
svn add (add new photos)
svn delete (delete bad photos)
svn revert (undo modifying/deleting photos)
svn commit (commit changes to the repository)

Basic photo management is as simple as these commands. Subversion is a powerful tool, with which we can do more jobs to organize our photos more nicely and safely. We will discuss this in Organize and version control digital photos with Subversion (advanced).

Reference

[1] Version Control with Subversion (can be obtained from http://svnbook.red-bean.com)
[2] http://subversion.tigris.org

Share/Save/Bookmark
Last edited by Wen at 2009-06-01 14:38:27

Posted at 2007-09-01 23:39:21

Comment: Re: Organize and version control digital photos with Subversion (basic)

good(0) bad(0) #1
milasch (guest)

Very interesting article. Now, is there a way of working with a SVN repository on an "online" mode?

I have 30GB+ of digital photos and videos taken along the last years and I'm thinking of using SVN to control it and make life easier to share those photos among other computers at home. I'm considering to use my laptop as a server and it only has 50GB hd space free. If I import all pictures to the repository, I'll have to checkout a "working copy" in my own laptop, which will duplicate the files and leave me with only 20GB disk space. For remote computers it would be nice and easy to get picutres, but for my laptop it would leave me with little free space.

So I was thinking if I could work with the local repository "online", just like some other vcs' work.

I'm not sure i'm entirely clear.

Thanks


Posted at 2009-05-29 11:57:43

Comment: Re: Organize and version control digital photos with Subversion (basic)

good(0) bad(0) #2
Wen Send a message to Wen Send an email to Wen
poster face
Level: ◆◆◆◆◇◇◇

First of all, since your laptop's hard drive is small, I don't think using it as a server is a good idea. Because photos and videos are hard to compress, the SVN repository will be as big as the original files themselves. So if you put the repository on your laptop, it alone will take 30GB+ space. Checking out a working copy will need another 30GB+ space.

Second of all, putting all 30GB+ photos into a single SVN repository will make it hard to backup and migrate.

Here's what I would do in your situation:

  • Use a computer with a large hard drive as a server.
  • Categorize the photos and videos by year or month. Each category should have less than 10GB files. Make an SVN repository for each category.
  • You can checkout working copies on the server computer locally.
  • On other computers, checkout working copies from the repository server through network. I will describe the details below.
  • Since your laptop has a small hard drive, you can choose only the repositories you want to work on the laptop to checkout. And as long as you commit all changes of the working copies to the repositories, you can delete the working copies on your laptop.

How to checkout a working copy from a repository server through network:

  1. Set up the repository computer as an SSH server. This computer should have a constant host name. If it has a constant IP address, the address can be used as the host name. If the IP address changes, then a constant domain name is needed. Some websites provide dynamic domain name (a constant domain name for dynamic IP addresses) service.
  2. Make sure from a client computer you can SSH to the server correctly with the constant host name.
  3. Now on the client computer we can checkout a working copy using this command:

    svn checkout svn+ssh://USERNAME@HOST_NAME/PATH

    where HOST_NAME is the host name of the server, USERNAME is the username you use to logon the SSH server, and PATH is the repository path on the server. You'll be prompted to type the password (sometiems more than once) during the checkout.

  4. There would be no change to the usage of these commands:
    svn add (add new photos)
    svn delete (delete bad photos)
    svn revert (undo modifying/deleting photos)
    svn commit (commit changes to the repository)
Last edited by Wen at 2009-06-01 12:00:16

Posted at 2009-06-01 11:55:35

Comment: Re: Organize and version control digital photos with Subversion (basic)

good(0) bad(0) #3
milasch (guest)

I guess I'll just simply wait until I'm back home (another 2 months) where I can start versioning my photos on my quad core desktop with 500GB of hd space. For now I'll try using rsync to keep my photos syncronized with my wife's laptop.

BTW, have you experienced using such large repositories with 30GB+ of files? Does it get too slow on svn updates and commits?

Thanks for your help


Posted at 2009-06-02 07:18:23

Comment: Re: Organize and version control digital photos with Subversion (basic)

good(0) bad(0) #4
Wen Send a message to Wen Send an email to Wen
poster face
Level: ◆◆◆◆◇◇◇

Well, rsync is a good tool.

I just realize that I do have a large repository with 37G of files. I make one repository for each year's photos. I didn't know I took that many photos last year. I didn't experience any svn updates or svn commits that were too slow due to the large size of the repository. It seems these operations are well-optimized.

I think the problem of having a huge repository is the inconvenience of backing up, migration, and other operations related to the whole repository. It will take forever to backup a huge repository, especially if it's through network. But if you split it up into multiple repositories, you can backup the 1st one today, the 2nd one tomorrow, etc. And if you don't make changes to a repository, you don't need to make a new backup of it.


Posted at 2009-06-03 18:14:24

Comment: There is a specialized tool for versioning photos

good(0) bad(0) #5
MEkberg (guest)

I have started a project to create a version control system for images, photos and large binary files in general (even video). It is under development, but it is already quite usable, especially if you are comfortable with the command line. Check it out on google code: http://code.google.com/p/boar/


Posted at 2010-12-08 05:20:14

Comment: Re: Organize and version control digital photos with Subversion (basic)

good(0) bad(0) #6
Wen Send a message to Wen Send an email to Wen
poster face
Level: ◆◆◆◆◇◇◇

Very interesting software, MEkberg. I'll keep an eye on it.


Posted at 2010-12-13 00:07:24

Comment: Re: Organize and version control digital photos with Subversion (basic)

good(0) bad(0) #7
MrSnowflake (guest)

Doesn't svn leave your working directory with a .svn directory with the size of (almost) all the files you want to backup, rendering it twice the size?


Posted at 2011-03-10 06:39:00

Comment: Re: Organize and version control digital photos with Subversion (basic)

good(0) bad(0) #8
Wen Send a message to Wen Send an email to Wen
poster face
Level: ◆◆◆◆◇◇◇

MrSnowflake, you're right, but I wouldn't worry about it much because hard drives are so cheap these days and I don't shoot that many photos.


Posted at 2011-03-10 13:22:54
Tech - Reading a topic
<<  <  1  >  >>
Browse topics by category: Footprints & Travel | Recipes | Fun | Tech | Science | Entertainment | Life | WENSH Affairs | Guests' Msgs |

Valid XHTML 1.0 | Valid CSS2 | WAI-A WCAG 1.0

Copyright 2005-2014 WEN'S Horizon [40/0.038]