New KB Article: Synchronizing the Projects Using Rsync

[rbzend]

Recently we published a new Knowledge Base article:
How to Keep Your Zend Studio Project Synchronized with a Linux Server

The article gives an example of a setup for synchronizing your Zend Studio projects with a remote Linux/UNIX server via rsync. Who should be interested in reading this article?

- Anyone who has trouble using the "conventional" methods for remote connectivity - FTP/SFTP.
- Developers who want to sync one or more staging servers with their Zend Studio projects.
- Developers who want to have on-demand sync of a Zend Studio project with the production server. In this case a slight modification of the procedure is required - the article suggests a procedure for constant synchronization.
- Developers who just want to backup their projects either remotely or locally. In case of local backup an obvious change in the rsync command line is required.

This article also makes it easier to hold on to our "best practices" recommendations - always working with the projects locally, not via a remote connection.

[dafish]

Thanks for this. However, I have one important question: How do I specify another port than the standard SSH port 22?
I'm running my ssh daemon on 60022 instead of 22.

[rbzend]

IMO the easiest way would be creating the file <your home>/.ssh/config and adding the following lines:

Code: Select all

Host <sever name or address>
Port 60022

See man ssh_config about this.

Alternatively, look up the -e (--rsh) parameter description in the rsync manual. It lists all the available options.

[dafish]

IMO the easiest way would be creating the file <your home>/.ssh/config and adding the following lines:

Code: Select all

Host <sever name or address>
Port 60022

See man ssh_config about this.

Alternatively, look up the -e (--rsh) parameter description in the rsync manual. It lists all the available options.

Thanks for the hint. I've created my config an it's working great. Testing the rsync-functionality now.

[i3vi3v]

I've tried to follow those KB article and I wish to share my experience...

Firstly: why do I need it? isn't Remote Server Support enough?

• Because Zend Studio 9 Remote Server Support is unable to transfer dotted folders. It just ignores them. It looks like its impossible to change this.
  (I've posted this question here:
  viewtopic.php?f=59&t=105783&p=196043
  , but no one replied yet)
  
• At best, I'd like to have a two-way sync, or at least a notification, that something on the server is updated

So...Let's start:

On the Windows workstation cwRsync a Windows port of rsync is used.

I don't know, which version of cwRsync author have used. For now, I can see two versions: newer commercial and free, cwRsync 4.0.5. I tried to use the latter.

First, you need to create an authentication key pair. But before you do this, make sure that you don't have the keys already in:
Windows - %USERPROFILE%\.ssh

This is where the story begins. The thing is - my cwRsync 4.0.5 won't try to get a profile from there. You may put your keys to anywhere - you still have to provide a full path to the key to cwRsync. (Putting your keys to "%USERPROFILE%\.ssh" is still a good idea, though).

So, we've created our keys and we're now going to test them.
Here's what we can see in the article:

C:\> cd C:\Program Files\cwRsync\bin
C:\Program Files\cwRsync\bin> ssh developer@staging-server
Welcome to the staging server!
developer@staging-server:~$ pwd
/home/developer

As I've just mentioned, I don't get, how ssh choses key in the example above. Even if we suppose that it's aware about "%USERPROFILE%\.ssh" folder, there might be several private keys there...

So, as for me, this example is wrong:

Code: Select all

C:\Program Files\cwRsync\bin>ssh developer@staging-server
Could not create directory '/home/John/.ssh'.
The authenticity of host 'staging-server (88.88.888.888)' can't be establi
shed.
RSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:x:xx:xx:xx:xx:xx:xx:xx:xx.
Are you sure you want to continue connecting (yes/no)? yes
Failed to add the host to the list of known hosts (/home/John/.ssh/known_hosts).

Password:

After reading some documentation I've managed to create an appropriate example:

Code: Select all

C:\Program Files\cwRsync\bin>ssh -i "C:\Users\John\.ssh\id_rsa" developer@staging-server
developer@staging-server ~ $ pwd
/home/developer

Next... concerning arguments list:
Here's what we can see in the article:

-vcrz
--delete-after
--exclude-from=rsyncExclude.txt
.
developer@staging-server:public_html

there's even no "ssh" in it... I'm puzzled. Of course, it wasn't working for me. But here's an argument list that worked:

Code: Select all

-vaurPcz
--delete-after
--exclude-from=rsyncExclude.txt
-e "ssh -i C:\Users\John\.ssh\id_rsa" developer@staging-server:public_html
./my_project_public_html/

As for "-vaurPcz" - one can find definitions of this options in rsync manual (for example, try to google "rsync samba manual" - I'm unable to post links here).
The last parameter "./my_project_public_html/" refers to the folder you're going to sync, relative to your ${build_project}

And the last thing, concerning Environment - have you mentioned "Failed to add the host to the list of known hosts (/home/John/.ssh/known_hosts)" above in my post? So, we need to define "HOME" variable in "Enviroment" tab.

• Variable = "HOME"
• Value = "/cygdrive/c/Users/John"

Here "/cygdrive/c/Users/John" stands for "C:/Users/John" . This is how you should pass a path to cwRsync 4.0.5 .... (But ssh accepts "ssh -i C:\Users\John\.ssh\id_rsa").

Hope this would help somebody.
Or even a miracle would happen and those article would get fixed...

Still, I've got two final notes:

• You may also add a second "builder" - a copy of this one, but with swapped source and target to get a two-way sync.
• rsync is designed for one-way syncing. Although there's an "-u" option (do not overwrite a newer file with older), you'll might run into problems with file deletion.
• you may try to set up "unison" or any other two-way sync tool,similarly to cwRsync. This is what I'm going to do now. I'll try to share my experience afterwards...

[rbzend]

Or even a miracle would happen and those article would get fixed...

The article got fixed - a small change that was long due. Thanks for poking me, your post was the last straw.
Basically, all you need to do is adding the environment variable %HOME% - this solves most of the issues you pointed out. Now the article should work as is, without additional steps.


Some comments to your post:

Firstly: why do I need it? isn't Remote Server Support enough?

In addition to what you wrote - Remote Server Support uses the good old protocols FTP or SFTP. Unfortunately, these are not optimal for synchronization. rsync is way more efficient in this regard. FTP and SFTP are good at what they do - file transfer.

I don't get, how ssh choses key

It's simpler than you may think. Unless instructed otherwise (command line, config file or ssh-agent), SSH will just look for ~/.ssh/id_rsa or ~/.ssh/id_dsa

You may also add a second "builder" - a copy of this one, but with swapped source and target to get a two-way sync.

In theory, this is possible. However, note, rsync may need user input in case of conflicts, which you cannot do from the Eclipse Console view. Therefore you need to make sure that the rsync operation is strictly non-interactive.

[i3vi3v]

Hi rbzend,
thanks for your concern!

Although I ended up using unison (I've even posted the tutorial here: http://i3vi3v.wordpress.___com/2013/01/ ... ng-unison/ )....
Hm...

Basically, all you need to do is adding the environment variable %HOME% - this solves most of the issues you pointed out.

+

It's simpler than you may think. Unless instructed otherwise (command line, config file or ssh-agent), SSH will just look for ~/.ssh/id_rsa or ~/.ssh/id_dsa

Hm, ok, but... It's a common situation, when developer works with several independent servers... Should he use the same rsa key for all of them? Sometimes it's possible, though, this doesn't sound like best practice...

[rbzend]

It is normal practice. You have the private part of your key (the key), the server only has the public part (the key hole). As long as you don't distribute your private key, it's ok.

Of course, you can use several keys and tell SSH to use them in one of the ways I mentioned:

1. Command line (like you did).

2. Configuration file ~/.ssh/config , for example:

Code: Select all

Host acme.com
User john
Port 22022
IdentityFile ~/.ssh/acme_com_id_rsa

Host example.com
User devteam
IdentityFile ~/.ssh/example_com_key.pem

3. ssh-agent. For example, on Linux / OSX you would do it like this:
$ ssh-add ~/.ssh/acme_com_id_rsa
On Windows you can use Pageant.

[i3vi3v]

It is normal practice. You have the private part of your key (the key), the server only has the public part (the key hole). As long as you don't distribute your private key, it's ok.

I'm not trying to say it's not safe, I don't like the idea just because it limits "adjustability". What if someone is GIVEN a key for a server (from a manager, or a server owner, or someone)?
Also, I often use several keys per server (a separate key for unison-on-prefix, for example)

Thanks for the article fix, anyway!