Through the mirror
By: sifbuilder (admin) 2007.08.15

WHITE RABBIT lyrics -- Jefferson Airplane
One pill makes you larger
And one pill makes you small
And the ones that mother gives you
Don't do anything at all
Go ask Alice
When she's ten feet tall

And if you go chasing rabbits
And you know you're going to fall
Tell 'em a hookah smoking caterpillar
Has given you the call
Call Alice
When she was just small

When men on the chessboard
Get up and tell you where to go
And you've just had some kind of mushroom
And your mind is moving low
Go ask Alice
I think she'll know

When logic and proportion
Have fallen sloppy dead
And the White Knight is talking backwards
And the Red Queen's "off with her head!"
Remember what the dormouse said:
"Feed your head
Feed your head
Feed your head"

When in 2004 sifbuilder moved from an oscommerce contribution to a multi core site generator, it lost carine's kind comments and had to look for a host server. At that time, sourceforge supported the distribution of os programs and was introducing the web, shell and database services for hosted projects. Access to shell services was available through SSH connections authenticated with SSH2 keys. sifbuilder supported the automatic generation of web sites on the localhost and those requirements seemed reasonable to try to automate the installation of sites on remote hosts, such as those of sourceforge. This was done introducing the sifnet concept based on the sifnethost, sifnetrelation and sifnetsite entities.

A sifnetrelation establishes a relationhip between a lochost -usually instantiated by the local devhost- and a nethost -usually a remote host. This note will take as motiv the installation of a site on a nethost instantiated to sourceforge defined by the sfhost class in the sifnetconfigfile file-. The sifnetsite describes the site to be installed on the nethost. The relation is reflexive and transitive but, unfortunately, not commutative. The non equivalence may be read as 'having the site hosted on Linux matters'.

Currently, a sifnetrelation is based on a SSH connection -in the case of sourceforge, the connection is controlled by a SS2 pair of keys-. The origin of the relation -identified by the lochost- is a winbox hosting the sifbuilder development package; the destination point -identified by the nethost- is a Linux host offering the following services: a shell to the sifdir where the sifbuilder package will be transferred to, a database hosting the remote site database, and a webdir hosting the remote site file system.

On the origin winbox, we have available a SSH client -ex. putty among various ssh clients - and a tar/untar tool -in our case 7-zip - and we have created the private and public ppk keys following the sf instructions.

The information on the lochost, the nethost, the sifnetsite and the type of sifnet relation (SS2 keys or login/password) is passed to sifbuilder in the sifnet_config.php file. A model of this file is included in the sifbuilder root folder. Normally, the customized file is stored in a parallel file tree as defined with the --sifnetconfigfile option. The following classes show the information assigned there.

The sfhost describes the target host at sourceforge. It extends a sifnet_linux_host and this a sifnet_host.

class sfhost extends sifnet_linux_host
{
    var $hostParams = array(
     'hosttype'=>'linux'
    ,'access'=>'ssh2key'
    ,'userdir'=>'/home/users/m/my/myuser'
    ,'sifdir'=>'/home/users/m/my/myuser/myproject/'
    ,'webdir'=>'/home/groups/m/my/myproject/htdocs/'
    ,'host_domain'=>'myproject.sf.net'
    ,'user_login'=>'my_user_login'
    ,'user_pwd'=>'my_user_pwd'
    ,'user_key'=>'D:\\sf_private_key.ppk'
    ,'php_bin'=>'php '
    ,'rexec_bin'=>'ssh '
    ,'rcp_bin'=>'scp '
    );
}

The devhost describes the local host, including the syntax of the local php, tar and remote exec commands.

class devhost extends sifnet_windows_host
{
    var $hostParams = array(
    'hosttype'=>'windows'
    ,'access'=>'local'
    ,'userdir'=>'.\\'
    ,'sifdir'=>'.\\'
    ,'webdir'=>'D:\\webroot\\htdocs\\'
    ,'downloadsdir'=>'D:\\webroot\\htdocs\\downloads\\'
    ,'packagesdir'=>'D:\\webroot\\htdocs\\downloads\\packages\\'
    ,'host_domain'=>'localhost'
    ,'user_login'=>''
    ,'user_pwd'=>''
    ,'user_key'=>''
    ,'php_bin'=>'php '
    ,'zip_bin'=>'"C:\\Program Files\\7-Zip\\7z.exe " a -tzip '
    ,'unzip_bin'=>'"C:\\Program Files\\7-Zip\\7z.exe" x '
    ,'rexec_bin'=>'D:\\BIN\\plink.exe '
    ,'rcp_bin'=>'D:\\BIN\\pscp.exe '
    );
}

The sfhost describes the remote host, including the syntax of the remote php, tar and remote exec commands, and the location of the SSH2 keys to be passed to the SSH client of the local host.

class sfhost extends sifnet_linux_host
{
    var $hostParams = array(
     'hosttype'=>'linux'
    ,'access'=>'ssh2key'
    ,'userdir'=>'/home/users/m/my/myuser'
    ,'sifdir'=>'/home/users/m/my/myuser/myproject/'
    ,'webdir'=>'/home/groups/m/my/myproject/htdocs/'
    ,'host_domain'=>'myproject.sf.net'
    ,'user_login'=>'my_user_login'
    ,'user_pwd'=>'my_user_pwd'
    ,'user_key'=>'D:\\sf_private_key.ppk'
    ,'php_bin'=>'php '
    ,'rexec_bin'=>'ssh '
    ,'rcp_bin'=>'scp '
    );
}

The remote site describes the site to be created on the remote host. This instance describes the structure of a sourceforge site but mirrors the structure and semantics of any site created or updated with sifbuilder.

class sbd501_at_sfhost  extends sifnet_site
{
    var $siteParams = array(
    'site_host'=>'myproject.sourceforge.net'
    ,'site_name'=>'mysite'
    ,'site_pod'=>'d501'
    ,'db_server'=>'mysql4-m'
    ,'db_database'=>'db_myproject'
    ,'db_prefix'=>'d501_'
    ,'db_prefix_sep'=>''
    ,'db_dbtype'=>'mysql'
    ,'db_tabletype'=>'myisam'
    ,'db_username'=>'db_admin'
    ,'db_password'=>'db_admin_pwd'
    ,'site_admin_username'=>'my_user_login'
    ,'site_admin_password'=>'my_user_pwd'
    ,'dir_fs_document_root'=>'/home/groups/m/my/myproject/htdocs/'
    ,'sys_temp'=>''
    ,'encodedpwd'=>1
    ,'enable_ssl'=>0
    );
}

To create the site on the remote sourceforge host server for a sourceforge project, we follow the following steps. Note that, when running in sifnet mode, the -v option launches the sifnet command in real mode. Without that option, the translated command will appear in simulation mode. Before running any of those commands, pick up any of the disclaimers in here and apply them to the rest of the note. Age quod agis.

Test the connection to the remote host with the ac001h admin case, as described in sifnet_usecases.php:

>php sifcmd.php a ac001h --sifnetconfigfile=../sifbuilder_local/sifnet_config.php --sifnethost=sbhost -v

This command will list the entries in the home dir in the remote host. Since the local sifbuilder distribution may have changed because of the development process or because new installation packages (sip) or installation profiles (sib) have been added, we create a new distribution. We create it with

>php sifcmd.php build_program --sifnetconfigfile=../sifbuilder_local/sifnet_config.php

and its tar file.

>php sifcmd.php sifnet_create_program_zip --sifnethost=devhost -v

Up to here the steps followed to create each new sifbuilder release package. To create the remote site, we upload the tar file, located in the ./siftmp/tmpUplaod folder from the sifbuilder distribution root, to the remote host.

>php sifcmd.php sifnet_upload_program --sifnetconfigfile=../sifbuilder_local/sifnet_config.php --sifnethost=sfhost -v

and we explode the tar there.

>php sifcmd.php sifnet_explode_package --sifnethost=sfhost --sifnetconfigfile=../sifbuilder_local/sifnet_config.php -v

As it appears in that command and most of the examples of the site, the local sifnet config file is located in a parallel file tree. The same happens with the site packages that constitute the generated site. This is the time to create it. It can be done connecting to the site or with

>php sifcmd.php a ac001i --sifnetconfigfile=../sifbuilder_local/sifnet_config.php --sifnethost=sfhost -v

Since we want to use a single and locally edited customized config file, we upload it to the remote sifbuilder_local/ folder:

> php sifcmd.php a ac023 --sifnetconfigfile=../sifbuilder_local/sifnet_config.php --sifnethost=sfhost -v

sifbuilder takes care to download the site packages required to build the site. However, sourceforge does not allow establishing an outgoing httpd connection. It is therefore necessary to upload them to the sifbuilder_local folder. First we copy them to siftmp/tmpUpload/packages folder -since we plan to create a drupal site using the sib_drupal5_sif_sif.xml sib, we copy the site packages that correspond to the drupal5_core_drupal, drupal5_mod_views and drupal5_mod_site_map installation packages (sips) - and run

> php sifcmd.php sifnet_upload_packages --sifnethost=sfhost --sifnetconfigfile=../sifbuilder_local/sifnet_config.php --sifpacksdir=../sifbuilder_local/packages 

Then, the site may be created.

> php sifcmd.php b --sifpacksdir=../sifbuilder_local/packages --sifnetconfigfile=../sifbuilder_local/sifnet_config.php --sifnetsite=sbd501_at_sfhost --sifnethost=sfhost --siburi=./sibs/sib_drupal5_sif_sif.xml -x -f -v

Running a sif command with -x means erasing the target site file system and dropping all the tables in the target database. -f means doing that without the predefine settings and without the possibility to  fix them. -v means  confirming those options in a remote site. Collect all the disclaimers in the site above before doing this.

The difference between that command and the usual sif commands used to generate a local site resides on setting the sifnethost option. By setting the sifnethost option the command is translated into a remote instruction, in this case the command that creates the remote target site.

If the -v option is ommitted, the remote command is shown but not run. It shows

> D:\BIN\plink.exe -ssh -2  -l my_user_login -i D:\LOCAL\sf_private_key.ppk myproject.sourceforge.net " php /home/users/m/my/myuser/myproject/sifcmd.php  sif_sip_build_site --dev --expandsiteparams --noupd --emptypod --force --complete --dbgtype=both --siburi=./sibs/sib_drupal5_sif_sif.xml --site_host=myproject.sourceforge.net --site_name=mysite --site_pod=d501 --db_server=mysql4-m --db_database=db_myproject --db_prefix=d501_ --db_dbtype=mysql --db_tabletype=myisam --db_username=db_admin --db_password=db_admin_pwd --site_admin_username=my_user_login --site_admin_password=my_user_pwd --dir_fs_document_root=/home/groups/m/my/myproject/htdocs/ --encodedpwd=1 --sifnetconfigfile=../sifbuilder_local/sifnet_config.php --sifpacksdir=../sifbuilder_local/packages --siflochost=devhost   drupal5_core_drupal  drupal5_theme_sif  drupal5_mod_views  drupal5_mod_site_map  drupal5_script_locals "

The sifbuilder site was created and is regularly maintained following these steps. if not automatic, the process is systematic. And every systematic process requires a table of definitions.

domain
terms
definition
hostparams
usedir
root folder of the use work environment. Usually, the sifbuilder tar will be downloaded to this location.

sifdir
folder containing the working sifbuilder distribution. Usually, the result of exploding the sifbuilder tar to this location.

webdir
folder containing the root to the web environment.

access
type of authentication, currently ssh2key or ssh2pwd.
siteParams
--site_name
folder under the webdir that will contain the target sites.

--site_pod
folder under that the site_name folder that will contain the specific target site. Plays, in a certain way, for the file system, the role of the prefix in the target database.
runParams
--sifpacksdir
indicates where the site packages (core, modules and themes) are downloaded. By storing them outside the sifbuilder folder, updating new versions of sifbuilder will not remove them.

--sifnetconfigfile
points to the file hosting the classes describing the target sites referenced by sifnetsite. The sample sifnet_config.php is included in the sifbuilder root folder. It should always be adapted to the local configuration before starting using sifbuilder.

--sifnetsite
refers to the class in the sifnetconfigfile with properties the site configuration options typically set during the installation of a new content management site. They coincide with the elements in the sifSiteParams.xml file in the sifbuilder root. Having these options on file permits to refer to each site configuration by name. Wrong setting of those options may erase a whole system. No need to look for a demonstration.

--sifnethost
refers to the (usually remote) host where the sifcommands will be run. By setting this option, the normal sif commands are translated into remote execution commands using the information in the associated class defined in the sifnetconfig file.

-x
may be added to the command to remove the site root and empty the site database before starting the installation. Do that only if you have the possibility of and know how to restore your working environment.

-f
may be used to force the execution of the command in non-interactive mode. It can be used and will be used if you are a programmer when you know to be in control of what you are doing.

-v
sifnet commands are run on the nethost by default in simulation mode.  When the -v option is set, they run in real mode.