sifbuilder

WHITE RABBIT lyrics -- Jefferson Airplane
One pill makes you larger
And one pill makes you small
And the ones that mother gives you
Don't do anything at all
Go ask Alice
When she's ten feet tall

And if you go chasing rabbits
And you know you're going to fall
Tell 'em a hookah smoking caterpillar
Has given you the call
Call Alice
When she was just small

When men on the chessboard
Get up and tell you where to go
And you've just had some kind of mushroom
And your mind is moving low
Go ask Alice
I think she'll know

When logic and proportion
Have fallen sloppy dead
And the White Knight is talking backwards
And the Red Queen's "off with her head!"
Remember what the dormouse said:
"Feed your head
Feed your head
Feed your head"

When in 2004 sifbuilder moved from an oscommerce contribution to a multi core site generator, it lost carine's kind comments and had to look for a host server. At that time, sourceforge supported the distribution of os programs and was introducing the web, shell and database services for hosted projects. Access to shell services was available through SSH connections authenticated with SSH2 keys. sifbuilder supported the automatic generation of web sites on the localhost and those requirements seemed reasonable to try to automate the installation of sites on remote hosts, such as those of sourceforge. This was done introducing the sifnet concept based on the sifnethost, sifnetrelation and sifnetsite entities.

A sifnetrelation establishes a relationhip between a lochost -usually instantiated by the local devhost- and a nethost -usually a remote host. This note will take as motiv the installation of a site on a nethost instantiated to sourceforge defined by the sfhost class in the sifnetconfigfile file-. The sifnetsite describes the site to be installed on the nethost. The relation is reflexive and transitive but, unfortunately, not commutative. The non equivalence may be read as 'having the site hosted on Linux matters'.

Currently, a sifnetrelation is based on a SSH connection -in the case of sourceforge, the connection is controlled by a SS2 pair of keys-. The origin of the relation -identified by the lochost- is a winbox hosting the sifbuilder development package; the destination point -identified by the nethost- is a Linux host offering the following services: a shell to the sifdir where the sifbuilder package will be transferred to, a database hosting the remote site database, and a webdir hosting the remote site file system.

On the origin winbox, we have available a SSH client -ex. putty among various ssh clients - and a tar/untar tool -in our case 7-zip - and we have created the private and public ppk keys following the sf instructions.

The information on the lochost, the nethost, the sifnetsite and the type of sifnet relation (SS2 keys or login/password) is passed to sifbuilder in the sifnet_config.php file. A model of this file is included in the sifbuilder root folder. Normally, the customized file is stored in a parallel file tree as defined with the --sifnetconfigfile option. The following classes show the information assigned there.

The sfhost describes the target host at sourceforge. It extends a sifnet_linux_host and this a sifnet_host.

class sfhost extends sifnet_linux_host
{
    var $hostParams = array(
     'hosttype'=>'linux'
    ,'access'=>'ssh2key'
    ,'userdir'=>'/home/users/m/my/myuser'
    ,'sifdir'=>'/home/users/m/my/myuser/myproject/'
    ,'webdir'=>'/home/groups/m/my/myproject/htdocs/'
    ,'host_domain'=>'myproject.sf.net'
    ,'user_login'=>'my_user_login'
    ,'user_pwd'=>'my_user_pwd'
    ,'user_key'=>'D:\\sf_private_key.ppk'
    ,'php_bin'=>'php '
    ,'rexec_bin'=>'ssh '
    ,'rcp_bin'=>'scp '
    );
}

The devhost describes the local host, including the syntax of the local php, tar and remote exec commands.

class devhost extends sifnet_windows_host
{
    var $hostParams = array(
    'hosttype'=>'windows'
    ,'access'=>'local'
    ,'userdir'=>'.\\'
    ,'sifdir'=>'.\\'
    ,'webdir'=>'D:\\webroot\\htdocs\\'
    ,'downloadsdir'=>'D:\\webroot\\htdocs\\downloads\\'
    ,'packagesdir'=>'D:\\webroot\\htdocs\\downloads\\packages\\'
    ,'host_domain'=>'localhost'
    ,'user_login'=>''
    ,'user_pwd'=>''
    ,'user_key'=>''
    ,'php_bin'=>'php '
    ,'zip_bin'=>'"C:\\Program Files\\7-Zip\\7z.exe " a -tzip '
    ,'unzip_bin'=>'"C:\\Program Files\\7-Zip\\7z.exe" x '
    ,'rexec_bin'=>'D:\\BIN\\plink.exe '
    ,'rcp_bin'=>'D:\\BIN\\pscp.exe '
    );
}

The sfhost describes the remote host, including the syntax of the remote php, tar and remote exec commands, and the location of the SSH2 keys to be passed to the SSH client of the local host.

class sfhost extends sifnet_linux_host
{
    var $hostParams = array(
     'hosttype'=>'linux'
    ,'access'=>'ssh2key'
    ,'userdir'=>'/home/users/m/my/myuser'
    ,'sifdir'=>'/home/users/m/my/myuser/myproject/'
    ,'webdir'=>'/home/groups/m/my/myproject/htdocs/'
    ,'host_domain'=>'myproject.sf.net'
    ,'user_login'=>'my_user_login'
    ,'user_pwd'=>'my_user_pwd'
    ,'user_key'=>'D:\\sf_private_key.ppk'
    ,'php_bin'=>'php '
    ,'rexec_bin'=>'ssh '
    ,'rcp_bin'=>'scp '
    );
}

The remote site describes the site to be created on the remote host. This instance describes the structure of a sourceforge site but mirrors the structure and semantics of any site created or updated with sifbuilder.

class sbd501_at_sfhost  extends sifnet_site
{
    var $siteParams = array(
    'site_host'=>'myproject.sourceforge.net'
    ,'site_name'=>'mysite'
    ,'site_pod'=>'d501'
    ,'db_server'=>'mysql4-m'
    ,'db_database'=>'db_myproject'
    ,'db_prefix'=>'d501_'
    ,'db_prefix_sep'=>''
    ,'db_dbtype'=>'mysql'
    ,'db_tabletype'=>'myisam'
    ,'db_username'=>'db_admin'
    ,'db_password'=>'db_admin_pwd'
    ,'site_admin_username'=>'my_user_login'
    ,'site_admin_password'=>'my_user_pwd'
    ,'dir_fs_document_root'=>'/home/groups/m/my/myproject/htdocs/'
    ,'sys_temp'=>''
    ,'encodedpwd'=>1
    ,'enable_ssl'=>0
    );
}

To create the site on the remote sourceforge host server for a sourceforge project, we follow the following steps. Note that, when running in sifnet mode, the -v option launches the sifnet command in real mode. Without that option, the translated command will appear in simulation mode. Before running any of those commands, pick up any of the disclaimers in here and apply them to the rest of the note. Age quod agis.

Test the connection to the remote host with the ac001h admin case, as described in sifnet_usecases.php:

>php sifcmd.php a ac001h --sifnetconfigfile=../sifbuilder_local/sifnet_config.php --sifnethost=sbhost -v

This command will list the entries in the home dir in the remote host. Since the local sifbuilder distribution may have changed because of the development process or because new installation packages (sip) or installation profiles (sib) have been added, we create a new distribution. We create it with

>php sifcmd.php build_program --sifnetconfigfile=../sifbuilder_local/sifnet_config.php

and its tar file.

>php sifcmd.php sifnet_create_program_zip --sifnethost=devhost -v

Up to here the steps followed to create each new sifbuilder release package. To create the remote site, we upload the tar file, located in the ./siftmp/tmpUplaod folder from the sifbuilder distribution root, to the remote host.

>php sifcmd.php sifnet_upload_program --sifnetconfigfile=../sifbuilder_local/sifnet_config.php --sifnethost=sfhost -v

and we explode the tar there.

>php sifcmd.php sifnet_explode_package --sifnethost=sfhost --sifnetconfigfile=../sifbuilder_local/sifnet_config.php -v

As it appears in that command and most of the examples of the site, the local sifnet config file is located in a parallel file tree. The same happens with the site packages that constitute the generated site. This is the time to create it. It can be done connecting to the site or with

>php sifcmd.php a ac001i --sifnetconfigfile=../sifbuilder_local/sifnet_config.php --sifnethost=sfhost -v

Since we want to use a single and locally edited customized config file, we upload it to the remote sifbuilder_local/ folder:

> php sifcmd.php a ac023 --sifnetconfigfile=../sifbuilder_local/sifnet_config.php --sifnethost=sfhost -v

sifbuilder takes care to download the site packages required to build the site. However, sourceforge does not allow establishing an outgoing httpd connection. It is therefore necessary to upload them to the sifbuilder_local folder. First we copy them to siftmp/tmpUpload/packages folder -since we plan to create a drupal site using the sib_drupal5_sif_sif.xml sib, we copy the site packages that correspond to the drupal5_core_drupal, drupal5_mod_views and drupal5_mod_site_map installation packages (sips) - and run

> php sifcmd.php sifnet_upload_packages --sifnethost=sfhost --sifnetconfigfile=../sifbuilder_local/sifnet_config.php --sifpacksdir=../sifbuilder_local/packages 

Then, the site may be created.

> php sifcmd.php b --sifpacksdir=../sifbuilder_local/packages --sifnetconfigfile=../sifbuilder_local/sifnet_config.php --sifnetsite=sbd501_at_sfhost --sifnethost=sfhost --siburi=./sibs/sib_drupal5_sif_sif.xml -x -f -v

Running a sif command with -x means erasing the target site file system and dropping all the tables in the target database. -f means doing that without the predefine settings and without the possibility to  fix them. -v means  confirming those options in a remote site. Collect all the disclaimers in the site above before doing this.

The difference between that command and the usual sif commands used to generate a local site resides on setting the sifnethost option. By setting the sifnethost option the command is translated into a remote instruction, in this case the command that creates the remote target site.

If the -v option is ommitted, the remote command is shown but not run. It shows

> D:\BIN\plink.exe -ssh -2  -l my_user_login -i D:\LOCAL\sf_private_key.ppk myproject.sourceforge.net " php /home/users/m/my/myuser/myproject/sifcmd.php  sif_sip_build_site --dev --expandsiteparams --noupd --emptypod --force --complete --dbgtype=both --siburi=./sibs/sib_drupal5_sif_sif.xml --site_host=myproject.sourceforge.net --site_name=mysite --site_pod=d501 --db_server=mysql4-m --db_database=db_myproject --db_prefix=d501_ --db_dbtype=mysql --db_tabletype=myisam --db_username=db_admin --db_password=db_admin_pwd --site_admin_username=my_user_login --site_admin_password=my_user_pwd --dir_fs_document_root=/home/groups/m/my/myproject/htdocs/ --encodedpwd=1 --sifnetconfigfile=../sifbuilder_local/sifnet_config.php --sifpacksdir=../sifbuilder_local/packages --siflochost=devhost   drupal5_core_drupal  drupal5_theme_sif  drupal5_mod_views  drupal5_mod_site_map  drupal5_script_locals "

The sifbuilder site was created and is regularly maintained following these steps. if not automatic, the process is systematic. And every systematic process requires a table of definitions.