Developer Guide edits.

Converted the guide to use the "r/2018.10" branch rather than the
default "master" branch.  Cleaned up a lot writing with general
edits.  Applied consistent formatting across various items such
as filenames, pathnames, etc.  I went through the entire file and
did some clean-up work.

Change-Id: If53ab66187307c087f5ae755f44d6080d1aa8245
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
This commit is contained in:
Scott Rifenbark 2018-10-26 14:32:26 -07:00 committed by Abraham Arce
parent 2399131b01
commit 8c1361ebd3
1 changed files with 193 additions and 163 deletions

View File

@ -4,8 +4,8 @@
Developer Guide Developer Guide
=============== ===============
This section contains the steps for building a StarlingX ISO from Master This section contains the steps for building a StarlingX ISO from
branch. the "r/2018.10" branch.
------------ ------------
Requirements Requirements
@ -46,7 +46,7 @@ Development Environment Setup
----------------------------- -----------------------------
This section describes how to set up a StarlingX development system on a This section describes how to set up a StarlingX development system on a
workstation computer. After completing these steps, you will be able to workstation computer. After completing these steps, you can
build a StarlingX ISO image on the following Linux distribution: build a StarlingX ISO image on the following Linux distribution:
- Ubuntu 16.04 LTS 64-bit - Ubuntu 16.04 LTS 64-bit
@ -55,8 +55,8 @@ build a StarlingX ISO image on the following Linux distribution:
Update Your Operating System Update Your Operating System
**************************** ****************************
Before proceeding with the build, ensure your OS is up to date. Youll Before proceeding with the build, ensure your Ubuntu distribution is up to date.
first need to update the local database list of available packages: You first need to update the local database list of available packages:
.. code:: sh .. code:: sh
@ -70,13 +70,14 @@ Installation Requirements and Dependencies
Git Git
^^^ ^^^
1. Install the required packages in an Ubuntu host system with: 1. Install the required packages on the Ubuntu host system:
.. code:: sh .. code:: sh
$ sudo apt-get install make git curl $ sudo apt-get install make git curl
2. Make sure to setup your identity 2. Make sure to setup your identity with the following two commands.
Be sure to provide your actual name and email address:
.. code:: sh .. code:: sh
@ -87,7 +88,7 @@ Git
Docker CE Docker CE
^^^^^^^^^ ^^^^^^^^^
3. Install the required Docker CE packages in an Ubuntu host system. See 3. Install the required Docker CE packages in the Ubuntu host system. See
`Get Docker CE for `Get Docker CE for
Ubuntu <https://docs.docker.com/install/linux/docker-ce/ubuntu/#os-requirements>`__ Ubuntu <https://docs.docker.com/install/linux/docker-ce/ubuntu/#os-requirements>`__
for more information. for more information.
@ -96,18 +97,18 @@ Docker CE
Android Repo Tool Android Repo Tool
^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^
4. Install the required Android Repo Tool in an Ubuntu host system. Follow 4. Install the required Android Repo Tool in the Ubuntu host system. Follow
the 2 steps in "Installing Repo" section from `Installing the steps in the `Installing
Repo <https://source.android.com/setup/build/downloading#installing-repo>`__ Repo <https://source.android.com/setup/build/downloading#installing-repo>`__
to have Andriod Repo Tool installed. section.
********************** **********************
Install Public SSH Key Install Public SSH Key
********************** **********************
#. Follow these instructions on GitHub to `Generate a Public SSH #. Follow these instructions on GitHub to `Generate a Public SSH
Key <https://help.github.com/articles/connecting-to-github-with-ssh>`__ Key <https://help.github.com/articles/connecting-to-github-with-ssh>`__.
and then upload your public key to your GitHub and Gerrit account Then upload your public key to your GitHub and Gerrit account
profiles: profiles:
- `Upload to - `Upload to
@ -119,20 +120,21 @@ Install Public SSH Key
Install stx-tools Project Install stx-tools Project
************************* *************************
#. Under your $HOME directory, clone the <stx-tools> project #. While in your $HOME directory, create a local copy of the stx-tools project
based on the "r/2018.10" branch:
.. code:: sh .. code:: sh
$ cd $HOME $ cd $HOME
$ git clone https://git.starlingx.io/stx-tools $ git clone -b r/2018.10 https://git.starlingx.io/stx-tools
**************************** ****************************
Create a Workspace Directory Create a Workspace Directory
**************************** ****************************
#. Create a *starlingx* workspace directory on your workstation #. Create a *starlingx* workspace directory on your system.
computer. Usually, youll want to create it somewhere under your Best practices dictate creating the workspace directory
users home directory. in your $HOME directory:
.. code:: sh .. code:: sh
@ -148,16 +150,16 @@ This section describes how to build the CentOS Mirror Repository.
Setup Repository Docker Container Setup Repository Docker Container
********************************* *********************************
Run the following commands under a terminal identified as "One". | Run the following commands under a terminal identified as "**One**".
#. Navigate to the *<$HOME/stx-tools>/centos-mirror-tool* project #. Navigate to the *$HOME/stx-tools/centos-mirror-tool* project
directory: directory:
.. code:: sh .. code:: sh
$ cd $HOME/stx-tools/centos-mirror-tools/ $ cd $HOME/stx-tools/centos-mirror-tools/
#. If necessary you might have to set http/https proxy in your #. If necessary, set the http/https proxy in your
Dockerfile before building the docker image. Dockerfile before building the docker image.
.. code:: sh .. code:: sh
@ -167,34 +169,35 @@ Run the following commands under a terminal identified as "One".
ENV ftp_proxy " http://your.actual_ftp_proxy.com:your_port " ENV ftp_proxy " http://your.actual_ftp_proxy.com:your_port "
RUN echo " proxy=http://your-proxy.com:port " >> /etc/yum.conf RUN echo " proxy=http://your-proxy.com:port " >> /etc/yum.conf
#. Build your *<user>:<tag>* base container image with **e.g.** #. Build your *<user>:<tag>* base container image. For example, use
*user:centos-mirror-repository* *user:centos-mirror-repository*:
.. code:: sh .. code:: sh
$ docker build --tag $USER:centos-mirror-repository --file Dockerfile . $ docker build --tag $USER:centos-mirror-repository --file Dockerfile .
#. Launch a *<user>* docker container using previously created Docker #. Launch a *<user>* docker container using the previously created Docker
base container image *<user>:<tag>* **e.g.** base container image *<user>:<tag>* (e.g.
*-centos-mirror-repository*. As /localdisk is defined as the workdir *-centos-mirror-repository*). As /localdisk is defined as the workdir
of the container, the same folder name should be used to define the of the container, the same folder name should be used to define the
volume. The container will start to run and populate a logs and volume. The container will start to run and populate the output folders
output folders in this directory. The container shall be run from the *logs* and *output* in this directory. The container is run from the
same directory where the other scripts are stored. same directory where the other scripts are stored:
.. code:: sh .. code:: sh
$ docker run -itd --name $USER-centos-mirror-repository --volume $(pwd):/localdisk $USER:centos-mirror-repository $ docker run -itd --name $USER-centos-mirror-repository --volume $(pwd):/localdisk $USER:centos-mirror-repository
**Note**: the above command will create the container in background, **NOTE**: The above command creates the container in the background.
this mean that you need to attach it manually. The advantage of this With the container running in the background, you must attach to it
is that you can enter/exit from the container many times as you want. manually. An advantage of attaching to the container manually is that
you can enter and exit from the container as many times as you want.
***************** *****************
Download Packages Download Packages
***************** *****************
#. Attach to the docker repository previously created #. Attach to the docker repository previously created:
:: ::
@ -208,8 +211,8 @@ Download Packages
# bash download_mirror.sh # bash download_mirror.sh
#. Monitor the download of packages until it is complete. When download #. Monitor the download of packages until it is complete. When the download
is complete, the following message is displayed: is complete, the following message appears:
:: ::
@ -225,24 +228,25 @@ Download Packages
Verify Packages Verify Packages
*************** ***************
#. Verify there are no missing or failed packages: #. Verify no missing or failed packages exist:
:: ::
# cat logs/*_missing_*.log # cat logs/*_missing_*.log
# cat logs/*_failmove_*.log # cat logs/*_failmove_*.log
#. In case there are missing or failed ones due to network instability #. In case missing or failed packages do exist, which is usually caused by
(or timeout), you should download them manually, to assure you get network instability (or timeout), you need to download the packages
all RPMs listed in manually.
**rpms_3rdparties.lst**/**rpms_centos.lst**/**rpms_centos3rdparties.lst**. Doing so assures you get all RPMs listed in
*rpms_3rdparties.lst*/*rpms_centos.lst*/*rpms_centos3rdparties.lst*.
****************** ******************
Packages Structure Packages Structure
****************** ******************
The following is a general overview of the packages structure that you The following is a general overview of the packages structure resulting
will have after having downloaded the packages from downloading the packages:
:: ::
@ -278,7 +282,7 @@ as "**Two**", run the following commands:
#. Copy the built CentOS Mirror Repository built under #. Copy the built CentOS Mirror Repository built under
*$HOME/stx-tools/centos-mirror-tool* to the *$HOME/starlingx/mirror/* *$HOME/stx-tools/centos-mirror-tool* to the *$HOME/starlingx/mirror/*
workspace directory. workspace directory:
.. code:: sh .. code:: sh
@ -293,25 +297,25 @@ Create StarlingX Packages
Setup Building Docker Container Setup Building Docker Container
******************************* *******************************
#. From terminal identified as "**Two**", create the workspace folder #. From the terminal identified as "**Two**", create the workspace folder:
.. code:: sh .. code:: sh
$ mkdir -p $HOME/starlingx/workspace $ mkdir -p $HOME/starlingx/workspace
#. Navigate to the '' $HOME/stx-tools'' project directory: #. Navigate to the *$HOME/stx-tools* project directory:
.. code:: sh .. code:: sh
$ cd $HOME/stx-tools $ cd $HOME/stx-tools
#. Copy your git options to "toCopy" folder #. Copy your Git options to *toCopy* folder:
.. code:: sh .. code:: sh
$ cp ~/.gitconfig toCOPY $ cp ~/.gitconfig toCOPY
#. Create a *<localrc>* file #. Create a *localrc* file:
.. code:: sh .. code:: sh
@ -323,8 +327,8 @@ Setup Building Docker Container
HOST_MIRROR_DIR=$HOME/starlingx/mirror HOST_MIRROR_DIR=$HOME/starlingx/mirror
EOF EOF
#. If necessary you might have to set http/https proxy in your #. If necessary, you might have to set the http/https proxy in your
Dockerfile.centos73 before building the docker image. *Dockerfile.centos73* file before building the docker image:
.. code:: sh .. code:: sh
@ -335,9 +339,16 @@ Setup Building Docker Container
echo -e "export http_proxy=$http_proxy\nexport https_proxy=$https_proxy\n\ echo -e "export http_proxy=$http_proxy\nexport https_proxy=$https_proxy\n\
export ftp_proxy=$ftp_proxy" >> /root/.bashrc export ftp_proxy=$ftp_proxy" >> /root/.bashrc
#. Base container setup If you are running in fedora system, you will #. Set up the base container:
see " .makeenv:88: \**\* missing separator. Stop. " error, to
continue : If you are using a Fedora distribution, you will
see the following error:
.. code:: sh
.makeenv:88: \**\* missing separator. Stop.
To continue, do the following:
- delete the functions define in the .makeenv ( module () { ... } ) - delete the functions define in the .makeenv ( module () { ... } )
- delete the line 19 in the Makefile and ( NULL := $(shell bash -c - delete the line 19 in the Makefile and ( NULL := $(shell bash -c
@ -347,19 +358,19 @@ Setup Building Docker Container
$ make base-build $ make base-build
#. Build container setup #. Set up the build container:
.. code:: sh .. code:: sh
$ make build $ make build
#. Verify environment variables #. Verify environment variables:
.. code:: sh .. code:: sh
$ bash tb.sh env $ bash tb.sh env
#. Build container run #. Run the build container:
.. code:: sh .. code:: sh
@ -375,35 +386,44 @@ Setup Building Docker Container
Download Source Code Repositories Download Source Code Repositories
********************************* *********************************
#. From terminal identified as "Two", now inside the Building Docker #. From the terminal identified as "**Two**", which is now inside the
container, Internal environment Building Docker container, start the internal environment:
.. code:: sh .. code:: sh
$ eval $(ssh-agent) $ eval $(ssh-agent)
$ ssh-add $ ssh-add
#. Repo init #. Use the repo tool to create a local clone of the stx-manifest
Git repository based on the "r/2018.10" branch:
.. code:: sh .. code:: sh
$ cd $MY_REPO_ROOT_DIR $ cd $MY_REPO_ROOT_DIR
$ repo init -u https://git.starlingx.io/stx-manifest -m default.xml -b r/2018.10
**NOTE:** To use the "repo" command to clone the stx-manifest repository and
check out the "master" branch, omit the "-b r/2018.10" option.
Following is an example:
.. code:: sh
$ repo init -u https://git.starlingx.io/stx-manifest -m default.xml $ repo init -u https://git.starlingx.io/stx-manifest -m default.xml
#. Repo sync #. Synchronize the repository:
.. code:: sh .. code:: sh
$ repo sync -j`nproc` $ repo sync -j`nproc`
#. Tarballs Repository #. Create a tarballs repository:
.. code:: sh .. code:: sh
$ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/ $ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/
Alternatively you can run the populate_downloads.sh script to copy Alternatively, you can run the "populate_downloads.sh" script to copy
the tarballs instead of using a symlink. the tarballs instead of using a symlink:
.. code:: sh .. code:: sh
@ -411,7 +431,7 @@ Download Source Code Repositories
Outside the container Outside the container
#. From another terminal identified as "Three", Mirror Binaries #. From another terminal identified as "**Three**", create mirror binaries:
.. code:: sh .. code:: sh
@ -424,30 +444,32 @@ Download Source Code Repositories
Build Packages Build Packages
************** **************
#. Back to the Building Docker container, terminal identified as #. Go back to the terminal identified as "**Two**", which is the Building Docker container.
"**Two**"
#. **Temporal!** Build-Pkgs Errors Be prepared to have some missing / #. **Temporal!** Build-Pkgs Errors. Be prepared to have some missing /
corrupted rpm and tarball packages generated during corrupted rpm and tarball packages generated during
`Build the CentOS Mirror Repository`_ which will make the next step `Build the CentOS Mirror Repository`_, which will cause the next step
to fail, if that happens please download manually those missing / to fail. If that step does fail, manually download those missing /
corrupted packages. corrupted packages.
#. **Update the symbolic links**
#. Update the symbolic links:
.. code:: sh .. code:: sh
$ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/ $ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
#. Build-Pkgs #. Build the packages:
.. code:: sh .. code:: sh
$ build-pkgs $ build-pkgs
#. **Optional!** Generate-Cgcs-Tis-Repo #. **Optional!** Generate-Cgcs-Tis-Repo:
This step is optional but will improve performance on subsequent
While this step is optional, it improves performance on subsequent
builds. The cgcs-tis-repo has the dependency information that builds. The cgcs-tis-repo has the dependency information that
sequences the build order; To generate or update the information the sequences the build order. To generate or update the information, you
following command needs to be executed after building modified or new need to execute the following command after building modified or new
packages. packages.
.. code:: sh .. code:: sh
@ -458,7 +480,7 @@ Build Packages
Build StarlingX ISO Build StarlingX ISO
------------------- -------------------
#. Build-Iso #. Build the image:
.. code:: sh .. code:: sh
@ -468,26 +490,27 @@ Build StarlingX ISO
Build Installer Build Installer
--------------- ---------------
To get your StarlingX ISO ready to use, you will need to create the init To get your StarlingX ISO ready to use, you must create the initialization
files that will be used to boot the ISO as well to boot additional files used to boot the ISO, additional controllers, and compute nodes.
controllers and compute nodes. Note that this procedure only is needed
in your first build and every time the kernel is upgraded.
Once you had run build-iso, run: **NOTE:** You only need this procedure during your first build and
every time you upgrade the kernel.
After running "build-iso", run:
.. code:: sh .. code:: sh
$ build-pkgs --installer $ build-pkgs --installer
This will build *rpm* and *anaconda* packages. Then run: This builds *rpm* and *anaconda* packages. Then run:
.. code:: sh .. code:: sh
$ update-pxe-network-installer $ update-pxe-network-installer
The *update-pxe-network-installer* covers the steps detailed in The *update-pxe-network-installer* covers the steps detailed in
*$MY_REPO/stx/stx-metal/installer/initrd/README*. This script will *$MY_REPO/stx/stx-metal/installer/initrd/README*. This script
create three files on creates three files on
*/localdisk/loadbuild///pxe-network-installer/output*. */localdisk/loadbuild///pxe-network-installer/output*.
:: ::
@ -496,7 +519,7 @@ create three files on
new-squashfs.img new-squashfs.img
new-vmlinuz new-vmlinuz
Then, rename them to: Rename the files as follows:
:: ::
@ -504,16 +527,15 @@ Then, rename them to:
squashfs.img-stx-0.2 squashfs.img-stx-0.2
vmlinuz-stx-0.2 vmlinuz-stx-0.2
There are two ways to use these files: Two ways exist for using these files:
#. Store the files in the */import/mirror/CentOS/stx-installer/* folder #. Store the files in the */import/mirror/CentOS/stx-installer/* folder
for future use. for future use.
#. Store it in an arbitrary location and modify the #. Store the files in an arbitrary location and modify the
*$MY_REPO/stx/stx-metal/installer/pxe-network-installer/centos/build_srpm.data* *$MY_REPO/stx/stx-metal/installer/pxe-network-installer/centos/build_srpm.data*
file to point to these files. file to point to these files.
Now, the *pxe-network-installer* package needs to be recreated and the Recreate the *pxe-network-installer* package and rebuild the image:
ISO regenerated.
.. code:: sh .. code:: sh
@ -521,18 +543,18 @@ ISO regenerated.
$ build-pkgs pxe-network-installer $ build-pkgs pxe-network-installer
$ build-iso $ build-iso
Now your ISO should be able to boot. Your ISO image should be able to boot.
**************** ****************
Additional Notes Additional Notes
**************** ****************
- In order to get the first boot working this complete procedure needs - In order to get the first boot working, this complete procedure needs
to be done. However, once the init files are created, these can be to be done. However, once the init files are created, these can be
stored in a shared location where different developers can make use stored in a shared location where different developers can make use
of them. Updating these files is not a frequent task and should be of them. Updating these files is not a frequent task and should be
done whenever the kernel is upgraded. done whenever the kernel is upgraded.
- StarlingX is in active development, so it is possible that in the - StarlingX is in active development. Consequently, it is possible that in the
future the **0.2** version will change to a more generic solution. future the **0.2** version will change to a more generic solution.
--------------- ---------------
@ -543,23 +565,25 @@ Build Avoidance
Purpose Purpose
******* *******
Greatly reduce build times after a repo sync for designers working Greatly reduce build times after using "repo" to syncronized a local
within a regional office. Starting from a new workspace, build-pkgs repository with an upstream source (i.e. "repo sync").
typically requires 3+ hours. Build avoidance typically reduces this step Build avoidance works well for designers working
to ~20min within a regional office. Starting from a new workspace, "build-pkgs"
typically requires three or more hours to complete. Build avoidance
reduces this step to approximately 20 minutes.
*********** ***********
Limitations Limitations
*********** ***********
- Little or no benefit for designers who refresh a pre-existing - Little or no benefit for designers who refresh a pre-existing
workspace at least daily. (download_mirror.sh, repo sync, workspace at least daily (e.g. download_mirror.sh, repo sync,
generate-cgcs-centos-repo.sh, build-pkgs, build-iso). In these cases generate-cgcs-centos-repo.sh, build-pkgs, build-iso). In these cases,
an incremental build (reuse of same workspace without a 'build-pkgs an incremental build (i.e. reuse of same workspace without a "build-pkgs
--clean') is often just as efficient. --clean") is often just as efficient.
- Not likely to be useful to solo designers, or teleworkers that wish - Not likely to be useful to solo designers, or teleworkers that wish
to compile on there home computers. Build avoidance downloads build to compile on using their home computers. Build avoidance downloads build
artifacts from a reference build, and WAN speeds are generally to artifacts from a reference build, and WAN speeds are generally too
slow. slow.
***************** *****************
@ -568,26 +592,28 @@ Method (in brief)
#. Reference Builds #. Reference Builds
- A server in the regional office performs a regular (daily?), - A server in the regional office performs regular (e.g. daily)
automated builds using existing methods. Call these the reference automated builds using existing methods. These builds are called
builds. "reference builds".
- The builds are timestamped, and preserved for some time. (a few - The builds are timestamped and preserved for some time (i.e. a
weeks) number of weeks).
- A build CONTEXT is captured. This is a file produced by build-pkgs - A build CONTEXT, which is a file produced by "build-pkgs"
at location '$MY_WORKSPACE/CONTEXT'. It is a bash script that can at location *$MY_WORKSPACE/CONTEXT*, is captured. It is a bash script that can
cd to each and every git and checkout the SHA that contributed to cd to each and every Git and checkout the SHA that contributed to
the build. the build.
- For each package built, a file shall capture he md5sums of all the - For each package built, a file captures the md5sums of all the
source code inputs to the build of that package. These files are source code inputs required to build that package. These files are
also produced by build-pkgs at location also produced by "build-pkgs" at location
'$MY_WORKSPACE//rpmbuild/SOURCES//srpm_reference.md5'. *$MY_WORKSPACE//rpmbuild/SOURCES//srpm_reference.md5*.
- All these build products are accessible locally (e.g. a regional - All these build products are accessible locally (e.g. a regional
office) via rsync (other protocols can be added later) office) using "rsync".
**NOTE:** Other protocols can be added later.
#. Designers #. Designers
- Request a build avoidance build. Recommended after you have just - Request a build avoidance build. Recommended after you have
done a repo sync. e.g. done synchronized the repository (i.e. "repo sync").
:: ::
@ -596,11 +622,11 @@ Method (in brief)
populate_downloads.sh populate_downloads.sh
build-pkgs --build-avoidance build-pkgs --build-avoidance
- Additional arguments, and/or environment variables, and/or a - Use combinations of additional arguments, environment variables, and a
config file unique to the regional office, are used to specify a URL configuration file unique to the regional office to specify an URL
to the reference builds. to the reference builds.
- Using a config file to specify location of your reference build - Using a configuration file to specify the location of your reference build:
:: ::
@ -621,14 +647,16 @@ Method (in brief)
BUILD_AVOIDANCE_DIR="/localdisk/loadbuild/jenkins/StarlingX_Reference_Build" BUILD_AVOIDANCE_DIR="/localdisk/loadbuild/jenkins/StarlingX_Reference_Build"
EOF EOF
- Using command line args to specify location of your reference - Using command-line arguments to specify the location of your reference
build build:
:: ::
build-pkgs --build-avoidance --build-avoidance-dir /localdisk/loadbuild/jenkins/StarlingX_Reference_Build --build-avoidance-host stx-builder.mycompany.com --build-avoidance-user jenkins build-pkgs --build-avoidance --build-avoidance-dir /localdisk/loadbuild/jenkins/StarlingX_Reference_Build --build-avoidance-host stx-builder.mycompany.com --build-avoidance-user jenkins
- Prior to your build attempt, you need to accept the host key. This will prevent rsync failures on a yes/no prompt. (you should only have to do this once) - Prior to your build attempt, you need to accept the host key.
Doing so prevents "rsync" failures on a "yes/no" prompt.
You only have to do this once.
:: ::
@ -638,23 +666,25 @@ Method (in brief)
fi fi
- build-pkgs will: - "build-pkgs" does the following:
- From newest to oldest, scan the CONTEXTs of the various - From newest to oldest, scans the CONTEXTs of the various
reference builds. Select the first (most recent) context which reference builds. Selects the first (i.e. most recent) context that
satisfies the following requirement. For every git, the SHA satisfies the following requirement: every Git the SHA
specified in the CONTEXT is present. specifies in the CONTEXT is present.
- The selected context might be slightly out of date, but not by - The selected context might be slightly out of date, but not by
more than a day (assuming daily reference builds). more than a day. This assumes daily reference builds are run.
- If the context has not been previously downloaded, then - If the context has not been previously downloaded, then
download it now. Meaning download select portions of the download it now. This means you need to download select portions of the
reference build workspace into the designer's workspace. This reference build workspace into the designer's workspace. This
includes all the SRPMS, RPMS, MD5SUMS, and misc supporting includes all the SRPMS, RPMS, MD5SUMS, and miscellaneous supporting
files. (~10 min over office LAN) files. Downloading these files usually takes about 10 minutes
- The designer may have additional commits not present in the over an office LAN.
reference build, or uncommitted changes. Affected packages will - The designer could have additional commits or uncommitted changes
identified by the differing md5sum's, and the package is not present in the reference builds. Affected packages are
re-built. (5+ min, depending on what packages have changed) identified by the differing md5sum's. In these cases, the packages
are re-built. Re-builds usually take five or more minutes,
depending on the packages that have changed.
- What if no valid reference build is found? Then build-pkgs will fall - What if no valid reference build is found? Then build-pkgs will fall
back to a regular build. back to a regular build.
@ -664,16 +694,16 @@ Reference Builds
**************** ****************
- The regional office implements an automated build that pulls the - The regional office implements an automated build that pulls the
latest StarlingX software and builds it on a regular basis. e.g. a latest StarlingX software and builds it on a regular basis (e.g.
daily. Perhaps implemented by Jenkins, cron, or similar tools. daily builds). Jenkins, cron, or similar tools can trigger these builds.
- Each build is saved to a unique directory, and preserved for a time - Each build is saved to a unique directory, and preserved for a time
that is reflective of how long a designer might be expected to work that is reflective of how long a designer might be expected to work
on a private branch without syncronizing with the master branch. e.g. on a private branch without syncronizing with the master branch.
2 weeks. This takes about two weeks.
- The MY_WORKSPACE directory for the build shall have a common root - The *MY_WORKSPACE* directory for the build shall have a common root
directory, and a leaf directory that is a sortable time stamp. Suggested directory, and a leaf directory that is a sortable time stamp. The
format YYYYMMDDThhmmss. e.g. suggested format is *YYYYMMDDThhmmss*.
.. code:: sh .. code:: sh
@ -683,24 +713,25 @@ Reference Builds
MY_WORKSPACE=${BUILD_AVOIDANCE_DIR}/${BUILD_TIMESTAMP} MY_WORKSPACE=${BUILD_AVOIDANCE_DIR}/${BUILD_TIMESTAMP}
- Designers can access all build products over the internal network of - Designers can access all build products over the internal network of
the regional office. The current prototype employs rsync. Other the regional office. The current prototype employs "rsync". Other
protocols that can efficiently share/copy/transfer large directories protocols that can efficiently share, copy, or transfer large directories
of content can be added as needed. of content can be added as needed.
^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^
Advanced Usage Advanced Usage
^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^
Can the reference build itself use build avoidance? Yes Can the reference build itself use build avoidance? Yes it can.
Can it reference itself? Yes. Can it reference itself? Yes it can.
In either case we advise caution. To protect against any possible In both these cases, caution is advised. To protect against any possible
'divergence from reality', you should limit how many steps removed a 'divergence from reality', you should limit how many steps you remove
build avoidance build is from a full build. a build avoidance build from a full build.
Suppose we want to implement a self referencing daily build, except Suppose we want to implement a self-referencing daily build in an
that a full build occurs every Saturday. To protect ourselves from a environment where a full build already occurs every Saturday.
build failure on Saturday we also want a limit of 7 days since last To protect ourselves from a
full build. You build script might look like this ... build failure on Saturday we also want a limit of seven days since
the last full build. Your build script might look like this ...
:: ::
@ -779,24 +810,23 @@ full build. You build script might look like this ...
fi fi
... ...
One final wrinkle. A final note....
We can ask build avoidance to preferentially use the full build day To use the full build day as your avoidance build reference point,
rather than the most recent build, as the reference point of the next modify the "build-pkgs" commands above to use "--build-avoidance-day ",
avoidance build via use of '--build-avoidance-day '. e.g. substitute as shown in the following two examples:
this line into the above.
:: ::
build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER --build-avoidance-day $FULL_BUILD_DAY build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER --build-avoidance-day $FULL_BUILD_DAY
# or perhaps, with a bit more shuffling of the above script. # Here is another example with a bit more shuffling of the above script.
build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER --build-avoidance-day $LAST_FULL_BUILD_DAY build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER --build-avoidance-day $LAST_FULL_BUILD_DAY
The advantage is that our build is never more than one step removed The advantage is that our build is never more than one step removed
from a full build (assuming the full build was successful). from a full build. This assumes the full build was successful.
The disadvantage is that by end of week the reference build is getting The disadvantage is that by the end of the week, the reference build is getting
rather old. During active weeks, builds times might be approaching rather old. During active weeks, build times could approach build times for
that of a full build. full builds.