Story: 2003389
Task: 26436
new systemd depends on lz4. And systemd is included in squashfs, so need to add lz4 to squashfs also.
Removed rpms and rt kernel from the image, since they are not used.
Change-Id: Ic0d8be91b747369bb9a068f41e7c4785d29f8881
Signed-off-by: slin14 <shuicheng.lin@intel.com>
the scripts contained hardcoded references to resources
that are not visible outside of the environment
where the scripts were originally created and used
The scripts sign-rpms was also updated with the original version
that was intended to be submitted. The initial submission contained
the wrong version.
Closes-Bug: #1791343
Change-Id: I8ce5884ad75156d3730cf30a451051d32445e136
Signed-off-by: Paul-Emile Element <Paul-Emile.Element@windriver.com>
Story: 2003680
Task: 26204
It is due to $DIR doesn't exist, and cause clean cmd fail and exit.
$DIR has be renamed to $BUILD_PKGS_PARALLEL_DIR. Remove the obsolete line.
Change-Id: I1b6f55fc7d08ab58cdf89ed81b5eef4ea0de7199
Signed-off-by: slin14 <shuicheng.lin@intel.com>
Fix below issues:
E003 Indent not multiple of 4
E011 Then keyword is not on same line as if or elif keyword
E020: Function declaration not in format "^function name {$":
'function check_all_explicit_deps_installed'
add ignore in tox env
E006 Line too long
E010: Do not on same line as command
Change-Id: I13fb83452324c0db74c485f304827a6c9aacf9ae
Story: 2003370
Task: 24431
Signed-off-by: Sun Austin <austin.sun@intel.com>
For Python 3.0, "oldoctinteger" isn't supported already, and an exception
will be raised if a literal has a leading "0" and a second character
which is a digit.
Story: 2003595
Task: 25735
Change-Id: Iec0146f9287f9eab95984608528f6535c8301cf3
Signed-off-by: zhangyangyang <zhangyangyang@unionpay.com>
During the build-srpm phase of build-pkgs
- for each package
- determine the unique sorted list of all input files
- calculate md5 sums of all input files and save to file
$MY_WORKSPACE/std/rpmbuild/SOURCES/<pkg-name>/srpm_input.md5
- Compare vs reference md5sums of previous builds
$MY_WORKSPACE/std/rpmbuild/SOURCES/<pkg-name>/srpm_reference.md5
- If different, rebuild package
- If build successful, cp srpm_input.md5 -> srpm_reference.md5
Build-rpms still uses timestamps, but we will switch from
'find --newer' to 'find -newermm' to identify updates files
which is compatable with how rsync sets the timestamps in the
next build-avoidance update.
Finally bring build-rpms-serial up to date, relative to
build-rpms-parallel, with respect to spec cache, signal handling,
process cleanup, and mock environment cleanup. The remaining
delta between the two should reflect just the differences for
parallel building.
Story: 2002835
Task: 22754
Change-Id: I19c5594c84f026861e913a3af8af7dd0dc676ced
Signed-off-by: Scott Little <scott.little@windriver.com>
First step in build avoidance is the creation of a reference build.
We need to capture the state of all the git trees that contribute
to the reference build.
This update will save the git context to $MY_WORKSPACE/CONTEXT
The context file format is actually a shell script that can be run to
recreate the build context.
repo init ...
repo sync
cd $MY_REPO/..
source .../CONTEXT
Sample CONTEXT file ....
(cd ./cgcs-root && git checkout -f 12a1595941)
(cd ./cgcs-root/stx/git/calico && git checkout -f 12a1595941)
(cd ./cgcs-root/stx/git/ceilometer && git checkout -f 12a1595941)
...
(cd ./stx-tools && git checkout -f 12a1595941)
(cd ./.repo/manifests && git checkout -f 12a1595941)
(cd ./.repo/repo && git checkout -f 12a1595941)
This update also the git_list function, which populates the previously
introduced GIT_LIST variable, and can be used in several other code
paths where we want to find all gits in a directory tree.
Tested build/clean/edit parallel/serial
Story: 2002835
Task: 22754
Change-Id: I5009b8a360bdb4a03e5a4e83430b7f2ef135115e
Signed-off-by: Scott Little <scott.little@windriver.com>
Serial build scripts use 'rm' without leading '\' which
is used to prevent user defined aliases from interfering
with the rm command.
Also convert from use of `` to $() to run subshell commands.
This will match the coding standard used in the parallel build
scripts.
Change-Id: I903d753c9ebbf9fc2f03a43cf8b27cbb6e2ecba3
Story: 2002835
Task: 24519
Signed-off-by: Scott Little <scott.little@windriver.com>
Add function and line number to error logs.
Log git commands that are issured when using --edit.
Replace global DIR variable with a more unique variable name.
DIR is used in many scripts that get sourced, possibly overwriting
the value in the calling script.
Make sure scripts won't fail if a redundant --parallel or --serial
argurement is seen.
Source files once at top of script, rather than in multiple places.
Source build_srpm.data once per package, and handle case where
build_srpm.data file is missing.
Fix handling of '<build-cmd> --installer <pkg-name>'.
Add handling of 'BUILD_IS_BIG' and 'UILD_IS_SLOW' to serial build.
Fix handling of packages that are built twice, once with a -rt
extension, during srpm audit.
Fix some exit code handling.
Prioritize Name over Service when searching for a package.
Delete some commented out code paths.
Change-Id: Ib5153cecf7b586d68aa382d382bc5a1a03a6b326
Story: 2002835
Task: 24519
Signed-off-by: Scott Little <scott.little@windriver.com>
Problem:
The kernel is sporadically reporting build failures due to
'no space on device' when compiled in a 10 GB tmpfs.
Solution:
Two parts:
1) Increase required size to 11 GB in the build_srpm.data.
2) Modify build system to allow alocation of a 11 GB tmpfs.
Change-Id: I4e1ee60a3522c3f13cf39b3115aa6881185735ed
Story: 2002835
Task: 24519
Signed-off-by: Scott Little <scott.little@windriver.com>
This is an enhancement request to add the screen package to the system
Story: 2003061
Task: 23100
Depends-on: https://review.openstack.org/#/c/595249/
Change-Id: I355d517ba0d0392d40fe78991798ddf6e5d16fde
Signed-off-by: Paul-Emile Element <Paul-Emile.Element@windriver.com>
Problem:
In the absence of a dependency cache, build-pkgs has
not information on which packages should be built first.
All it can do is try to build each package in a arbitrary order.
If the build fails, it is set aside and will be retired on
the next iteration. Builds fail when a BuildRequired package
can't be found, because it hasn't been built yet.
This was not an issue when cgcs-tis-repo, the home of the cache,
was stored in a git.
Solution:
Have build-pkgs invoke create_dependancy_cache.py if the
cache is not present.
Note: We only run this automatically once. If the user is
adding packages, or modifying the BuildRequires or Requires of
an existing package, they should re-run create_dependancy_cache.py
manually.
Change-Id: Id63e42903d9c0884470a990fb62fe1bafacd1849
Story: 2002835
Task: 24519
Signed-off-by: Scott Little <scott.little@windriver.com>
Background:
drbd-kernel compiles from source directory drbd.
drbd compiles from source directory drbd-tools.
In one cleanup path the build tools incorrectly assume
package name = source directory name.
As a result "build-pkgs --clean drbd" was incorrectly deleting
a temporary directory named 'drbd' that was in fact the temporary
directory for package drbd-kernel.
For incremental parallel builds this can lead to build failures
if drbd is cleaned in paraller with drbd-kernel being built.
Change-Id: Ia0f5e9e226ce3ea0f771c2a4dc899a2b606f561f
Story: 2002835
Task: 24519
Signed-off-by: Scott Little <scott.little@windriver.com>
Problem:
build-guest fails for hard to determine reason.
Root cause, user was not running the command under the correct group.
They need to be in the mock group to run mock successfully.
Solution:
Verify that the command is being run under the correct group id.
Note: 'cgts' is our current alias for the 'mock' group. Should probably
be renamed as stx-build in a future update.
Story: 2002835
Task: 24519
Change-Id: I658800cbae001f5cb0fc81bff1ed1ab159f6f6a3
Signed-off-by: Scott Little <scott.little@windriver.com>
Depends on self creates false loops and results in sub-optimal
compile orders.
Change-Id: I429b79fed58d0edea20df68a883caafbddf8a07c
Story: 2002835
Task: 24519
Signed-off-by: Scott Little <scott.little@windriver.com>
The build host's yum.conf should not be allowed to influence the
iso we compile.
Story: 2002835
Task: 24519
Change-Id: I202ddbbe64d2dafc198fc7055e360f1c4b047bcc
Signed-off-by: Scott Little <scott.little@windriver.com>
Currently compiling a new package and adding it
to the iso still requires a multi-git update because
image.inc is a single centralized file in the root git.
It would be better to allow a single git update to add
a package. Too allow this, image.inc must be split across
the git repos and the build tools must be changed to
read/merge those files to arrive at the final package list.
Current scheme is to name the image.inc files using this
schema.
${distro}_${build_target}_image_${build_type}.inc
distro = centos, ...
build_target = iso, guest ...
build_type = std, rt ...
Traditionally build_type=std is omitted from config files,
so we instread use ${distro}_${build_target}_image.inc.
Story: 2003447
Task: 24649
Depends-On: Ib39b8063e7759842ba15330c68503bfe2dea6e20
Change-Id: I37c63b86a7b4e420ca5ee1cdd343f9ad78c5e36a
Signed-off-by: Scott Little <scott.little@windriver.com>
Currently compiling a new package and adding it
to the iso still requires a multi-git update because
image.inc is a single centralized file in the root git.
It would be better to allow a single git update to add
a package. Too allow this, image.inc must be split across
the git repos and the build tools must be changed to
read/merge those files to arrive at the final package list.
Current scheme is to name the image.inc files using this
schema.
${distro}_${build_target}_image_${build_type}.inc
distro = centos, ...
build_target = iso, guest ...
build_type = std, rt ...
Traditionally build_type=std is omitted from config files,
so we instread use ${distro}_${build_target}_image.inc.
Story: 2003447
Task: 24650
Change-Id: Ib39b8063e7759842ba15330c68503bfe2dea6e20
Signed-off-by: Scott Little <scott.little@windriver.com>
Problem:
Hung on a spec file that contains a seemingly self referential macro ...
Version: %{_version}
Normally 'Version: x.y.z' has the effect of setting macro %{_version}.
In effect we have 'x=x'. Not sure how rpm handles this, but we need to
prevent a recursion in our evaluation scripts.
Solution:
1) Test for recursion in spec_evaluate() and fail rather than
entering into a recursion.
2) Record the version from the srpm file name into an environment variable.
If we fail to read the version from the spec file directly, substitute
the one from the environment variable, if defined.
Note: Tripped up on mismatched '`' during testing, so substituted
all occurances with $(). Now a good browser should be able to
highlight an future mismatches.
Story: 2002839
Task: 22778
Change-Id: Ib69f879e531d842ff007d473bd3ad34db46cabf1
Signed-off-by: Scott Little <scott.little@windriver.com>
Directory stx will now be owned by stx-root.
Relocate .gitignore from stx-utils for that directory.
Change-Id: I75ac350600e5dcce286e6a76cf3a985d9297a2a8
Story: 2002801
Task: 22687
Depends-On: https://review.openstack.org/588634
Signed-off-by: Scott Little <scott.little@windriver.com>
Package build-info is referenced in man build scripts.
It's relocation requires that the build scripts be modified
to reflect the new location.
from: stx/middleware/recipes-common/build-info/release-info.inc
to: stx/stx-integ/utilities/build-info/release-info.inc
Change-Id: I82a9c12cd1e5607c1c903fc942a3c2007aca4920
Story: 2002801
Task: 22687
Signed-off-by: Scott Little <scott.little@windriver.com>
This is used by ETCD for logging.
Story: 2002841
Task: 22784
Change-Id: If99d7b0844d3fccd4f04360d60319f689bc171f0
Signed-off-by: Jack Ding <jack.ding@windriver.com>
These files are needed to setup the cgcs-centos-repo, in a first
approach both were planned to be created in the fly by the generator-
cgcs-centos-repo script, however having them in a location and
copied by the script seems to be a better solution.
Story: 2002936
Task: 23033
Change-Id: I301925434e005461fe39f4bfcbf0ee9bc936ef7c
Signed-off-by: Erich Cordoba <erich.cordoba.malibran@intel.com>
All content formerly hosted under directory addons has been moved to
the 'stx' subdirectory. It is now safe to delete the now empty
'addons' directory.
Change-Id: Iea4607dccd9b11e81766d0d2faf60af725415a4d
Signed-off-by: Scott Little <scott.little@windriver.com>
Define PATCHES_BASE, GIT_BASE and STX_BASE within the build tools,
so that the can be refereced within build_srpm.data files.
STX_BASE: Synonym for i$CGCS_BASE.
CGCS_BASE will be obsoleted in a future update.
GIT_BASE: New, points to the root of the git that contains the package.
This will facilitate inter-package references that are
co-located in the same git.
PATCHES_BASE: New, same as $PKG_BASE/$DISTRO/patches
Change-Id: I0636bb38b347374fa5ffe03ea67fd9d3422e2eea
Signed-off-by: Scott Little <scott.little@windriver.com>
OpenDev Sysadmins