From: seungkyun.kim@lge.com
To: openembedded-core@lists.openembedded.org
Cc: "seungkyun.kim" <seungkyun.kim@lge.com>
Subject: [PATCH v2] ipk/rootfs: run sanity test of multilib in parallel
Date: Wed, 15 May 2024 13:02:05 +0000 [thread overview]
Message-ID: <20240515130205.1547168-1-seungkyun.kim@lge.com> (raw)
From: "seungkyun.kim" <seungkyun.kim@lge.com>
For multilib type packages, there is an additional temporary
installation before the actual installation. It makes almost doubles
the do_rootfs time if having many multilib type packages.
To avoid this overhead, run sanity test in parallel.
Installing package groups through opkg takes much more time than
copying directory.
- Changes in V2:
Fix FileNotFoundError exception when copying rootfs
Signed-off-by: seungkyun.kim <seungkyun.kim@lge.com>
---
meta/lib/oe/package_manager/ipk/rootfs.py | 38 +++++++++++++++++++++--
1 file changed, 36 insertions(+), 2 deletions(-)
diff --git a/meta/lib/oe/package_manager/ipk/rootfs.py b/meta/lib/oe/package_manager/ipk/rootfs.py
index ba93eb62ea..530de740d1 100644
--- a/meta/lib/oe/package_manager/ipk/rootfs.py
+++ b/meta/lib/oe/package_manager/ipk/rootfs.py
@@ -6,7 +6,9 @@
import re
import filecmp
+import multiprocessing
import shutil
+import stat
from oe.rootfs import Rootfs
from oe.manifest import Manifest
from oe.utils import execute_pre_post_process
@@ -197,11 +199,34 @@ class PkgRootfs(DpkgOpkgRootfs):
files[key] = item
def _multilib_test_install(self, pkgs):
+ def _copy_rootfs(src, dst):
+ if os.path.islink(src):
+ linkto = os.readlink(src)
+ if os.path.isabs(linkto):
+ linkto = os.path.normpath(os.path.join(os.path.dirname(dst),
+ os.path.relpath(linkto, src)))
+ os.symlink(linkto, dst)
+ elif os.path.isfile(src):
+ shutil.copy2(src, dst)
+ elif stat.S_ISFIFO(os.stat(src).st_mode):
+ os.mkfifo(dst)
+ else:
+ bb.warn("Skip unsupported file type: %s" % src)
+
ml_temp = self.d.getVar("MULTILIB_TEMP_ROOTFS")
+ rootfs_temp = os.path.join(ml_temp, "rootfs")
bb.utils.mkdirhier(ml_temp)
+ bb.utils.remove(rootfs_temp, True)
- dirs = [self.image_rootfs]
+ if os.path.exists(self.image_rootfs):
+ shutil.copytree(self.image_rootfs, rootfs_temp, copy_function=_copy_rootfs)
+ else:
+ bb.utils.mkdirhier(rootfs_temp)
+ dirs = [rootfs_temp]
+ return multiprocessing.Process(target=self._multilib_test_pkg_install,
+ args=(pkgs, ml_temp, dirs, False))
+ def _multilib_test_pkg_install(self, pkgs, ml_temp, dirs):
for variant in self.d.getVar("MULTILIB_VARIANTS").split():
ml_target_rootfs = os.path.join(ml_temp, variant)
@@ -218,6 +243,8 @@ class PkgRootfs(DpkgOpkgRootfs):
dirs.append(ml_target_rootfs)
self._multilib_sanity_test(dirs)
+ rootfs_temp = os.path.join(ml_temp, "rootfs")
+ bb.utils.remove(rootfs_temp)
'''
While ipk incremental image generation is enabled, it will remove the
@@ -300,15 +327,22 @@ class PkgRootfs(DpkgOpkgRootfs):
for pkg_type in self.install_order:
if pkg_type in pkgs_to_install:
+ sanity_test = None
# For multilib, we perform a sanity test before final install
# If sanity test fails, it will automatically do a bb.fatal()
# and the installation will stop
if pkg_type == Manifest.PKG_TYPE_MULTILIB:
- self._multilib_test_install(pkgs_to_install[pkg_type])
+ sanity_test= self._multilib_test_install(pkgs_to_install[pkg_type])
+ sanity_test.start()
self.pm.install(pkgs_to_install[pkg_type],
[False, True][pkg_type == Manifest.PKG_TYPE_ATTEMPT_ONLY])
+ self._multilib_test_install(pkgs_to_install[pkg_type])
+
+ if sanity_test is not None:
+ sanity_test.join()
+
if self.progress_reporter:
self.progress_reporter.next_stage()
--
2.34.1
reply other threads:[~2024-05-15 13:02 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240515130205.1547168-1-seungkyun.kim@lge.com \
--to=seungkyun.kim@lge.com \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).