Hi all, recently a couple of users have been finding cases where ybd was failing to build known-good definitions. I'm struggling to debug what's going on, because the problem doesn't occur on any of my normal test scenarios (i.e. AWS, Ubuntu VirtualBox on Mac, Ubuntu on HP Moonshot AARCH64 and ARMv7 chroot). It *does* occur reasonably frequently on one of our Concourse setups, and has been happening reproducibly for Tristan on his work deep in the outback with Aboriginal Linux. I've tweaked master YBD to ensure that uncaught exceptions explode properly (16.01 and previous releases are hiding them, sadly) and confirmed that an example of the problem is as follows. From the exception, it seems somehow sandboxlib is trying to an already mounted or busy proc directory. It's obviously possible I've broken something, but I'm wondering if this could be somehow as a result of whatever container approach Concourse is using? Any ideas? br Paul 16-01-12 03:34:52 [14/121/121] [linux-api-headers] Running command: case "$MORPH_ARCH" in armv5l) ARCH="arm" ;; armv7b|armv7l|armv7lhf) ARCH="arm" ;; armv8l64|armv8b64) ARCH="arm64" ;; x86_32) ARCH="i386" ;; x86_64) ARCH="x86_64" ;; ppc64) ARCH="powerpc" ;; *) echo "Error: unsupported Morph architecture: $MORPH_ARCH" >&2 exit 1 esac ARCH=$ARCH make INSTALL_HDR_PATH=dest headers_install WARNING: ['umount', '/root/ybd/tmp/tmpSuZip3/dev/shm'] failed: umount: tmpfs: mountpoint not found WARNING: ['umount', '/root/ybd/tmp/tmpSuZip3/proc'] failed: umount: proc: mountpoint not found 16-01-12 03:38:44 [14/121/121] [linux-api-headers] Running command: install -d "$DESTDIR${PREFIX-/usr}/include" Traceback (most recent call last): File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/__main__.py", line 65, in <module> assemble(defs, target) File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/assembly.py", line 92, in assemble preinstall(defs, component, subcomponent) File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/assembly.py", line 154, in preinstall preinstall(defs, component, it) File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/assembly.py", line 147, in preinstall preinstall(defs, component, it) File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/assembly.py", line 147, in preinstall preinstall(defs, component, it) File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/assembly.py", line 154, in preinstall preinstall(defs, component, it) File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/assembly.py", line 147, in preinstall preinstall(defs, component, it) File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/assembly.py", line 154, in preinstall preinstall(defs, component, it) File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/assembly.py", line 147, in preinstall preinstall(defs, component, it) File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/assembly.py", line 154, in preinstall preinstall(defs, component, it) File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/assembly.py", line 156, in preinstall assemble(defs, dependency) File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/assembly.py", line 101, in assemble do_build(defs, component) File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/assembly.py", line 111, in do_build build(defs, component) File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/assembly.py", line 187, in build allow_parallel=('build' in build_step)) File "/tmp/build/641c0e3f-a9d8-4e7f-5533-e94ca07e9810/ybd/ybd/sandbox.py", line 155, in run_sandboxed env=env, **config) File "/usr/local/lib/python2.7/dist-packages/sandboxlib/chroot.py", line 227, in run_sandbox_with_redirection exit, out, err = run_sandbox(command, **sandbox_config) File "/usr/local/lib/python2.7/dist-packages/sandboxlib/chroot.py", line 208, in run_sandbox with mount_all(filesystem_root, extra_mounts): File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/usr/local/lib/python2.7/dist-packages/sandboxlib/chroot.py", line 139, in mount_all mount(source, path, mount_type, mount_options) File "/usr/local/lib/python2.7/dist-packages/sandboxlib/chroot.py", line 112, in mount argv, err.decode('utf-8'))) RuntimeError: ['mount', '-t', 'proc', '-o', '', '', '/root/ybd/tmp/tmpSuZip3/proc'] failed: mount: is already mounted or /root/ybd/tmp/tmpSuZip3/proc busy