Issues with ccache, URL escaping and local repos

Jonathan Maw jonathan.maw at codethink.co.uk
Wed Jan 2 16:27:59 GMT 2013


On 2012-12-28 19:28, Jannis Pohlmann wrote:
> Hi,
>
> this is a mail mainly to remind us that there is a problem involving 
> the
> new ccache support, URL escaping and system branch repos. The problem 
> is
> this:
>
> When you run "morph build <system>" in a system branch and the build
> involves locally cloned repos with colons in the filename (e.g.
> "upstream:linux"), morph will attempt to create a
>
>   /src/cache/ccache/upstream:linux
>
> directory and will try to bind-mount that directory into the
> staging area using
>
>   mount --bind /src/cache/ccache/upstream:linux
>
> There are two problems with this:
>
> a) mainly, this doesn't work: the mount command will assume that
>    "/src/cache/ccache/upstream" is a socket family or someting and
>    will fail with a "bad address" error.
>
> b) secondly, the ccache directory name is not "unique", i.e., we
>    are not using the same repo URL escaping mechanism as we are
>    using for the local and remote repo caches.
>
> The way this currently works is that the LocalRepoCache is asked to
> create a repo from a file:// URL, it will simply use the local path. 
> If
> it is asked to do the same for a non-file URL, it will cache the repo
> etc. The Builder classes uses this to obtain the full local path of 
> e.g.
> the /src/workspace/your/system/branch/upstream:linux repo, then takes
> the basename and creates /src/cache/ccache/upstream:linux for ccache.
> A quick fix is to simply replace : with _ in Builder. A better fix 
> would
> probably be to use URL escaping based on the resolved original repo 
> URL.
> So not escape file:///src/workspace/your/branch/upstream:linux but
> resolve it's morph.repository URL and use that.
>
> Opinions?
>
>   - Jannis

Hi Jannis,

I've had a go at making it use the escaped URL to cache with with mixed
results.

The good news is that I can make it use a properly escaped path, and
`morph build` will build successfully.

The bad news is that using 'self.artifact.source' is the repository at
file:///src/workspace/blah/blah:etc, so making it use the same ccache
for both the clone of the remote repository, and its local equivalent
will be a bit complicated.

I could check if the path starts with 'file://', and contains
"ALIAS:foo", where ALIAS is on the list of repo-aliases, but I'm not
sure about the syntax of the file paths created in the workspace.

Are there any ways that a colon can end up in the path other than by
a repo-alias?

What do paths look like when it's a local repository for a repo that
doesn't use a repo-alias?

- Jonathan




More information about the baserock-dev mailing list