11:51 a.m.
Repo: git://gitano.org.uk/personal/richardmaw/gitano.git
Ref: refs/heads/shallow
This unifies how git subcommands are called,
and allows the command to be augmented based on config.
The test suite doesn't cover all the places where I had to make changes,
but I assert that they are ok from visual inspection,
and I'm on the hook to provide tests for those eventually anyway.
It then adds special-case logic for the git.receive.shallowUpdate config key,
setting it on the git command-line with the `-c foo=bar` syntax,
so if it's set in the gitano config, it gets set in the git config.
If we get many options it would make sense to include.path=config-file,
to avoid potentially creating a command too large to execute,
but that requires caching logic or generating a temp file every time.
We *could* do a more generic thing and set every git.* key set in gitano config,
but I'd rather keep it to a whitelist we're aware of,
since you can badly break a repository if you set config willy-nilly.
Technically git is case-insensitive when handling this config option,
but receive.shallowUpdate is the form that is documented.
Richard Maw (7):
gitano.repository: Add git_command() method
Call git commands through repo:git_command()
gitano.repository.git_command: Handle shallowUpdate config
testing: Extend cloneviassh to accept depth
testing: extend gtt to support pushing
testing: Support checking for the existence of files in repos
server-side
testing: Add yarn for shallow pushes
Makefile | 3 ++-
bin/gitano-smart-http.cgi.in | 16 +++++++++-------
lib/gitano/command.lua | 15 ++++++---------
lib/gitano/repocommand.lua | 7 ++-----
lib/gitano/repository.lua | 24 ++++++++++++++++++++++++
plugins/archive.lua | 8 +++-----
testing/03-shallow-push.yarn | 19 +++++++++++++++++++
testing/gitano-test-tool.in | 11 +++++++++--
testing/library.yarn | 11 +++++++++--
9 files changed, 83 insertions(+), 31 deletions(-)
create mode 100644 testing/03-shallow-push.yarn
--
2.7.0
11:51 a.m.
New subject: [PATCH 1/7] gitano.repository: Add git_command() method
This takes a git command to run, minus the "git" argument.
The intention is that if it needs to insert config options,
then they can all be done in one place.
---
lib/gitano/repository.lua | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/lib/gitano/repository.lua b/lib/gitano/repository.lua
index 3ee85e1..982dc4d 100644
--- a/lib/gitano/repository.lua
+++ b/lib/gitano/repository.lua
@@ -722,6 +722,13 @@ function repo_method:save_admin(reason, author, committer)
return self:run_checks()
end
+function repo_method:git_command(t)
+ local t_copy = util.deep_copy(t)
+ table.insert(t_copy, 1, "git")
+ local proc = sp.spawn(t_copy)
+ return proc:wait()
+end
+
local repo_meta = {
__index = repo_method,
}
--
2.7.0
11:51 a.m.
New subject: [PATCH 2/7] Call git commands through repo:git_command()
---
bin/gitano-smart-http.cgi.in | 16 +++++++++-------
lib/gitano/command.lua | 15 ++++++---------
lib/gitano/repocommand.lua | 7 ++-----
plugins/archive.lua | 8 +++-----
4 files changed, 20 insertions(+), 26 deletions(-)
diff --git a/bin/gitano-smart-http.cgi.in b/bin/gitano-smart-http.cgi.in
index 6f8fdee..31749ec 100755
--- a/bin/gitano-smart-http.cgi.in
+++ b/bin/gitano-smart-http.cgi.in
@@ -13,7 +13,6 @@
local gitano = require "gitano"
local gall = require "gall"
local luxio = require "luxio"
-local subprocess = require "luxio.subprocess"
local sio = require "luxio.simple"
-- @@GITANO_BIN_PATH
@@ -63,14 +62,17 @@ if request_method == "GET" or request_method ==
"POST" then
gitano.log.cap_level(gitano.log.level.INFO)
local transactionid = gitano.log.syslog.open()
- if cmdline and gitano.auth.is_authorized(user, "http", cmdline,
- os.getenv("GITANO_ROOT"),
- transactionid,
- start_log_level) then
- local proc = subprocess.spawn_simple({"git", "http-backend"})
+ local authorized, cmd, parsed_cmdline, config, env, repo
+ if cmdline then
+ authorized, cmd, parsed_cmdline, config, env, repo =
+ gitano.auth.is_authorized(user, "http", cmdline,
+ os.getenv("GITANO_ROOT"),
+ transactionid,
+ start_log_level)
+ if cmdline and authorized then
local exit_code
- _, exit_code = proc:wait()
+ _, exit_code = repo:git_command{"http-backend"}
if exit_code ~= 0 then
stdout:write("Status: 500 Internal Server Error\r\n\r\n")
diff --git a/lib/gitano/command.lua b/lib/gitano/command.lua
index b2e04dd..b46613c 100644
--- a/lib/gitano/command.lua
+++ b/lib/gitano/command.lua
@@ -8,7 +8,6 @@ local log = require 'gitano.log'
local util = require 'gitano.util'
local repository = require 'gitano.repository'
-local sp = require "luxio.subprocess"
local sio = require "luxio.simple"
local cmds = {}
@@ -190,10 +189,9 @@ local function builtin_upload_pack_prep(config, repo, cmdline,
context)
end
local function builtin_upload_pack_run(config, repo, cmdline, env)
- local cmdcopy = {env=env}
- for i = 1, #cmdline do cmdcopy[i] = cmdline[i] end
- local proc = sp.spawn(cmdcopy)
- return proc:wait()
+ local cmdcopy = {"upload-pack", env=env}
+ for i = 2, #cmdline do cmdcopy[i] = cmdline[i] end
+ return repo:git_command(cmdcopy)
end
assert(register_cmd("git-upload-pack", nil, nil,
@@ -221,10 +219,9 @@ local function builtin_receive_pack_prep(config, repo, cmdline,
context)
end
local function builtin_receive_pack_run(config, repo, cmdline, env)
- local cmdcopy = {env=env}
- for i = 1, #cmdline do cmdcopy[i] = cmdline[i] end
- local proc = sp.spawn(cmdcopy)
- local how, why = proc:wait()
+ local cmdcopy = {"receive-pack", env=env}
+ for i = 2, #cmdline do cmdcopy[i] = cmdline[i] end
+ local how, why = repo:git_command(cmdcopy)
-- Check that HEAD is now resolvable in the repo
if how == "exit" and why == 0 and not repo.git:get("HEAD") then
log.warn("")
diff --git a/lib/gitano/repocommand.lua b/lib/gitano/repocommand.lua
index 4d70a8b..cc81146 100644
--- a/lib/gitano/repocommand.lua
+++ b/lib/gitano/repocommand.lua
@@ -8,8 +8,6 @@ local log = require 'gitano.log'
local util = require 'gitano.util'
local repository = require 'gitano.repository'
-local sp = require "luxio.subprocess"
-
local builtin_gc_short = "Invoke git gc on your repository"
local builtin_gc_helptext = [[
usage: gc repo [options]
@@ -62,11 +60,10 @@ local function builtin_simple_validate(config, repo, cmdline)
end
local function builtin_simple_run(config, repo, cmdline, env)
- local cmdcopy = {env=util.deep_copy(env), "git", cmdline[1]}
+ local cmdcopy = {cmdline[1], env=util.deep_copy(env)}
cmdcopy.env.GIT_DIR=repo:fs_path()
for i = 3, #cmdline do cmdcopy[#cmdcopy+1] = cmdline[i] end
- local proc = sp.spawn(cmdcopy)
- return proc:wait()
+ return repo:git_command(cmdcopy)
end
local function register_repocommand(register_cmd)
diff --git a/plugins/archive.lua b/plugins/archive.lua
index e117201..026050b 100644
--- a/plugins/archive.lua
+++ b/plugins/archive.lua
@@ -1,5 +1,4 @@
local gitano = require "gitano"
-local sp = require "luxio.subprocess"
local function builtin_upload_archive_validate(config, repo, cmdline)
-- git-upload-archive repo
@@ -20,10 +19,9 @@ local function builtin_upload_archive_prep(config, repo, cmdline,
context)
end
local function builtin_upload_archive_run(config, repo, cmdline, env)
- local cmdcopy = {env=env}
- for i = 1, #cmdline do cmdcopy[i] = cmdline[i] end
- local proc = sp.spawn(cmdcopy)
- return proc:wait()
+ local cmdcopy = {"upload-archive", env=env}
+ for i = 2, #cmdline do cmdcopy[i] = cmdline[i] end
+ return repo:git_command(cmdcopy)
end
assert(gitano.command.register("git-upload-archive", nil, nil,
--
2.7.0
11:51 a.m.
New subject: [PATCH 3/7] gitano.repository.git_command: Handle shallowUpdate config
If the repository is configured with git.receive.shallowUpdate
(as set by `gitanocmd config set git.receive.shallowUpdate true`)
then git will be run with receive.shallowUpdate set,
which allows shallow commits to be pushed.
---
lib/gitano/repository.lua | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/lib/gitano/repository.lua b/lib/gitano/repository.lua
index 982dc4d..e5f7927 100644
--- a/lib/gitano/repository.lua
+++ b/lib/gitano/repository.lua
@@ -722,9 +722,26 @@ function repo_method:save_admin(reason, author, committer)
return self:run_checks()
end
+local _git_booleans = {
+ ["true"] = true, yes = true, on = true,
+ ["false"] = false, no = false, off = false,
+}
+local function _git_config_bool(v)
+ if type(v) == "string" then
+ return _git_booleans[v:lower()]
+ elseif type(v) == "boolean" then
+ return v
+ end
+ -- Intentional fall-through to return nil
+end
+
function repo_method:git_command(t)
local t_copy = util.deep_copy(t)
table.insert(t_copy, 1, "git")
+ if _git_config_bool(self:conf_get "git.receive.shallowUpdate") then
+ table.insert(t_copy, 2, "-c")
+ table.insert(t_copy, 3, "receive.shallowUpdate=true")
+ end
local proc = sp.spawn(t_copy)
return proc:wait()
end
--
2.7.0
11:51 a.m.
New subject: [PATCH 4/7] testing: Extend cloneviassh to accept depth
---
testing/gitano-test-tool.in | 4 ++--
testing/library.yarn | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/testing/gitano-test-tool.in b/testing/gitano-test-tool.in
index 8ed7546..e07ad15 100644
--- a/testing/gitano-test-tool.in
+++ b/testing/gitano-test-tool.in
@@ -150,10 +150,10 @@ function cmd_setupstandard(owning_user, master_key)
}
end
-function cmd_cloneviassh(user, key, repo, localname)
+function cmd_cloneviassh(user, key, repo, localname, ...)
local exturl = generate_exturl(user, key, repo)
run_program {
- "git", "clone", exturl, user_home(user) .. "/" ..
localname,
+ "git", "clone", exturl, user_home(user) .. "/" ..
localname, ...
}
end
diff --git a/testing/library.yarn b/testing/library.yarn
index bda2aa5..aeeb0bf 100644
--- a/testing/library.yarn
+++ b/testing/library.yarn
@@ -34,8 +34,8 @@ General instance management
Repository access
-----------------
- IMPLEMENTS WHEN ([a-z][a-z0-9]*),? using ([a-z][a-z0-9]*),? clones ([^ ]+) as ([^ ]+)
- $GTT cloneviassh $MATCH_1 $MATCH_2 "$MATCH_3" "$MATCH_4"
+ IMPLEMENTS WHEN ([a-z][a-z0-9]*),? using ([a-z][a-z0-9]*),? clones ([^ ]+) as ([^ ]+)(
with depth (\d+))?
+ $GTT cloneviassh $MATCH_1 $MATCH_2 "$MATCH_3" "$MATCH_4" ${MATCH_5:+
--no-local --depth="$MATCH_6"}
Server-side repository checking for behind-the-scenes work
----------------------------------------------------------
--
2.7.0
12:17 p.m.
New subject: [PATCH 4/7] testing: Extend cloneviassh to accept depth
On Wed, Jan 13, 2016 at 04:51:16PM +0000, Richard Maw wrote:
<snip>
+ $GTT cloneviassh $MATCH_1 $MATCH_2 "$MATCH_3"
"$MATCH_4" ${MATCH_5:+ --no-local --depth="$MATCH_6"}
--no-local is recommended because --depth is ignored when it's a file path.
Potentially we could later have an implementation
that calls the auth check then clones via file path,
and I'd rather avoid the confusion that would result.
11:51 a.m.
New subject: [PATCH 5/7] testing: extend gtt to support pushing
---
testing/gitano-test-tool.in | 7 +++++++
testing/library.yarn | 3 +++
2 files changed, 10 insertions(+)
diff --git a/testing/gitano-test-tool.in b/testing/gitano-test-tool.in
index e07ad15..fc422c6 100644
--- a/testing/gitano-test-tool.in
+++ b/testing/gitano-test-tool.in
@@ -157,6 +157,13 @@ function cmd_cloneviassh(user, key, repo, localname, ...)
}
end
+function cmd_pushviassh(user, key, localname, repo, ...)
+ local exturl = generate_exturl(user, key, repo)
+ run_program {
+ "git", "push", exturl, cwd = user_home(user) .. "/"
.. localname, ...
+ }
+end
+
function cmd_cloneexists(user, localname)
run_program {
"git", "fsck", cwd = user_home(user) .. "/" ..
localname
diff --git a/testing/library.yarn b/testing/library.yarn
index aeeb0bf..b795580 100644
--- a/testing/library.yarn
+++ b/testing/library.yarn
@@ -37,6 +37,9 @@ Repository access
IMPLEMENTS WHEN ([a-z][a-z0-9]*),? using ([a-z][a-z0-9]*),? clones ([^ ]+) as ([^ ]+)(
with depth (\d+))?
$GTT cloneviassh $MATCH_1 $MATCH_2 "$MATCH_3" "$MATCH_4" ${MATCH_5:+
--no-local --depth="$MATCH_6"}
+ IMPLEMENTS WHEN ([a-z][a-z0-9]*),? using ([a-z][a-z0-9]*),? pushes ([^ ]+) to ([^ ]+)
+ $GTT pushviassh $MATCH_1 $MATCH_2 "$MATCH_3" "$MATCH_4"
+
Server-side repository checking for behind-the-scenes work
----------------------------------------------------------
--
2.7.0
11:51 a.m.
New subject: [PATCH 6/7] testing: Support checking for the existence of files in repos server-side
Technically checking whether a file contains "." will work,
but this is more readable.
---
testing/library.yarn | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/testing/library.yarn b/testing/library.yarn
index b795580..371cc02 100644
--- a/testing/library.yarn
+++ b/testing/library.yarn
@@ -46,6 +46,10 @@ Server-side repository checking for behind-the-scenes work
IMPLEMENTS THEN server-side ([^ ]+) file ([^ ]+) contains (.+)
cd "$($GTT serverlocation $MATCH_1)"
grep -q "$MATCH_3" "$MATCH_2"
+
+ IMPLEMENTS THEN server-side ([^ ]+) file ([^ ]+) exists
+ cd "$($GTT serverlocation $MATCH_1)"
+ test -e "$MATCH_2"
Clone manipulation
--
2.7.0
11:51 a.m.
New subject: [PATCH 7/7] testing: Add yarn for shallow pushes
---
Makefile | 3 ++-
testing/03-shallow-push.yarn | 19 +++++++++++++++++++
2 files changed, 21 insertions(+), 1 deletion(-)
create mode 100644 testing/03-shallow-push.yarn
diff --git a/Makefile b/Makefile
index d82eec1..4bda393 100644
--- a/Makefile
+++ b/Makefile
@@ -35,7 +35,8 @@ TESTS := 01-basics 02-commands-as 02-commands-config
02-commands-copy \
02-commands-fsck 02-commands-gc 02-commands-graveyard \
02-commands-git-upload-archive 02-commands-group 02-commands-help \
02-commands-ls 02-commands-readme 02-commands-rename 02-commands-rsync \
- 02-commands-sshkey 02-commands-user 02-commands-whoami 03-cgit-support
+ 02-commands-sshkey 02-commands-user 02-commands-whoami 03-cgit-support \
+ 03-shallow-push
MODS := gitano \
diff --git a/testing/03-shallow-push.yarn b/testing/03-shallow-push.yarn
new file mode 100644
index 0000000..7a30a06
--- /dev/null
+++ b/testing/03-shallow-push.yarn
@@ -0,0 +1,19 @@
+<!-- -*- markdown -*- -->
+
+Pushing from shallow clones
+===========================
+
+ SCENARIO Pushing shallow history
+ GIVEN a standard instance
+ WHEN testinstance, using adminkey, clones gitano-admin.git as local-shallow with depth
1
+
+To push you need to set `git.receive.shallowUpdate=true`
+after the repository is created.
+
+ WHEN testinstance adminkey runs create remote-shallow.git
+ AND testinstance adminkey runs config remote-shallow.git set git.receive.shallowUpdate
true
+
+Then pushing shallow history succeeds.
+
+ WHEN testinstance, using adminkey, pushes local-shallow to remote-shallow.git
+ THEN server-side remote-shallow.git file shallow exists
--
2.7.0
12:30 p.m.
New subject: [PATCH 0/7] Support configuring a repository to accept shallow pushes.
On Wed, Jan 13, 2016 at 04:51:12PM +0000, Richard Maw wrote:
It then adds special-case logic for the git.receive.shallowUpdate
config key,
setting it on the git command-line with the `-c foo=bar` syntax,
so if it's set in the gitano config, it gets set in the git config.
The discussion for why we're doing config like this,
rather than setting it in the `.git/config` flie in the repository
happened off-list.
The short version, is that I'd like to allow forks to share objects.
The most reliable way to do this is to have them share a repository,
and use git namespaces for each logical repository.
You would then want to deal with some repositories having different config,
but they can't all share the `.git/config` file.
We already have a way to set gitano config,
so rather than having some magic to populate a config file for each fork
and configure git to use that based on the namespace,
we read config out of gitano, and add it to the command-line.
4:33 a.m.
New subject: [PATCH 0/7] Support configuring a repository to accept shallow pushes.
On Wed, Jan 13, 2016 at 16:51:12 +0000, Richard Maw wrote:
Repo: git://gitano.org.uk/personal/richardmaw/gitano.git
Ref: refs/heads/shallow
Over-all I'm okay with this series, but I can't find that ref.
D.
--
Daniel Silverstone http://www.digital-scurf.org/
PGP mail accepted and encouraged. Key Id: 3CCE BABE 206C 3B69
5:02 a.m.
New subject: [PATCH 0/7] Support configuring a repository to accept shallow pushes.
On Tue, Feb 02, 2016 at 09:33:38 +0000, Daniel Silverstone wrote:
On Wed, Jan 13, 2016 at 16:51:12 +0000, Richard Maw wrote:
> Repo: git://gitano.org.uk/personal/richardmaw/gitano.git
> Ref: refs/heads/shallow
Over-all I'm okay with this series, but I can't find that ref.
Turns out Richard meant 'config' not 'shallow'
Merged.
--
Daniel Silverstone http://www.digital-scurf.org/
PGP mail accepted and encouraged. Key Id: 3CCE BABE 206C 3B69
2672
days inactive
2692
days old
11 comments
3 participants
participants (3)
-
Daniel Silverstone -
Richard Maw -
Richard Maw