[RFC] git-config-batch: a new builtin for tools reading config

.gitignore

@@ -44,6 +44,7 @@
 /git-commit-graph
 /git-commit-tree
 /git-config
+/git-config-batch
 /git-count-objects
 /git-credential
 /git-credential-cache

Documentation/git-config-batch.adoc

@@ -0,0 +1,162 @@
+git-config-batch(1)
+===================
+
+NAME
+----
+git-config-batch - Get and set options using machine-parseable interface
+
+
+SYNOPSIS
+--------
+[verse]
+'git config-batch' <options>
+
+DESCRIPTION
+-----------
+Tools frequently need to change their behavior based on values stored in
+Git's configuration files. These files may have complicated conditions
+for including extra files, so it is difficult to produce an independent
+parser. To avoid executing multiple processes to discover or modify
+multiple configuration values, the `git config-batch` command allows a
+single process to handle multiple requests using a machine-parseable
+interface across `stdin` and `stdout`.
+
+OPTIONS
+-------
+
+`-z`::
+	If specified, then use the NUL-terminated input and output
+	format instead of the space and newline format. This format is
+	useful when the strings involved may include spaces or newlines.
+	See PROTOCOL for more details.
+
+PROTOCOL
+--------
+By default, the protocol uses line feeds (`LF`) to signal the end of a
+command over `stdin` or a response over `stdout`.
+
+The protocol will be extended in the future, and consumers should be
+resilient to older Git versions not understanding the latest command
+set. Thus, if the Git version includes the `git config-batch` builtin
+but doesn't understand an input command, it will return a single line
+response:
+
+------------
+unknown_command LF
+------------
+
+These are the commands that are currently understood:
+
+`help` version 1::
+	The `help` command lists the currently-available commands in
+	this version of Git. The output is multi-line, but the first
+	line provides the count of possible commands via `help 1 count <N>`.
+	The next `<N>` lines are of the form `help 1 <command> <version>`
+	to state that this Git version supports that `<command>` at
+	version `<version>`. Note that the same command may have multiple
+	available versions.
++
+Here is the current output of the help text at the latest version:
++
+------------
+help 1 count 2
+help 1 help 1
+help 1 get 1
+------------
+
+`get` version 1::
+	The `get` command searches the config key-value pairs within a
+	given `<scope>` for values that match the fixed `<key>` and
+	filters the resulting value based on an optional `<value-filter>`.
+	This can either be a regex or a fixed value. The command format
+	is one of the following formats:
++
+------------
+get 1 <scope> <key>
+get 1 <scope> <key> arg:regex <value-pattern>
+get 1 <scope> <key> arg:fixed-value <value>
+------------
++
+The `<scope>` value can be one of `inherited`, `system`, `global`,
+`local`, `worktree`, `submodule`, or `command`. If `inherited`, then all
+config key-value pairs will be considered regardless of scope. Otherwise,
+only the given scope will be considered.
++
+If no optional arguments are given, then the value will not be filtered
+by any pattern matching. If `arg:regex` is specified, then the rest of
+the line is considered a single string, `<value-pattern>`, and is
+interpreted as a regular expression for matching against stored values,
+similar to specifying a value to `get config --get <key> "<value-pattern>"`.
+If `arg:fixed-value` is specified, then the rest of the line is
+considered a single string, `<value>`, and is checked for an exact
+match against the key-value pairs, simmilar to `git config --get <key>
+--fixed-value "<value>"`.
++
+At mmost one key-value pair is returned, that being the last key-value
+pair in the standard config order by scope and sequence within each scope.
++
+If a key-value pair is found, then the following output is given:
++
+------------
+get 1 found <key> <scope> <value>
+------------
++
+If no matching key-value pair is found, then the following output is
+given:
++
+------------
+get 1 missing <key> [<value-pattern>|<value>]
+------------
++
+where `<value-pattern>` or `<value>` is only supplied if provided in
+the command.
+
+NUL-Terminated Format
+~~~~~~~~~~~~~~~~~~~~~
+
+When `-z` is given, the protocol changes in some structural ways.
+
+First, each command is terminated with two NUL bytes, providing a clear
+boundary between commands regardless of future possibilities of new
+command formats.
+
+Second, any time that a space _would_ be used to partition tokens in a
+command, a NUL byte is used instead. Further, each token is prefixed
+with `<N>:` where `<N>` is a decimal representation of the length of
+the string between the `:` and the next NUL byte. Any disagreement in
+these lengths is treated as a parsing error. This use of a length does
+imply that "`0:`" is the representation of an empty string, if relevant.
+
+The decimal representation must have at most five numerals, thus the
+maximum length of a string token can have 99999 characters.
+
+For example, the `get` command, version 1, could have any of the
+following forms:
+
+------------
+3:get NUL 1:1 NUL 5:local NUL 14:key.with space NUL NUL
+3:get NUL 1:1 NUL 9:inherit NUL 8:test.key NUL 9:arg:regex NUL 6:.*\ .* NUL NUL
+3:get NUL 1:1 NUL 6:global NUL 8:test.key NUL 15:arg:fixed-value NUL 3:a b NUL NUL
+------------
+
+The output is modified similarly, such as the following output examples,
+as if the input has a parse error, a valid `help` command, a `get`
+command that had a match, and a `get` command that did not match.
+
+------------
+15:unknown_command NUL NUL
+4:help NUL 1:1 NUL 5:count NUL 1:2 NUL NUL
+4:help NUL 1:1 NUL 4:help NUL 1:1 NUL NUL
+4:help NUL 1:1 NUL 3:get NUL 1:1 NUL NUL
+3:get NUL 1:1 NUL 5:found NUL 8:test.key NUL 5:value NUL NUL
+3:get NUL 1:1 NUL 7:missing NUL 8:test.key NUL NUL
+------------
+
+
+SEE ALSO
+--------
+linkgit:git-config[1]
+
+GIT
+---
+Part of the linkgit:git[1] suite

Documentation/meson.build

@@ -29,6 +29,7 @@ manpages = {
   'git-commit-tree.adoc' : 1,
   'git-commit.adoc' : 1,
   'git-config.adoc' : 1,
+  'git-config-batch.adoc' : 1,
   'git-count-objects.adoc' : 1,
   'git-credential-cache--daemon.adoc' : 1,
   'git-credential-cache.adoc' : 1,

Documentation/technical/config-batch.adoc

@@ -0,0 +1,70 @@
+Git Config-Batch Design Notes
+=============================
+
+The `git config-batch` builtin has a robust protocol for parsing multiple
+commands over `stdin` and providing structured output over `stdout`. The
+intended use is for scripts or third-party software to interact with the
+config settings of a repository multiple times within the same Git process.
+The protocol is built with versioning that allows the consumer to know when
+a certain command is available and to fall back to single-use `git config`
+processes if the installed Git version does not have the latest commands
+at the required versions.
+
+Recommended interaction pattern
+-------------------------------
+
+This section provides a guide for ideal interaction with the `git
+config-batch` command and its protocol.
+
+For maximum compatibility, do not attempt parsing the output of `git
+version` to determine which commands are available. Instead, first check
+if the `git config-batch` command succeeds and does not die immediately
+due to the builtin being unavailable. Then, use the v1 of the `help`
+command to get a list of available commands and versions. Use this list to
+determine if your capabilities are available or should be replaced with an
+appropriate `git config` single-use process.
+
+Further, all automated tooling would be better off using the
+NUL-terminated format instead of the whitespace-delimited format, in case
+config keys contain spaces or config values contain newlines. The
+whitespace-delimited version is available for simpler integration and
+human inspection.
+
+Current commands
+----------------
+
+See the documentation in linkgit::config-batch[1] for the latest set of
+available commands and their protocols.
+
+Future commands
+---------------
+
+The following modes of `git config` are not currently available as commands
+in `git config-batch`, but are planned for future integration:
+
+`git config list [--<scope>]`::
+	Getting all values, regardless of config key, would require a
+	multi-valued output similar to the `help` command. This tool will
+	likely assume advanced options such as `--show-origin`.
+
+`git config set [--<scope>] <key> <value>`::
+	It will be desirable to set a config key at a given scope as a
+	single value, replacing the current value at that scope, if it
+	exists and is a single value. A `set` command could satisfy this
+	purpose.
+
+`git config set --all [<value-pattern>|--fixed-value=<fixedvalue>] <key> <value>`::
+	When replacing multiple values, it may be necessary to have a different
+	output describing the places those values were set, so it may need to
+	be implemented via a `set-all` command to differentiate from a `set`
+	command.
+
+`git config unset <key>`::
+
+`git config unset --all [<value-pattern>|--fixed-value=<fixedvalue>] <key>`::
+
+`git config get --all --rexexp <key-pattern> [<value-options>]`::
+
+`--replace-all` option::
+
+`--type=<type>` option::

Makefile

@@ -1390,6 +1390,7 @@ BUILTIN_OBJS += builtin/commit-graph.o
 BUILTIN_OBJS += builtin/commit-tree.o
 BUILTIN_OBJS += builtin/commit.o
 BUILTIN_OBJS += builtin/config.o
+BUILTIN_OBJS += builtin/config-batch.o
 BUILTIN_OBJS += builtin/count-objects.o
 BUILTIN_OBJS += builtin/credential-cache--daemon.o
 BUILTIN_OBJS += builtin/credential-cache.o

builtin.h

@@ -68,12 +68,18 @@
  *
  * . Add `builtin/foo.o` to `BUILTIN_OBJS` in `Makefile`.
  *
+ * . Add 'builtin/foo.c' to the 'builtin_sources' array in 'meson.build'.
+ *
  * Additionally, if `foo` is a new command, there are 4 more things to do:
  *
  * . Add tests to `t/` directory.
  *
+ * . Add the test script to 'integration_tests' in  't/meson.build'.
+ *
  * . Write documentation in `Documentation/git-foo.adoc`.
  *
+ * . Add 'git-foo.adoc' to the manpages list in 'Documentation/meson.build'.
+ *
  * . Add an entry for `git-foo` to `command-list.txt`.
  *
  * . Add an entry for `/git-foo` to `.gitignore`.
@@ -167,6 +173,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix, struct repositor
 int cmd_commit_graph(int argc, const char **argv, const char *prefix, struct repository *repo);
 int cmd_commit_tree(int argc, const char **argv, const char *prefix, struct repository *repo);
 int cmd_config(int argc, const char **argv, const char *prefix, struct repository *repo);
+int cmd_config_batch(int argc, const char **argv, const char *prefix, struct repository *repo);
 int cmd_count_objects(int argc, const char **argv, const char *prefix, struct repository *repo);
 int cmd_credential(int argc, const char **argv, const char *prefix, struct repository *repo);
 int cmd_credential_cache(int argc, const char **argv, const char *prefix, struct repository *repo);