diff options
Diffstat (limited to 'Documentation/public-inbox-clone.pod')
| -rw-r--r-- | Documentation/public-inbox-clone.pod | 146 |
1 files changed, 133 insertions, 13 deletions
diff --git a/Documentation/public-inbox-clone.pod b/Documentation/public-inbox-clone.pod index 2a7081ac..64ee3138 100644 --- a/Documentation/public-inbox-clone.pod +++ b/Documentation/public-inbox-clone.pod @@ -4,9 +4,9 @@ public-inbox-clone - "git clone --mirror" wrapper =head1 SYNOPSIS -public-inbox-clone INBOX_URL [INBOX_DIR] +public-inbox-clone [OPTIONS] INBOX_URL [INBOX_DIR] -public-inbox-clone ROOT_URL [DESTINATION] +public-inbox-clone [OPTIONS] ROOT_URL [DESTINATION] # public-inbox 2.0+ =head1 DESCRIPTION @@ -15,20 +15,29 @@ making the initial clone of a remote HTTP(S) public-inbox. It allows cloning multi-epoch v2 inboxes with a single command and zero configuration. +In public-inbox 2.0+, public-inbox-clone can create and maintain +a mirror of multiple inboxes or code repositories using manifest.js.gz +files like L<grok-pull(1)> from grokmirror. L<public-inbox-fetch(1)> is +NOT required when using this mode. + It does not run L<public-inbox-init(1)> nor L<public-inbox-index(1)>. Those commands must be run separately if serving/searching the mirror is required. As-is, -public-inbox-clone is suitable for creating a git-only backup. +public-inbox-clone is suitable for creating a git-only backup +without Xapian and SQLite indices. -public-inbox-clone creates a Makefile with handy targets to update the -inbox once indexed. This Makefile may be edited by the user; it will +When cloning a single inbox, public-inbox-clone creates a Makefile +with handy targets to update the inbox once indexed. +This Makefile may be edited by the user; it will not be rewritten by L<public-inbox-fetch(1)> unless it is removed completely. public-inbox-clone does not use nor require any extra -configuration files (not even C<~/.public-inbox/config>). +configuration files (not even C<~/.public-inbox/config>), +but it can download snippets suitable for adding to any +L<public-inbox-config(5)> file. -L<public-inbox-fetch(1)> may be used to keep C<INBOX_DIR> +L<public-inbox-fetch(1)> may be used to keep a single C<INBOX_DIR> up-to-date. For v2 inboxes, it will create a C<$INBOX_DIR/manifest.js.gz> @@ -57,15 +66,19 @@ Default: C<0..~0> or C<0..> or C<..~0> =item --include=PATTERN -When cloning a top-level with multiple inboxes, only clone inboxes and -repositories matching a given wildcard pattern (using C<*?> and C<[]> is -supported). +When cloning a top-level with multiple inboxes via manifest, +only clone inboxes and repositories matching a given wildcard pattern +(using C<*?> and C<[]> is supported). + +This is a new option in public-inbox 2.0+ =item --exclude=PATTERN -When cloning a top-level with multiple inboxes, ignore inboxes and -repositories matching the given wildcard pattern. Supports the same -wildcards as L</--include> +When cloning a top-level with multiple inboxes via manifest, +ignore inboxes and repositories matching the given wildcard pattern. +Supports the same wildcards as L</--include> + +This is a new option in public-inbox 2.0+ =item --inbox-config=always|v2|v1|never @@ -78,12 +91,16 @@ no v1 inboxes are present. Default: C<always> +This is a new option in public-inbox 2.0+ + =item --inbox-version=NUM Force a remote public-inbox version (must be C<1> or C<2>). This is auto-detected by default, and this option exists mainly for testing. +This is a new option in public-inbox 2.0+ + =item --objstore=DIR Enables space savings when the remote C<manifest.js.gz> @@ -94,6 +111,8 @@ as relative to the C<DESTINATION> directory. If only C<--objstore=> is specified where C<DIR> is an empty string (C<"">), then C<objstore> (C<$DESTINATION/objstore>) is the implied value of C<DIR>. +This is a new option in public-inbox 2.0+ + =item --manifest=FILE When incrementally updating an existing mirror, load the given @@ -108,6 +127,50 @@ as relative to the C<DESTINATION> directory. If only C<--manifest=> is specified where C<FILE> is an empty string (C<"">), then C<manifest.js.gz> (C<$DESTINATION/manifest.js.gz>) is the implied value of C<FILE>. +When updating manifests with many forks using the same objstore, +git 2.41+ is highly recommended for performance as we automatically +use the C<fetch.hideRefs> feature to speed up negotiation. + +C<--manifest=> is a new option in public-inbox 2.0+ + +=item --remote-manifest=URL|RELATIVE_PATH + +Use an alternate location for the remote manifest.js.gz file. +This may be specified as a full absolute URL (e.g +C<--remote-manifest=https://80x24.org/lore/pub/manifest.js.gz>), +or a pathname relative to the ROOT_URL (e.g +C<--remote-manifest=pub/manifest.js.gz> when ROOT_URL is +C<https://80x24.org/lore/> + +By default, C<ROOT_URL/manifest.js.gz> is used. + +This is a new option in public-inbox 2.0+ + +=item --project-list=FILE + +When cloning code repos from a manifest, generate a cgit-compatible +project list. + +If C<FILE> does not start with C</>, C<./>, or C<../>, it is treated +as relative to the C<DESTINATION> directory. If only C<--project-list=> +is specified where C<FILE> is an empty string (C<"">), then C<projects.list> +(C<$DESTINATION/projects.list>) is the implied value of C<FILE>. + +This is a new option in public-inbox 2.0+ + +=item --post-update-hook=COMMAND + +Hooks to run after a repository is cloned or updated, C<COMMAND> will +have the bare git repository destination given as its first and only +argument. + +For v2 inboxes, this operates on a per-epoch basis. + +May be specified multiple times to run multiple commands in the +order specified on the command-line. + +This is a new option in public-inbox 2.0+ + =item -p =item --prune @@ -115,18 +178,41 @@ is specified where C<FILE> is an empty string (C<"">), then C<manifest.js.gz> Pass the C<--prune> and C<--prune-tags> flags to L<git-fetch(1)> calls on incremental clones. +This is a new option in public-inbox 2.0+ + +=item --purge + +Deletes entire repos which no longer exist in the remote manifest, +or are filtered out by C<--include=> or C<--exclude=>. + +This is only useful when using C<--manifest> + +This is a new option in public-inbox 2.0+ + +=item --exit-code + +Exit with C<127> if no updates are done when relying on a manifest. +Updates include fingerprint mismatches in the manifest, new symlinks, +new repositories, and removed repositories from the L<--project-list> + +This is a new option in public-inbox 2.0+ + =item -k =item --keep-going Continue as much as possible after an error. +This is a new option in public-inbox 2.0+ + =item -n =item --dry-run Show what would be done, without making any changes. +This is a new option in public-inbox 2.0+ + =item -q =item --quiet @@ -147,6 +233,40 @@ Whether to wrap L<git(1)> and L<curl(1)> commands with L<torsocks(1)>. Default: C<auto> +=item -j JOBS + +=item --jobs=JOBS + +The number of parallel processes to spawn at once for various network +operations using L<git(1)> and/or L<curl(1)>. + +=back + +=head1 EXAMPLES + +=for comment +Sticking to smaller projects in examples to minimize load on servers + +=over + +=item To mirror the most recent epochs of dwarves and LTTng inboxes: + + public-inbox-clone --epoch=~0 \ + --include='*lttng*' --include='*dwarves' \ + https://80x24.org/lore/ /path/to/inbox-mirror + +C<https://lore.kernel.org/> may be used instead of C<https://80x24.org/lore/> + +=item To mirror all code repos of the sparse project: + + public-inbox-clone --objstore= --project-list= --prune \ + --include='*sparse*' --inbox-config=never \ + --remote-manifest=https://80x24.org/lore/pub/manifest.js.gz \ + https://80x24.org/lore/ /path/to/code-mirror + +C<https://git.kernel.org/> may be used instead of C<https://80x24.org/lore/> +and the C<--remote-manifest> option can be omitted. + =back =head1 CONTACT |