* [PATCH 0/6] refs: introduce support for partial reference transactions
@ 2025-02-07 7:34 Karthik Nayak
2025-02-07 7:34 ` [PATCH 1/6] refs/files: remove duplicate check in `split_symref_update()` Karthik Nayak
` (11 more replies)
0 siblings, 12 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-07 7:34 UTC (permalink / raw)
To: git; +Cc: ps, jltobler, Karthik Nayak
Git's reference updates are traditionally atomic - when updating
multiple references in a transaction, either all updates succeed or none
do. While this behavior is generally desirable, it can be limiting in
certain scenarios, particularly with the reftable backend where batching
multiple reference updates is more efficient than performing them
sequentially.
This series introduces support for partial reference transactions,
allowing individual reference updates to fail while letting others
proceed. This capability is exposed through git-update-ref's
`--allow-partial` flag, which can be used in `--stdin` mode to batch
updates and handle failures gracefully.
The changes are structured to carefully build up this functionality:
First, we clean up and consolidate the reference update checking logic.
This includes removing duplicate checks in the files backend and moving
refname tracking to the generic layer, which simplifies the codebase and
prepares it for the new feature.
We then restructure the reftable backend's transaction preparation code,
extracting the update validation logic into a dedicated function. This
not only improves code organization but sets the stage for implementing
partial transaction support.
With this groundwork in place, we implement the core partial transaction
support in the refs subsystem. This adds the necessary infrastructure to
track and report rejected updates while allowing transactions to proceed.
All reference backends are modified to support this behavior when enabled.
Finally, we expose this functionality to users through
git-update-ref(1)'s `--allow-partial` flag, complete with test coverage
and documentation. The flag is specifically limited to `--stdin` mode
where batching multiple updates is most relevant.
This enhancement improves Git's flexibility in handling reference
updates while maintaining the safety of atomic transactions by default.
It's particularly valuable for tools and workflows that need to handle
reference update failures gracefully without abandoning the entire batch
of updates.
This series is based on top of bc204b7427 (The seventh batch, 2025-02-03).
There were no conflicts noticed with topics in 'seen' or 'next'.
---
Karthik Nayak (6):
refs/files: remove duplicate check in `split_symref_update()`
refs: move duplicate refname update check to generic layer
refs/files: remove duplicate duplicates check
refs/reftable: extract code from the transaction preparation
refs: implement partial reference transaction support
update-ref: add --allow-partial flag for stdin mode
Documentation/git-update-ref.txt | 12 +-
builtin/update-ref.c | 53 ++++-
refs.c | 58 ++++-
refs.h | 22 ++
refs/files-backend.c | 97 ++------
refs/packed-backend.c | 49 ++--
refs/refs-internal.h | 19 +-
refs/reftable-backend.c | 494 +++++++++++++++++++--------------------
t/t1400-update-ref.sh | 191 +++++++++++++++
9 files changed, 633 insertions(+), 362 deletions(-)
---
---
base-commit: bc204b742735ae06f65bb20291c95985c9633b7f
change-id: 20241206-245-partially-atomic-ref-updates-9fe8b080345c
Thanks
- Karthik
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH 1/6] refs/files: remove duplicate check in `split_symref_update()`
2025-02-07 7:34 [PATCH 0/6] refs: introduce support for partial reference transactions Karthik Nayak
@ 2025-02-07 7:34 ` Karthik Nayak
2025-02-07 16:12 ` Patrick Steinhardt
2025-02-07 7:34 ` [PATCH 2/6] refs: move duplicate refname update check to generic layer Karthik Nayak
` (10 subsequent siblings)
11 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-02-07 7:34 UTC (permalink / raw)
To: git; +Cc: ps, jltobler, Karthik Nayak
In split_symref_update(), there were two redundant checks:
- At the start: checking if refname exists in `affected_refnames`.
- After adding refname: checking if the item added to
`affected_refnames` contains the util field.
Remove the second check since the first one already prevents duplicate
refnames from being added to the transaction updates.
Since this is the only place that utilizes the `item->util` value, avoid
setting the value in the first place and cleanup code around it.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs/files-backend.c | 20 +++-----------------
1 file changed, 3 insertions(+), 17 deletions(-)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 29f08dced40418eb815072c6335e0c3d1a45c7d8..c6a3f6d6261a894e1c294bb1329fdf8079a39eb4 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2387,7 +2387,6 @@ static int split_head_update(struct ref_update *update,
struct string_list *affected_refnames,
struct strbuf *err)
{
- struct string_list_item *item;
struct ref_update *new_update;
if ((update->flags & REF_LOG_ONLY) ||
@@ -2426,8 +2425,7 @@ static int split_head_update(struct ref_update *update,
*/
if (strcmp(new_update->refname, "HEAD"))
BUG("%s unexpectedly not 'HEAD'", new_update->refname);
- item = string_list_insert(affected_refnames, new_update->refname);
- item->util = new_update;
+ string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2446,7 +2444,6 @@ static int split_symref_update(struct ref_update *update,
struct string_list *affected_refnames,
struct strbuf *err)
{
- struct string_list_item *item;
struct ref_update *new_update;
unsigned int new_flags;
@@ -2501,11 +2498,7 @@ static int split_symref_update(struct ref_update *update,
* be valid as long as affected_refnames is in use, and NOT
* referent, which might soon be freed by our caller.
*/
- item = string_list_insert(affected_refnames, new_update->refname);
- if (item->util)
- BUG("%s unexpectedly found in affected_refnames",
- new_update->refname);
- item->util = new_update;
+ string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2837,7 +2830,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
*/
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
- struct string_list_item *item;
if ((update->flags & REF_IS_PRUNING) &&
!(update->flags & REF_NO_DEREF))
@@ -2846,13 +2838,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
if (update->flags & REF_LOG_ONLY)
continue;
- item = string_list_append(&affected_refnames, update->refname);
- /*
- * We store a pointer to update in item->util, but at
- * the moment we never use the value of this field
- * except to check whether it is non-NULL.
- */
- item->util = update;
+ string_list_append(&affected_refnames, update->refname);
}
string_list_sort(&affected_refnames);
if (ref_update_reject_duplicates(&affected_refnames, err)) {
--
2.47.0
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH 2/6] refs: move duplicate refname update check to generic layer
2025-02-07 7:34 [PATCH 0/6] refs: introduce support for partial reference transactions Karthik Nayak
2025-02-07 7:34 ` [PATCH 1/6] refs/files: remove duplicate check in `split_symref_update()` Karthik Nayak
@ 2025-02-07 7:34 ` Karthik Nayak
2025-02-07 16:12 ` Patrick Steinhardt
2025-02-07 7:34 ` [PATCH 3/6] refs/files: remove duplicate duplicates check Karthik Nayak
` (9 subsequent siblings)
11 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-02-07 7:34 UTC (permalink / raw)
To: git; +Cc: ps, jltobler, Karthik Nayak
Move the tracking of refnames in `affected_refnames` from individual
backends into the generic layer in 'refs.c'. This centralizes the
duplicate refname detection that was previously handled separately by
each backend.
Make some changes to accommodate this move:
- Add a `string_list` field `refnames` to `ref_transaction` to contain
all the references in a transaction. This field is updated whenever
a new update is added.
- Modify the backends to use this field internally as needed. The
backends need to check if an update for refname already exists when
splitting symrefs or adding an update for 'HEAD'.
- In the reftable backend, in `reftable_be_transaction_prepare()`,
move the instance of `string_list_has_string()` above
`ref_transaction_add_update()` to check before the reference is
added.
This helps reduce duplication of functionality between the backends and
makes it easier to make changes in a more centralized manner.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 17 ++++++++++++
refs/files-backend.c | 69 ++++++++++---------------------------------------
refs/packed-backend.c | 25 +-----------------
refs/refs-internal.h | 2 ++
refs/reftable-backend.c | 53 ++++++++++++-------------------------
5 files changed, 50 insertions(+), 116 deletions(-)
diff --git a/refs.c b/refs.c
index f4094a326a9f88f979654b668cc9c3d27d83cb5d..4c9b706461977995be1d55e7667f7fb708fbbb76 100644
--- a/refs.c
+++ b/refs.c
@@ -1175,6 +1175,7 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,
CALLOC_ARRAY(tr, 1);
tr->ref_store = refs;
tr->flags = flags;
+ string_list_init_dup(&tr->refnames);
return tr;
}
@@ -1205,6 +1206,7 @@ void ref_transaction_free(struct ref_transaction *transaction)
free((char *)transaction->updates[i]->old_target);
free(transaction->updates[i]);
}
+ string_list_clear(&transaction->refnames, 0);
free(transaction->updates);
free(transaction);
}
@@ -1218,6 +1220,7 @@ struct ref_update *ref_transaction_add_update(
const char *committer_info,
const char *msg)
{
+ struct string_list_item *item;
struct ref_update *update;
if (transaction->state != REF_TRANSACTION_OPEN)
@@ -1245,6 +1248,16 @@ struct ref_update *ref_transaction_add_update(
update->msg = normalize_reflog_message(msg);
}
+ /*
+ * This list is generally used by the backends to avoid duplicates.
+ * But we do support multiple log updates for a given refname within
+ * a single transaction.
+ */
+ if (!(update->flags & REF_LOG_ONLY)) {
+ item = string_list_append(&transaction->refnames, refname);
+ item->util = update;
+ }
+
return update;
}
@@ -2397,6 +2410,10 @@ int ref_transaction_prepare(struct ref_transaction *transaction,
return -1;
}
+ string_list_sort(&transaction->refnames);
+ if (ref_update_reject_duplicates(&transaction->refnames, err))
+ return TRANSACTION_GENERIC_ERROR;
+
ret = refs->be->transaction_prepare(refs, transaction, err);
if (ret)
return ret;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index c6a3f6d6261a894e1c294bb1329fdf8079a39eb4..18da30c3f37dc5c09f7d81a9083d6b41d0463bd5 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2383,9 +2383,7 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st
*/
static int split_head_update(struct ref_update *update,
struct ref_transaction *transaction,
- const char *head_ref,
- struct string_list *affected_refnames,
- struct strbuf *err)
+ const char *head_ref, struct strbuf *err)
{
struct ref_update *new_update;
@@ -2403,7 +2401,7 @@ static int split_head_update(struct ref_update *update,
* transaction. This check is O(lg N) in the transaction
* size, but it happens at most once per transaction.
*/
- if (string_list_has_string(affected_refnames, "HEAD")) {
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
/* An entry already existed */
strbuf_addf(err,
"multiple updates for 'HEAD' (including one "
@@ -2425,7 +2423,6 @@ static int split_head_update(struct ref_update *update,
*/
if (strcmp(new_update->refname, "HEAD"))
BUG("%s unexpectedly not 'HEAD'", new_update->refname);
- string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2441,7 +2438,6 @@ static int split_head_update(struct ref_update *update,
static int split_symref_update(struct ref_update *update,
const char *referent,
struct ref_transaction *transaction,
- struct string_list *affected_refnames,
struct strbuf *err)
{
struct ref_update *new_update;
@@ -2453,7 +2449,7 @@ static int split_symref_update(struct ref_update *update,
* size, but it happens at most once per symref in a
* transaction.
*/
- if (string_list_has_string(affected_refnames, referent)) {
+ if (string_list_has_string(&transaction->refnames, referent)) {
/* An entry already exists */
strbuf_addf(err,
"multiple updates for '%s' (including one "
@@ -2491,15 +2487,6 @@ static int split_symref_update(struct ref_update *update,
update->flags |= REF_LOG_ONLY | REF_NO_DEREF;
update->flags &= ~REF_HAVE_OLD;
- /*
- * Add the referent. This insertion is O(N) in the transaction
- * size, but it happens at most once per symref in a
- * transaction. Make sure to add new_update->refname, which will
- * be valid as long as affected_refnames is in use, and NOT
- * referent, which might soon be freed by our caller.
- */
- string_list_insert(affected_refnames, new_update->refname);
-
return 0;
}
@@ -2561,9 +2548,7 @@ struct files_transaction_backend_data {
static int lock_ref_for_update(struct files_ref_store *refs,
struct ref_update *update,
struct ref_transaction *transaction,
- const char *head_ref,
- struct string_list *affected_refnames,
- struct strbuf *err)
+ const char *head_ref, struct strbuf *err)
{
struct strbuf referent = STRBUF_INIT;
int mustexist = ref_update_expects_existing_old_ref(update);
@@ -2579,8 +2564,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
update->flags |= REF_DELETING;
if (head_ref) {
- ret = split_head_update(update, transaction, head_ref,
- affected_refnames, err);
+ ret = split_head_update(update, transaction, head_ref, err);
if (ret)
goto out;
}
@@ -2590,7 +2574,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
lock->count++;
} else {
ret = lock_raw_ref(refs, update->refname, mustexist,
- affected_refnames,
+ &transaction->refnames,
&lock, &referent,
&update->type, err);
if (ret) {
@@ -2646,9 +2630,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,
* of processing the split-off update, so we
* don't have to do it here.
*/
- ret = split_symref_update(update,
- referent.buf, transaction,
- affected_refnames, err);
+ ret = split_symref_update(update, referent.buf,
+ transaction, err);
if (ret)
goto out;
}
@@ -2803,7 +2786,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
"ref_transaction_prepare");
size_t i;
int ret = 0;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
char *head_ref = NULL;
int head_type;
struct files_transaction_backend_data *backend_data;
@@ -2821,12 +2803,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
transaction->backend_data = backend_data;
/*
- * Fail if a refname appears more than once in the
- * transaction. (If we end up splitting up any updates using
- * split_symref_update() or split_head_update(), those
- * functions will check that the new updates don't have the
- * same refname as any existing ones.) Also fail if any of the
- * updates use REF_IS_PRUNING without REF_NO_DEREF.
+ * Fail if any of the updates use REF_IS_PRUNING without REF_NO_DEREF.
*/
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
@@ -2834,16 +2811,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
if ((update->flags & REF_IS_PRUNING) &&
!(update->flags & REF_NO_DEREF))
BUG("REF_IS_PRUNING set without REF_NO_DEREF");
-
- if (update->flags & REF_LOG_ONLY)
- continue;
-
- string_list_append(&affected_refnames, update->refname);
- }
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
}
/*
@@ -2884,7 +2851,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
struct ref_update *update = transaction->updates[i];
ret = lock_ref_for_update(refs, update, transaction,
- head_ref, &affected_refnames, err);
+ head_ref, err);
if (ret)
goto cleanup;
@@ -2957,7 +2924,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
cleanup:
free(head_ref);
- string_list_clear(&affected_refnames, 0);
if (ret)
files_transaction_cleanup(refs, transaction);
@@ -3021,7 +2987,6 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
{
size_t i;
int ret = 0;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
struct ref_transaction *packed_transaction = NULL;
struct ref_transaction *loose_transaction = NULL;
@@ -3030,13 +2995,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (transaction->state != REF_TRANSACTION_PREPARED)
BUG("commit called for transaction that is not prepared");
- /* Fail if a refname appears more than once in the transaction: */
- for (i = 0; i < transaction->nr; i++)
- if (!(transaction->updates[i]->flags & REF_LOG_ONLY))
- string_list_append(&affected_refnames,
- transaction->updates[i]->refname);
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
+ string_list_sort(&transaction->refnames);
+ if (ref_update_reject_duplicates(&transaction->refnames, err)) {
ret = TRANSACTION_GENERIC_ERROR;
goto cleanup;
}
@@ -3054,7 +3014,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
* that we are creating already exists.
*/
if (refs_for_each_rawref(&refs->base, ref_present,
- &affected_refnames))
+ &transaction->refnames))
BUG("initial ref transaction called with existing refs");
packed_transaction = ref_store_transaction_begin(refs->packed_ref_store,
@@ -3072,7 +3032,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
BUG("initial ref transaction with old_sha1 set");
if (refs_verify_refname_available(&refs->base, update->refname,
- &affected_refnames, NULL, 1, err)) {
+ &transaction->refnames, NULL, 1, err)) {
ret = TRANSACTION_NAME_CONFLICT;
goto cleanup;
}
@@ -3132,7 +3092,6 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (packed_transaction)
ref_transaction_free(packed_transaction);
transaction->state = REF_TRANSACTION_CLOSED;
- string_list_clear(&affected_refnames, 0);
return ret;
}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index a7b6f74b6e35f897f619c540cbc600bbd888bc67..6e7acb077e81435715a1ca3cc928550147c8c56a 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1604,8 +1604,6 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
struct packed_transaction_backend_data {
/* True iff the transaction owns the packed-refs lock. */
int own_lock;
-
- struct string_list updates;
};
static void packed_transaction_cleanup(struct packed_ref_store *refs,
@@ -1614,8 +1612,6 @@ static void packed_transaction_cleanup(struct packed_ref_store *refs,
struct packed_transaction_backend_data *data = transaction->backend_data;
if (data) {
- string_list_clear(&data->updates, 0);
-
if (is_tempfile_active(refs->tempfile))
delete_tempfile(&refs->tempfile);
@@ -1640,7 +1636,6 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
"ref_transaction_prepare");
struct packed_transaction_backend_data *data;
- size_t i;
int ret = TRANSACTION_GENERIC_ERROR;
/*
@@ -1653,34 +1648,16 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
*/
CALLOC_ARRAY(data, 1);
- string_list_init_nodup(&data->updates);
transaction->backend_data = data;
- /*
- * Stick the updates in a string list by refname so that we
- * can sort them:
- */
- for (i = 0; i < transaction->nr; i++) {
- struct ref_update *update = transaction->updates[i];
- struct string_list_item *item =
- string_list_append(&data->updates, update->refname);
-
- /* Store a pointer to update in item->util: */
- item->util = update;
- }
- string_list_sort(&data->updates);
-
- if (ref_update_reject_duplicates(&data->updates, err))
- goto failure;
-
if (!is_lock_file_locked(&refs->lock)) {
if (packed_refs_lock(ref_store, 0, err))
goto failure;
data->own_lock = 1;
}
- if (write_with_updates(refs, &data->updates, err))
+ if (write_with_updates(refs, &transaction->refnames, err))
goto failure;
transaction->state = REF_TRANSACTION_PREPARED;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index aaab711bb96844755dfa600d37efdb25b30a0765..f43766e1f00443eb689685cf4df0fa0b80018a03 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -3,6 +3,7 @@
#include "refs.h"
#include "iterator.h"
+#include "string-list.h"
struct fsck_options;
struct ref_transaction;
@@ -198,6 +199,7 @@ enum ref_transaction_state {
struct ref_transaction {
struct ref_store *ref_store;
struct ref_update **updates;
+ struct string_list refnames;
size_t alloc;
size_t nr;
enum ref_transaction_state state;
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index d39a14c5a469d7d219362e9eae4f578784d65a5b..dd2099d94948a4f23fd9f7ddc06bf3d741229eba 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1068,7 +1068,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
struct reftable_ref_store *refs =
reftable_be_downcast(ref_store, REF_STORE_WRITE|REF_STORE_MAIN, "ref_transaction_prepare");
struct strbuf referent = STRBUF_INIT, head_referent = STRBUF_INIT;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
struct reftable_transaction_data *tx_data = NULL;
struct reftable_backend *be;
struct object_id head_oid;
@@ -1092,10 +1091,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->updates[i], err);
if (ret)
goto done;
-
- if (!(transaction->updates[i]->flags & REF_LOG_ONLY))
- string_list_append(&affected_refnames,
- transaction->updates[i]->refname);
}
/*
@@ -1107,17 +1102,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
tx_data->args[i].updates_alloc = tx_data->args[i].updates_expected;
}
- /*
- * Fail if a refname appears more than once in the transaction.
- * This code is taken from the files backend and is a good candidate to
- * be moved into the generic layer.
- */
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto done;
- }
-
/*
* TODO: it's dubious whether we should reload the stack that "HEAD"
* belongs to or not. In theory, it may happen that we only modify
@@ -1185,14 +1169,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
!(u->flags & REF_LOG_ONLY) &&
!(u->flags & REF_UPDATE_VIA_HEAD) &&
!strcmp(rewritten_ref, head_referent.buf)) {
- struct ref_update *new_update;
-
/*
* First make sure that HEAD is not already in the
* transaction. This check is O(lg N) in the transaction
* size, but it happens at most once per transaction.
*/
- if (string_list_has_string(&affected_refnames, "HEAD")) {
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
/* An entry already existed */
strbuf_addf(err,
_("multiple updates for 'HEAD' (including one "
@@ -1202,12 +1184,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
goto done;
}
- new_update = ref_transaction_add_update(
- transaction, "HEAD",
- u->flags | REF_LOG_ONLY | REF_NO_DEREF,
- &u->new_oid, &u->old_oid, NULL, NULL, NULL,
- u->msg);
- string_list_insert(&affected_refnames, new_update->refname);
+ ref_transaction_add_update(
+ transaction, "HEAD",
+ u->flags | REF_LOG_ONLY | REF_NO_DEREF,
+ &u->new_oid, &u->old_oid, NULL, NULL, NULL,
+ u->msg);
}
ret = reftable_backend_read_ref(be, rewritten_ref,
@@ -1225,7 +1206,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
* at a later point.
*/
ret = refs_verify_refname_available(ref_store, u->refname,
- &affected_refnames, NULL,
+ &transaction->refnames, NULL,
transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
err);
if (ret < 0)
@@ -1277,6 +1258,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
if (!strcmp(rewritten_ref, "HEAD"))
new_flags |= REF_UPDATE_VIA_HEAD;
+ if (string_list_has_string(&transaction->refnames, referent.buf)) {
+ strbuf_addf(err,
+ _("multiple updates for '%s' (including one "
+ "via symref '%s') are not allowed"),
+ referent.buf, u->refname);
+ ret = TRANSACTION_NAME_CONFLICT;
+ goto done;
+ }
+
/*
* If we are updating a symref (eg. HEAD), we should also
* update the branch that the symref points to.
@@ -1301,16 +1291,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
*/
u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
u->flags &= ~REF_HAVE_OLD;
-
- if (string_list_has_string(&affected_refnames, new_update->refname)) {
- strbuf_addf(err,
- _("multiple updates for '%s' (including one "
- "via symref '%s') are not allowed"),
- referent.buf, u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
- string_list_insert(&affected_refnames, new_update->refname);
}
}
@@ -1391,7 +1371,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
strbuf_addf(err, _("reftable: transaction prepare: %s"),
reftable_error_str(ret));
}
- string_list_clear(&affected_refnames, 0);
strbuf_release(&referent);
strbuf_release(&head_referent);
--
2.47.0
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH 3/6] refs/files: remove duplicate duplicates check
2025-02-07 7:34 [PATCH 0/6] refs: introduce support for partial reference transactions Karthik Nayak
2025-02-07 7:34 ` [PATCH 1/6] refs/files: remove duplicate check in `split_symref_update()` Karthik Nayak
2025-02-07 7:34 ` [PATCH 2/6] refs: move duplicate refname update check to generic layer Karthik Nayak
@ 2025-02-07 7:34 ` Karthik Nayak
2025-02-07 16:12 ` Patrick Steinhardt
2025-02-07 7:34 ` [PATCH 4/6] refs/reftable: extract code from the transaction preparation Karthik Nayak
` (8 subsequent siblings)
11 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-02-07 7:34 UTC (permalink / raw)
To: git; +Cc: ps, jltobler, Karthik Nayak
Within the files reference backend's transaction's 'finish' phase, a
verification step is currently performed wherein the refnames list is
sorted and examined for multiple updates targeting the same refname.
It has been observed that this verification is redundant, as an
identical check is already executed during the transaction's 'prepare'
stage. Since the refnames list remains unmodified following the
'prepare' stage, this secondary verification can be safely eliminated.
The duplicate check has been removed accordingly, and the
`ref_update_reject_duplicates()` function has been marked as static, as
its usage is now confined to 'refs.c'.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 9 +++++++--
refs/files-backend.c | 6 ------
refs/refs-internal.h | 8 --------
3 files changed, 7 insertions(+), 16 deletions(-)
diff --git a/refs.c b/refs.c
index 4c9b706461977995be1d55e7667f7fb708fbbb76..b420a120102b3793168598b885bba68e4f5f5f03 100644
--- a/refs.c
+++ b/refs.c
@@ -2295,8 +2295,13 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
return ret;
}
-int ref_update_reject_duplicates(struct string_list *refnames,
- struct strbuf *err)
+/*
+ * Write an error to `err` and return a nonzero value iff the same
+ * refname appears multiple times in `refnames`. `refnames` must be
+ * sorted on entry to this function.
+ */
+static int ref_update_reject_duplicates(struct string_list *refnames,
+ struct strbuf *err)
{
size_t i, n = refnames->nr;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 18da30c3f37dc5c09f7d81a9083d6b41d0463bd5..9fc5454678340dd7c72539bfa0f15ee7eb24b1ff 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2995,12 +2995,6 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (transaction->state != REF_TRANSACTION_PREPARED)
BUG("commit called for transaction that is not prepared");
- string_list_sort(&transaction->refnames);
- if (ref_update_reject_duplicates(&transaction->refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
- }
-
/*
* It's really undefined to call this function in an active
* repository or when there are existing references: we are
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index f43766e1f00443eb689685cf4df0fa0b80018a03..434362b6099a35f92906a04ddd65365140147572 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -142,14 +142,6 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
struct object_id *oid, struct strbuf *referent,
unsigned int *type, int *failure_errno);
-/*
- * Write an error to `err` and return a nonzero value iff the same
- * refname appears multiple times in `refnames`. `refnames` must be
- * sorted on entry to this function.
- */
-int ref_update_reject_duplicates(struct string_list *refnames,
- struct strbuf *err);
-
/*
* Add a ref_update with the specified properties to transaction, and
* return a pointer to the new object. This function does not verify
--
2.47.0
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH 4/6] refs/reftable: extract code from the transaction preparation
2025-02-07 7:34 [PATCH 0/6] refs: introduce support for partial reference transactions Karthik Nayak
` (2 preceding siblings ...)
2025-02-07 7:34 ` [PATCH 3/6] refs/files: remove duplicate duplicates check Karthik Nayak
@ 2025-02-07 7:34 ` Karthik Nayak
2025-02-07 7:34 ` [PATCH 5/6] refs: implement partial reference transaction support Karthik Nayak
` (7 subsequent siblings)
11 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-07 7:34 UTC (permalink / raw)
To: git; +Cc: ps, jltobler, Karthik Nayak
Extract the core logic for preparing individual reference updates from
`reftable_be_transaction_prepare()` into `prepare_single_update()`. This
dedicated function now handles all validation and preparation steps for
each reference update in the transaction, including object ID
verification, HEAD reference handling, and symref processing.
The refactoring consolidates all reference update validation into a
single logical block, which improves code maintainability and
readability. More importantly, this restructuring lays the groundwork
for implementing partial transaction support in the reftable backend,
which will be introduced in the following commit.
No functional changes are included in this commit - it is purely a code
reorganization to support future enhancements.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs/reftable-backend.c | 471 ++++++++++++++++++++++++------------------------
1 file changed, 240 insertions(+), 231 deletions(-)
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index dd2099d94948a4f23fd9f7ddc06bf3d741229eba..5533acfaf9027765d5a270abfce96225e42cc823 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1061,6 +1061,242 @@ static int queue_transaction_update(struct reftable_ref_store *refs,
return 0;
}
+static int prepare_single_update(struct ref_store *ref_store,
+ struct reftable_ref_store *refs,
+ struct reftable_transaction_data *tx_data,
+ struct ref_transaction *transaction,
+ struct reftable_backend *be,
+ struct ref_update *u,
+ unsigned int head_type,
+ struct strbuf *head_referent,
+ struct strbuf *referent,
+ struct strbuf *err)
+{
+ struct object_id current_oid = {0};
+ const char *rewritten_ref;
+ int ret = 0;
+
+ /*
+ * There is no need to reload the respective backends here as
+ * we have already reloaded them when preparing the transaction
+ * update. And given that the stacks have been locked there
+ * shouldn't have been any concurrent modifications of the
+ * stack.
+ */
+ ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
+ if (ret)
+ return ret;
+
+ /* Verify that the new object ID is valid. */
+ if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
+ !(u->flags & REF_SKIP_OID_VERIFICATION) &&
+ !(u->flags & REF_LOG_ONLY)) {
+ struct object *o = parse_object(refs->base.repo, &u->new_oid);
+ if (!o) {
+ strbuf_addf(err,
+ _("trying to write ref '%s' with nonexistent object %s"),
+ u->refname, oid_to_hex(&u->new_oid));
+ return -1;
+ }
+
+ if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
+ strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
+ oid_to_hex(&u->new_oid), u->refname);
+ return -1;
+ }
+ }
+
+ /*
+ * When we update the reference that HEAD points to we enqueue
+ * a second log-only update for HEAD so that its reflog is
+ * updated accordingly.
+ */
+ if (head_type == REF_ISSYMREF &&
+ !(u->flags & REF_LOG_ONLY) &&
+ !(u->flags & REF_UPDATE_VIA_HEAD) &&
+ !strcmp(rewritten_ref, head_referent->buf)) {
+ /*
+ * First make sure that HEAD is not already in the
+ * transaction. This check is O(lg N) in the transaction
+ * size, but it happens at most once per transaction.
+ */
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
+ /* An entry already existed */
+ strbuf_addf(err,
+ _("multiple updates for 'HEAD' (including one "
+ "via its referent '%s') are not allowed"),
+ u->refname);
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ ref_transaction_add_update(
+ transaction, "HEAD",
+ u->flags | REF_LOG_ONLY | REF_NO_DEREF,
+ &u->new_oid, &u->old_oid, NULL, NULL, NULL,
+ u->msg);
+ }
+
+ ret = reftable_backend_read_ref(be, rewritten_ref,
+ ¤t_oid, referent, &u->type);
+ if (ret < 0)
+ return ret;
+ if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
+ /*
+ * The reference does not exist, and we either have no
+ * old object ID or expect the reference to not exist.
+ * We can thus skip below safety checks as well as the
+ * symref splitting. But we do want to verify that
+ * there is no conflicting reference here so that we
+ * can output a proper error message instead of failing
+ * at a later point.
+ */
+ ret = refs_verify_refname_available(ref_store, u->refname,
+ &transaction->refnames, NULL,
+ transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
+ err);
+ if (ret < 0)
+ return ret;
+
+ /*
+ * There is no need to write the reference deletion
+ * when the reference in question doesn't exist.
+ */
+ if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
+ ret = queue_transaction_update(refs, tx_data, u,
+ ¤t_oid, err);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+ }
+ if (ret > 0) {
+ /* The reference does not exist, but we expected it to. */
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "unable to resolve reference '%s'"),
+ ref_update_original_update_refname(u), u->refname);
+ return -1;
+ }
+
+ if (u->type & REF_ISSYMREF) {
+ /*
+ * The reftable stack is locked at this point already,
+ * so it is safe to call `refs_resolve_ref_unsafe()`
+ * here without causing races.
+ */
+ const char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,
+ ¤t_oid, NULL);
+
+ if (u->flags & REF_NO_DEREF) {
+ if (u->flags & REF_HAVE_OLD && !resolved) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "error reading reference"), u->refname);
+ return -1;
+ }
+ } else {
+ struct ref_update *new_update;
+ int new_flags;
+
+ new_flags = u->flags;
+ if (!strcmp(rewritten_ref, "HEAD"))
+ new_flags |= REF_UPDATE_VIA_HEAD;
+
+ if (string_list_has_string(&transaction->refnames, referent->buf)) {
+ strbuf_addf(err,
+ _("multiple updates for '%s' (including one "
+ "via symref '%s') are not allowed"),
+ referent->buf, u->refname);
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ /*
+ * If we are updating a symref (eg. HEAD), we should also
+ * update the branch that the symref points to.
+ *
+ * This is generic functionality, and would be better
+ * done in refs.c, but the current implementation is
+ * intertwined with the locking in files-backend.c.
+ */
+ new_update = ref_transaction_add_update(
+ transaction, referent->buf, new_flags,
+ u->new_target ? NULL : &u->new_oid,
+ u->old_target ? NULL : &u->old_oid,
+ u->new_target, u->old_target,
+ u->committer_info, u->msg);
+
+ new_update->parent_update = u;
+
+ /*
+ * Change the symbolic ref update to log only. Also, it
+ * doesn't need to check its old OID value, as that will be
+ * done when new_update is processed.
+ */
+ u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
+ u->flags &= ~REF_HAVE_OLD;
+ }
+ }
+
+ /*
+ * Verify that the old object matches our expectations. Note
+ * that the error messages here do not make a lot of sense in
+ * the context of the reftable backend as we never lock
+ * individual refs. But the error messages match what the files
+ * backend returns, which keeps our tests happy.
+ */
+ if (u->old_target) {
+ if (!(u->type & REF_ISSYMREF)) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "expected symref with target '%s': "
+ "but is a regular ref"),
+ ref_update_original_update_refname(u),
+ u->old_target);
+ return -1;
+ }
+
+ if (ref_update_check_old_target(referent->buf, u, err)) {
+ return -1;
+ }
+ } else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
+ if (is_null_oid(&u->old_oid)) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "reference already exists"),
+ ref_update_original_update_refname(u));
+ return TRANSACTION_CREATE_EXISTS;
+ }
+ else if (is_null_oid(¤t_oid))
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "reference is missing but expected %s"),
+ ref_update_original_update_refname(u),
+ oid_to_hex(&u->old_oid));
+ else
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "is at %s but expected %s"),
+ ref_update_original_update_refname(u),
+ oid_to_hex(¤t_oid),
+ oid_to_hex(&u->old_oid));
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ /*
+ * If all of the following conditions are true:
+ *
+ * - We're not about to write a symref.
+ * - We're not about to write a log-only entry.
+ * - Old and new object ID are different.
+ *
+ * Then we're essentially doing a no-op update that can be
+ * skipped. This is not only for the sake of efficiency, but
+ * also skips writing unneeded reflog entries.
+ */
+ if ((u->type & REF_ISSYMREF) ||
+ (u->flags & REF_LOG_ONLY) ||
+ (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))
+ return queue_transaction_update(refs, tx_data, u,
+ ¤t_oid, err);
+
+ return 0;
+}
+
static int reftable_be_transaction_prepare(struct ref_store *ref_store,
struct ref_transaction *transaction,
struct strbuf *err)
@@ -1124,239 +1360,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
ret = 0;
for (i = 0; i < transaction->nr; i++) {
- struct ref_update *u = transaction->updates[i];
- struct object_id current_oid = {0};
- const char *rewritten_ref;
-
- /*
- * There is no need to reload the respective backends here as
- * we have already reloaded them when preparing the transaction
- * update. And given that the stacks have been locked there
- * shouldn't have been any concurrent modifications of the
- * stack.
- */
- ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
+ ret = prepare_single_update(ref_store, refs, tx_data,
+ transaction, be,
+ transaction->updates[i], head_type,
+ &head_referent, &referent, err);
if (ret)
goto done;
-
- /* Verify that the new object ID is valid. */
- if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
- !(u->flags & REF_SKIP_OID_VERIFICATION) &&
- !(u->flags & REF_LOG_ONLY)) {
- struct object *o = parse_object(refs->base.repo, &u->new_oid);
- if (!o) {
- strbuf_addf(err,
- _("trying to write ref '%s' with nonexistent object %s"),
- u->refname, oid_to_hex(&u->new_oid));
- ret = -1;
- goto done;
- }
-
- if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
- strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
- oid_to_hex(&u->new_oid), u->refname);
- ret = -1;
- goto done;
- }
- }
-
- /*
- * When we update the reference that HEAD points to we enqueue
- * a second log-only update for HEAD so that its reflog is
- * updated accordingly.
- */
- if (head_type == REF_ISSYMREF &&
- !(u->flags & REF_LOG_ONLY) &&
- !(u->flags & REF_UPDATE_VIA_HEAD) &&
- !strcmp(rewritten_ref, head_referent.buf)) {
- /*
- * First make sure that HEAD is not already in the
- * transaction. This check is O(lg N) in the transaction
- * size, but it happens at most once per transaction.
- */
- if (string_list_has_string(&transaction->refnames, "HEAD")) {
- /* An entry already existed */
- strbuf_addf(err,
- _("multiple updates for 'HEAD' (including one "
- "via its referent '%s') are not allowed"),
- u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
-
- ref_transaction_add_update(
- transaction, "HEAD",
- u->flags | REF_LOG_ONLY | REF_NO_DEREF,
- &u->new_oid, &u->old_oid, NULL, NULL, NULL,
- u->msg);
- }
-
- ret = reftable_backend_read_ref(be, rewritten_ref,
- ¤t_oid, &referent, &u->type);
- if (ret < 0)
- goto done;
- if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
- /*
- * The reference does not exist, and we either have no
- * old object ID or expect the reference to not exist.
- * We can thus skip below safety checks as well as the
- * symref splitting. But we do want to verify that
- * there is no conflicting reference here so that we
- * can output a proper error message instead of failing
- * at a later point.
- */
- ret = refs_verify_refname_available(ref_store, u->refname,
- &transaction->refnames, NULL,
- transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
- err);
- if (ret < 0)
- goto done;
-
- /*
- * There is no need to write the reference deletion
- * when the reference in question doesn't exist.
- */
- if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
- ret = queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
- if (ret)
- goto done;
- }
-
- continue;
- }
- if (ret > 0) {
- /* The reference does not exist, but we expected it to. */
- strbuf_addf(err, _("cannot lock ref '%s': "
- "unable to resolve reference '%s'"),
- ref_update_original_update_refname(u), u->refname);
- ret = -1;
- goto done;
- }
-
- if (u->type & REF_ISSYMREF) {
- /*
- * The reftable stack is locked at this point already,
- * so it is safe to call `refs_resolve_ref_unsafe()`
- * here without causing races.
- */
- const char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,
- ¤t_oid, NULL);
-
- if (u->flags & REF_NO_DEREF) {
- if (u->flags & REF_HAVE_OLD && !resolved) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "error reading reference"), u->refname);
- ret = -1;
- goto done;
- }
- } else {
- struct ref_update *new_update;
- int new_flags;
-
- new_flags = u->flags;
- if (!strcmp(rewritten_ref, "HEAD"))
- new_flags |= REF_UPDATE_VIA_HEAD;
-
- if (string_list_has_string(&transaction->refnames, referent.buf)) {
- strbuf_addf(err,
- _("multiple updates for '%s' (including one "
- "via symref '%s') are not allowed"),
- referent.buf, u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
-
- /*
- * If we are updating a symref (eg. HEAD), we should also
- * update the branch that the symref points to.
- *
- * This is generic functionality, and would be better
- * done in refs.c, but the current implementation is
- * intertwined with the locking in files-backend.c.
- */
- new_update = ref_transaction_add_update(
- transaction, referent.buf, new_flags,
- u->new_target ? NULL : &u->new_oid,
- u->old_target ? NULL : &u->old_oid,
- u->new_target, u->old_target,
- u->committer_info, u->msg);
-
- new_update->parent_update = u;
-
- /*
- * Change the symbolic ref update to log only. Also, it
- * doesn't need to check its old OID value, as that will be
- * done when new_update is processed.
- */
- u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
- u->flags &= ~REF_HAVE_OLD;
- }
- }
-
- /*
- * Verify that the old object matches our expectations. Note
- * that the error messages here do not make a lot of sense in
- * the context of the reftable backend as we never lock
- * individual refs. But the error messages match what the files
- * backend returns, which keeps our tests happy.
- */
- if (u->old_target) {
- if (!(u->type & REF_ISSYMREF)) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "expected symref with target '%s': "
- "but is a regular ref"),
- ref_update_original_update_refname(u),
- u->old_target);
- ret = -1;
- goto done;
- }
-
- if (ref_update_check_old_target(referent.buf, u, err)) {
- ret = -1;
- goto done;
- }
- } else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
- ret = TRANSACTION_NAME_CONFLICT;
- if (is_null_oid(&u->old_oid)) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "reference already exists"),
- ref_update_original_update_refname(u));
- ret = TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(¤t_oid))
- strbuf_addf(err, _("cannot lock ref '%s': "
- "reference is missing but expected %s"),
- ref_update_original_update_refname(u),
- oid_to_hex(&u->old_oid));
- else
- strbuf_addf(err, _("cannot lock ref '%s': "
- "is at %s but expected %s"),
- ref_update_original_update_refname(u),
- oid_to_hex(¤t_oid),
- oid_to_hex(&u->old_oid));
- goto done;
- }
-
- /*
- * If all of the following conditions are true:
- *
- * - We're not about to write a symref.
- * - We're not about to write a log-only entry.
- * - Old and new object ID are different.
- *
- * Then we're essentially doing a no-op update that can be
- * skipped. This is not only for the sake of efficiency, but
- * also skips writing unneeded reflog entries.
- */
- if ((u->type & REF_ISSYMREF) ||
- (u->flags & REF_LOG_ONLY) ||
- (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid))) {
- ret = queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
- if (ret)
- goto done;
- }
}
transaction->backend_data = tx_data;
--
2.47.0
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH 5/6] refs: implement partial reference transaction support
2025-02-07 7:34 [PATCH 0/6] refs: introduce support for partial reference transactions Karthik Nayak
` (3 preceding siblings ...)
2025-02-07 7:34 ` [PATCH 4/6] refs/reftable: extract code from the transaction preparation Karthik Nayak
@ 2025-02-07 7:34 ` Karthik Nayak
2025-02-07 16:12 ` Patrick Steinhardt
2025-02-07 7:34 ` [PATCH 6/6] update-ref: add --allow-partial flag for stdin mode Karthik Nayak
` (6 subsequent siblings)
11 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-02-07 7:34 UTC (permalink / raw)
To: git; +Cc: ps, jltobler, Karthik Nayak
Git's reference transactions are all-or-nothing: either all updates
succeed, or none do. While this atomic behavior is generally desirable,
it can be suboptimal when using the reftable backend, where batching
multiple reference updates into a single transaction is more efficient
than performing them sequentially.
Introduce partial transaction support through a new flag
`REF_TRANSACTION_ALLOW_PARTIAL`. When this flag is set, individual
reference updates that would normally fail the entire transaction are
instead marked as rejected while allowing other updates to proceed. This
provides more flexibility while maintaining transactional integrity
where needed.
The implementation introduces several key components:
- Add 'rejected' and 'rejection_err' fields to struct `ref_update` to
track failed updates and their failure reasons.
- Modify reference backends (files, packed, reftable) to handle
partial transactions by using `ref_transaction_add_rejection()`
instead of failing the entire transaction when
`REF_TRANSACTION_ALLOW_PARTIAL` is set.
- Add `ref_transaction_for_each_rejected_update()` to let callers
examine which updates were rejected and why.
This foundational change enables partial transaction support throughout
the reference subsystem. The next commit will expose this capability to
users by adding a `--allow-partial` flag to 'git-update-ref(1)',
providing both a user-facing feature and a testable implementation.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 32 ++++++++++++++++++++++++++++++++
refs.h | 22 ++++++++++++++++++++++
refs/files-backend.c | 12 +++++++++++-
refs/packed-backend.c | 26 ++++++++++++++++++++++++--
refs/refs-internal.h | 15 +++++++++++++++
refs/reftable-backend.c | 12 +++++++++++-
6 files changed, 115 insertions(+), 4 deletions(-)
diff --git a/refs.c b/refs.c
index b420a120102b3793168598b885bba68e4f5f5f03..75dbd84acbc41658d4b8b6b5e7763c04e78d0061 100644
--- a/refs.c
+++ b/refs.c
@@ -1204,6 +1204,7 @@ void ref_transaction_free(struct ref_transaction *transaction)
free(transaction->updates[i]->committer_info);
free((char *)transaction->updates[i]->new_target);
free((char *)transaction->updates[i]->old_target);
+ strbuf_release(&transaction->updates[i]->rejection_err);
free(transaction->updates[i]);
}
string_list_clear(&transaction->refnames, 0);
@@ -1211,6 +1212,14 @@ void ref_transaction_free(struct ref_transaction *transaction)
free(transaction);
}
+void ref_transaction_add_rejection(struct ref_transaction *transaction,
+ size_t update_idx, struct strbuf *err)
+{
+ struct ref_update *update = transaction->updates[update_idx];
+ update->rejected = 1;
+ strbuf_addbuf(&update->rejection_err, err);
+}
+
struct ref_update *ref_transaction_add_update(
struct ref_transaction *transaction,
const char *refname, unsigned int flags,
@@ -1237,6 +1246,8 @@ struct ref_update *ref_transaction_add_update(
update->flags = flags;
+ strbuf_init(&update->rejection_err, 0);
+
update->new_target = xstrdup_or_null(new_target);
update->old_target = xstrdup_or_null(old_target);
if ((flags & REF_HAVE_NEW) && new_oid)
@@ -2676,6 +2687,27 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
}
}
+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
+ ref_transaction_for_each_rejected_update_fn cb,
+ void *cb_data)
+{
+ if (!(transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL))
+ return;
+
+ for (size_t i = 0; i < transaction->nr; i++) {
+ struct ref_update *update = transaction->updates[i];
+
+ if (!update->rejected)
+ continue;
+
+ cb(update->refname,
+ (update->flags & REF_HAVE_OLD) ? &update->old_oid : NULL,
+ (update->flags & REF_HAVE_NEW) ? &update->new_oid : NULL,
+ update->old_target, update->new_target,
+ &update->rejection_err, cb_data);
+ }
+}
+
int refs_delete_refs(struct ref_store *refs, const char *logmsg,
struct string_list *refnames, unsigned int flags)
{
diff --git a/refs.h b/refs.h
index a0cdd99250e8286b55808b697b0a94afac5d8319..a0f15fdea024527fcfdb478f78cbf6fd6568a25b 100644
--- a/refs.h
+++ b/refs.h
@@ -638,6 +638,13 @@ enum ref_transaction_flag {
* either be absent or null_oid.
*/
REF_TRANSACTION_FLAG_INITIAL = (1 << 0),
+
+ /*
+ * The transaction mechanism by default fails all updates if any conflict
+ * is detected. This flag allows transactions to partially apply updates
+ * while rejecting updates which do not match the expected state.
+ */
+ REF_TRANSACTION_ALLOW_PARTIAL = (1 << 1),
};
/*
@@ -889,6 +896,21 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
ref_transaction_for_each_queued_update_fn cb,
void *cb_data);
+/*
+ * Execute the given callback function for each of the reference updates which
+ * have been rejected in the given transaction.
+ */
+typedef void ref_transaction_for_each_rejected_update_fn(const char *refname,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
+ const struct strbuf *reason,
+ void *cb_data);
+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
+ ref_transaction_for_each_rejected_update_fn cb,
+ void *cb_data);
+
/*
* Free `*transaction` and all associated data.
*/
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 9fc5454678340dd7c72539bfa0f15ee7eb24b1ff..99ec29164fbd30635125cc2325aab3d300cf906c 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2852,8 +2852,18 @@ static int files_transaction_prepare(struct ref_store *ref_store,
ret = lock_ref_for_update(refs, update, transaction,
head_ref, err);
- if (ret)
+ if (ret) {
+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
+ ref_transaction_add_rejection(transaction, i, err);
+
+ strbuf_setlen(err, 0);
+ ret = 0;
+
+ continue;
+ }
goto cleanup;
+ }
+
if (update->flags & REF_DELETING &&
!(update->flags & REF_LOG_ONLY) &&
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 6e7acb077e81435715a1ca3cc928550147c8c56a..cb9b6f0a620eaa59941f6fbc653600304f2bae8c 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1313,9 +1313,10 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
* remain locked when it is done.
*/
static int write_with_updates(struct packed_ref_store *refs,
- struct string_list *updates,
+ struct ref_transaction *transaction,
struct strbuf *err)
{
+ struct string_list *updates = &transaction->refnames;
struct ref_iterator *iter = NULL;
size_t i;
int ok;
@@ -1393,6 +1394,13 @@ static int write_with_updates(struct packed_ref_store *refs,
strbuf_addf(err, "cannot update ref '%s': "
"reference already exists",
update->refname);
+
+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
+ ref_transaction_add_rejection(transaction, i, err);
+ strbuf_setlen(err, 0);
+ continue;
+ }
+
goto error;
} else if (!oideq(&update->old_oid, iter->oid)) {
strbuf_addf(err, "cannot update ref '%s': "
@@ -1400,6 +1408,13 @@ static int write_with_updates(struct packed_ref_store *refs,
update->refname,
oid_to_hex(iter->oid),
oid_to_hex(&update->old_oid));
+
+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
+ ref_transaction_add_rejection(transaction, i, err);
+ strbuf_setlen(err, 0);
+ continue;
+ }
+
goto error;
}
}
@@ -1434,6 +1449,13 @@ static int write_with_updates(struct packed_ref_store *refs,
"reference is missing but expected %s",
update->refname,
oid_to_hex(&update->old_oid));
+
+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
+ ref_transaction_add_rejection(transaction, i, err);
+ strbuf_setlen(err, 0);
+ continue;
+ }
+
goto error;
}
}
@@ -1657,7 +1679,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
data->own_lock = 1;
}
- if (write_with_updates(refs, &transaction->refnames, err))
+ if (write_with_updates(refs, transaction, err))
goto failure;
transaction->state = REF_TRANSACTION_PREPARED;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 434362b6099a35f92906a04ddd65365140147572..6b8f5b2bd83baa22480083e1002daba9300f1b70 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -3,6 +3,7 @@
#include "refs.h"
#include "iterator.h"
+#include "strbuf.h"
#include "string-list.h"
struct fsck_options;
@@ -123,6 +124,13 @@ struct ref_update {
*/
unsigned int index;
+ /*
+ * Used in partial transactions to mark a given update as rejected,
+ * with rejection reason.
+ */
+ unsigned int rejected;
+ struct strbuf rejection_err;
+
/*
* If this ref_update was split off of a symref update via
* split_symref_update(), then this member points at that
@@ -142,6 +150,13 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
struct object_id *oid, struct strbuf *referent,
unsigned int *type, int *failure_errno);
+/*
+ * Mark a given update as rejected with a given reason. To be used in conjuction
+ * with the `REF_TRANSACTION_ALLOW_PARTIAL` flag to allow partial transactions.
+ */
+void ref_transaction_add_rejection(struct ref_transaction *transaction,
+ size_t update_idx, struct strbuf *err);
+
/*
* Add a ref_update with the specified properties to transaction, and
* return a pointer to the new object. This function does not verify
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 5533acfaf9027765d5a270abfce96225e42cc823..a2d86d1c5098b30bd212fc12a3708d2c0a60c677 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1364,8 +1364,18 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction, be,
transaction->updates[i], head_type,
&head_referent, &referent, err);
- if (ret)
+
+ if (ret) {
+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
+ ref_transaction_add_rejection(transaction, i, err);
+
+ strbuf_setlen(err, 0);
+ ret = 0;
+
+ continue;
+ }
goto done;
+ }
}
transaction->backend_data = tx_data;
--
2.47.0
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH 6/6] update-ref: add --allow-partial flag for stdin mode
2025-02-07 7:34 [PATCH 0/6] refs: introduce support for partial reference transactions Karthik Nayak
` (4 preceding siblings ...)
2025-02-07 7:34 ` [PATCH 5/6] refs: implement partial reference transaction support Karthik Nayak
@ 2025-02-07 7:34 ` Karthik Nayak
2025-02-07 16:12 ` Patrick Steinhardt
2025-02-11 17:03 ` [PATCH 0/6] refs: introduce support for partial reference transactions Phillip Wood
` (5 subsequent siblings)
11 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-02-07 7:34 UTC (permalink / raw)
To: git; +Cc: ps, jltobler, Karthik Nayak
When updating multiple references through stdin, Git's update-ref
command normally aborts the entire transaction if any single update
fails. While this atomic behavior prevents partial updates by default,
there are cases where applying successful updates while reporting
failures is desirable.
Add a new `--allow-partial` flag that allows the transaction to continue
even when individual reference updates fail. This flag can only be used
in `--stdin` mode and builds upon the partial transaction support added
to the refs subsystem. When enabled, failed updates are reported in the
following format:
rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
or with `-z`:
rejected NUL (<old-oid> | <old-target>) NUL (<new-oid> | <new-target>) NUL <rejection-reason> NUL
Update the documentation to reflect this change and also tests to cover
different scenarios where an update could be rejected.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
Documentation/git-update-ref.txt | 12 ++-
builtin/update-ref.c | 53 +++++++++--
t/t1400-update-ref.sh | 191 +++++++++++++++++++++++++++++++++++++++
3 files changed, 248 insertions(+), 8 deletions(-)
diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 9e6935d38d031b4890135e0cce36fffcc349ac1d..529d3c15404cdc13216219fba6f56dde91f4909c 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -8,7 +8,7 @@ git-update-ref - Update the object name stored in a ref safely
SYNOPSIS
--------
[verse]
-'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z])
+'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z] [--allow-partial])
DESCRIPTION
-----------
@@ -57,6 +57,12 @@ performs all modifications together. Specify commands of the form:
With `--create-reflog`, update-ref will create a reflog for each ref
even if one would not ordinarily be created.
+With `--allow-partial`, update-ref will process the transaction even if
+some of the updates fail, allowing remaining updates to be applied.
+Failed updates will be printed in the following format:
+
+ rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
+
Quote fields containing whitespace as if they were strings in C source
code; i.e., surrounded by double-quotes and with backslash escapes.
Use 40 "0" characters or the empty string to specify a zero value. To
@@ -82,6 +88,10 @@ quoting:
In this format, use 40 "0" to specify a zero value, and use the empty
string to specify a missing value.
+With `-z`, `--allow-partial` will print rejections in the following form:
+
+ rejected NUL (<old-oid> | <old-target>) NUL (<new-oid> | <new-target>) NUL <rejection-reason> NUL
+
In either format, values can be specified in any form that Git
recognizes as an object name. Commands in any other format or a
repeated <ref> produce an error. Command meanings are:
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 4d35bdc4b4b57937112e6c4c9740420b1f1771e5..83dcb7d8d73f423226c36b61374c86c6b29ec756 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -5,6 +5,7 @@
#include "config.h"
#include "gettext.h"
#include "hash.h"
+#include "hex.h"
#include "refs.h"
#include "object-name.h"
#include "parse-options.h"
@@ -13,7 +14,7 @@
static const char * const git_update_ref_usage[] = {
N_("git update-ref [<options>] -d <refname> [<old-oid>]"),
N_("git update-ref [<options>] <refname> <new-oid> [<old-oid>]"),
- N_("git update-ref [<options>] --stdin [-z]"),
+ N_("git update-ref [<options>] --stdin [-z] [--allow-partial]"),
NULL
};
@@ -562,6 +563,30 @@ static void parse_cmd_abort(struct ref_transaction *transaction,
report_ok("abort");
}
+static void print_rejected_refs(const char *refname,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
+ const struct strbuf *reason,
+ void *cb_data UNUSED)
+{
+ struct strbuf sb = STRBUF_INIT;
+ char space = ' ';
+
+ if (!line_termination)
+ space = line_termination;
+
+ strbuf_addf(&sb, "rejected%c%s%c%s%c%c%s%c%s%c", space,
+ refname, space, new_oid ? oid_to_hex(new_oid) : new_target,
+ space, space, old_oid ? oid_to_hex(old_oid) : old_target,
+ space, reason->buf, line_termination);
+
+ fwrite(sb.buf, sb.len, 1, stdout);
+ strbuf_release(&sb);
+ fflush(stdout);
+}
+
static void parse_cmd_commit(struct ref_transaction *transaction,
const char *next, const char *end UNUSED)
{
@@ -570,6 +595,10 @@ static void parse_cmd_commit(struct ref_transaction *transaction,
die("commit: extra input: %s", next);
if (ref_transaction_commit(transaction, &error))
die("commit: %s", error.buf);
+
+ ref_transaction_for_each_rejected_update(transaction,
+ print_rejected_refs, NULL);
+
report_ok("commit");
ref_transaction_free(transaction);
}
@@ -606,7 +635,7 @@ static const struct parse_cmd {
{ "commit", parse_cmd_commit, 0, UPDATE_REFS_CLOSED },
};
-static void update_refs_stdin(void)
+static void update_refs_stdin(unsigned int flags)
{
struct strbuf input = STRBUF_INIT, err = STRBUF_INIT;
enum update_refs_state state = UPDATE_REFS_OPEN;
@@ -614,7 +643,7 @@ static void update_refs_stdin(void)
int i, j;
transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
- 0, &err);
+ flags, &err);
if (!transaction)
die("%s", err.buf);
@@ -682,7 +711,7 @@ static void update_refs_stdin(void)
*/
state = cmd->state;
transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
- 0, &err);
+ flags, &err);
if (!transaction)
die("%s", err.buf);
@@ -698,6 +727,8 @@ static void update_refs_stdin(void)
/* Commit by default if no transaction was requested. */
if (ref_transaction_commit(transaction, &err))
die("%s", err.buf);
+ ref_transaction_for_each_rejected_update(transaction,
+ print_rejected_refs, NULL);
ref_transaction_free(transaction);
break;
case UPDATE_REFS_STARTED:
@@ -723,7 +754,8 @@ int cmd_update_ref(int argc,
const char *refname, *oldval;
struct object_id oid, oldoid;
int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;
- int create_reflog = 0;
+ int create_reflog = 0, allow_partial = 0;
+
struct option options[] = {
OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
OPT_BOOL('d', NULL, &delete, N_("delete the reference")),
@@ -732,6 +764,7 @@ int cmd_update_ref(int argc,
OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
OPT_BOOL( 0 , "create-reflog", &create_reflog, N_("create a reflog")),
+ OPT_BOOL('0', "allow-partial", &allow_partial, N_("allow partial transactions")),
OPT_END(),
};
@@ -749,13 +782,19 @@ int cmd_update_ref(int argc,
}
if (read_stdin) {
+ unsigned int flags = 0;
+
+ if (allow_partial)
+ flags |= REF_TRANSACTION_ALLOW_PARTIAL;
+
if (delete || argc > 0)
usage_with_options(git_update_ref_usage, options);
if (end_null)
line_termination = '\0';
- update_refs_stdin();
+ update_refs_stdin(flags);
return 0;
- }
+ } else if (allow_partial)
+ die("--allow-partial can only be used with --stdin");
if (end_null)
usage_with_options(git_update_ref_usage, options);
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 29045aad43906fce3f64fb82ee98fb5f80d4796b..4f02f1974de4164442507a2eaec258edf6574f1f 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -2066,6 +2066,197 @@ do
grep "$(git rev-parse $a) $(git rev-parse $a)" actual
'
+ test_expect_success "stdin $type allow-partial" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit commit &&
+ head=$(git rev-parse HEAD) &&
+
+ format_command $type "update refs/heads/ref1" "$head" "$Z" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head" "$Z" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with invalid new_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$(test_oid 001)" "$head" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "trying to write ref ${SQ}refs/heads/ref2${SQ} with nonexistent object" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with non-commit new_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ head_tree=$(git rev-parse HEAD^{tree}) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head_tree" "$head" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "trying to write non-commit object $head_tree to branch ${SQ}refs/heads/ref2${SQ}" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with non-existent ref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ test_must_fail git rev-parse refs/heads/ref2 &&
+ test_grep -q "unable to resolve reference" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with dangling symref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git symbolic-ref refs/heads/ref2 refs/heads/nonexistent &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
+ git update-ref $type --no-deref --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ test_must_fail git rev-parse refs/heads/ref2 &&
+ test_grep -q "reference is missing but expected $head" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with regular ref as symref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "symref-update refs/heads/ref2" "$old_head" "ref" "refs/heads/nonexistent" >>stdin &&
+ git update-ref $type --no-deref --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "expected symref with target ${SQ}refs/heads/nonexistent${SQ}: but is a regular ref" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with invalid old_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$Z" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "reference already exists" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with incorrect old oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head" "$old_head" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "${SQ}refs/heads/ref2${SQ}: is at $head but expected $old_head" stdout
+ )
+ '
done
test_expect_success 'update-ref should also create reflog for HEAD' '
--
2.47.0
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH 1/6] refs/files: remove duplicate check in `split_symref_update()`
2025-02-07 7:34 ` [PATCH 1/6] refs/files: remove duplicate check in `split_symref_update()` Karthik Nayak
@ 2025-02-07 16:12 ` Patrick Steinhardt
2025-02-11 6:35 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Patrick Steinhardt @ 2025-02-07 16:12 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler
On Fri, Feb 07, 2025 at 08:34:36AM +0100, Karthik Nayak wrote:
> In split_symref_update(), there were two redundant checks:
> - At the start: checking if refname exists in `affected_refnames`.
> - After adding refname: checking if the item added to
> `affected_refnames` contains the util field.
Okay, it took me a bit longer to understand what's going on here. What
you're saying is that we already use `string_list_has_string()` at the
start of `split_symref_update()`, and if that returns true then we would
bail out. Consequently, it is impossible for `string_list_insert()` to
find a preexisting values.
Makes sense, but I think that could be explained a bit better.
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index 29f08dced40418eb815072c6335e0c3d1a45c7d8..c6a3f6d6261a894e1c294bb1329fdf8079a39eb4 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -2846,13 +2838,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
> if (update->flags & REF_LOG_ONLY)
> continue;
>
> - item = string_list_append(&affected_refnames, update->refname);
> - /*
> - * We store a pointer to update in item->util, but at
> - * the moment we never use the value of this field
> - * except to check whether it is non-NULL.
> - */
> - item->util = update;
> + string_list_append(&affected_refnames, update->refname);
Nice to see this and other code removed.
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 2/6] refs: move duplicate refname update check to generic layer
2025-02-07 7:34 ` [PATCH 2/6] refs: move duplicate refname update check to generic layer Karthik Nayak
@ 2025-02-07 16:12 ` Patrick Steinhardt
2025-02-11 10:33 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Patrick Steinhardt @ 2025-02-07 16:12 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler
On Fri, Feb 07, 2025 at 08:34:37AM +0100, Karthik Nayak wrote:
> Move the tracking of refnames in `affected_refnames` from individual
> backends into the generic layer in 'refs.c'. This centralizes the
> duplicate refname detection that was previously handled separately by
> each backend.
Exciting, this has been on my TODO list for quite a while already.
> Make some changes to accommodate this move:
>
> - Add a `string_list` field `refnames` to `ref_transaction` to contain
> all the references in a transaction. This field is updated whenever
> a new update is added.
>
> - Modify the backends to use this field internally as needed. The
> backends need to check if an update for refname already exists when
> splitting symrefs or adding an update for 'HEAD'.
Okay. Is this actually necessary to be handled by the backends? I
would've expected that it is possible to split up symref updates so that
we insert both symref and target into the list. I wouldn't be surprised
if this wasn't easily possible though -- the logic here is surprisingly
intricate.
> - In the reftable backend, in `reftable_be_transaction_prepare()`,
> move the instance of `string_list_has_string()` above
> `ref_transaction_add_update()` to check before the reference is
> added.
>
> This helps reduce duplication of functionality between the backends and
> makes it easier to make changes in a more centralized manner.
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
> ---
> refs.c | 17 ++++++++++++
> refs/files-backend.c | 69 ++++++++++---------------------------------------
> refs/packed-backend.c | 25 +-----------------
> refs/refs-internal.h | 2 ++
> refs/reftable-backend.c | 53 ++++++++++++-------------------------
> 5 files changed, 50 insertions(+), 116 deletions(-)
Nice.
> diff --git a/refs.c b/refs.c
> index f4094a326a9f88f979654b668cc9c3d27d83cb5d..4c9b706461977995be1d55e7667f7fb708fbbb76 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1175,6 +1175,7 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,
> CALLOC_ARRAY(tr, 1);
> tr->ref_store = refs;
> tr->flags = flags;
> + string_list_init_dup(&tr->refnames);
Do we actually have to duplicate strings? I would've expected that we
keep strings alive via the `ref_update`s anyway during the transaction's
lifetime.
It might also be interesting to check whether using a strset for this
is more efficient. But that is certainly outside the scope of your patch
series and can be done at a later point. #leftoverbit
> @@ -1245,6 +1248,16 @@ struct ref_update *ref_transaction_add_update(
> update->msg = normalize_reflog_message(msg);
> }
>
> + /*
> + * This list is generally used by the backends to avoid duplicates.
> + * But we do support multiple log updates for a given refname within
> + * a single transaction.
> + */
> + if (!(update->flags & REF_LOG_ONLY)) {
> + item = string_list_append(&transaction->refnames, refname);
> + item->util = update;
> + }
> +
> return update;
> }
> @@ -2397,6 +2410,10 @@ int ref_transaction_prepare(struct ref_transaction *transaction,
> return -1;
> }
>
> + string_list_sort(&transaction->refnames);
> + if (ref_update_reject_duplicates(&transaction->refnames, err))
> + return TRANSACTION_GENERIC_ERROR;
> +
> ret = refs->be->transaction_prepare(refs, transaction, err);
> if (ret)
> return ret;
Okay, we keep the list unserted initially, but sort it later before
passing it to the backends so that `string_list_has_string()` works
correctly. Good.
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index c6a3f6d6261a894e1c294bb1329fdf8079a39eb4..18da30c3f37dc5c09f7d81a9083d6b41d0463bd5 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -2425,7 +2423,6 @@ static int split_head_update(struct ref_update *update,
> */
> if (strcmp(new_update->refname, "HEAD"))
> BUG("%s unexpectedly not 'HEAD'", new_update->refname);
> - string_list_insert(affected_refnames, new_update->refname);
>
> return 0;
> }
Previously we would've inserted "HEAD" into the list of affected
refnames even if it wasn't directly updated. Why don't we have to do
that now anymore?
> @@ -2441,7 +2438,6 @@ static int split_head_update(struct ref_update *update,
> @@ -2491,15 +2487,6 @@ static int split_symref_update(struct ref_update *update,
> update->flags |= REF_LOG_ONLY | REF_NO_DEREF;
> update->flags &= ~REF_HAVE_OLD;
>
> - /*
> - * Add the referent. This insertion is O(N) in the transaction
> - * size, but it happens at most once per symref in a
> - * transaction. Make sure to add new_update->refname, which will
> - * be valid as long as affected_refnames is in use, and NOT
> - * referent, which might soon be freed by our caller.
> - */
> - string_list_insert(affected_refnames, new_update->refname);
> -
> return 0;
> }
Same question here, but for symref updates.
> @@ -3030,13 +2995,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
> if (transaction->state != REF_TRANSACTION_PREPARED)
> BUG("commit called for transaction that is not prepared");
>
> - /* Fail if a refname appears more than once in the transaction: */
> - for (i = 0; i < transaction->nr; i++)
> - if (!(transaction->updates[i]->flags & REF_LOG_ONLY))
> - string_list_append(&affected_refnames,
> - transaction->updates[i]->refname);
> - string_list_sort(&affected_refnames);
> - if (ref_update_reject_duplicates(&affected_refnames, err)) {
> + string_list_sort(&transaction->refnames);
> + if (ref_update_reject_duplicates(&transaction->refnames, err)) {
> ret = TRANSACTION_GENERIC_ERROR;
> goto cleanup;
> }
Can't we also make this check generic for initial transactions?
> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
> index a7b6f74b6e35f897f619c540cbc600bbd888bc67..6e7acb077e81435715a1ca3cc928550147c8c56a 100644
> --- a/refs/packed-backend.c
> +++ b/refs/packed-backend.c
> @@ -1653,34 +1648,16 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
> */
>
> CALLOC_ARRAY(data, 1);
> - string_list_init_nodup(&data->updates);
>
> transaction->backend_data = data;
>
> - /*
> - * Stick the updates in a string list by refname so that we
> - * can sort them:
> - */
> - for (i = 0; i < transaction->nr; i++) {
> - struct ref_update *update = transaction->updates[i];
> - struct string_list_item *item =
> - string_list_append(&data->updates, update->refname);
> -
> - /* Store a pointer to update in item->util: */
> - item->util = update;
> - }
> - string_list_sort(&data->updates);
> -
> - if (ref_update_reject_duplicates(&data->updates, err))
> - goto failure;
> -
> if (!is_lock_file_locked(&refs->lock)) {
> if (packed_refs_lock(ref_store, 0, err))
> goto failure;
> data->own_lock = 1;
> }
>
> - if (write_with_updates(refs, &data->updates, err))
> + if (write_with_updates(refs, &transaction->refnames, err))
> goto failure;
>
> transaction->state = REF_TRANSACTION_PREPARED;
This change is a lot more straight-forward because the packed backend
does not support symrefs at all. Nice.
> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
> index d39a14c5a469d7d219362e9eae4f578784d65a5b..dd2099d94948a4f23fd9f7ddc06bf3d741229eba 100644
> --- a/refs/reftable-backend.c
> +++ b/refs/reftable-backend.c
> @@ -1202,12 +1184,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
> goto done;
> }
>
> - new_update = ref_transaction_add_update(
> - transaction, "HEAD",
> - u->flags | REF_LOG_ONLY | REF_NO_DEREF,
> - &u->new_oid, &u->old_oid, NULL, NULL, NULL,
> - u->msg);
> - string_list_insert(&affected_refnames, new_update->refname);
> + ref_transaction_add_update(
> + transaction, "HEAD",
> + u->flags | REF_LOG_ONLY | REF_NO_DEREF,
> + &u->new_oid, &u->old_oid, NULL, NULL, NULL,
> + u->msg);
> }
>
> ret = reftable_backend_read_ref(be, rewritten_ref,
Equivalent question as for the files backend.
> @@ -1277,6 +1258,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
> if (!strcmp(rewritten_ref, "HEAD"))
> new_flags |= REF_UPDATE_VIA_HEAD;
>
> + if (string_list_has_string(&transaction->refnames, referent.buf)) {
> + strbuf_addf(err,
> + _("multiple updates for '%s' (including one "
> + "via symref '%s') are not allowed"),
> + referent.buf, u->refname);
> + ret = TRANSACTION_NAME_CONFLICT;
> + goto done;
> + }
> +
> /*
> * If we are updating a symref (eg. HEAD), we should also
> * update the branch that the symref points to.
This change surprised me a bit. You mention it in the commit message,
but don't state a reason why you do it.
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 3/6] refs/files: remove duplicate duplicates check
2025-02-07 7:34 ` [PATCH 3/6] refs/files: remove duplicate duplicates check Karthik Nayak
@ 2025-02-07 16:12 ` Patrick Steinhardt
0 siblings, 0 replies; 143+ messages in thread
From: Patrick Steinhardt @ 2025-02-07 16:12 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler
On Fri, Feb 07, 2025 at 08:34:38AM +0100, Karthik Nayak wrote:
> Within the files reference backend's transaction's 'finish' phase, a
> verification step is currently performed wherein the refnames list is
> sorted and examined for multiple updates targeting the same refname.
>
> It has been observed that this verification is redundant, as an
> identical check is already executed during the transaction's 'prepare'
> stage. Since the refnames list remains unmodified following the
> 'prepare' stage, this secondary verification can be safely eliminated.
>
> The duplicate check has been removed accordingly, and the
> `ref_update_reject_duplicates()` function has been marked as static, as
> its usage is now confined to 'refs.c'.
Nice, I had been wondering about this code in the preceding commit.
> diff --git a/refs.c b/refs.c
> index 4c9b706461977995be1d55e7667f7fb708fbbb76..b420a120102b3793168598b885bba68e4f5f5f03 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -2295,8 +2295,13 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
> return ret;
> }
>
> -int ref_update_reject_duplicates(struct string_list *refnames,
> - struct strbuf *err)
> +/*
> + * Write an error to `err` and return a nonzero value iff the same
> + * refname appears multiple times in `refnames`. `refnames` must be
> + * sorted on entry to this function.
> + */
> +static int ref_update_reject_duplicates(struct string_list *refnames,
> + struct strbuf *err)
> {
> size_t i, n = refnames->nr;
>
Doubly nice that this can now be static.
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 5/6] refs: implement partial reference transaction support
2025-02-07 7:34 ` [PATCH 5/6] refs: implement partial reference transaction support Karthik Nayak
@ 2025-02-07 16:12 ` Patrick Steinhardt
2025-02-21 10:33 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Patrick Steinhardt @ 2025-02-07 16:12 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler
On Fri, Feb 07, 2025 at 08:34:40AM +0100, Karthik Nayak wrote:
> Git's reference transactions are all-or-nothing: either all updates
> succeed, or none do. While this atomic behavior is generally desirable,
> it can be suboptimal when using the reftable backend, where batching
> multiple reference updates into a single transaction is more efficient
> than performing them sequentially.
In fact it's even inefficient for the "files" backend. The whole
machinery around creating a new transaction, preparing it, committing it
and then cleaning up its state does bring a bunch of overhead with it.
But true, for the "reftable" backend it's way more impactful.
> diff --git a/refs.c b/refs.c
> index b420a120102b3793168598b885bba68e4f5f5f03..75dbd84acbc41658d4b8b6b5e7763c04e78d0061 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1211,6 +1212,14 @@ void ref_transaction_free(struct ref_transaction *transaction)
> free(transaction);
> }
>
> +void ref_transaction_add_rejection(struct ref_transaction *transaction,
> + size_t update_idx, struct strbuf *err)
"add" to me sounds like you're adding a new thingy to the transaction,
but you rather update something. How about `ref_update_set_rejected()`
or `ref_transacton_set_rejected()`?
> +{
> + struct ref_update *update = transaction->updates[update_idx];
Do we want to `BUG()` in case `update_idx >= transaction->nr`?
> + update->rejected = 1;
> + strbuf_addbuf(&update->rejection_err, err);
> +}
Do we really need a string as rejection error? I'd expect that the set
of failures that lead to rejection should be rather limited, which means
that we could use an enum instead. This would unify the errors across
backends and also allows us to figure out the root cause of rejection in
other subsystems.
If we introduced an enum, we could eventually even iterate a bit on the
mechanism and rather trivially tell the backends which kind of failures
are acceptable. As an example, a conflicting ref update may for example
be ignored and not cause failure, a conflicting path name might cause
failure.
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index 9fc5454678340dd7c72539bfa0f15ee7eb24b1ff..99ec29164fbd30635125cc2325aab3d300cf906c 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -2852,8 +2852,18 @@ static int files_transaction_prepare(struct ref_store *ref_store,
>
> ret = lock_ref_for_update(refs, update, transaction,
> head_ref, err);
> - if (ret)
> + if (ret) {
I wonder whether we want to accept all failures. Some failures are
certainly benign, like for example mismatching expected OIDs or a
conflict due to a preexisting ref that blocks the path. But other kinds
of failures which are unexpected might be a bit more on the dangerous
side to accept, so I think we should be careful here.
The same comment also applies to the other backends.
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 6/6] update-ref: add --allow-partial flag for stdin mode
2025-02-07 7:34 ` [PATCH 6/6] update-ref: add --allow-partial flag for stdin mode Karthik Nayak
@ 2025-02-07 16:12 ` Patrick Steinhardt
2025-02-21 11:45 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Patrick Steinhardt @ 2025-02-07 16:12 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler
On Fri, Feb 07, 2025 at 08:34:41AM +0100, Karthik Nayak wrote:
> diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
> index 9e6935d38d031b4890135e0cce36fffcc349ac1d..529d3c15404cdc13216219fba6f56dde91f4909c 100644
> --- a/Documentation/git-update-ref.txt
> +++ b/Documentation/git-update-ref.txt
> @@ -8,7 +8,7 @@ git-update-ref - Update the object name stored in a ref safely
> SYNOPSIS
> --------
> [verse]
> -'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z])
> +'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z] [--allow-partial])
I think it's time that we start to split this line into multiple lines :)
> diff --git a/builtin/update-ref.c b/builtin/update-ref.c
> index 4d35bdc4b4b57937112e6c4c9740420b1f1771e5..83dcb7d8d73f423226c36b61374c86c6b29ec756 100644
> --- a/builtin/update-ref.c
> +++ b/builtin/update-ref.c
> @@ -562,6 +563,30 @@ static void parse_cmd_abort(struct ref_transaction *transaction,
> report_ok("abort");
> }
>
> +static void print_rejected_refs(const char *refname,
> + const struct object_id *old_oid,
> + const struct object_id *new_oid,
> + const char *old_target,
> + const char *new_target,
> + const struct strbuf *reason,
> + void *cb_data UNUSED)
> +{
> + struct strbuf sb = STRBUF_INIT;
> + char space = ' ';
> +
> + if (!line_termination)
> + space = line_termination;
> +
> + strbuf_addf(&sb, "rejected%c%s%c%s%c%c%s%c%s%c", space,
Whew, that's a lot of placeholders.
> @@ -723,7 +754,8 @@ int cmd_update_ref(int argc,
> const char *refname, *oldval;
> struct object_id oid, oldoid;
> int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;
> - int create_reflog = 0;
> + int create_reflog = 0, allow_partial = 0;
> +
> struct option options[] = {
> OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
> OPT_BOOL('d', NULL, &delete, N_("delete the reference")),
> @@ -732,6 +764,7 @@ int cmd_update_ref(int argc,
> OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
> OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
> OPT_BOOL( 0 , "create-reflog", &create_reflog, N_("create a reflog")),
> + OPT_BOOL('0', "allow-partial", &allow_partial, N_("allow partial transactions")),
You can use `OPT_BIT()` to set a specific bit in a flags field..
> @@ -749,13 +782,19 @@ int cmd_update_ref(int argc,
> }
>
> if (read_stdin) {
> + unsigned int flags = 0;
> +
> + if (allow_partial)
> + flags |= REF_TRANSACTION_ALLOW_PARTIAL;
> +
> if (delete || argc > 0)
> usage_with_options(git_update_ref_usage, options);
> if (end_null)
> line_termination = '\0';
> - update_refs_stdin();
> + update_refs_stdin(flags);
> return 0;
> - }
> + } else if (allow_partial)
> + die("--allow-partial can only be used with --stdin");
>
> if (end_null)
> usage_with_options(git_update_ref_usage, options);
The implementation is quite simple, nice.
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 1/6] refs/files: remove duplicate check in `split_symref_update()`
2025-02-07 16:12 ` Patrick Steinhardt
@ 2025-02-11 6:35 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-11 6:35 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git, jltobler
[-- Attachment #1: Type: text/plain, Size: 1585 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> On Fri, Feb 07, 2025 at 08:34:36AM +0100, Karthik Nayak wrote:
>> In split_symref_update(), there were two redundant checks:
>> - At the start: checking if refname exists in `affected_refnames`.
>> - After adding refname: checking if the item added to
>> `affected_refnames` contains the util field.
>
> Okay, it took me a bit longer to understand what's going on here. What
> you're saying is that we already use `string_list_has_string()` at the
> start of `split_symref_update()`, and if that returns true then we would
> bail out. Consequently, it is impossible for `string_list_insert()` to
> find a preexisting values.
>
> Makes sense, but I think that could be explained a bit better.
>
That's correct.
I'll rewrite it to make it clearer. Thanks.
>> diff --git a/refs/files-backend.c b/refs/files-backend.c
>> index 29f08dced40418eb815072c6335e0c3d1a45c7d8..c6a3f6d6261a894e1c294bb1329fdf8079a39eb4 100644
>> --- a/refs/files-backend.c
>> +++ b/refs/files-backend.c
>> @@ -2846,13 +2838,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
>> if (update->flags & REF_LOG_ONLY)
>> continue;
>>
>> - item = string_list_append(&affected_refnames, update->refname);
>> - /*
>> - * We store a pointer to update in item->util, but at
>> - * the moment we never use the value of this field
>> - * except to check whether it is non-NULL.
>> - */
>> - item->util = update;
>> + string_list_append(&affected_refnames, update->refname);
>
>
> Nice to see this and other code removed.
>
> Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 2/6] refs: move duplicate refname update check to generic layer
2025-02-07 16:12 ` Patrick Steinhardt
@ 2025-02-11 10:33 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-11 10:33 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git, jltobler
[-- Attachment #1: Type: text/plain, Size: 10753 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> On Fri, Feb 07, 2025 at 08:34:37AM +0100, Karthik Nayak wrote:
>> Move the tracking of refnames in `affected_refnames` from individual
>> backends into the generic layer in 'refs.c'. This centralizes the
>> duplicate refname detection that was previously handled separately by
>> each backend.
>
> Exciting, this has been on my TODO list for quite a while already.
>
Yeah, I saw that you left a TODO in the reftable backend too. This
change was not really needed for partial transactions. But it does make
things a bit nicer and easier.
>> Make some changes to accommodate this move:
>>
>> - Add a `string_list` field `refnames` to `ref_transaction` to contain
>> all the references in a transaction. This field is updated whenever
>> a new update is added.
>>
>> - Modify the backends to use this field internally as needed. The
>> backends need to check if an update for refname already exists when
>> splitting symrefs or adding an update for 'HEAD'.
>
> Okay. Is this actually necessary to be handled by the backends? I
> would've expected that it is possible to split up symref updates so that
> we insert both symref and target into the list. I wouldn't be surprised
> if this wasn't easily possible though -- the logic here is surprisingly
> intricate.
It is a bit intricate and requires a bit of unwinding to move it to the
generic layer. But it is possible, I tried to timebox it for this patch
series, but unfortunately it needs a lot more time. So perhaps,
something for later.
>
>> - In the reftable backend, in `reftable_be_transaction_prepare()`,
>> move the instance of `string_list_has_string()` above
>> `ref_transaction_add_update()` to check before the reference is
>> added.
>>
>> This helps reduce duplication of functionality between the backends and
>> makes it easier to make changes in a more centralized manner.
>
>> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
>> ---
>> refs.c | 17 ++++++++++++
>> refs/files-backend.c | 69 ++++++++++---------------------------------------
>> refs/packed-backend.c | 25 +-----------------
>> refs/refs-internal.h | 2 ++
>> refs/reftable-backend.c | 53 ++++++++++++-------------------------
>> 5 files changed, 50 insertions(+), 116 deletions(-)
>
> Nice.
>
>> diff --git a/refs.c b/refs.c
>> index f4094a326a9f88f979654b668cc9c3d27d83cb5d..4c9b706461977995be1d55e7667f7fb708fbbb76 100644
>> --- a/refs.c
>> +++ b/refs.c
>> @@ -1175,6 +1175,7 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,
>> CALLOC_ARRAY(tr, 1);
>> tr->ref_store = refs;
>> tr->flags = flags;
>> + string_list_init_dup(&tr->refnames);
>
> Do we actually have to duplicate strings? I would've expected that we
> keep strings alive via the `ref_update`s anyway during the transaction's
> lifetime.
>
True. I was more thinking along the lines of the keeping the memory
concerns separate. Also, I sure if there are any scenario's that
a `ref_transaction` could outlive a `ref_update`.
> It might also be interesting to check whether using a strset for this
> is more efficient. But that is certainly outside the scope of your patch
> series and can be done at a later point. #leftoverbit
>
Agreed.
>> @@ -1245,6 +1248,16 @@ struct ref_update *ref_transaction_add_update(
>> update->msg = normalize_reflog_message(msg);
>> }
>>
>> + /*
>> + * This list is generally used by the backends to avoid duplicates.
>> + * But we do support multiple log updates for a given refname within
>> + * a single transaction.
>> + */
>> + if (!(update->flags & REF_LOG_ONLY)) {
>> + item = string_list_append(&transaction->refnames, refname);
>> + item->util = update;
>> + }
>> +
>> return update;
>> }
>> @@ -2397,6 +2410,10 @@ int ref_transaction_prepare(struct ref_transaction *transaction,
>> return -1;
>> }
>>
>> + string_list_sort(&transaction->refnames);
>> + if (ref_update_reject_duplicates(&transaction->refnames, err))
>> + return TRANSACTION_GENERIC_ERROR;
>> +
>> ret = refs->be->transaction_prepare(refs, transaction, err);
>> if (ret)
>> return ret;
>
> Okay, we keep the list unserted initially, but sort it later before
> passing it to the backends so that `string_list_has_string()` works
> correctly. Good.
>
>> diff --git a/refs/files-backend.c b/refs/files-backend.c
>> index c6a3f6d6261a894e1c294bb1329fdf8079a39eb4..18da30c3f37dc5c09f7d81a9083d6b41d0463bd5 100644
>> --- a/refs/files-backend.c
>> +++ b/refs/files-backend.c
>> @@ -2425,7 +2423,6 @@ static int split_head_update(struct ref_update *update,
>> */
>> if (strcmp(new_update->refname, "HEAD"))
>> BUG("%s unexpectedly not 'HEAD'", new_update->refname);
>> - string_list_insert(affected_refnames, new_update->refname);
>>
>> return 0;
>> }
>
> Previously we would've inserted "HEAD" into the list of affected
> refnames even if it wasn't directly updated. Why don't we have to do
> that now anymore?
>
We still do, right above this code, there is a call to
`ref_transaction_add_update()`. So any ref_update added to the
transaction via `ref_transaction_add_update()` will also add the refname
to `transaction->refnames`.
>> @@ -2441,7 +2438,6 @@ static int split_head_update(struct ref_update *update,
>> @@ -2491,15 +2487,6 @@ static int split_symref_update(struct ref_update *update,
>> update->flags |= REF_LOG_ONLY | REF_NO_DEREF;
>> update->flags &= ~REF_HAVE_OLD;
>>
>> - /*
>> - * Add the referent. This insertion is O(N) in the transaction
>> - * size, but it happens at most once per symref in a
>> - * transaction. Make sure to add new_update->refname, which will
>> - * be valid as long as affected_refnames is in use, and NOT
>> - * referent, which might soon be freed by our caller.
>> - */
>> - string_list_insert(affected_refnames, new_update->refname);
>> -
>> return 0;
>> }
>
> Same question here, but for symref updates.
>
Same as above. In summary, the need for adding new refnames to the list
is now centralized and a part of adding a ref update to the transaction.
It's a good question, so I'll also add a hint in the commit message.
>
>> @@ -3030,13 +2995,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
>> if (transaction->state != REF_TRANSACTION_PREPARED)
>> BUG("commit called for transaction that is not prepared");
>>
>> - /* Fail if a refname appears more than once in the transaction: */
>> - for (i = 0; i < transaction->nr; i++)
>> - if (!(transaction->updates[i]->flags & REF_LOG_ONLY))
>> - string_list_append(&affected_refnames,
>> - transaction->updates[i]->refname);
>> - string_list_sort(&affected_refnames);
>> - if (ref_update_reject_duplicates(&affected_refnames, err)) {
>> + string_list_sort(&transaction->refnames);
>> + if (ref_update_reject_duplicates(&transaction->refnames, err)) {
>> ret = TRANSACTION_GENERIC_ERROR;
>> goto cleanup;
>> }
>
> Can't we also make this check generic for initial transactions?
>
This one is handled in the next commit, I mostly separated them out
because I was not sure why this needs to be here and to draw attention
if I'm missing something when removing this.
>> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
>> index a7b6f74b6e35f897f619c540cbc600bbd888bc67..6e7acb077e81435715a1ca3cc928550147c8c56a 100644
>> --- a/refs/packed-backend.c
>> +++ b/refs/packed-backend.c
>> @@ -1653,34 +1648,16 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
>> */
>>
>> CALLOC_ARRAY(data, 1);
>> - string_list_init_nodup(&data->updates);
>>
>> transaction->backend_data = data;
>>
>> - /*
>> - * Stick the updates in a string list by refname so that we
>> - * can sort them:
>> - */
>> - for (i = 0; i < transaction->nr; i++) {
>> - struct ref_update *update = transaction->updates[i];
>> - struct string_list_item *item =
>> - string_list_append(&data->updates, update->refname);
>> -
>> - /* Store a pointer to update in item->util: */
>> - item->util = update;
>> - }
>> - string_list_sort(&data->updates);
>> -
>> - if (ref_update_reject_duplicates(&data->updates, err))
>> - goto failure;
>> -
>> if (!is_lock_file_locked(&refs->lock)) {
>> if (packed_refs_lock(ref_store, 0, err))
>> goto failure;
>> data->own_lock = 1;
>> }
>>
>> - if (write_with_updates(refs, &data->updates, err))
>> + if (write_with_updates(refs, &transaction->refnames, err))
>> goto failure;
>>
>> transaction->state = REF_TRANSACTION_PREPARED;
>
> This change is a lot more straight-forward because the packed backend
> does not support symrefs at all. Nice.
>
Yes indeed.
>> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
>> index d39a14c5a469d7d219362e9eae4f578784d65a5b..dd2099d94948a4f23fd9f7ddc06bf3d741229eba 100644
>> --- a/refs/reftable-backend.c
>> +++ b/refs/reftable-backend.c
>> @@ -1202,12 +1184,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>> goto done;
>> }
>>
>> - new_update = ref_transaction_add_update(
>> - transaction, "HEAD",
>> - u->flags | REF_LOG_ONLY | REF_NO_DEREF,
>> - &u->new_oid, &u->old_oid, NULL, NULL, NULL,
>> - u->msg);
>> - string_list_insert(&affected_refnames, new_update->refname);
>> + ref_transaction_add_update(
>> + transaction, "HEAD",
>> + u->flags | REF_LOG_ONLY | REF_NO_DEREF,
>> + &u->new_oid, &u->old_oid, NULL, NULL, NULL,
>> + u->msg);
>> }
>>
>> ret = reftable_backend_read_ref(be, rewritten_ref,
>
> Equivalent question as for the files backend.
>
I hope my answer earlier helps, especially since the diff here shows the
call to `ref_transaction_add_update()`.
>> @@ -1277,6 +1258,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>> if (!strcmp(rewritten_ref, "HEAD"))
>> new_flags |= REF_UPDATE_VIA_HEAD;
>>
>> + if (string_list_has_string(&transaction->refnames, referent.buf)) {
>> + strbuf_addf(err,
>> + _("multiple updates for '%s' (including one "
>> + "via symref '%s') are not allowed"),
>> + referent.buf, u->refname);
>> + ret = TRANSACTION_NAME_CONFLICT;
>> + goto done;
>> + }
>> +
>> /*
>> * If we are updating a symref (eg. HEAD), we should also
>> * update the branch that the symref points to.
>
> This change surprised me a bit. You mention it in the commit message,
> but don't state a reason why you do it.
>
When a `ref_update` is added to the `transaction` via
`ref_transaction_add_update()`, the refname is automatically added to
`transaction->refnames`. As a result, checking for the refname in this
list will always return true. I'll clarify this in the commit message.
> Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 0/6] refs: introduce support for partial reference transactions
2025-02-07 7:34 [PATCH 0/6] refs: introduce support for partial reference transactions Karthik Nayak
` (5 preceding siblings ...)
2025-02-07 7:34 ` [PATCH 6/6] update-ref: add --allow-partial flag for stdin mode Karthik Nayak
@ 2025-02-11 17:03 ` Phillip Wood
2025-02-11 17:40 ` Phillip Wood
2025-02-12 12:34 ` Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 0/7] " Karthik Nayak
` (4 subsequent siblings)
11 siblings, 2 replies; 143+ messages in thread
From: Phillip Wood @ 2025-02-11 17:03 UTC (permalink / raw)
To: Karthik Nayak, git; +Cc: ps, jltobler
Hi Karthik
On 07/02/2025 07:34, Karthik Nayak wrote:
> Git's reference updates are traditionally atomic
I'm nitpicking but the updates aren't actually atomic, if a transaction
updates two refs then it is possible for another process to see the one
ref pointing to the new value and the other pointing to the old value.
> - when updating
> multiple references in a transaction, either all updates succeed or none
> do. While this behavior is generally desirable,
Isn't that the whole point of transactions?
> it can be limiting in> certain scenarios, particularly with the reftable backend where batching
> multiple reference updates is more efficient than performing them
> sequentially.
>
> This series introduces support for partial reference transactions,
> allowing individual reference updates to fail while letting others
> proceed.
This sounds like it's abusing ref transactions to implement a
performance optimization. I wonder if it would be better to provide that
via a different interface than shares the same underling implementation
as transactions. That would make it clear to someone reading the code
that individual ref updates can fail without affecting the rest. Burying
that detail in a flag makes it rather easy to miss.
Best Wishes
Phillip
> This capability is exposed through git-update-ref's
> `--allow-partial` flag, which can be used in `--stdin` mode to batch
> updates and handle failures gracefully.
> The changes are structured to carefully build up this functionality:
>
> First, we clean up and consolidate the reference update checking logic.
> This includes removing duplicate checks in the files backend and moving
> refname tracking to the generic layer, which simplifies the codebase and
> prepares it for the new feature.
>
> We then restructure the reftable backend's transaction preparation code,
> extracting the update validation logic into a dedicated function. This
> not only improves code organization but sets the stage for implementing
> partial transaction support.
>
> With this groundwork in place, we implement the core partial transaction
> support in the refs subsystem. This adds the necessary infrastructure to
> track and report rejected updates while allowing transactions to proceed.
> All reference backends are modified to support this behavior when enabled.
>
> Finally, we expose this functionality to users through
> git-update-ref(1)'s `--allow-partial` flag, complete with test coverage
> and documentation. The flag is specifically limited to `--stdin` mode
> where batching multiple updates is most relevant.
>
> This enhancement improves Git's flexibility in handling reference
> updates while maintaining the safety of atomic transactions by default.
> It's particularly valuable for tools and workflows that need to handle
> reference update failures gracefully without abandoning the entire batch
> of updates.
>
> This series is based on top of bc204b7427 (The seventh batch, 2025-02-03).
> There were no conflicts noticed with topics in 'seen' or 'next'.
>
> ---
> Karthik Nayak (6):
> refs/files: remove duplicate check in `split_symref_update()`
> refs: move duplicate refname update check to generic layer
> refs/files: remove duplicate duplicates check
> refs/reftable: extract code from the transaction preparation
> refs: implement partial reference transaction support
> update-ref: add --allow-partial flag for stdin mode
>
> Documentation/git-update-ref.txt | 12 +-
> builtin/update-ref.c | 53 ++++-
> refs.c | 58 ++++-
> refs.h | 22 ++
> refs/files-backend.c | 97 ++------
> refs/packed-backend.c | 49 ++--
> refs/refs-internal.h | 19 +-
> refs/reftable-backend.c | 494 +++++++++++++++++++--------------------
> t/t1400-update-ref.sh | 191 +++++++++++++++
> 9 files changed, 633 insertions(+), 362 deletions(-)
> ---
>
>
>
> ---
>
> base-commit: bc204b742735ae06f65bb20291c95985c9633b7f
> change-id: 20241206-245-partially-atomic-ref-updates-9fe8b080345c
>
> Thanks
> - Karthik
>
>
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 0/6] refs: introduce support for partial reference transactions
2025-02-11 17:03 ` [PATCH 0/6] refs: introduce support for partial reference transactions Phillip Wood
@ 2025-02-11 17:40 ` Phillip Wood
2025-02-12 12:36 ` Karthik Nayak
2025-02-12 12:34 ` Karthik Nayak
1 sibling, 1 reply; 143+ messages in thread
From: Phillip Wood @ 2025-02-11 17:40 UTC (permalink / raw)
To: Karthik Nayak, git; +Cc: ps, jltobler
On 11/02/2025 17:03, Phillip Wood wrote:
> On 07/02/2025 07:34, Karthik Nayak wrote:
>
>> This series introduces support for partial reference transactions,
>> allowing individual reference updates to fail while letting others
>> proceed.
Thinking about this some more it is possible to skip the checking the
current value of the ref so what is making the transaction fail? Is it
D/F conflicts or something else?
Best Wishes
Phillip
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 0/6] refs: introduce support for partial reference transactions
2025-02-11 17:03 ` [PATCH 0/6] refs: introduce support for partial reference transactions Phillip Wood
2025-02-11 17:40 ` Phillip Wood
@ 2025-02-12 12:34 ` Karthik Nayak
2025-02-19 14:34 ` Phillip Wood
1 sibling, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-02-12 12:34 UTC (permalink / raw)
To: phillip.wood, git; +Cc: ps, jltobler
[-- Attachment #1: Type: text/plain, Size: 2701 bytes --]
Phillip Wood <phillip.wood123@gmail.com> writes:
> Hi Karthik
>
> On 07/02/2025 07:34, Karthik Nayak wrote:
>> Git's reference updates are traditionally atomic
>
> I'm nitpicking but the updates aren't actually atomic, if a transaction
> updates two refs then it is possible for another process to see the one
> ref pointing to the new value and the other pointing to the old value.
>
Good point. This is true in the case of the files backend, since updates
involve locking individual files and during the commit phase, there is a
possibility that one ref is updated while the other is yet to be
(committing of the lock is not global but rather per ref file).
However this is not the case with the reftable backend, there, updates
are written to a new table and committed at the end after locking the
table. So in the reftable backend, this is indeed atomic.
>> - when updating
>> multiple references in a transaction, either all updates succeed or none
>> do. While this behavior is generally desirable,
>
> Isn't that the whole point of transactions?
>
Yup, this is the point of having a transaction indeed.
>> it can be limiting in> certain scenarios, particularly with the reftable backend where batching
>> multiple reference updates is more efficient than performing them
>> sequentially.
>>
>> This series introduces support for partial reference transactions,
>> allowing individual reference updates to fail while letting others
>> proceed.
>
> This sounds like it's abusing ref transactions to implement a
> performance optimization.
I understand where you're coming from. This is definitely a stray from
the regular atomic behavior, that transactions promise. But I would say
this is more of an exception handling for the regular transaction
mechanism and AFAIK this is also something that some of the databases
support (see EXCEPTION in PostgreSQL).
Overall, we're adding an exception handling support to the existing
transaction interface.
> I wonder if it would be better to provide that
> via a different interface than shares the same underling implementation
> as transactions. That would make it clear to someone reading the code
> that individual ref updates can fail without affecting the rest. Burying
> that detail in a flag makes it rather easy to miss.
>
Thinking this out, having a different interface sound good, but I feel
we'd end up with the same structure as currently presented in this
series. Only other way is to really split the implementation to support
partial transactions as a entity of its own. In that case, we'd end up
with code duplication.
Do you think you can expand a little more here?
> Best Wishes
>
> Phillip
>
Thanks for engaging!
[snip]
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 0/6] refs: introduce support for partial reference transactions
2025-02-11 17:40 ` Phillip Wood
@ 2025-02-12 12:36 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-12 12:36 UTC (permalink / raw)
To: phillip.wood, git; +Cc: ps, jltobler
[-- Attachment #1: Type: text/plain, Size: 752 bytes --]
Phillip Wood <phillip.wood123@gmail.com> writes:
> On 11/02/2025 17:03, Phillip Wood wrote:
>> On 07/02/2025 07:34, Karthik Nayak wrote:
> >
>>> This series introduces support for partial reference transactions,
>>> allowing individual reference updates to fail while letting others
>>> proceed.
>
> Thinking about this some more it is possible to skip the checking the
> current value of the ref so what is making the transaction fail? Is it
> D/F conflicts or something else?
>
It could be a multitude of issues, to name a some:
- D/F conflicts
- Unexpected old oid/target
- dangling symrefs
So this gives a hatch to partially commit parts of a transaction while
also notifying the user about parts which failed.
> Best Wishes
>
> Phillip
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 0/6] refs: introduce support for partial reference transactions
2025-02-12 12:34 ` Karthik Nayak
@ 2025-02-19 14:34 ` Phillip Wood
2025-02-19 15:10 ` Patrick Steinhardt
2025-02-21 11:50 ` Karthik Nayak
0 siblings, 2 replies; 143+ messages in thread
From: Phillip Wood @ 2025-02-19 14:34 UTC (permalink / raw)
To: Karthik Nayak, phillip.wood, git; +Cc: ps, jltobler
Hi Karthik
On 12/02/2025 12:34, Karthik Nayak wrote:
>> On 07/02/2025 07:34, Karthik Nayak wrote:
>>> Git's reference updates are traditionally atomic
>>
>> I'm nitpicking but the updates aren't actually atomic, if a transaction
>> updates two refs then it is possible for another process to see the one
>> ref pointing to the new value and the other pointing to the old value.
>>
>
> Good point. This is true in the case of the files backend, since updates
> involve locking individual files and during the commit phase, there is a
> possibility that one ref is updated while the other is yet to be
> (committing of the lock is not global but rather per ref file).
>
> However this is not the case with the reftable backend, there, updates
> are written to a new table and committed at the end after locking the
> table. So in the reftable backend, this is indeed atomic.
Ah, interesting. That explains why batching updates is so much more
efficient when using the reftable backend.
>>> This series introduces support for partial reference transactions,
>>> allowing individual reference updates to fail while letting others
>>> proceed.
>>
>> This sounds like it's abusing ref transactions to implement a
>> performance optimization.
>
> I understand where you're coming from. This is definitely a stray from
> the regular atomic behavior, that transactions promise. But I would say
> this is more of an exception handling for the regular transaction
> mechanism and AFAIK this is also something that some of the databases
> support (see EXCEPTION in PostgreSQL).
>
> Overall, we're adding an exception handling support to the existing
> transaction interface.
My understanding of exception handling is that if an error occurs then
an error handler is called (reading [1] that seems to be what PostgreSQL
does as well). Is that what is being proposed here? I thought this
series added a flag to ignore errors rather than provide a way to handle
them.
[1]
https://www.postgresql.org/docs/current/plpgsql-control-structures.html#PLPGSQL-ERROR-TRAPPING
>> I wonder if it would be better to provide that
>> via a different interface than shares the same underling implementation
>> as transactions. That would make it clear to someone reading the code
>> that individual ref updates can fail without affecting the rest. Burying
>> that detail in a flag makes it rather easy to miss.
>>
>
> Thinking this out, having a different interface sound good, but I feel
> we'd end up with the same structure as currently presented in this
> series. Only other way is to really split the implementation to support
> partial transactions as a entity of its own. In that case, we'd end up
> with code duplication.
>
> Do you think you can expand a little more here?
I was thinking of a function that took a list of refs to update and made
a best effort to update them, ignoring any updates that fail.
My concern with adding a flag to ignore errors in the transaction api is
that a partial transaction is a contradiction in terms. I'm also
concerned that it seems to be ignoring all errors. I'd be happier if
there was someway for the caller to specify which errors to ignore or if
the caller could provide a callback to handle any errors. That way a
caller could ignore d/f conflicts but still cause the transaction to
fail if there was an i/o or could create a reference if it did not exist
but leave it unchanged if it did exist.
Best Wishes
Phillip
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 0/6] refs: introduce support for partial reference transactions
2025-02-19 14:34 ` Phillip Wood
@ 2025-02-19 15:10 ` Patrick Steinhardt
2025-02-21 11:50 ` Karthik Nayak
1 sibling, 0 replies; 143+ messages in thread
From: Patrick Steinhardt @ 2025-02-19 15:10 UTC (permalink / raw)
To: Phillip Wood; +Cc: Karthik Nayak, phillip.wood, git, jltobler
On Wed, Feb 19, 2025 at 02:34:15PM +0000, Phillip Wood wrote:
> On 12/02/2025 12:34, Karthik Nayak wrote:
> > Thinking this out, having a different interface sound good, but I feel
> > we'd end up with the same structure as currently presented in this
> > series. Only other way is to really split the implementation to support
> > partial transactions as a entity of its own. In that case, we'd end up
> > with code duplication.
> >
> > Do you think you can expand a little more here?
>
> I was thinking of a function that took a list of refs to update and made a
> best effort to update them, ignoring any updates that fail.
>
> My concern with adding a flag to ignore errors in the transaction api is
> that a partial transaction is a contradiction in terms. I'm also concerned
> that it seems to be ignoring all errors. I'd be happier if there was someway
> for the caller to specify which errors to ignore or if the caller could
> provide a callback to handle any errors. That way a caller could ignore d/f
> conflicts but still cause the transaction to fail if there was an i/o or
> could create a reference if it did not exist but leave it unchanged if it
> did exist.
This is a fair point I think, and it's also something that I called out
in [1]. We shouldn't blanket-ignore all errors, but instead only ignore
a well-known subset of errors and then record the exact failure reason.
This allows for better and more unified error reporting, and would also
allow us to easily build mechanisms where callers can specify that we
are only expected to ignore a subset of those well-defined errors.
But I also think that this is another selling point for continuing to
build on top of the ref transaction. If we now want to record and ignore
specific errors, only, then this falls out quite naturally from the
current design of ref transactions. We already have all the ref updates
in there, so we can then simply set an `enum ref_transaction_error`
field for each of the failed updates.
Eventually, I think we could also allow for modes where we declare only
a subset of reference updates to be allowed to fail. I don't have a
specific usecase for this, and don't think it needs to be implemented
without one. But it's another thing that we could implement on top of
transactions rather trivially.
Patrick
[1]: <Z6YxA4BlhNwbeYk-@pks.im>
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 5/6] refs: implement partial reference transaction support
2025-02-07 16:12 ` Patrick Steinhardt
@ 2025-02-21 10:33 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-21 10:33 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git, jltobler
[-- Attachment #1: Type: text/plain, Size: 3761 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> On Fri, Feb 07, 2025 at 08:34:40AM +0100, Karthik Nayak wrote:
>> Git's reference transactions are all-or-nothing: either all updates
>> succeed, or none do. While this atomic behavior is generally desirable,
>> it can be suboptimal when using the reftable backend, where batching
>> multiple reference updates into a single transaction is more efficient
>> than performing them sequentially.
>
> In fact it's even inefficient for the "files" backend. The whole
> machinery around creating a new transaction, preparing it, committing it
> and then cleaning up its state does bring a bunch of overhead with it.
> But true, for the "reftable" backend it's way more impactful.
>
>> diff --git a/refs.c b/refs.c
>> index b420a120102b3793168598b885bba68e4f5f5f03..75dbd84acbc41658d4b8b6b5e7763c04e78d0061 100644
>> --- a/refs.c
>> +++ b/refs.c
>> @@ -1211,6 +1212,14 @@ void ref_transaction_free(struct ref_transaction *transaction)
>> free(transaction);
>> }
>>
>> +void ref_transaction_add_rejection(struct ref_transaction *transaction,
>> + size_t update_idx, struct strbuf *err)
>
> "add" to me sounds like you're adding a new thingy to the transaction,
> but you rather update something. How about `ref_update_set_rejected()`
> or `ref_transacton_set_rejected()`?
>
Fair enough, I've changed it to `ref_transacton_set_rejected()`.
>> +{
>> + struct ref_update *update = transaction->updates[update_idx];
>
> Do we want to `BUG()` in case `update_idx >= transaction->nr`?
>
Good point, let me add that in.
>> + update->rejected = 1;
>> + strbuf_addbuf(&update->rejection_err, err);
>> +}
>
> Do we really need a string as rejection error? I'd expect that the set
> of failures that lead to rejection should be rather limited, which means
> that we could use an enum instead. This would unify the errors across
> backends and also allows us to figure out the root cause of rejection in
> other subsystems.
>
> If we introduced an enum, we could eventually even iterate a bit on the
> mechanism and rather trivially tell the backends which kind of failures
> are acceptable. As an example, a conflicting ref update may for example
> be ignored and not cause failure, a conflicting path name might cause
> failure.
>
That's a good point, This also allows us to eventually extend the flag
to do something like you mentioned where `--allow-partial=all` would
skip all errors. But one could optimize to also say
`--allow-partial=name_conflict,old_value` to only skip errors due to
refname conflicts and invalid/incorrect old_value.
I'll add another commit to introduce and add 'enum transaction_error'
and build around it.
>> diff --git a/refs/files-backend.c b/refs/files-backend.c
>> index 9fc5454678340dd7c72539bfa0f15ee7eb24b1ff..99ec29164fbd30635125cc2325aab3d300cf906c 100644
>> --- a/refs/files-backend.c
>> +++ b/refs/files-backend.c
>> @@ -2852,8 +2852,18 @@ static int files_transaction_prepare(struct ref_store *ref_store,
>>
>> ret = lock_ref_for_update(refs, update, transaction,
>> head_ref, err);
>> - if (ret)
>> + if (ret) {
>
> I wonder whether we want to accept all failures. Some failures are
> certainly benign, like for example mismatching expected OIDs or a
> conflict due to a preexisting ref that blocks the path. But other kinds
> of failures which are unexpected might be a bit more on the dangerous
> side to accept, so I think we should be careful here.
>
Fair point. For the current implementation with the enum design
discussed above. I think it would be best to skip over all user
oriented errors. But any system errors would actually cease the
transaction. We can further iterate on this later.
> The same comment also applies to the other backends.
>
> Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 6/6] update-ref: add --allow-partial flag for stdin mode
2025-02-07 16:12 ` Patrick Steinhardt
@ 2025-02-21 11:45 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-21 11:45 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git, jltobler
[-- Attachment #1: Type: text/plain, Size: 3335 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> On Fri, Feb 07, 2025 at 08:34:41AM +0100, Karthik Nayak wrote:
>> diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
>> index 9e6935d38d031b4890135e0cce36fffcc349ac1d..529d3c15404cdc13216219fba6f56dde91f4909c 100644
>> --- a/Documentation/git-update-ref.txt
>> +++ b/Documentation/git-update-ref.txt
>> @@ -8,7 +8,7 @@ git-update-ref - Update the object name stored in a ref safely
>> SYNOPSIS
>> --------
>> [verse]
>> -'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z])
>> +'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z] [--allow-partial])
>
> I think it's time that we start to split this line into multiple lines :)
>
Yes, indeed, will do.
>> diff --git a/builtin/update-ref.c b/builtin/update-ref.c
>> index 4d35bdc4b4b57937112e6c4c9740420b1f1771e5..83dcb7d8d73f423226c36b61374c86c6b29ec756 100644
>> --- a/builtin/update-ref.c
>> +++ b/builtin/update-ref.c
>> @@ -562,6 +563,30 @@ static void parse_cmd_abort(struct ref_transaction *transaction,
>> report_ok("abort");
>> }
>>
>> +static void print_rejected_refs(const char *refname,
>> + const struct object_id *old_oid,
>> + const struct object_id *new_oid,
>> + const char *old_target,
>> + const char *new_target,
>> + const struct strbuf *reason,
>> + void *cb_data UNUSED)
>> +{
>> + struct strbuf sb = STRBUF_INIT;
>> + char space = ' ';
>> +
>> + if (!line_termination)
>> + space = line_termination;
>> +
>> + strbuf_addf(&sb, "rejected%c%s%c%s%c%c%s%c%s%c", space,
>
> Whew, that's a lot of placeholders.
>
True. More prone to errors too.
>> @@ -723,7 +754,8 @@ int cmd_update_ref(int argc,
>> const char *refname, *oldval;
>> struct object_id oid, oldoid;
>> int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;
>> - int create_reflog = 0;
>> + int create_reflog = 0, allow_partial = 0;
>> +
>> struct option options[] = {
>> OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
>> OPT_BOOL('d', NULL, &delete, N_("delete the reference")),
>> @@ -732,6 +764,7 @@ int cmd_update_ref(int argc,
>> OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
>> OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
>> OPT_BOOL( 0 , "create-reflog", &create_reflog, N_("create a reflog")),
>> + OPT_BOOL('0', "allow-partial", &allow_partial, N_("allow partial transactions")),
>
> You can use `OPT_BIT()` to set a specific bit in a flags field..
>
That would be cleaner, will fix.
>> @@ -749,13 +782,19 @@ int cmd_update_ref(int argc,
>> }
>>
>> if (read_stdin) {
>> + unsigned int flags = 0;
>> +
>> + if (allow_partial)
>> + flags |= REF_TRANSACTION_ALLOW_PARTIAL;
>> +
>> if (delete || argc > 0)
>> usage_with_options(git_update_ref_usage, options);
>> if (end_null)
>> line_termination = '\0';
>> - update_refs_stdin();
>> + update_refs_stdin(flags);
>> return 0;
>> - }
>> + } else if (allow_partial)
>> + die("--allow-partial can only be used with --stdin");
>>
>> if (end_null)
>> usage_with_options(git_update_ref_usage, options);
>
> The implementation is quite simple, nice.
>
> Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH 0/6] refs: introduce support for partial reference transactions
2025-02-19 14:34 ` Phillip Wood
2025-02-19 15:10 ` Patrick Steinhardt
@ 2025-02-21 11:50 ` Karthik Nayak
1 sibling, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-21 11:50 UTC (permalink / raw)
To: Phillip Wood, phillip.wood, git; +Cc: ps, jltobler
[-- Attachment #1: Type: text/plain, Size: 4382 bytes --]
Phillip Wood <phillip.wood123@gmail.com> writes:
> Hi Karthik
>
> On 12/02/2025 12:34, Karthik Nayak wrote:
>>> On 07/02/2025 07:34, Karthik Nayak wrote:
>>>> Git's reference updates are traditionally atomic
>>>
>>> I'm nitpicking but the updates aren't actually atomic, if a transaction
>>> updates two refs then it is possible for another process to see the one
>>> ref pointing to the new value and the other pointing to the old value.
>>>
>>
>> Good point. This is true in the case of the files backend, since updates
>> involve locking individual files and during the commit phase, there is a
>> possibility that one ref is updated while the other is yet to be
>> (committing of the lock is not global but rather per ref file).
>>
>> However this is not the case with the reftable backend, there, updates
>> are written to a new table and committed at the end after locking the
>> table. So in the reftable backend, this is indeed atomic.
>
> Ah, interesting. That explains why batching updates is so much more
> efficient when using the reftable backend.
>
>>>> This series introduces support for partial reference transactions,
>>>> allowing individual reference updates to fail while letting others
>>>> proceed.
>>>
>>> This sounds like it's abusing ref transactions to implement a
>>> performance optimization.
>>
>> I understand where you're coming from. This is definitely a stray from
>> the regular atomic behavior, that transactions promise. But I would say
>> this is more of an exception handling for the regular transaction
>> mechanism and AFAIK this is also something that some of the databases
>> support (see EXCEPTION in PostgreSQL).
>>
>> Overall, we're adding an exception handling support to the existing
>> transaction interface.
>
> My understanding of exception handling is that if an error occurs then
> an error handler is called (reading [1] that seems to be what PostgreSQL
> does as well). Is that what is being proposed here? I thought this
> series added a flag to ignore errors rather than provide a way to handle
> them.
>
That is correct, and while the current implementation is to ignore them.
It does make way for building something like that in the future.
> [1]
> https://www.postgresql.org/docs/current/plpgsql-control-structures.html#PLPGSQL-ERROR-TRAPPING
>
>>> I wonder if it would be better to provide that
>>> via a different interface than shares the same underling implementation
>>> as transactions. That would make it clear to someone reading the code
>>> that individual ref updates can fail without affecting the rest. Burying
>>> that detail in a flag makes it rather easy to miss.
>>>
>>
>> Thinking this out, having a different interface sound good, but I feel
>> we'd end up with the same structure as currently presented in this
>> series. Only other way is to really split the implementation to support
>> partial transactions as a entity of its own. In that case, we'd end up
>> with code duplication.
>>
>> Do you think you can expand a little more here?
>
> I was thinking of a function that took a list of refs to update and made
> a best effort to update them, ignoring any updates that fail.
>
Yes, this is what we want too, but in the context of transactions.
Without transactions, this is already possible to build albeit on the
user side with some simple scripts.
> My concern with adding a flag to ignore errors in the transaction api is
> that a partial transaction is a contradiction in terms. I'm also
> concerned that it seems to be ignoring all errors. I'd be happier if
> there was someway for the caller to specify which errors to ignore or if
> the caller could provide a callback to handle any errors. That way a
> caller could ignore d/f conflicts but still cause the transaction to
> fail if there was an i/o or could create a reference if it did not exist
> but leave it unchanged if it did exist.
>
Yeah, I think this is also something that Patrick raised and something I
will tackle in the next version of this patch series.
The next version will skip all user oriented errors (errors which can be
fixed by changing the user input), while still catch system errors (I/O
, memory ...). I also see how this can be extended to allow users to
nit-pick which errors they don't care about. But that is something I
don't plan to tackle for now.
> Best Wishes
>
> Phillip
Thanks for your inputs!
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH v2 0/7] refs: introduce support for partial reference transactions
2025-02-07 7:34 [PATCH 0/6] refs: introduce support for partial reference transactions Karthik Nayak
` (6 preceding siblings ...)
2025-02-11 17:03 ` [PATCH 0/6] refs: introduce support for partial reference transactions Phillip Wood
@ 2025-02-25 9:29 ` Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 1/7] refs/files: remove redundant check in split_symref_update() Karthik Nayak
` (6 more replies)
2025-03-05 17:38 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Karthik Nayak
` (3 subsequent siblings)
11 siblings, 7 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-25 9:29 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Documentation/git-update-ref.adoc | 21 +-
builtin/update-ref.c | 74 +++++-
refs.c | 101 ++++++--
refs.h | 78 ++++--
refs/files-backend.c | 260 ++++++++------------
refs/packed-backend.c | 70 +++---
refs/refs-internal.h | 22 +-
refs/reftable-backend.c | 496 +++++++++++++++++++-------------------
t/t1400-update-ref.sh | 216 +++++++++++++++++
9 files changed, 849 insertions(+), 489 deletions(-)
Karthik Nayak (7):
refs/files: remove redundant check in split_symref_update()
refs: move duplicate refname update check to generic layer
refs/files: remove duplicate duplicates check
refs/reftable: extract code from the transaction preparation
refs: introduce enum-based transaction error types
refs: implement partial reference transaction support
update-ref: add --allow-partial flag for stdin mode
Git's reference updates are traditionally all or nothing - when updating
multiple references in a transaction, either all updates succeed or none
do. While this behavior is generally desirable, it can be limiting in
certain scenarios, particularly with the reftable backend where batching
multiple reference updates is more efficient than performing them
sequentially.
This series introduces support for partial reference transactions,
allowing individual reference updates to fail while letting others
proceed. This capability is exposed through git-update-ref's
`--allow-partial` flag, which can be used in `--stdin` mode to batch
updates and handle failures gracefully.
The changes are structured to carefully build up this functionality:
First, we clean up and consolidate the reference update checking logic.
This includes removing duplicate checks in the files backend and moving
refname tracking to the generic layer, which simplifies the codebase and
prepares it for the new feature.
We then restructure the reftable backend's transaction preparation code,
extracting the update validation logic into a dedicated function. This
not only improves code organization but sets the stage for implementing
partial transaction support.
To ensure we only skip errors which are user-oriented, we introduce
typed errors for transactions with 'enum transaction_error'. We extend
the existing errors to include other scenarios and use this new errors
throughout the refs code.
With this groundwork in place, we implement the core partial transaction
support in the refs subsystem. This adds the necessary infrastructure to
track and report rejected updates while allowing transactions to proceed.
All reference backends are modified to support this behavior when enabled.
Finally, we expose this functionality to users through
git-update-ref(1)'s `--allow-partial` flag, complete with test coverage
and documentation. The flag is specifically limited to `--stdin` mode
where batching multiple updates is most relevant.
This enhancement improves Git's flexibility in handling reference
updates while maintaining the safety of atomic transactions by default.
It's particularly valuable for tools and workflows that need to handle
reference update failures gracefully without abandoning the entire batch
of updates.
This series is based on top of b838bf1938 (Merge branch 'master' of
https://github.com/j6t/gitk, 2025-02-20) with Patrick's series 'refs:
batch refname availability checks' [1] merged in.
[1]: https://lore.kernel.org/all/20250217-pks-update-ref-optimization-v1-0-a2b6d87a24af@pks.im/
---
Changes in v2:
- Introduce and use structured errors. This consolidates the errors
and their handling between the ref backends.
- In the previous version, we skipped over all failures. This include
system failures such as low memory or IO problems. Let's instead, only
skip user-oriented failures, such as invalid old OID and so on.
- Change the rejection function name to `ref_transaction_set_rejected()`.
- Modify the commit messages and documentation to be a little more
verbose.
- Link to v1: https://lore.kernel.org/r/20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com
Range-diff versus v1:
1: e48e562f27 ! 1: 4a1b748e7a refs/files: remove duplicate check in `split_symref_update()`
@@ Metadata
Author: Karthik Nayak <karthik.188@gmail.com>
## Commit message ##
- refs/files: remove duplicate check in `split_symref_update()`
+ refs/files: remove redundant check in split_symref_update()
- In split_symref_update(), there were two redundant checks:
- - At the start: checking if refname exists in `affected_refnames`.
- - After adding refname: checking if the item added to
- `affected_refnames` contains the util field.
+ In `split_symref_update()`, there were two checks for duplicate
+ refnames:
- Remove the second check since the first one already prevents duplicate
- refnames from being added to the transaction updates.
+ - At the start, `string_list_has_string()` ensures the refname is not
+ already in `affected_refnames`, preventing duplicates from being
+ added.
- Since this is the only place that utilizes the `item->util` value, avoid
- setting the value in the first place and cleanup code around it.
+ - After adding the refname, another check verifies whether the newly
+ inserted item has a `util` value.
+
+ The second check is unnecessary because the first one guarantees that
+ `string_list_insert()` will never encounter a preexisting entry.
+
+ Since `item->util` is only used in this context, remove the assignment and
+ simplify the surrounding code.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
2: b5efdd3149 ! 2: 1cfb4f91b5 refs: move duplicate refname update check to generic layer
@@ Commit message
- Add a `string_list` field `refnames` to `ref_transaction` to contain
all the references in a transaction. This field is updated whenever
- a new update is added.
+ a new update is added via `ref_transaction_add_update`, so manual
+ additions in reference backends are dropped.
- Modify the backends to use this field internally as needed. The
backends need to check if an update for refname already exists when
splitting symrefs or adding an update for 'HEAD'.
- - In the reftable backend, in `reftable_be_transaction_prepare()`,
- move the instance of `string_list_has_string()` above
- `ref_transaction_add_update()` to check before the reference is
- added.
+ - In the reftable backend, within `reftable_be_transaction_prepare()`,
+ move the `string_list_has_string()` check above
+ `ref_transaction_add_update()`. Since `ref_transaction_add_update()`
+ automatically adds the refname to `transaction->refnames`,
+ performing the check after will always return true, so we perform
+ the check before adding the update.
This helps reduce duplication of functionality between the backends and
makes it easier to make changes in a more centralized manner.
@@ refs/files-backend.c: static int split_symref_update(struct ref_update *update,
return 0;
}
-@@ refs/files-backend.c: struct files_transaction_backend_data {
- static int lock_ref_for_update(struct files_ref_store *refs,
- struct ref_update *update,
+@@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
struct ref_transaction *transaction,
-- const char *head_ref,
+ const char *head_ref,
+ struct string_list *refnames_to_check,
- struct string_list *affected_refnames,
-- struct strbuf *err)
-+ const char *head_ref, struct strbuf *err)
+ struct strbuf *err)
{
struct strbuf referent = STRBUF_INIT;
- int mustexist = ref_update_expects_existing_old_ref(update);
@@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
update->flags |= REF_DELETING;
@@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
lock->count++;
} else {
ret = lock_raw_ref(refs, update->refname, mustexist,
-- affected_refnames,
-+ &transaction->refnames,
- &lock, &referent,
- &update->type, err);
+- refnames_to_check, affected_refnames,
+- &lock, &referent,
+- &update->type, err);
++ refnames_to_check, &transaction->refnames,
++ &lock, &referent, &update->type, err);
if (ret) {
+ char *reason;
+
@@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
* of processing the split-off update, so we
* don't have to do it here.
@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref
size_t i;
int ret = 0;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
+ struct string_list refnames_to_check = STRING_LIST_INIT_NODUP;
char *head_ref = NULL;
int head_type;
- struct files_transaction_backend_data *backend_data;
@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref_store,
transaction->backend_data = backend_data;
@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref
/*
@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref_store,
- struct ref_update *update = transaction->updates[i];
ret = lock_ref_for_update(refs, update, transaction,
-- head_ref, &affected_refnames, err);
-+ head_ref, err);
+ head_ref, &refnames_to_check,
+- &affected_refnames, err);
++ err);
if (ret)
goto cleanup;
+@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref_store,
+ * So instead, we accept the race for now.
+ */
+ if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
+- &affected_refnames, NULL, 0, err)) {
++ &transaction->refnames, NULL, 0, err)) {
+ ret = TRANSACTION_NAME_CONFLICT;
+ goto cleanup;
+ }
@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref_store,
cleanup:
free(head_ref);
- string_list_clear(&affected_refnames, 0);
+ string_list_clear(&refnames_to_check, 0);
if (ret)
- files_transaction_cleanup(refs, transaction);
-@@ refs/files-backend.c: static int files_transaction_finish_initial(struct files_ref_store *refs,
- {
- size_t i;
- int ret = 0;
-- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
- struct ref_transaction *packed_transaction = NULL;
- struct ref_transaction *loose_transaction = NULL;
-
@@ refs/files-backend.c: static int files_transaction_finish_initial(struct files_ref_store *refs,
if (transaction->state != REF_TRANSACTION_PREPARED)
BUG("commit called for transaction that is not prepared");
@@ refs/files-backend.c: static int files_transaction_finish_initial(struct files_r
BUG("initial ref transaction called with existing refs");
packed_transaction = ref_store_transaction_begin(refs->packed_ref_store,
-@@ refs/files-backend.c: static int files_transaction_finish_initial(struct files_ref_store *refs,
- BUG("initial ref transaction with old_sha1 set");
-
- if (refs_verify_refname_available(&refs->base, update->refname,
-- &affected_refnames, NULL, 1, err)) {
-+ &transaction->refnames, NULL, 1, err)) {
- ret = TRANSACTION_NAME_CONFLICT;
- goto cleanup;
- }
-@@ refs/files-backend.c: static int files_transaction_finish_initial(struct files_ref_store *refs,
- if (packed_transaction)
- ref_transaction_free(packed_transaction);
- transaction->state = REF_TRANSACTION_CLOSED;
-- string_list_clear(&affected_refnames, 0);
- return ret;
- }
-
## refs/packed-backend.c ##
@@ refs/packed-backend.c: int is_packed_transaction_needed(struct ref_store *ref_store,
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
reftable_be_downcast(ref_store, REF_STORE_WRITE|REF_STORE_MAIN, "ref_transaction_prepare");
struct strbuf referent = STRBUF_INIT, head_referent = STRBUF_INIT;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
+ struct string_list refnames_to_check = STRING_LIST_INIT_NODUP;
struct reftable_transaction_data *tx_data = NULL;
struct reftable_backend *be;
- struct object_id head_oid;
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->updates[i], err);
if (ret)
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
}
ret = reftable_backend_read_ref(be, rewritten_ref,
-@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_store *ref_store,
- * at a later point.
- */
- ret = refs_verify_refname_available(ref_store, u->refname,
-- &affected_refnames, NULL,
-+ &transaction->refnames, NULL,
- transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
- err);
- if (ret < 0)
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_store *ref_store,
if (!strcmp(rewritten_ref, "HEAD"))
new_flags |= REF_UPDATE_VIA_HEAD;
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
}
}
+@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_store *ref_store,
+ }
+
+ string_list_sort(&refnames_to_check);
+- ret = refs_verify_refnames_available(ref_store, &refnames_to_check, &affected_refnames, NULL,
++ ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
++ &transaction->refnames, NULL,
+ transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
+ err);
+ if (ret < 0)
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_store *ref_store,
strbuf_addf(err, _("reftable: transaction prepare: %s"),
reftable_error_str(ret));
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
- string_list_clear(&affected_refnames, 0);
strbuf_release(&referent);
strbuf_release(&head_referent);
-
+ string_list_clear(&refnames_to_check, 0);
3: 1f1c261afd = 3: 91e96c9048 refs/files: remove duplicate duplicates check
4: 556fd87651 ! 4: c47a020dc5 refs/reftable: extract code from the transaction preparation
@@ refs/reftable-backend.c: static int queue_transaction_update(struct reftable_ref
return 0;
}
-+static int prepare_single_update(struct ref_store *ref_store,
-+ struct reftable_ref_store *refs,
++static int prepare_single_update(struct reftable_ref_store *refs,
+ struct reftable_transaction_data *tx_data,
+ struct ref_transaction *transaction,
+ struct reftable_backend *be,
+ struct ref_update *u,
++ struct string_list *refnames_to_check,
+ unsigned int head_type,
+ struct strbuf *head_referent,
+ struct strbuf *referent,
@@ refs/reftable-backend.c: static int queue_transaction_update(struct reftable_ref
+ * can output a proper error message instead of failing
+ * at a later point.
+ */
-+ ret = refs_verify_refname_available(ref_store, u->refname,
-+ &transaction->refnames, NULL,
-+ transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
-+ err);
-+ if (ret < 0)
-+ return ret;
++ string_list_append(refnames_to_check, u->refname);
+
+ /*
+ * There is no need to write the reference deletion
@@ refs/reftable-backend.c: static int queue_transaction_update(struct reftable_ref
+ if (ret > 0) {
+ /* The reference does not exist, but we expected it to. */
+ strbuf_addf(err, _("cannot lock ref '%s': "
++
++
+ "unable to resolve reference '%s'"),
+ ref_update_original_update_refname(u), u->refname);
+ return -1;
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
- * stack.
- */
- ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
-+ ret = prepare_single_update(ref_store, refs, tx_data,
-+ transaction, be,
-+ transaction->updates[i], head_type,
++ ret = prepare_single_update(refs, tx_data, transaction, be,
++ transaction->updates[i],
++ &refnames_to_check, head_type,
+ &head_referent, &referent, err);
if (ret)
goto done;
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
- * can output a proper error message instead of failing
- * at a later point.
- */
-- ret = refs_verify_refname_available(ref_store, u->refname,
-- &transaction->refnames, NULL,
-- transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
-- err);
-- if (ret < 0)
-- goto done;
+- string_list_append(&refnames_to_check, u->refname);
-
- /*
- * There is no need to write the reference deletion
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
- }
}
- transaction->backend_data = tx_data;
+ string_list_sort(&refnames_to_check);
-: ---------- > 5: ff7235f2b8 refs: introduce enum-based transaction error types
5: 14b9657c99 ! 6: 74dce2ea61 refs: implement partial reference transaction support
@@ Commit message
Git's reference transactions are all-or-nothing: either all updates
succeed, or none do. While this atomic behavior is generally desirable,
- it can be suboptimal when using the reftable backend, where batching
- multiple reference updates into a single transaction is more efficient
- than performing them sequentially.
+ it can be suboptimal especially when using the reftable backend, where
+ batching multiple reference updates into a single transaction is more
+ efficient than performing them sequentially.
- Introduce partial transaction support through a new flag
- `REF_TRANSACTION_ALLOW_PARTIAL`. When this flag is set, individual
- reference updates that would normally fail the entire transaction are
- instead marked as rejected while allowing other updates to proceed. This
- provides more flexibility while maintaining transactional integrity
- where needed.
+ Introduce partial transaction support with a new flag,
+ 'REF_TRANSACTION_ALLOW_PARTIAL'. When enabled, this flag allows
+ individual reference updates that would typically cause the entire
+ transaction to fail due to non-system-related errors to be marked as
+ rejected while permitting other updates to proceed. Non-system-related
+ errors include issues caused by user-provided input values, whereas
+ system-related errors, such as I/O failures or memory issues, continue
+ to result in a full transaction failure. This approach enhances
+ flexibility while preserving transactional integrity where necessary.
The implementation introduces several key components:
- - Add 'rejected' and 'rejection_err' fields to struct `ref_update` to
- track failed updates and their failure reasons.
+ - Add 'rejection_err' field to struct `ref_update` to track failed
+ updates with failure reason.
- Modify reference backends (files, packed, reftable) to handle
- partial transactions by using `ref_transaction_add_rejection()`
+ partial transactions by using `ref_transaction_set_rejected()`
instead of failing the entire transaction when
`REF_TRANSACTION_ALLOW_PARTIAL` is set.
@@ Commit message
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
## refs.c ##
-@@ refs.c: void ref_transaction_free(struct ref_transaction *transaction)
- free(transaction->updates[i]->committer_info);
- free((char *)transaction->updates[i]->new_target);
- free((char *)transaction->updates[i]->old_target);
-+ strbuf_release(&transaction->updates[i]->rejection_err);
- free(transaction->updates[i]);
- }
- string_list_clear(&transaction->refnames, 0);
@@ refs.c: void ref_transaction_free(struct ref_transaction *transaction)
free(transaction);
}
-+void ref_transaction_add_rejection(struct ref_transaction *transaction,
-+ size_t update_idx, struct strbuf *err)
++void ref_transaction_set_rejected(struct ref_transaction *transaction,
++ size_t update_idx,
++ enum transaction_error err)
+{
-+ struct ref_update *update = transaction->updates[update_idx];
-+ update->rejected = 1;
-+ strbuf_addbuf(&update->rejection_err, err);
++ if (update_idx >= transaction->nr)
++ BUG("trying to set rejection on invalid update index");
++ transaction->updates[update_idx]->rejection_err = err;
+}
+
struct ref_update *ref_transaction_add_update(
struct ref_transaction *transaction,
const char *refname, unsigned int flags,
@@ refs.c: struct ref_update *ref_transaction_add_update(
+ transaction->updates[transaction->nr++] = update;
update->flags = flags;
++ update->rejection_err = TRANSACTION_OK;
-+ strbuf_init(&update->rejection_err, 0);
-+
update->new_target = xstrdup_or_null(new_target);
update->old_target = xstrdup_or_null(old_target);
- if ((flags & REF_HAVE_NEW) && new_oid)
@@ refs.c: void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
}
}
@@ refs.c: void ref_transaction_for_each_queued_update(struct ref_transaction *tran
+ for (size_t i = 0; i < transaction->nr; i++) {
+ struct ref_update *update = transaction->updates[i];
+
-+ if (!update->rejected)
++ if (!update->rejection_err)
+ continue;
+
+ cb(update->refname,
+ (update->flags & REF_HAVE_OLD) ? &update->old_oid : NULL,
+ (update->flags & REF_HAVE_NEW) ? &update->new_oid : NULL,
+ update->old_target, update->new_target,
-+ &update->rejection_err, cb_data);
++ update->rejection_err, cb_data);
+ }
+}
+
@@ refs.h: void ref_transaction_for_each_queued_update(struct ref_transaction *tran
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
-+ const struct strbuf *reason,
++ enum transaction_error err,
+ void *cb_data);
+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
+ ref_transaction_for_each_rejected_update_fn cb,
@@ refs.h: void ref_transaction_for_each_queued_update(struct ref_transaction *tran
## refs/files-backend.c ##
@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref_store,
-
ret = lock_ref_for_update(refs, update, transaction,
- head_ref, err);
+ head_ref, &refnames_to_check,
+ err);
- if (ret)
+ if (ret) {
-+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
-+ ref_transaction_add_rejection(transaction, i, err);
++ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL &&
++ ret != TRANSACTION_GENERIC_ERROR) {
++ ref_transaction_set_rejected(transaction, i, ret);
+
+ strbuf_setlen(err, 0);
-+ ret = 0;
++ ret = TRANSACTION_OK;
+
+ continue;
+ }
goto cleanup;
+ }
-+
if (update->flags & REF_DELETING &&
!(update->flags & REF_LOG_ONLY) &&
@@ refs/packed-backend.c
@@ refs/packed-backend.c: static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
* remain locked when it is done.
*/
- static int write_with_updates(struct packed_ref_store *refs,
-- struct string_list *updates,
-+ struct ref_transaction *transaction,
- struct strbuf *err)
+ static enum transaction_error write_with_updates(struct packed_ref_store *refs,
+- struct string_list *updates,
++ struct ref_transaction *transaction,
+ struct strbuf *err)
{
+ enum transaction_error ret = TRANSACTION_GENERIC_ERROR;
+ struct string_list *updates = &transaction->refnames;
struct ref_iterator *iter = NULL;
size_t i;
int ok;
-@@ refs/packed-backend.c: static int write_with_updates(struct packed_ref_store *refs,
- strbuf_addf(err, "cannot update ref '%s': "
+@@ refs/packed-backend.c: static enum transaction_error write_with_updates(struct packed_ref_store *refs,
"reference already exists",
update->refname);
+ ret = TRANSACTION_CREATE_EXISTS;
+
+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
-+ ref_transaction_add_rejection(transaction, i, err);
++ ref_transaction_set_rejected(transaction, i, ret);
+ strbuf_setlen(err, 0);
++ ret = 0;
+ continue;
+ }
+
goto error;
} else if (!oideq(&update->old_oid, iter->oid)) {
strbuf_addf(err, "cannot update ref '%s': "
-@@ refs/packed-backend.c: static int write_with_updates(struct packed_ref_store *refs,
- update->refname,
+@@ refs/packed-backend.c: static enum transaction_error write_with_updates(struct packed_ref_store *refs,
oid_to_hex(iter->oid),
oid_to_hex(&update->old_oid));
+ ret = TRANSACTION_INCORRECT_OLD_VALUE;
+
+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
-+ ref_transaction_add_rejection(transaction, i, err);
++ ref_transaction_set_rejected(transaction, i, ret);
+ strbuf_setlen(err, 0);
++ ret = 0;
+ continue;
+ }
+
goto error;
}
}
-@@ refs/packed-backend.c: static int write_with_updates(struct packed_ref_store *refs,
- "reference is missing but expected %s",
+@@ refs/packed-backend.c: static enum transaction_error write_with_updates(struct packed_ref_store *refs,
update->refname,
oid_to_hex(&update->old_oid));
+ return TRANSACTION_NONEXISTENT_REF;
+
+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
-+ ref_transaction_add_rejection(transaction, i, err);
++ ref_transaction_set_rejected(transaction, i, ret);
+ strbuf_setlen(err, 0);
++ ret = 0;
+ continue;
+ }
+
goto error;
}
}
+@@ refs/packed-backend.c: static enum transaction_error write_with_updates(struct packed_ref_store *refs,
+ write_error:
+ strbuf_addf(err, "error writing to %s: %s",
+ get_tempfile_path(refs->tempfile), strerror(errno));
++ ret = TRANSACTION_GENERIC_ERROR;
+
+ error:
+ ref_iterator_free(iter);
@@ refs/packed-backend.c: static int packed_transaction_prepare(struct ref_store *ref_store,
data->own_lock = 1;
}
-- if (write_with_updates(refs, &transaction->refnames, err))
-+ if (write_with_updates(refs, transaction, err))
+- ret = write_with_updates(refs, &transaction->refnames, err);
++ ret = write_with_updates(refs, transaction, err);
+ if (ret)
goto failure;
- transaction->state = REF_TRANSACTION_PREPARED;
## refs/refs-internal.h ##
-@@
-
- #include "refs.h"
- #include "iterator.h"
-+#include "strbuf.h"
- #include "string-list.h"
-
- struct fsck_options;
@@ refs/refs-internal.h: struct ref_update {
*/
- unsigned int index;
+ uint64_t index;
+ /*
-+ * Used in partial transactions to mark a given update as rejected,
-+ * with rejection reason.
++ * Used in partial transactions to mark if a given update was rejected.
+ */
-+ unsigned int rejected;
-+ struct strbuf rejection_err;
++ enum transaction_error rejection_err;
+
/*
* If this ref_update was split off of a symref update via
@@ refs/refs-internal.h: int refs_read_raw_ref(struct ref_store *ref_store, const c
+ * Mark a given update as rejected with a given reason. To be used in conjuction
+ * with the `REF_TRANSACTION_ALLOW_PARTIAL` flag to allow partial transactions.
+ */
-+void ref_transaction_add_rejection(struct ref_transaction *transaction,
-+ size_t update_idx, struct strbuf *err);
++void ref_transaction_set_rejected(struct ref_transaction *transaction,
++ size_t update_idx,
++ enum transaction_error err);
+
/*
* Add a ref_update with the specified properties to transaction, and
@@ refs/refs-internal.h: int refs_read_raw_ref(struct ref_store *ref_store, const c
## refs/reftable-backend.c ##
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_store *ref_store,
- transaction, be,
- transaction->updates[i], head_type,
+ transaction->updates[i],
+ &refnames_to_check, head_type,
&head_referent, &referent, err);
- if (ret)
-+
+ if (ret) {
-+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
-+ ref_transaction_add_rejection(transaction, i, err);
++ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL &&
++ ret != TRANSACTION_GENERIC_ERROR) {
++ ref_transaction_set_rejected(transaction, i, ret);
+
+ strbuf_setlen(err, 0);
-+ ret = 0;
++ ret = TRANSACTION_OK;
+
+ continue;
+ }
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
+ }
}
- transaction->backend_data = tx_data;
+ string_list_sort(&refnames_to_check);
6: 8c90e4201a ! 7: d79851e041 update-ref: add --allow-partial flag for stdin mode
@@ Commit message
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
- ## Documentation/git-update-ref.txt ##
-@@ Documentation/git-update-ref.txt: git-update-ref - Update the object name stored in a ref safely
+ ## Documentation/git-update-ref.adoc ##
+@@ Documentation/git-update-ref.adoc: git-update-ref - Update the object name stored in a ref safely
+
SYNOPSIS
--------
- [verse]
+-[verse]
-'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z])
-+'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z] [--allow-partial])
++[synopsis]
++git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
++ [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
++ [-m <reason>] [--no-deref] --stdin [-z] [--allow-partial]
DESCRIPTION
-----------
-@@ Documentation/git-update-ref.txt: performs all modifications together. Specify commands of the form:
+@@ Documentation/git-update-ref.adoc: performs all modifications together. Specify commands of the form:
With `--create-reflog`, update-ref will create a reflog for each ref
even if one would not ordinarily be created.
-+With `--allow-partial`, update-ref will process the transaction even if
-+some of the updates fail, allowing remaining updates to be applied.
-+Failed updates will be printed in the following format:
++With `--allow-partial`, update-ref continues executing the transaction even if
++some updates fail due to invalid or incorrect user input, applying only the
++successful updates. Errors resulting from user-provided input are treated as
++non-system-related and do not cause the entire transaction to be aborted.
++However, system-related errors—such as I/O failures or memory issues—will still
++result in a full failure. Additionally, errors like F/D conflicts are batched
++for performance optimization and will also cause a full failure. Any failed
++updates will be reported in the following format:
+
+ rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
+
Quote fields containing whitespace as if they were strings in C source
code; i.e., surrounded by double-quotes and with backslash escapes.
Use 40 "0" characters or the empty string to specify a zero value. To
-@@ Documentation/git-update-ref.txt: quoting:
+@@ Documentation/git-update-ref.adoc: quoting:
In this format, use 40 "0" to specify a zero value, and use the empty
string to specify a missing value.
@@ builtin/update-ref.c: static void parse_cmd_abort(struct ref_transaction *transa
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
-+ const struct strbuf *reason,
++ enum transaction_error err,
+ void *cb_data UNUSED)
+{
+ struct strbuf sb = STRBUF_INIT;
+ char space = ' ';
++ const char *reason = "";
++
++ switch (err) {
++ case TRANSACTION_NAME_CONFLICT:
++ reason = _("refname conflict");
++ break;
++ case TRANSACTION_CREATE_EXISTS:
++ reason = _("reference already exists");
++ break;
++ case TRANSACTION_NONEXISTENT_REF:
++ reason = _("reference does not exist");
++ break;
++ case TRANSACTION_INCORRECT_OLD_VALUE:
++ reason = _("incorrect old value provided");
++ break;
++ case TRANSACTION_INVALID_NEW_VALUE:
++ reason = _("invalid new value provided");
++ break;
++ case TRANSACTION_EXPECTED_SYMREF:
++ reason = _("expected symref but found regular ref");
++ break;
++ default:
++ reason = _("unkown failure");
++ }
+
+ if (!line_termination)
+ space = line_termination;
@@ builtin/update-ref.c: static void parse_cmd_abort(struct ref_transaction *transa
+ strbuf_addf(&sb, "rejected%c%s%c%s%c%c%s%c%s%c", space,
+ refname, space, new_oid ? oid_to_hex(new_oid) : new_target,
+ space, space, old_oid ? oid_to_hex(old_oid) : old_target,
-+ space, reason->buf, line_termination);
++ space, reason, line_termination);
+
+ fwrite(sb.buf, sb.len, 1, stdout);
+ strbuf_release(&sb);
@@ builtin/update-ref.c: int cmd_update_ref(int argc,
int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;
- int create_reflog = 0;
+ int create_reflog = 0, allow_partial = 0;
++ unsigned int flags = 0;
+
struct option options[] = {
OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
@@ builtin/update-ref.c: int cmd_update_ref(int argc,
OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
OPT_BOOL( 0 , "create-reflog", &create_reflog, N_("create a reflog")),
-+ OPT_BOOL('0', "allow-partial", &allow_partial, N_("allow partial transactions")),
++ OPT_BIT('0', "allow-partial", &flags, N_("allow partial transactions"),
++ REF_TRANSACTION_ALLOW_PARTIAL),
OPT_END(),
};
@@ builtin/update-ref.c: int cmd_update_ref(int argc,
- }
-
- if (read_stdin) {
-+ unsigned int flags = 0;
-+
-+ if (allow_partial)
-+ flags |= REF_TRANSACTION_ALLOW_PARTIAL;
-+
- if (delete || argc > 0)
usage_with_options(git_update_ref_usage, options);
if (end_null)
line_termination = '\0';
@@ t/t1400-update-ref.sh: do
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
-+ test_grep -q "trying to write ref ${SQ}refs/heads/ref2${SQ} with nonexistent object" stdout
++ test_grep -q "invalid new value provided" stdout
+ )
+ '
+
@@ t/t1400-update-ref.sh: do
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
-+ test_grep -q "trying to write non-commit object $head_tree to branch ${SQ}refs/heads/ref2${SQ}" stdout
++ test_grep -q "invalid new value provided" stdout
+ )
+ '
+
@@ t/t1400-update-ref.sh: do
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ test_must_fail git rev-parse refs/heads/ref2 &&
-+ test_grep -q "unable to resolve reference" stdout
++ test_grep -q "reference does not exist" stdout
+ )
+ '
+
@@ t/t1400-update-ref.sh: do
+ test_cmp expect actual &&
+ echo $head >expect &&
+ test_must_fail git rev-parse refs/heads/ref2 &&
-+ test_grep -q "reference is missing but expected $head" stdout
++ test_grep -q "reference does not exist" stdout
+ )
+ '
+
@@ t/t1400-update-ref.sh: do
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
-+ test_grep -q "expected symref with target ${SQ}refs/heads/nonexistent${SQ}: but is a regular ref" stdout
++ test_grep -q "expected symref but found regular ref" stdout
+ )
+ '
+
@@ t/t1400-update-ref.sh: do
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
-+ test_grep -q "${SQ}refs/heads/ref2${SQ}: is at $head but expected $old_head" stdout
++ test_grep -q "incorrect old value provided" stdout
++ )
++ '
++
++ # F/D conflicts on the files backend are resolved on an individual
++ # update level since refs are stored as files. On the reftable backend
++ # this check is batched to optimize for performance, so failures cannot
++ # be isolated to a single update.
++ test_expect_success REFFILES "stdin $type allow-partial refname conflict" '
++ git init repo &&
++ test_when_finished "rm -fr repo" &&
++ (
++ cd repo &&
++ test_commit one &&
++ old_head=$(git rev-parse HEAD) &&
++ test_commit two &&
++ head=$(git rev-parse HEAD) &&
++ git update-ref refs/heads/ref/foo $head &&
++
++ format_command $type "update refs/heads/ref/foo" "$old_head" "$head" >stdin &&
++ format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
++ git update-ref $type --stdin --allow-partial <stdin >stdout &&
++ echo $old_head >expect &&
++ git rev-parse refs/heads/ref/foo >actual &&
++ test_cmp expect actual &&
++ test_grep -q "refname conflict" stdout
+ )
+ '
done
base-commit: 408c44885d5b61a728dfc1df462490487cb01dae
change-id: 20241206-245-partially-atomic-ref-updates-9fe8b080345c
Thanks
- Karthik
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH v2 1/7] refs/files: remove redundant check in split_symref_update()
2025-02-25 9:29 ` [PATCH v2 0/7] " Karthik Nayak
@ 2025-02-25 9:29 ` Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 2/7] refs: move duplicate refname update check to generic layer Karthik Nayak
` (5 subsequent siblings)
6 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-25 9:29 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
In `split_symref_update()`, there were two checks for duplicate
refnames:
- At the start, `string_list_has_string()` ensures the refname is not
already in `affected_refnames`, preventing duplicates from being
added.
- After adding the refname, another check verifies whether the newly
inserted item has a `util` value.
The second check is unnecessary because the first one guarantees that
`string_list_insert()` will never encounter a preexisting entry.
Since `item->util` is only used in this context, remove the assignment and
simplify the surrounding code.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs/files-backend.c | 20 +++-----------------
1 file changed, 3 insertions(+), 17 deletions(-)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 4e1c50fead..6c7df30738 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2382,7 +2382,6 @@ static int split_head_update(struct ref_update *update,
struct string_list *affected_refnames,
struct strbuf *err)
{
- struct string_list_item *item;
struct ref_update *new_update;
if ((update->flags & REF_LOG_ONLY) ||
@@ -2421,8 +2420,7 @@ static int split_head_update(struct ref_update *update,
*/
if (strcmp(new_update->refname, "HEAD"))
BUG("%s unexpectedly not 'HEAD'", new_update->refname);
- item = string_list_insert(affected_refnames, new_update->refname);
- item->util = new_update;
+ string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2441,7 +2439,6 @@ static int split_symref_update(struct ref_update *update,
struct string_list *affected_refnames,
struct strbuf *err)
{
- struct string_list_item *item;
struct ref_update *new_update;
unsigned int new_flags;
@@ -2496,11 +2493,7 @@ static int split_symref_update(struct ref_update *update,
* be valid as long as affected_refnames is in use, and NOT
* referent, which might soon be freed by our caller.
*/
- item = string_list_insert(affected_refnames, new_update->refname);
- if (item->util)
- BUG("%s unexpectedly found in affected_refnames",
- new_update->refname);
- item->util = new_update;
+ string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2834,7 +2827,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
*/
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
- struct string_list_item *item;
if ((update->flags & REF_IS_PRUNING) &&
!(update->flags & REF_NO_DEREF))
@@ -2843,13 +2835,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
if (update->flags & REF_LOG_ONLY)
continue;
- item = string_list_append(&affected_refnames, update->refname);
- /*
- * We store a pointer to update in item->util, but at
- * the moment we never use the value of this field
- * except to check whether it is non-NULL.
- */
- item->util = update;
+ string_list_append(&affected_refnames, update->refname);
}
string_list_sort(&affected_refnames);
if (ref_update_reject_duplicates(&affected_refnames, err)) {
--
2.47.2
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v2 2/7] refs: move duplicate refname update check to generic layer
2025-02-25 9:29 ` [PATCH v2 0/7] " Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 1/7] refs/files: remove redundant check in split_symref_update() Karthik Nayak
@ 2025-02-25 9:29 ` Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 3/7] refs/files: remove duplicate duplicates check Karthik Nayak
` (4 subsequent siblings)
6 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-25 9:29 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Move the tracking of refnames in `affected_refnames` from individual
backends into the generic layer in 'refs.c'. This centralizes the
duplicate refname detection that was previously handled separately by
each backend.
Make some changes to accommodate this move:
- Add a `string_list` field `refnames` to `ref_transaction` to contain
all the references in a transaction. This field is updated whenever
a new update is added via `ref_transaction_add_update`, so manual
additions in reference backends are dropped.
- Modify the backends to use this field internally as needed. The
backends need to check if an update for refname already exists when
splitting symrefs or adding an update for 'HEAD'.
- In the reftable backend, within `reftable_be_transaction_prepare()`,
move the `string_list_has_string()` check above
`ref_transaction_add_update()`. Since `ref_transaction_add_update()`
automatically adds the refname to `transaction->refnames`,
performing the check after will always return true, so we perform
the check before adding the update.
This helps reduce duplication of functionality between the backends and
makes it easier to make changes in a more centralized manner.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 17 +++++++++++++
refs/files-backend.c | 67 +++++++++++--------------------------------------
refs/packed-backend.c | 25 +-----------------
refs/refs-internal.h | 2 ++
refs/reftable-backend.c | 54 +++++++++++++--------------------------
5 files changed, 51 insertions(+), 114 deletions(-)
diff --git a/refs.c b/refs.c
index 54fd5ce21e..ab69746947 100644
--- a/refs.c
+++ b/refs.c
@@ -1175,6 +1175,7 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,
CALLOC_ARRAY(tr, 1);
tr->ref_store = refs;
tr->flags = flags;
+ string_list_init_dup(&tr->refnames);
return tr;
}
@@ -1205,6 +1206,7 @@ void ref_transaction_free(struct ref_transaction *transaction)
free((char *)transaction->updates[i]->old_target);
free(transaction->updates[i]);
}
+ string_list_clear(&transaction->refnames, 0);
free(transaction->updates);
free(transaction);
}
@@ -1218,6 +1220,7 @@ struct ref_update *ref_transaction_add_update(
const char *committer_info,
const char *msg)
{
+ struct string_list_item *item;
struct ref_update *update;
if (transaction->state != REF_TRANSACTION_OPEN)
@@ -1245,6 +1248,16 @@ struct ref_update *ref_transaction_add_update(
update->msg = normalize_reflog_message(msg);
}
+ /*
+ * This list is generally used by the backends to avoid duplicates.
+ * But we do support multiple log updates for a given refname within
+ * a single transaction.
+ */
+ if (!(update->flags & REF_LOG_ONLY)) {
+ item = string_list_append(&transaction->refnames, refname);
+ item->util = update;
+ }
+
return update;
}
@@ -2405,6 +2418,10 @@ int ref_transaction_prepare(struct ref_transaction *transaction,
return -1;
}
+ string_list_sort(&transaction->refnames);
+ if (ref_update_reject_duplicates(&transaction->refnames, err))
+ return TRANSACTION_GENERIC_ERROR;
+
ret = refs->be->transaction_prepare(refs, transaction, err);
if (ret)
return ret;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 6c7df30738..85ed85ad87 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2378,9 +2378,7 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st
*/
static int split_head_update(struct ref_update *update,
struct ref_transaction *transaction,
- const char *head_ref,
- struct string_list *affected_refnames,
- struct strbuf *err)
+ const char *head_ref, struct strbuf *err)
{
struct ref_update *new_update;
@@ -2398,7 +2396,7 @@ static int split_head_update(struct ref_update *update,
* transaction. This check is O(lg N) in the transaction
* size, but it happens at most once per transaction.
*/
- if (string_list_has_string(affected_refnames, "HEAD")) {
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
/* An entry already existed */
strbuf_addf(err,
"multiple updates for 'HEAD' (including one "
@@ -2420,7 +2418,6 @@ static int split_head_update(struct ref_update *update,
*/
if (strcmp(new_update->refname, "HEAD"))
BUG("%s unexpectedly not 'HEAD'", new_update->refname);
- string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2436,7 +2433,6 @@ static int split_head_update(struct ref_update *update,
static int split_symref_update(struct ref_update *update,
const char *referent,
struct ref_transaction *transaction,
- struct string_list *affected_refnames,
struct strbuf *err)
{
struct ref_update *new_update;
@@ -2448,7 +2444,7 @@ static int split_symref_update(struct ref_update *update,
* size, but it happens at most once per symref in a
* transaction.
*/
- if (string_list_has_string(affected_refnames, referent)) {
+ if (string_list_has_string(&transaction->refnames, referent)) {
/* An entry already exists */
strbuf_addf(err,
"multiple updates for '%s' (including one "
@@ -2486,15 +2482,6 @@ static int split_symref_update(struct ref_update *update,
update->flags |= REF_LOG_ONLY | REF_NO_DEREF;
update->flags &= ~REF_HAVE_OLD;
- /*
- * Add the referent. This insertion is O(N) in the transaction
- * size, but it happens at most once per symref in a
- * transaction. Make sure to add new_update->refname, which will
- * be valid as long as affected_refnames is in use, and NOT
- * referent, which might soon be freed by our caller.
- */
- string_list_insert(affected_refnames, new_update->refname);
-
return 0;
}
@@ -2558,7 +2545,6 @@ static int lock_ref_for_update(struct files_ref_store *refs,
struct ref_transaction *transaction,
const char *head_ref,
struct string_list *refnames_to_check,
- struct string_list *affected_refnames,
struct strbuf *err)
{
struct strbuf referent = STRBUF_INIT;
@@ -2575,8 +2561,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
update->flags |= REF_DELETING;
if (head_ref) {
- ret = split_head_update(update, transaction, head_ref,
- affected_refnames, err);
+ ret = split_head_update(update, transaction, head_ref, err);
if (ret)
goto out;
}
@@ -2586,9 +2571,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,
lock->count++;
} else {
ret = lock_raw_ref(refs, update->refname, mustexist,
- refnames_to_check, affected_refnames,
- &lock, &referent,
- &update->type, err);
+ refnames_to_check, &transaction->refnames,
+ &lock, &referent, &update->type, err);
if (ret) {
char *reason;
@@ -2642,9 +2626,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,
* of processing the split-off update, so we
* don't have to do it here.
*/
- ret = split_symref_update(update,
- referent.buf, transaction,
- affected_refnames, err);
+ ret = split_symref_update(update, referent.buf,
+ transaction, err);
if (ret)
goto out;
}
@@ -2799,7 +2782,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
"ref_transaction_prepare");
size_t i;
int ret = 0;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
struct string_list refnames_to_check = STRING_LIST_INIT_NODUP;
char *head_ref = NULL;
int head_type;
@@ -2818,12 +2800,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
transaction->backend_data = backend_data;
/*
- * Fail if a refname appears more than once in the
- * transaction. (If we end up splitting up any updates using
- * split_symref_update() or split_head_update(), those
- * functions will check that the new updates don't have the
- * same refname as any existing ones.) Also fail if any of the
- * updates use REF_IS_PRUNING without REF_NO_DEREF.
+ * Fail if any of the updates use REF_IS_PRUNING without REF_NO_DEREF.
*/
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
@@ -2831,16 +2808,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
if ((update->flags & REF_IS_PRUNING) &&
!(update->flags & REF_NO_DEREF))
BUG("REF_IS_PRUNING set without REF_NO_DEREF");
-
- if (update->flags & REF_LOG_ONLY)
- continue;
-
- string_list_append(&affected_refnames, update->refname);
- }
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
}
/*
@@ -2882,7 +2849,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
ret = lock_ref_for_update(refs, update, transaction,
head_ref, &refnames_to_check,
- &affected_refnames, err);
+ err);
if (ret)
goto cleanup;
@@ -2929,7 +2896,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
* So instead, we accept the race for now.
*/
if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
- &affected_refnames, NULL, 0, err)) {
+ &transaction->refnames, NULL, 0, err)) {
ret = TRANSACTION_NAME_CONFLICT;
goto cleanup;
}
@@ -2975,7 +2942,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
cleanup:
free(head_ref);
- string_list_clear(&affected_refnames, 0);
string_list_clear(&refnames_to_check, 0);
if (ret)
@@ -3050,13 +3016,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (transaction->state != REF_TRANSACTION_PREPARED)
BUG("commit called for transaction that is not prepared");
- /* Fail if a refname appears more than once in the transaction: */
- for (i = 0; i < transaction->nr; i++)
- if (!(transaction->updates[i]->flags & REF_LOG_ONLY))
- string_list_append(&affected_refnames,
- transaction->updates[i]->refname);
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
+ string_list_sort(&transaction->refnames);
+ if (ref_update_reject_duplicates(&transaction->refnames, err)) {
ret = TRANSACTION_GENERIC_ERROR;
goto cleanup;
}
@@ -3074,7 +3035,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
* that we are creating already exists.
*/
if (refs_for_each_rawref(&refs->base, ref_present,
- &affected_refnames))
+ &transaction->refnames))
BUG("initial ref transaction called with existing refs");
packed_transaction = ref_store_transaction_begin(refs->packed_ref_store,
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 71a38acfed..3247871574 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1619,8 +1619,6 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
struct packed_transaction_backend_data {
/* True iff the transaction owns the packed-refs lock. */
int own_lock;
-
- struct string_list updates;
};
static void packed_transaction_cleanup(struct packed_ref_store *refs,
@@ -1629,8 +1627,6 @@ static void packed_transaction_cleanup(struct packed_ref_store *refs,
struct packed_transaction_backend_data *data = transaction->backend_data;
if (data) {
- string_list_clear(&data->updates, 0);
-
if (is_tempfile_active(refs->tempfile))
delete_tempfile(&refs->tempfile);
@@ -1655,7 +1651,6 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
"ref_transaction_prepare");
struct packed_transaction_backend_data *data;
- size_t i;
int ret = TRANSACTION_GENERIC_ERROR;
/*
@@ -1668,34 +1663,16 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
*/
CALLOC_ARRAY(data, 1);
- string_list_init_nodup(&data->updates);
transaction->backend_data = data;
- /*
- * Stick the updates in a string list by refname so that we
- * can sort them:
- */
- for (i = 0; i < transaction->nr; i++) {
- struct ref_update *update = transaction->updates[i];
- struct string_list_item *item =
- string_list_append(&data->updates, update->refname);
-
- /* Store a pointer to update in item->util: */
- item->util = update;
- }
- string_list_sort(&data->updates);
-
- if (ref_update_reject_duplicates(&data->updates, err))
- goto failure;
-
if (!is_lock_file_locked(&refs->lock)) {
if (packed_refs_lock(ref_store, 0, err))
goto failure;
data->own_lock = 1;
}
- if (write_with_updates(refs, &data->updates, err))
+ if (write_with_updates(refs, &transaction->refnames, err))
goto failure;
transaction->state = REF_TRANSACTION_PREPARED;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index cadd9808fd..a6e05eaecd 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -3,6 +3,7 @@
#include "refs.h"
#include "iterator.h"
+#include "string-list.h"
struct fsck_options;
struct ref_transaction;
@@ -198,6 +199,7 @@ enum ref_transaction_state {
struct ref_transaction {
struct ref_store *ref_store;
struct ref_update **updates;
+ struct string_list refnames;
size_t alloc;
size_t nr;
enum ref_transaction_state state;
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 441b8c69c1..f616d9aabe 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1076,7 +1076,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
struct reftable_ref_store *refs =
reftable_be_downcast(ref_store, REF_STORE_WRITE|REF_STORE_MAIN, "ref_transaction_prepare");
struct strbuf referent = STRBUF_INIT, head_referent = STRBUF_INIT;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
struct string_list refnames_to_check = STRING_LIST_INIT_NODUP;
struct reftable_transaction_data *tx_data = NULL;
struct reftable_backend *be;
@@ -1101,10 +1100,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->updates[i], err);
if (ret)
goto done;
-
- if (!(transaction->updates[i]->flags & REF_LOG_ONLY))
- string_list_append(&affected_refnames,
- transaction->updates[i]->refname);
}
/*
@@ -1116,17 +1111,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
tx_data->args[i].updates_alloc = tx_data->args[i].updates_expected;
}
- /*
- * Fail if a refname appears more than once in the transaction.
- * This code is taken from the files backend and is a good candidate to
- * be moved into the generic layer.
- */
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto done;
- }
-
/*
* TODO: it's dubious whether we should reload the stack that "HEAD"
* belongs to or not. In theory, it may happen that we only modify
@@ -1194,14 +1178,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
!(u->flags & REF_LOG_ONLY) &&
!(u->flags & REF_UPDATE_VIA_HEAD) &&
!strcmp(rewritten_ref, head_referent.buf)) {
- struct ref_update *new_update;
-
/*
* First make sure that HEAD is not already in the
* transaction. This check is O(lg N) in the transaction
* size, but it happens at most once per transaction.
*/
- if (string_list_has_string(&affected_refnames, "HEAD")) {
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
/* An entry already existed */
strbuf_addf(err,
_("multiple updates for 'HEAD' (including one "
@@ -1211,12 +1193,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
goto done;
}
- new_update = ref_transaction_add_update(
- transaction, "HEAD",
- u->flags | REF_LOG_ONLY | REF_NO_DEREF,
- &u->new_oid, &u->old_oid, NULL, NULL, NULL,
- u->msg);
- string_list_insert(&affected_refnames, new_update->refname);
+ ref_transaction_add_update(
+ transaction, "HEAD",
+ u->flags | REF_LOG_ONLY | REF_NO_DEREF,
+ &u->new_oid, &u->old_oid, NULL, NULL, NULL,
+ u->msg);
}
ret = reftable_backend_read_ref(be, rewritten_ref,
@@ -1281,6 +1262,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
if (!strcmp(rewritten_ref, "HEAD"))
new_flags |= REF_UPDATE_VIA_HEAD;
+ if (string_list_has_string(&transaction->refnames, referent.buf)) {
+ strbuf_addf(err,
+ _("multiple updates for '%s' (including one "
+ "via symref '%s') are not allowed"),
+ referent.buf, u->refname);
+ ret = TRANSACTION_NAME_CONFLICT;
+ goto done;
+ }
+
/*
* If we are updating a symref (eg. HEAD), we should also
* update the branch that the symref points to.
@@ -1305,16 +1295,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
*/
u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
u->flags &= ~REF_HAVE_OLD;
-
- if (string_list_has_string(&affected_refnames, new_update->refname)) {
- strbuf_addf(err,
- _("multiple updates for '%s' (including one "
- "via symref '%s') are not allowed"),
- referent.buf, u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
- string_list_insert(&affected_refnames, new_update->refname);
}
}
@@ -1384,7 +1364,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
}
string_list_sort(&refnames_to_check);
- ret = refs_verify_refnames_available(ref_store, &refnames_to_check, &affected_refnames, NULL,
+ ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
+ &transaction->refnames, NULL,
transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
err);
if (ret < 0)
@@ -1402,7 +1383,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
strbuf_addf(err, _("reftable: transaction prepare: %s"),
reftable_error_str(ret));
}
- string_list_clear(&affected_refnames, 0);
strbuf_release(&referent);
strbuf_release(&head_referent);
string_list_clear(&refnames_to_check, 0);
--
2.47.2
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v2 3/7] refs/files: remove duplicate duplicates check
2025-02-25 9:29 ` [PATCH v2 0/7] " Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 1/7] refs/files: remove redundant check in split_symref_update() Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 2/7] refs: move duplicate refname update check to generic layer Karthik Nayak
@ 2025-02-25 9:29 ` Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 4/7] refs/reftable: extract code from the transaction preparation Karthik Nayak
` (3 subsequent siblings)
6 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-25 9:29 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Within the files reference backend's transaction's 'finish' phase, a
verification step is currently performed wherein the refnames list is
sorted and examined for multiple updates targeting the same refname.
It has been observed that this verification is redundant, as an
identical check is already executed during the transaction's 'prepare'
stage. Since the refnames list remains unmodified following the
'prepare' stage, this secondary verification can be safely eliminated.
The duplicate check has been removed accordingly, and the
`ref_update_reject_duplicates()` function has been marked as static, as
its usage is now confined to 'refs.c'.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 9 +++++++--
refs/files-backend.c | 6 ------
refs/refs-internal.h | 8 --------
3 files changed, 7 insertions(+), 16 deletions(-)
diff --git a/refs.c b/refs.c
index ab69746947..69f385f344 100644
--- a/refs.c
+++ b/refs.c
@@ -2303,8 +2303,13 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
return ret;
}
-int ref_update_reject_duplicates(struct string_list *refnames,
- struct strbuf *err)
+/*
+ * Write an error to `err` and return a nonzero value iff the same
+ * refname appears multiple times in `refnames`. `refnames` must be
+ * sorted on entry to this function.
+ */
+static int ref_update_reject_duplicates(struct string_list *refnames,
+ struct strbuf *err)
{
size_t i, n = refnames->nr;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 85ed85ad87..7c6a0b3478 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3016,12 +3016,6 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (transaction->state != REF_TRANSACTION_PREPARED)
BUG("commit called for transaction that is not prepared");
- string_list_sort(&transaction->refnames);
- if (ref_update_reject_duplicates(&transaction->refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
- }
-
/*
* It's really undefined to call this function in an active
* repository or when there are existing references: we are
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index a6e05eaecd..4b7fc8f1ab 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -142,14 +142,6 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
struct object_id *oid, struct strbuf *referent,
unsigned int *type, int *failure_errno);
-/*
- * Write an error to `err` and return a nonzero value iff the same
- * refname appears multiple times in `refnames`. `refnames` must be
- * sorted on entry to this function.
- */
-int ref_update_reject_duplicates(struct string_list *refnames,
- struct strbuf *err);
-
/*
* Add a ref_update with the specified properties to transaction, and
* return a pointer to the new object. This function does not verify
--
2.47.2
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v2 4/7] refs/reftable: extract code from the transaction preparation
2025-02-25 9:29 ` [PATCH v2 0/7] " Karthik Nayak
` (2 preceding siblings ...)
2025-02-25 9:29 ` [PATCH v2 3/7] refs/files: remove duplicate duplicates check Karthik Nayak
@ 2025-02-25 9:29 ` Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 5/7] refs: introduce enum-based transaction error types Karthik Nayak
` (2 subsequent siblings)
6 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-25 9:29 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Extract the core logic for preparing individual reference updates from
`reftable_be_transaction_prepare()` into `prepare_single_update()`. This
dedicated function now handles all validation and preparation steps for
each reference update in the transaction, including object ID
verification, HEAD reference handling, and symref processing.
The refactoring consolidates all reference update validation into a
single logical block, which improves code maintainability and
readability. More importantly, this restructuring lays the groundwork
for implementing partial transaction support in the reftable backend,
which will be introduced in the following commit.
No functional changes are included in this commit - it is purely a code
reorganization to support future enhancements.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs/reftable-backend.c | 463 +++++++++++++++++++++++++-----------------------
1 file changed, 237 insertions(+), 226 deletions(-)
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index f616d9aabe..2c1e2995de 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1069,6 +1069,239 @@ static int queue_transaction_update(struct reftable_ref_store *refs,
return 0;
}
+static int prepare_single_update(struct reftable_ref_store *refs,
+ struct reftable_transaction_data *tx_data,
+ struct ref_transaction *transaction,
+ struct reftable_backend *be,
+ struct ref_update *u,
+ struct string_list *refnames_to_check,
+ unsigned int head_type,
+ struct strbuf *head_referent,
+ struct strbuf *referent,
+ struct strbuf *err)
+{
+ struct object_id current_oid = {0};
+ const char *rewritten_ref;
+ int ret = 0;
+
+ /*
+ * There is no need to reload the respective backends here as
+ * we have already reloaded them when preparing the transaction
+ * update. And given that the stacks have been locked there
+ * shouldn't have been any concurrent modifications of the
+ * stack.
+ */
+ ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
+ if (ret)
+ return ret;
+
+ /* Verify that the new object ID is valid. */
+ if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
+ !(u->flags & REF_SKIP_OID_VERIFICATION) &&
+ !(u->flags & REF_LOG_ONLY)) {
+ struct object *o = parse_object(refs->base.repo, &u->new_oid);
+ if (!o) {
+ strbuf_addf(err,
+ _("trying to write ref '%s' with nonexistent object %s"),
+ u->refname, oid_to_hex(&u->new_oid));
+ return -1;
+ }
+
+ if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
+ strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
+ oid_to_hex(&u->new_oid), u->refname);
+ return -1;
+ }
+ }
+
+ /*
+ * When we update the reference that HEAD points to we enqueue
+ * a second log-only update for HEAD so that its reflog is
+ * updated accordingly.
+ */
+ if (head_type == REF_ISSYMREF &&
+ !(u->flags & REF_LOG_ONLY) &&
+ !(u->flags & REF_UPDATE_VIA_HEAD) &&
+ !strcmp(rewritten_ref, head_referent->buf)) {
+ /*
+ * First make sure that HEAD is not already in the
+ * transaction. This check is O(lg N) in the transaction
+ * size, but it happens at most once per transaction.
+ */
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
+ /* An entry already existed */
+ strbuf_addf(err,
+ _("multiple updates for 'HEAD' (including one "
+ "via its referent '%s') are not allowed"),
+ u->refname);
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ ref_transaction_add_update(
+ transaction, "HEAD",
+ u->flags | REF_LOG_ONLY | REF_NO_DEREF,
+ &u->new_oid, &u->old_oid, NULL, NULL, NULL,
+ u->msg);
+ }
+
+ ret = reftable_backend_read_ref(be, rewritten_ref,
+ ¤t_oid, referent, &u->type);
+ if (ret < 0)
+ return ret;
+ if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
+ /*
+ * The reference does not exist, and we either have no
+ * old object ID or expect the reference to not exist.
+ * We can thus skip below safety checks as well as the
+ * symref splitting. But we do want to verify that
+ * there is no conflicting reference here so that we
+ * can output a proper error message instead of failing
+ * at a later point.
+ */
+ string_list_append(refnames_to_check, u->refname);
+
+ /*
+ * There is no need to write the reference deletion
+ * when the reference in question doesn't exist.
+ */
+ if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
+ ret = queue_transaction_update(refs, tx_data, u,
+ ¤t_oid, err);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+ }
+ if (ret > 0) {
+ /* The reference does not exist, but we expected it to. */
+ strbuf_addf(err, _("cannot lock ref '%s': "
+
+
+ "unable to resolve reference '%s'"),
+ ref_update_original_update_refname(u), u->refname);
+ return -1;
+ }
+
+ if (u->type & REF_ISSYMREF) {
+ /*
+ * The reftable stack is locked at this point already,
+ * so it is safe to call `refs_resolve_ref_unsafe()`
+ * here without causing races.
+ */
+ const char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,
+ ¤t_oid, NULL);
+
+ if (u->flags & REF_NO_DEREF) {
+ if (u->flags & REF_HAVE_OLD && !resolved) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "error reading reference"), u->refname);
+ return -1;
+ }
+ } else {
+ struct ref_update *new_update;
+ int new_flags;
+
+ new_flags = u->flags;
+ if (!strcmp(rewritten_ref, "HEAD"))
+ new_flags |= REF_UPDATE_VIA_HEAD;
+
+ if (string_list_has_string(&transaction->refnames, referent->buf)) {
+ strbuf_addf(err,
+ _("multiple updates for '%s' (including one "
+ "via symref '%s') are not allowed"),
+ referent->buf, u->refname);
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ /*
+ * If we are updating a symref (eg. HEAD), we should also
+ * update the branch that the symref points to.
+ *
+ * This is generic functionality, and would be better
+ * done in refs.c, but the current implementation is
+ * intertwined with the locking in files-backend.c.
+ */
+ new_update = ref_transaction_add_update(
+ transaction, referent->buf, new_flags,
+ u->new_target ? NULL : &u->new_oid,
+ u->old_target ? NULL : &u->old_oid,
+ u->new_target, u->old_target,
+ u->committer_info, u->msg);
+
+ new_update->parent_update = u;
+
+ /*
+ * Change the symbolic ref update to log only. Also, it
+ * doesn't need to check its old OID value, as that will be
+ * done when new_update is processed.
+ */
+ u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
+ u->flags &= ~REF_HAVE_OLD;
+ }
+ }
+
+ /*
+ * Verify that the old object matches our expectations. Note
+ * that the error messages here do not make a lot of sense in
+ * the context of the reftable backend as we never lock
+ * individual refs. But the error messages match what the files
+ * backend returns, which keeps our tests happy.
+ */
+ if (u->old_target) {
+ if (!(u->type & REF_ISSYMREF)) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "expected symref with target '%s': "
+ "but is a regular ref"),
+ ref_update_original_update_refname(u),
+ u->old_target);
+ return -1;
+ }
+
+ if (ref_update_check_old_target(referent->buf, u, err)) {
+ return -1;
+ }
+ } else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
+ if (is_null_oid(&u->old_oid)) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "reference already exists"),
+ ref_update_original_update_refname(u));
+ return TRANSACTION_CREATE_EXISTS;
+ }
+ else if (is_null_oid(¤t_oid))
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "reference is missing but expected %s"),
+ ref_update_original_update_refname(u),
+ oid_to_hex(&u->old_oid));
+ else
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "is at %s but expected %s"),
+ ref_update_original_update_refname(u),
+ oid_to_hex(¤t_oid),
+ oid_to_hex(&u->old_oid));
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ /*
+ * If all of the following conditions are true:
+ *
+ * - We're not about to write a symref.
+ * - We're not about to write a log-only entry.
+ * - Old and new object ID are different.
+ *
+ * Then we're essentially doing a no-op update that can be
+ * skipped. This is not only for the sake of efficiency, but
+ * also skips writing unneeded reflog entries.
+ */
+ if ((u->type & REF_ISSYMREF) ||
+ (u->flags & REF_LOG_ONLY) ||
+ (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))
+ return queue_transaction_update(refs, tx_data, u,
+ ¤t_oid, err);
+
+ return 0;
+}
+
static int reftable_be_transaction_prepare(struct ref_store *ref_store,
struct ref_transaction *transaction,
struct strbuf *err)
@@ -1133,234 +1366,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
ret = 0;
for (i = 0; i < transaction->nr; i++) {
- struct ref_update *u = transaction->updates[i];
- struct object_id current_oid = {0};
- const char *rewritten_ref;
-
- /*
- * There is no need to reload the respective backends here as
- * we have already reloaded them when preparing the transaction
- * update. And given that the stacks have been locked there
- * shouldn't have been any concurrent modifications of the
- * stack.
- */
- ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
+ ret = prepare_single_update(refs, tx_data, transaction, be,
+ transaction->updates[i],
+ &refnames_to_check, head_type,
+ &head_referent, &referent, err);
if (ret)
goto done;
-
- /* Verify that the new object ID is valid. */
- if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
- !(u->flags & REF_SKIP_OID_VERIFICATION) &&
- !(u->flags & REF_LOG_ONLY)) {
- struct object *o = parse_object(refs->base.repo, &u->new_oid);
- if (!o) {
- strbuf_addf(err,
- _("trying to write ref '%s' with nonexistent object %s"),
- u->refname, oid_to_hex(&u->new_oid));
- ret = -1;
- goto done;
- }
-
- if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
- strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
- oid_to_hex(&u->new_oid), u->refname);
- ret = -1;
- goto done;
- }
- }
-
- /*
- * When we update the reference that HEAD points to we enqueue
- * a second log-only update for HEAD so that its reflog is
- * updated accordingly.
- */
- if (head_type == REF_ISSYMREF &&
- !(u->flags & REF_LOG_ONLY) &&
- !(u->flags & REF_UPDATE_VIA_HEAD) &&
- !strcmp(rewritten_ref, head_referent.buf)) {
- /*
- * First make sure that HEAD is not already in the
- * transaction. This check is O(lg N) in the transaction
- * size, but it happens at most once per transaction.
- */
- if (string_list_has_string(&transaction->refnames, "HEAD")) {
- /* An entry already existed */
- strbuf_addf(err,
- _("multiple updates for 'HEAD' (including one "
- "via its referent '%s') are not allowed"),
- u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
-
- ref_transaction_add_update(
- transaction, "HEAD",
- u->flags | REF_LOG_ONLY | REF_NO_DEREF,
- &u->new_oid, &u->old_oid, NULL, NULL, NULL,
- u->msg);
- }
-
- ret = reftable_backend_read_ref(be, rewritten_ref,
- ¤t_oid, &referent, &u->type);
- if (ret < 0)
- goto done;
- if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
- /*
- * The reference does not exist, and we either have no
- * old object ID or expect the reference to not exist.
- * We can thus skip below safety checks as well as the
- * symref splitting. But we do want to verify that
- * there is no conflicting reference here so that we
- * can output a proper error message instead of failing
- * at a later point.
- */
- string_list_append(&refnames_to_check, u->refname);
-
- /*
- * There is no need to write the reference deletion
- * when the reference in question doesn't exist.
- */
- if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
- ret = queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
- if (ret)
- goto done;
- }
-
- continue;
- }
- if (ret > 0) {
- /* The reference does not exist, but we expected it to. */
- strbuf_addf(err, _("cannot lock ref '%s': "
- "unable to resolve reference '%s'"),
- ref_update_original_update_refname(u), u->refname);
- ret = -1;
- goto done;
- }
-
- if (u->type & REF_ISSYMREF) {
- /*
- * The reftable stack is locked at this point already,
- * so it is safe to call `refs_resolve_ref_unsafe()`
- * here without causing races.
- */
- const char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,
- ¤t_oid, NULL);
-
- if (u->flags & REF_NO_DEREF) {
- if (u->flags & REF_HAVE_OLD && !resolved) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "error reading reference"), u->refname);
- ret = -1;
- goto done;
- }
- } else {
- struct ref_update *new_update;
- int new_flags;
-
- new_flags = u->flags;
- if (!strcmp(rewritten_ref, "HEAD"))
- new_flags |= REF_UPDATE_VIA_HEAD;
-
- if (string_list_has_string(&transaction->refnames, referent.buf)) {
- strbuf_addf(err,
- _("multiple updates for '%s' (including one "
- "via symref '%s') are not allowed"),
- referent.buf, u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
-
- /*
- * If we are updating a symref (eg. HEAD), we should also
- * update the branch that the symref points to.
- *
- * This is generic functionality, and would be better
- * done in refs.c, but the current implementation is
- * intertwined with the locking in files-backend.c.
- */
- new_update = ref_transaction_add_update(
- transaction, referent.buf, new_flags,
- u->new_target ? NULL : &u->new_oid,
- u->old_target ? NULL : &u->old_oid,
- u->new_target, u->old_target,
- u->committer_info, u->msg);
-
- new_update->parent_update = u;
-
- /*
- * Change the symbolic ref update to log only. Also, it
- * doesn't need to check its old OID value, as that will be
- * done when new_update is processed.
- */
- u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
- u->flags &= ~REF_HAVE_OLD;
- }
- }
-
- /*
- * Verify that the old object matches our expectations. Note
- * that the error messages here do not make a lot of sense in
- * the context of the reftable backend as we never lock
- * individual refs. But the error messages match what the files
- * backend returns, which keeps our tests happy.
- */
- if (u->old_target) {
- if (!(u->type & REF_ISSYMREF)) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "expected symref with target '%s': "
- "but is a regular ref"),
- ref_update_original_update_refname(u),
- u->old_target);
- ret = -1;
- goto done;
- }
-
- if (ref_update_check_old_target(referent.buf, u, err)) {
- ret = -1;
- goto done;
- }
- } else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
- ret = TRANSACTION_NAME_CONFLICT;
- if (is_null_oid(&u->old_oid)) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "reference already exists"),
- ref_update_original_update_refname(u));
- ret = TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(¤t_oid))
- strbuf_addf(err, _("cannot lock ref '%s': "
- "reference is missing but expected %s"),
- ref_update_original_update_refname(u),
- oid_to_hex(&u->old_oid));
- else
- strbuf_addf(err, _("cannot lock ref '%s': "
- "is at %s but expected %s"),
- ref_update_original_update_refname(u),
- oid_to_hex(¤t_oid),
- oid_to_hex(&u->old_oid));
- goto done;
- }
-
- /*
- * If all of the following conditions are true:
- *
- * - We're not about to write a symref.
- * - We're not about to write a log-only entry.
- * - Old and new object ID are different.
- *
- * Then we're essentially doing a no-op update that can be
- * skipped. This is not only for the sake of efficiency, but
- * also skips writing unneeded reflog entries.
- */
- if ((u->type & REF_ISSYMREF) ||
- (u->flags & REF_LOG_ONLY) ||
- (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid))) {
- ret = queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
- if (ret)
- goto done;
- }
}
string_list_sort(&refnames_to_check);
--
2.47.2
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v2 5/7] refs: introduce enum-based transaction error types
2025-02-25 9:29 ` [PATCH v2 0/7] " Karthik Nayak
` (3 preceding siblings ...)
2025-02-25 9:29 ` [PATCH v2 4/7] refs/reftable: extract code from the transaction preparation Karthik Nayak
@ 2025-02-25 9:29 ` Karthik Nayak
2025-02-25 11:08 ` Patrick Steinhardt
2025-02-25 9:29 ` [PATCH v2 6/7] refs: implement partial reference transaction support Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 7/7] update-ref: add --allow-partial flag for stdin mode Karthik Nayak
6 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-02-25 9:29 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Replace preprocessor-defined transaction errors with a strongly-typed
enum `transaction_error`. This change:
- Improves type safety and function signature clarity.
- Makes error handling more explicit and discoverable.
- Maintains existing error cases, while adding new error cases for
common scenarios.
This refactoring paves the way for more comprehensive error handling
which we will utilize in the upcoming commits to add partial transaction
support.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 44 +++++++------
refs.h | 56 ++++++++++------
refs/files-backend.c | 167 ++++++++++++++++++++++++------------------------
refs/packed-backend.c | 21 +++---
refs/refs-internal.h | 5 +-
refs/reftable-backend.c | 59 +++++++++--------
6 files changed, 191 insertions(+), 161 deletions(-)
diff --git a/refs.c b/refs.c
index 69f385f344..f989a46a5a 100644
--- a/refs.c
+++ b/refs.c
@@ -2497,18 +2497,18 @@ int ref_transaction_commit(struct ref_transaction *transaction,
return ret;
}
-int refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *refnames,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err)
+enum transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err)
{
struct strbuf dirname = STRBUF_INIT;
struct strbuf referent = STRBUF_INIT;
struct ref_iterator *iter = NULL;
struct strset dirnames;
- int ret = -1;
+ int ret = TRANSACTION_NAME_CONFLICT;
/*
* For the sake of comments in this function, suppose that
@@ -2624,12 +2624,12 @@ int refs_verify_refnames_available(struct ref_store *refs,
return ret;
}
-int refs_verify_refname_available(struct ref_store *refs,
- const char *refname,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err)
+enum transaction_error refs_verify_refname_available(struct ref_store *refs,
+ const char *refname,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err)
{
struct string_list_item item = { .string = (char *) refname };
struct string_list refnames = {
@@ -2817,26 +2817,28 @@ int ref_update_has_null_new_value(struct ref_update *update)
return !update->new_target && is_null_oid(&update->new_oid);
}
-int ref_update_check_old_target(const char *referent, struct ref_update *update,
- struct strbuf *err)
+enum transaction_error ref_update_check_old_target(const char *referent,
+ struct ref_update *update,
+ struct strbuf *err)
{
if (!update->old_target)
BUG("called without old_target set");
if (!strcmp(referent, update->old_target))
- return 0;
+ return TRANSACTION_OK;
- if (!strcmp(referent, ""))
+ if (!strcmp(referent, "")) {
strbuf_addf(err, "verifying symref target: '%s': "
"reference is missing but expected %s",
ref_update_original_update_refname(update),
update->old_target);
- else
- strbuf_addf(err, "verifying symref target: '%s': "
- "is at %s but expected %s",
+ return TRANSACTION_NONEXISTENT_REF;
+ }
+
+ strbuf_addf(err, "verifying symref target: '%s': is at %s but expected %s",
ref_update_original_update_refname(update),
referent, update->old_target);
- return -1;
+ return TRANSACTION_INCORRECT_OLD_VALUE;
}
struct migration_data {
diff --git a/refs.h b/refs.h
index b14ba1f9ff..8e9ead174c 100644
--- a/refs.h
+++ b/refs.h
@@ -16,6 +16,31 @@ struct worktree;
enum ref_storage_format ref_storage_format_by_name(const char *name);
const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);
+/*
+ * enum transaction_error represents the following return codes:
+ * TRANSACTION_OK: success code.
+ * TRANSACTION_GENERIC_ERROR error_code: default error code.
+ * TRANSACTION_NAME_CONFLICT error_code: ref name conflict like A vs A/B.
+ * TRANSACTION_CREATE_EXISTS error_code: ref to be created already exists.
+ * TRANSACTION_NONEXISTENT_REF error_code: ref expected but doesn't exist.
+ * TRANSACTION_INCORRECT_OLD_VALUE error_code: provided old_oid or old_target of
+ * reference doesn't match actual.
+ * TRANSACTION_INVALID_NEW_VALUE error_code: provided new_oid or new_target is
+ * invalid.
+ * TRANSACTION_EXPECTED_SYMREF error_code: expected ref to be symref, but is a
+ * regular ref.
+ */
+enum transaction_error {
+ TRANSACTION_OK = 0,
+ TRANSACTION_GENERIC_ERROR = -1,
+ TRANSACTION_NAME_CONFLICT = -2,
+ TRANSACTION_CREATE_EXISTS = -3,
+ TRANSACTION_NONEXISTENT_REF = -4,
+ TRANSACTION_INCORRECT_OLD_VALUE = -5,
+ TRANSACTION_INVALID_NEW_VALUE = -6,
+ TRANSACTION_EXPECTED_SYMREF = -7,
+};
+
/*
* Resolve a reference, recursively following symbolic references.
*
@@ -117,24 +142,24 @@ int refs_read_symbolic_ref(struct ref_store *ref_store, const char *refname,
*
* extras and skip must be sorted.
*/
-int refs_verify_refname_available(struct ref_store *refs,
- const char *refname,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
+enum transaction_error refs_verify_refname_available(struct ref_store *refs,
+ const char *refname,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err);
/*
* Same as `refs_verify_refname_available()`, but checking for a list of
* refnames instead of only a single item. This is more efficient in the case
* where one needs to check multiple refnames.
*/
-int refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *refnames,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
+enum transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err);
int refs_ref_exists(struct ref_store *refs, const char *refname);
@@ -830,13 +855,6 @@ int ref_transaction_verify(struct ref_transaction *transaction,
unsigned int flags,
struct strbuf *err);
-/* Naming conflict (for example, the ref names A and A/B conflict). */
-#define TRANSACTION_NAME_CONFLICT -1
-/* When only creation was requested, but the ref already exists. */
-#define TRANSACTION_CREATE_EXISTS -2
-/* All other errors. */
-#define TRANSACTION_GENERIC_ERROR -3
-
/*
* Perform the preparatory stages of committing `transaction`. Acquire
* any needed locks, check preconditions, etc.; basically, do as much
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 7c6a0b3478..3b0adf8bb2 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -676,19 +676,19 @@ static void unlock_ref(struct ref_lock *lock)
* avoided, namely if we were successfully able to read the ref
* - Generate informative error messages in the case of failure
*/
-static int lock_raw_ref(struct files_ref_store *refs,
- const char *refname, int mustexist,
- struct string_list *refnames_to_check,
- const struct string_list *extras,
- struct ref_lock **lock_p,
- struct strbuf *referent,
- unsigned int *type,
- struct strbuf *err)
+static enum transaction_error lock_raw_ref(struct files_ref_store *refs,
+ const char *refname, int mustexist,
+ struct string_list *refnames_to_check,
+ const struct string_list *extras,
+ struct ref_lock **lock_p,
+ struct strbuf *referent,
+ unsigned int *type,
+ struct strbuf *err)
{
+ enum transaction_error ret = TRANSACTION_GENERIC_ERROR;
struct ref_lock *lock;
struct strbuf ref_file = STRBUF_INIT;
int attempts_remaining = 3;
- int ret = TRANSACTION_GENERIC_ERROR;
int failure_errno;
assert(err);
@@ -728,6 +728,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
strbuf_reset(err);
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = TRANSACTION_NONEXISTENT_REF;
} else {
/*
* The error message set by
@@ -788,6 +789,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
/* Garden variety missing reference. */
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = TRANSACTION_NONEXISTENT_REF;
goto error_return;
} else {
/*
@@ -820,6 +822,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
/* Garden variety missing reference. */
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = TRANSACTION_NONEXISTENT_REF;
goto error_return;
} else if (remove_dir_recursively(&ref_file,
REMOVE_DIR_EMPTY_ONLY)) {
@@ -863,7 +866,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
string_list_insert(refnames_to_check, refname);
}
- ret = 0;
+ ret = TRANSACTION_OK;
goto out;
error_return:
@@ -1517,10 +1520,10 @@ static int rename_tmp_log(struct files_ref_store *refs, const char *newrefname)
return ret;
}
-static int write_ref_to_lockfile(struct files_ref_store *refs,
- struct ref_lock *lock,
- const struct object_id *oid,
- int skip_oid_verification, struct strbuf *err);
+static enum transaction_error write_ref_to_lockfile(struct files_ref_store *refs,
+ struct ref_lock *lock,
+ const struct object_id *oid,
+ int skip_oid_verification, struct strbuf *err);
static int commit_ref_update(struct files_ref_store *refs,
struct ref_lock *lock,
const struct object_id *oid, const char *logmsg,
@@ -1926,10 +1929,11 @@ static int files_log_ref_write(struct files_ref_store *refs,
* Write oid into the open lockfile, then close the lockfile. On
* errors, rollback the lockfile, fill in *err and return -1.
*/
-static int write_ref_to_lockfile(struct files_ref_store *refs,
- struct ref_lock *lock,
- const struct object_id *oid,
- int skip_oid_verification, struct strbuf *err)
+static enum transaction_error write_ref_to_lockfile(struct files_ref_store *refs,
+ struct ref_lock *lock,
+ const struct object_id *oid,
+ int skip_oid_verification,
+ struct strbuf *err)
{
static char term = '\n';
struct object *o;
@@ -1943,7 +1947,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
"trying to write ref '%s' with nonexistent object %s",
lock->ref_name, oid_to_hex(oid));
unlock_ref(lock);
- return -1;
+ return TRANSACTION_INVALID_NEW_VALUE;
}
if (o->type != OBJ_COMMIT && is_branch(lock->ref_name)) {
strbuf_addf(
@@ -1951,7 +1955,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
"trying to write non-commit object %s to branch '%s'",
oid_to_hex(oid), lock->ref_name);
unlock_ref(lock);
- return -1;
+ return TRANSACTION_INVALID_NEW_VALUE;
}
}
fd = get_lock_file_fd(&lock->lk);
@@ -1962,9 +1966,9 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
strbuf_addf(err,
"couldn't write '%s'", get_lock_file_path(&lock->lk));
unlock_ref(lock);
- return -1;
+ return TRANSACTION_GENERIC_ERROR;
}
- return 0;
+ return TRANSACTION_OK;
}
/*
@@ -2376,9 +2380,10 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st
* If update is a direct update of head_ref (the reference pointed to
* by HEAD), then add an extra REF_LOG_ONLY update for HEAD.
*/
-static int split_head_update(struct ref_update *update,
- struct ref_transaction *transaction,
- const char *head_ref, struct strbuf *err)
+static enum transaction_error split_head_update(struct ref_update *update,
+ struct ref_transaction *transaction,
+ const char *head_ref,
+ struct strbuf *err)
{
struct ref_update *new_update;
@@ -2386,10 +2391,10 @@ static int split_head_update(struct ref_update *update,
(update->flags & REF_SKIP_CREATE_REFLOG) ||
(update->flags & REF_IS_PRUNING) ||
(update->flags & REF_UPDATE_VIA_HEAD))
- return 0;
+ return TRANSACTION_OK;
if (strcmp(update->refname, head_ref))
- return 0;
+ return TRANSACTION_OK;
/*
* First make sure that HEAD is not already in the
@@ -2419,7 +2424,7 @@ static int split_head_update(struct ref_update *update,
if (strcmp(new_update->refname, "HEAD"))
BUG("%s unexpectedly not 'HEAD'", new_update->refname);
- return 0;
+ return TRANSACTION_OK;
}
/*
@@ -2430,10 +2435,10 @@ static int split_head_update(struct ref_update *update,
* Note that the new update will itself be subject to splitting when
* the iteration gets to it.
*/
-static int split_symref_update(struct ref_update *update,
- const char *referent,
- struct ref_transaction *transaction,
- struct strbuf *err)
+static enum transaction_error split_symref_update(struct ref_update *update,
+ const char *referent,
+ struct ref_transaction *transaction,
+ struct strbuf *err)
{
struct ref_update *new_update;
unsigned int new_flags;
@@ -2482,7 +2487,7 @@ static int split_symref_update(struct ref_update *update,
update->flags |= REF_LOG_ONLY | REF_NO_DEREF;
update->flags &= ~REF_HAVE_OLD;
- return 0;
+ return TRANSACTION_OK;
}
/*
@@ -2491,34 +2496,32 @@ static int split_symref_update(struct ref_update *update,
* everything is OK, return 0; otherwise, write an error message to
* err and return -1.
*/
-static int check_old_oid(struct ref_update *update, struct object_id *oid,
- struct strbuf *err)
+static enum transaction_error check_old_oid(struct ref_update *update,
+ struct object_id *oid,
+ struct strbuf *err)
{
- int ret = TRANSACTION_GENERIC_ERROR;
-
if (!(update->flags & REF_HAVE_OLD) ||
oideq(oid, &update->old_oid))
- return 0;
+ return TRANSACTION_OK;
if (is_null_oid(&update->old_oid)) {
strbuf_addf(err, "cannot lock ref '%s': "
"reference already exists",
ref_update_original_update_refname(update));
- ret = TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(oid))
+ return TRANSACTION_CREATE_EXISTS;
+ } else if (is_null_oid(oid)) {
strbuf_addf(err, "cannot lock ref '%s': "
"reference is missing but expected %s",
ref_update_original_update_refname(update),
oid_to_hex(&update->old_oid));
- else
- strbuf_addf(err, "cannot lock ref '%s': "
- "is at %s but expected %s",
- ref_update_original_update_refname(update),
- oid_to_hex(oid),
- oid_to_hex(&update->old_oid));
+ return TRANSACTION_NONEXISTENT_REF;
+ }
- return ret;
+ strbuf_addf(err, "cannot lock ref '%s': is at %s but expected %s",
+ ref_update_original_update_refname(update), oid_to_hex(oid),
+ oid_to_hex(&update->old_oid));
+
+ return TRANSACTION_INCORRECT_OLD_VALUE;
}
struct files_transaction_backend_data {
@@ -2540,17 +2543,17 @@ struct files_transaction_backend_data {
* - If it is an update of head_ref, add a corresponding REF_LOG_ONLY
* update of HEAD.
*/
-static int lock_ref_for_update(struct files_ref_store *refs,
- struct ref_update *update,
- struct ref_transaction *transaction,
- const char *head_ref,
- struct string_list *refnames_to_check,
- struct strbuf *err)
+static enum transaction_error lock_ref_for_update(struct files_ref_store *refs,
+ struct ref_update *update,
+ struct ref_transaction *transaction,
+ const char *head_ref,
+ struct string_list *refnames_to_check,
+ struct strbuf *err)
{
struct strbuf referent = STRBUF_INIT;
int mustexist = ref_update_expects_existing_old_ref(update);
struct files_transaction_backend_data *backend_data;
- int ret = 0;
+ enum transaction_error ret = TRANSACTION_OK;
struct ref_lock *lock;
files_assert_main_repository(refs, "lock_ref_for_update");
@@ -2607,16 +2610,12 @@ static int lock_ref_for_update(struct files_ref_store *refs,
}
}
- if (update->old_target) {
- if (ref_update_check_old_target(referent.buf, update, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto out;
- }
- } else {
+ if (update->old_target)
+ ret = ref_update_check_old_target(referent.buf, update, err);
+ else
ret = check_old_oid(update, &lock->old_oid, err);
- if (ret) {
- goto out;
- }
+ if (ret) {
+ goto out;
}
} else {
/*
@@ -2644,7 +2643,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
"but is a regular ref"),
ref_update_original_update_refname(update),
update->old_target);
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = TRANSACTION_EXPECTED_SYMREF;
goto out;
} else {
ret = check_old_oid(update, &lock->old_oid, err);
@@ -2693,25 +2692,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
* The reference already has the desired
* value, so we don't need to write it.
*/
- } else if (write_ref_to_lockfile(
- refs, lock, &update->new_oid,
- update->flags & REF_SKIP_OID_VERIFICATION,
- err)) {
- char *write_err = strbuf_detach(err, NULL);
-
- /*
- * The lock was freed upon failure of
- * write_ref_to_lockfile():
- */
- update->backend_data = NULL;
- strbuf_addf(err,
- "cannot update ref '%s': %s",
- update->refname, write_err);
- free(write_err);
- ret = TRANSACTION_GENERIC_ERROR;
- goto out;
} else {
- update->flags |= REF_NEEDS_COMMIT;
+ ret = write_ref_to_lockfile(
+ refs, lock, &update->new_oid,
+ update->flags & REF_SKIP_OID_VERIFICATION,
+ err);
+ if (ret) {
+ char *write_err = strbuf_detach(err, NULL);
+
+ /*
+ * The lock was freed upon failure of
+ * write_ref_to_lockfile():
+ */
+ update->backend_data = NULL;
+ strbuf_addf(err,
+ "cannot update ref '%s': %s",
+ update->refname, write_err);
+ free(write_err);
+ goto out;
+ } else {
+ update->flags |= REF_NEEDS_COMMIT;
+ }
}
}
if (!(update->flags & REF_NEEDS_COMMIT)) {
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 3247871574..75e1ebf67d 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1323,10 +1323,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
* The packfile must be locked before calling this function and will
* remain locked when it is done.
*/
-static int write_with_updates(struct packed_ref_store *refs,
- struct string_list *updates,
- struct strbuf *err)
+static enum transaction_error write_with_updates(struct packed_ref_store *refs,
+ struct string_list *updates,
+ struct strbuf *err)
{
+ enum transaction_error ret = TRANSACTION_GENERIC_ERROR;
struct ref_iterator *iter = NULL;
size_t i;
int ok;
@@ -1350,7 +1351,7 @@ static int write_with_updates(struct packed_ref_store *refs,
strbuf_addf(err, "unable to create file %s: %s",
sb.buf, strerror(errno));
strbuf_release(&sb);
- return -1;
+ return TRANSACTION_GENERIC_ERROR;
}
strbuf_release(&sb);
@@ -1406,6 +1407,7 @@ static int write_with_updates(struct packed_ref_store *refs,
strbuf_addf(err, "cannot update ref '%s': "
"reference already exists",
update->refname);
+ ret = TRANSACTION_CREATE_EXISTS;
goto error;
} else if (!oideq(&update->old_oid, iter->oid)) {
strbuf_addf(err, "cannot update ref '%s': "
@@ -1413,6 +1415,7 @@ static int write_with_updates(struct packed_ref_store *refs,
update->refname,
oid_to_hex(iter->oid),
oid_to_hex(&update->old_oid));
+ ret = TRANSACTION_INCORRECT_OLD_VALUE;
goto error;
}
}
@@ -1449,6 +1452,7 @@ static int write_with_updates(struct packed_ref_store *refs,
"reference is missing but expected %s",
update->refname,
oid_to_hex(&update->old_oid));
+ return TRANSACTION_NONEXISTENT_REF;
goto error;
}
}
@@ -1506,10 +1510,10 @@ static int write_with_updates(struct packed_ref_store *refs,
strerror(errno));
strbuf_release(&sb);
delete_tempfile(&refs->tempfile);
- return -1;
+ return TRANSACTION_GENERIC_ERROR;
}
- return 0;
+ return TRANSACTION_OK;
write_error:
strbuf_addf(err, "error writing to %s: %s",
@@ -1518,7 +1522,7 @@ static int write_with_updates(struct packed_ref_store *refs,
error:
ref_iterator_free(iter);
delete_tempfile(&refs->tempfile);
- return -1;
+ return ret;
}
int is_packed_transaction_needed(struct ref_store *ref_store,
@@ -1672,7 +1676,8 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
data->own_lock = 1;
}
- if (write_with_updates(refs, &transaction->refnames, err))
+ ret = write_with_updates(refs, &transaction->refnames, err);
+ if (ret)
goto failure;
transaction->state = REF_TRANSACTION_PREPARED;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 4b7fc8f1ab..c97045fbed 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -769,8 +769,9 @@ int ref_update_has_null_new_value(struct ref_update *update);
* If everything is OK, return 0; otherwise, write an error message to
* err and return -1.
*/
-int ref_update_check_old_target(const char *referent, struct ref_update *update,
- struct strbuf *err);
+enum transaction_error ref_update_check_old_target(const char *referent,
+ struct ref_update *update,
+ struct strbuf *err);
/*
* Check if the ref must exist, this means that the old_oid or
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 2c1e2995de..e1fd9c2de2 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1069,20 +1069,20 @@ static int queue_transaction_update(struct reftable_ref_store *refs,
return 0;
}
-static int prepare_single_update(struct reftable_ref_store *refs,
- struct reftable_transaction_data *tx_data,
- struct ref_transaction *transaction,
- struct reftable_backend *be,
- struct ref_update *u,
- struct string_list *refnames_to_check,
- unsigned int head_type,
- struct strbuf *head_referent,
- struct strbuf *referent,
- struct strbuf *err)
+static enum transaction_error prepare_single_update(struct reftable_ref_store *refs,
+ struct reftable_transaction_data *tx_data,
+ struct ref_transaction *transaction,
+ struct reftable_backend *be,
+ struct ref_update *u,
+ struct string_list *refnames_to_check,
+ unsigned int head_type,
+ struct strbuf *head_referent,
+ struct strbuf *referent,
+ struct strbuf *err)
{
+ enum transaction_error ret = TRANSACTION_OK;
struct object_id current_oid = {0};
const char *rewritten_ref;
- int ret = 0;
/*
* There is no need to reload the respective backends here as
@@ -1093,7 +1093,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
*/
ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
if (ret)
- return ret;
+ return TRANSACTION_GENERIC_ERROR;
/* Verify that the new object ID is valid. */
if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
@@ -1104,13 +1104,13 @@ static int prepare_single_update(struct reftable_ref_store *refs,
strbuf_addf(err,
_("trying to write ref '%s' with nonexistent object %s"),
u->refname, oid_to_hex(&u->new_oid));
- return -1;
+ return TRANSACTION_INVALID_NEW_VALUE;
}
if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
oid_to_hex(&u->new_oid), u->refname);
- return -1;
+ return TRANSACTION_INVALID_NEW_VALUE;
}
}
@@ -1147,7 +1147,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
ret = reftable_backend_read_ref(be, rewritten_ref,
¤t_oid, referent, &u->type);
if (ret < 0)
- return ret;
+ return TRANSACTION_GENERIC_ERROR;
if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
/*
* The reference does not exist, and we either have no
@@ -1168,7 +1168,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
ret = queue_transaction_update(refs, tx_data, u,
¤t_oid, err);
if (ret)
- return ret;
+ return TRANSACTION_GENERIC_ERROR;
}
return 0;
@@ -1180,7 +1180,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
"unable to resolve reference '%s'"),
ref_update_original_update_refname(u), u->refname);
- return -1;
+ return TRANSACTION_NONEXISTENT_REF;
}
if (u->type & REF_ISSYMREF) {
@@ -1196,7 +1196,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
if (u->flags & REF_HAVE_OLD && !resolved) {
strbuf_addf(err, _("cannot lock ref '%s': "
"error reading reference"), u->refname);
- return -1;
+ return TRANSACTION_GENERIC_ERROR;
}
} else {
struct ref_update *new_update;
@@ -1255,11 +1255,12 @@ static int prepare_single_update(struct reftable_ref_store *refs,
"but is a regular ref"),
ref_update_original_update_refname(u),
u->old_target);
- return -1;
+ return TRANSACTION_EXPECTED_SYMREF;
}
- if (ref_update_check_old_target(referent->buf, u, err)) {
- return -1;
+ ret = ref_update_check_old_target(referent->buf, u, err);
+ if (ret) {
+ return ret;
}
} else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
if (is_null_oid(&u->old_oid)) {
@@ -1268,18 +1269,21 @@ static int prepare_single_update(struct reftable_ref_store *refs,
ref_update_original_update_refname(u));
return TRANSACTION_CREATE_EXISTS;
}
- else if (is_null_oid(¤t_oid))
+ else if (is_null_oid(¤t_oid)) {
strbuf_addf(err, _("cannot lock ref '%s': "
"reference is missing but expected %s"),
ref_update_original_update_refname(u),
oid_to_hex(&u->old_oid));
- else
+ return TRANSACTION_NONEXISTENT_REF;
+
+ } else {
strbuf_addf(err, _("cannot lock ref '%s': "
"is at %s but expected %s"),
ref_update_original_update_refname(u),
oid_to_hex(¤t_oid),
oid_to_hex(&u->old_oid));
- return TRANSACTION_NAME_CONFLICT;
+ return TRANSACTION_INCORRECT_OLD_VALUE;
+ }
}
/*
@@ -1296,10 +1300,10 @@ static int prepare_single_update(struct reftable_ref_store *refs,
if ((u->type & REF_ISSYMREF) ||
(u->flags & REF_LOG_ONLY) ||
(u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))
- return queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
+ if (queue_transaction_update(refs, tx_data, u, ¤t_oid, err))
+ return TRANSACTION_GENERIC_ERROR;
- return 0;
+ return TRANSACTION_OK;
}
static int reftable_be_transaction_prepare(struct ref_store *ref_store,
@@ -1386,7 +1390,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->state = REF_TRANSACTION_PREPARED;
done:
- assert(ret != REFTABLE_API_ERROR);
if (ret < 0) {
free_transaction_data(tx_data);
transaction->state = REF_TRANSACTION_CLOSED;
--
2.47.2
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v2 6/7] refs: implement partial reference transaction support
2025-02-25 9:29 ` [PATCH v2 0/7] " Karthik Nayak
` (4 preceding siblings ...)
2025-02-25 9:29 ` [PATCH v2 5/7] refs: introduce enum-based transaction error types Karthik Nayak
@ 2025-02-25 9:29 ` Karthik Nayak
2025-02-25 11:07 ` Patrick Steinhardt
2025-02-25 14:57 ` Phillip Wood
2025-02-25 9:29 ` [PATCH v2 7/7] update-ref: add --allow-partial flag for stdin mode Karthik Nayak
6 siblings, 2 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-25 9:29 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Git's reference transactions are all-or-nothing: either all updates
succeed, or none do. While this atomic behavior is generally desirable,
it can be suboptimal especially when using the reftable backend, where
batching multiple reference updates into a single transaction is more
efficient than performing them sequentially.
Introduce partial transaction support with a new flag,
'REF_TRANSACTION_ALLOW_PARTIAL'. When enabled, this flag allows
individual reference updates that would typically cause the entire
transaction to fail due to non-system-related errors to be marked as
rejected while permitting other updates to proceed. Non-system-related
errors include issues caused by user-provided input values, whereas
system-related errors, such as I/O failures or memory issues, continue
to result in a full transaction failure. This approach enhances
flexibility while preserving transactional integrity where necessary.
The implementation introduces several key components:
- Add 'rejection_err' field to struct `ref_update` to track failed
updates with failure reason.
- Modify reference backends (files, packed, reftable) to handle
partial transactions by using `ref_transaction_set_rejected()`
instead of failing the entire transaction when
`REF_TRANSACTION_ALLOW_PARTIAL` is set.
- Add `ref_transaction_for_each_rejected_update()` to let callers
examine which updates were rejected and why.
This foundational change enables partial transaction support throughout
the reference subsystem. The next commit will expose this capability to
users by adding a `--allow-partial` flag to 'git-update-ref(1)',
providing both a user-facing feature and a testable implementation.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 31 +++++++++++++++++++++++++++++++
refs.h | 22 ++++++++++++++++++++++
refs/files-backend.c | 12 +++++++++++-
refs/packed-backend.c | 30 ++++++++++++++++++++++++++++--
refs/refs-internal.h | 13 +++++++++++++
refs/reftable-backend.c | 12 +++++++++++-
6 files changed, 116 insertions(+), 4 deletions(-)
diff --git a/refs.c b/refs.c
index f989a46a5a..243c09c368 100644
--- a/refs.c
+++ b/refs.c
@@ -1211,6 +1211,15 @@ void ref_transaction_free(struct ref_transaction *transaction)
free(transaction);
}
+void ref_transaction_set_rejected(struct ref_transaction *transaction,
+ size_t update_idx,
+ enum transaction_error err)
+{
+ if (update_idx >= transaction->nr)
+ BUG("trying to set rejection on invalid update index");
+ transaction->updates[update_idx]->rejection_err = err;
+}
+
struct ref_update *ref_transaction_add_update(
struct ref_transaction *transaction,
const char *refname, unsigned int flags,
@@ -1236,6 +1245,7 @@ struct ref_update *ref_transaction_add_update(
transaction->updates[transaction->nr++] = update;
update->flags = flags;
+ update->rejection_err = TRANSACTION_OK;
update->new_target = xstrdup_or_null(new_target);
update->old_target = xstrdup_or_null(old_target);
@@ -2726,6 +2736,27 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
}
}
+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
+ ref_transaction_for_each_rejected_update_fn cb,
+ void *cb_data)
+{
+ if (!(transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL))
+ return;
+
+ for (size_t i = 0; i < transaction->nr; i++) {
+ struct ref_update *update = transaction->updates[i];
+
+ if (!update->rejection_err)
+ continue;
+
+ cb(update->refname,
+ (update->flags & REF_HAVE_OLD) ? &update->old_oid : NULL,
+ (update->flags & REF_HAVE_NEW) ? &update->new_oid : NULL,
+ update->old_target, update->new_target,
+ update->rejection_err, cb_data);
+ }
+}
+
int refs_delete_refs(struct ref_store *refs, const char *logmsg,
struct string_list *refnames, unsigned int flags)
{
diff --git a/refs.h b/refs.h
index 8e9ead174c..e4a6a8218f 100644
--- a/refs.h
+++ b/refs.h
@@ -675,6 +675,13 @@ enum ref_transaction_flag {
* either be absent or null_oid.
*/
REF_TRANSACTION_FLAG_INITIAL = (1 << 0),
+
+ /*
+ * The transaction mechanism by default fails all updates if any conflict
+ * is detected. This flag allows transactions to partially apply updates
+ * while rejecting updates which do not match the expected state.
+ */
+ REF_TRANSACTION_ALLOW_PARTIAL = (1 << 1),
};
/*
@@ -905,6 +912,21 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
ref_transaction_for_each_queued_update_fn cb,
void *cb_data);
+/*
+ * Execute the given callback function for each of the reference updates which
+ * have been rejected in the given transaction.
+ */
+typedef void ref_transaction_for_each_rejected_update_fn(const char *refname,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
+ enum transaction_error err,
+ void *cb_data);
+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
+ ref_transaction_for_each_rejected_update_fn cb,
+ void *cb_data);
+
/*
* Free `*transaction` and all associated data.
*/
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 3b0adf8bb2..d0a53c9ace 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2851,8 +2851,18 @@ static int files_transaction_prepare(struct ref_store *ref_store,
ret = lock_ref_for_update(refs, update, transaction,
head_ref, &refnames_to_check,
err);
- if (ret)
+ if (ret) {
+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL &&
+ ret != TRANSACTION_GENERIC_ERROR) {
+ ref_transaction_set_rejected(transaction, i, ret);
+
+ strbuf_setlen(err, 0);
+ ret = TRANSACTION_OK;
+
+ continue;
+ }
goto cleanup;
+ }
if (update->flags & REF_DELETING &&
!(update->flags & REF_LOG_ONLY) &&
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 75e1ebf67d..0857204213 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1324,10 +1324,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
* remain locked when it is done.
*/
static enum transaction_error write_with_updates(struct packed_ref_store *refs,
- struct string_list *updates,
+ struct ref_transaction *transaction,
struct strbuf *err)
{
enum transaction_error ret = TRANSACTION_GENERIC_ERROR;
+ struct string_list *updates = &transaction->refnames;
struct ref_iterator *iter = NULL;
size_t i;
int ok;
@@ -1408,6 +1409,14 @@ static enum transaction_error write_with_updates(struct packed_ref_store *refs,
"reference already exists",
update->refname);
ret = TRANSACTION_CREATE_EXISTS;
+
+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
+ ref_transaction_set_rejected(transaction, i, ret);
+ strbuf_setlen(err, 0);
+ ret = 0;
+ continue;
+ }
+
goto error;
} else if (!oideq(&update->old_oid, iter->oid)) {
strbuf_addf(err, "cannot update ref '%s': "
@@ -1416,6 +1425,14 @@ static enum transaction_error write_with_updates(struct packed_ref_store *refs,
oid_to_hex(iter->oid),
oid_to_hex(&update->old_oid));
ret = TRANSACTION_INCORRECT_OLD_VALUE;
+
+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
+ ref_transaction_set_rejected(transaction, i, ret);
+ strbuf_setlen(err, 0);
+ ret = 0;
+ continue;
+ }
+
goto error;
}
}
@@ -1453,6 +1470,14 @@ static enum transaction_error write_with_updates(struct packed_ref_store *refs,
update->refname,
oid_to_hex(&update->old_oid));
return TRANSACTION_NONEXISTENT_REF;
+
+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
+ ref_transaction_set_rejected(transaction, i, ret);
+ strbuf_setlen(err, 0);
+ ret = 0;
+ continue;
+ }
+
goto error;
}
}
@@ -1518,6 +1543,7 @@ static enum transaction_error write_with_updates(struct packed_ref_store *refs,
write_error:
strbuf_addf(err, "error writing to %s: %s",
get_tempfile_path(refs->tempfile), strerror(errno));
+ ret = TRANSACTION_GENERIC_ERROR;
error:
ref_iterator_free(iter);
@@ -1676,7 +1702,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
data->own_lock = 1;
}
- ret = write_with_updates(refs, &transaction->refnames, err);
+ ret = write_with_updates(refs, transaction, err);
if (ret)
goto failure;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index c97045fbed..7196f2d880 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -123,6 +123,11 @@ struct ref_update {
*/
uint64_t index;
+ /*
+ * Used in partial transactions to mark if a given update was rejected.
+ */
+ enum transaction_error rejection_err;
+
/*
* If this ref_update was split off of a symref update via
* split_symref_update(), then this member points at that
@@ -142,6 +147,14 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
struct object_id *oid, struct strbuf *referent,
unsigned int *type, int *failure_errno);
+/*
+ * Mark a given update as rejected with a given reason. To be used in conjuction
+ * with the `REF_TRANSACTION_ALLOW_PARTIAL` flag to allow partial transactions.
+ */
+void ref_transaction_set_rejected(struct ref_transaction *transaction,
+ size_t update_idx,
+ enum transaction_error err);
+
/*
* Add a ref_update with the specified properties to transaction, and
* return a pointer to the new object. This function does not verify
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index e1fd9c2de2..83cf8d582b 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1374,8 +1374,18 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->updates[i],
&refnames_to_check, head_type,
&head_referent, &referent, err);
- if (ret)
+ if (ret) {
+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL &&
+ ret != TRANSACTION_GENERIC_ERROR) {
+ ref_transaction_set_rejected(transaction, i, ret);
+
+ strbuf_setlen(err, 0);
+ ret = TRANSACTION_OK;
+
+ continue;
+ }
goto done;
+ }
}
string_list_sort(&refnames_to_check);
--
2.47.2
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v2 7/7] update-ref: add --allow-partial flag for stdin mode
2025-02-25 9:29 ` [PATCH v2 0/7] " Karthik Nayak
` (5 preceding siblings ...)
2025-02-25 9:29 ` [PATCH v2 6/7] refs: implement partial reference transaction support Karthik Nayak
@ 2025-02-25 9:29 ` Karthik Nayak
2025-02-25 11:08 ` Patrick Steinhardt
2025-02-25 14:59 ` Phillip Wood
6 siblings, 2 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-02-25 9:29 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
When updating multiple references through stdin, Git's update-ref
command normally aborts the entire transaction if any single update
fails. While this atomic behavior prevents partial updates by default,
there are cases where applying successful updates while reporting
failures is desirable.
Add a new `--allow-partial` flag that allows the transaction to continue
even when individual reference updates fail. This flag can only be used
in `--stdin` mode and builds upon the partial transaction support added
to the refs subsystem. When enabled, failed updates are reported in the
following format:
rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
or with `-z`:
rejected NUL (<old-oid> | <old-target>) NUL (<new-oid> | <new-target>) NUL <rejection-reason> NUL
Update the documentation to reflect this change and also tests to cover
different scenarios where an update could be rejected.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
Documentation/git-update-ref.adoc | 21 +++-
builtin/update-ref.c | 74 +++++++++++--
t/t1400-update-ref.sh | 216 ++++++++++++++++++++++++++++++++++++++
3 files changed, 302 insertions(+), 9 deletions(-)
diff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc
index 9e6935d38d..fc73f1d8aa 100644
--- a/Documentation/git-update-ref.adoc
+++ b/Documentation/git-update-ref.adoc
@@ -7,8 +7,10 @@ git-update-ref - Update the object name stored in a ref safely
SYNOPSIS
--------
-[verse]
-'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z])
+[synopsis]
+git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
+ [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
+ [-m <reason>] [--no-deref] --stdin [-z] [--allow-partial]
DESCRIPTION
-----------
@@ -57,6 +59,17 @@ performs all modifications together. Specify commands of the form:
With `--create-reflog`, update-ref will create a reflog for each ref
even if one would not ordinarily be created.
+With `--allow-partial`, update-ref continues executing the transaction even if
+some updates fail due to invalid or incorrect user input, applying only the
+successful updates. Errors resulting from user-provided input are treated as
+non-system-related and do not cause the entire transaction to be aborted.
+However, system-related errors—such as I/O failures or memory issues—will still
+result in a full failure. Additionally, errors like F/D conflicts are batched
+for performance optimization and will also cause a full failure. Any failed
+updates will be reported in the following format:
+
+ rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
+
Quote fields containing whitespace as if they were strings in C source
code; i.e., surrounded by double-quotes and with backslash escapes.
Use 40 "0" characters or the empty string to specify a zero value. To
@@ -82,6 +95,10 @@ quoting:
In this format, use 40 "0" to specify a zero value, and use the empty
string to specify a missing value.
+With `-z`, `--allow-partial` will print rejections in the following form:
+
+ rejected NUL (<old-oid> | <old-target>) NUL (<new-oid> | <new-target>) NUL <rejection-reason> NUL
+
In either format, values can be specified in any form that Git
recognizes as an object name. Commands in any other format or a
repeated <ref> produce an error. Command meanings are:
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 1d541e13ad..b03b40eacb 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -5,6 +5,7 @@
#include "config.h"
#include "gettext.h"
#include "hash.h"
+#include "hex.h"
#include "refs.h"
#include "object-name.h"
#include "parse-options.h"
@@ -13,7 +14,7 @@
static const char * const git_update_ref_usage[] = {
N_("git update-ref [<options>] -d <refname> [<old-oid>]"),
N_("git update-ref [<options>] <refname> <new-oid> [<old-oid>]"),
- N_("git update-ref [<options>] --stdin [-z]"),
+ N_("git update-ref [<options>] --stdin [-z] [--allow-partial]"),
NULL
};
@@ -565,6 +566,54 @@ static void parse_cmd_abort(struct ref_transaction *transaction,
report_ok("abort");
}
+static void print_rejected_refs(const char *refname,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
+ enum transaction_error err,
+ void *cb_data UNUSED)
+{
+ struct strbuf sb = STRBUF_INIT;
+ char space = ' ';
+ const char *reason = "";
+
+ switch (err) {
+ case TRANSACTION_NAME_CONFLICT:
+ reason = _("refname conflict");
+ break;
+ case TRANSACTION_CREATE_EXISTS:
+ reason = _("reference already exists");
+ break;
+ case TRANSACTION_NONEXISTENT_REF:
+ reason = _("reference does not exist");
+ break;
+ case TRANSACTION_INCORRECT_OLD_VALUE:
+ reason = _("incorrect old value provided");
+ break;
+ case TRANSACTION_INVALID_NEW_VALUE:
+ reason = _("invalid new value provided");
+ break;
+ case TRANSACTION_EXPECTED_SYMREF:
+ reason = _("expected symref but found regular ref");
+ break;
+ default:
+ reason = _("unkown failure");
+ }
+
+ if (!line_termination)
+ space = line_termination;
+
+ strbuf_addf(&sb, "rejected%c%s%c%s%c%c%s%c%s%c", space,
+ refname, space, new_oid ? oid_to_hex(new_oid) : new_target,
+ space, space, old_oid ? oid_to_hex(old_oid) : old_target,
+ space, reason, line_termination);
+
+ fwrite(sb.buf, sb.len, 1, stdout);
+ strbuf_release(&sb);
+ fflush(stdout);
+}
+
static void parse_cmd_commit(struct ref_transaction *transaction,
const char *next, const char *end UNUSED)
{
@@ -573,6 +622,10 @@ static void parse_cmd_commit(struct ref_transaction *transaction,
die("commit: extra input: %s", next);
if (ref_transaction_commit(transaction, &error))
die("commit: %s", error.buf);
+
+ ref_transaction_for_each_rejected_update(transaction,
+ print_rejected_refs, NULL);
+
report_ok("commit");
ref_transaction_free(transaction);
}
@@ -609,7 +662,7 @@ static const struct parse_cmd {
{ "commit", parse_cmd_commit, 0, UPDATE_REFS_CLOSED },
};
-static void update_refs_stdin(void)
+static void update_refs_stdin(unsigned int flags)
{
struct strbuf input = STRBUF_INIT, err = STRBUF_INIT;
enum update_refs_state state = UPDATE_REFS_OPEN;
@@ -617,7 +670,7 @@ static void update_refs_stdin(void)
int i, j;
transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
- 0, &err);
+ flags, &err);
if (!transaction)
die("%s", err.buf);
@@ -685,7 +738,7 @@ static void update_refs_stdin(void)
*/
state = cmd->state;
transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
- 0, &err);
+ flags, &err);
if (!transaction)
die("%s", err.buf);
@@ -701,6 +754,8 @@ static void update_refs_stdin(void)
/* Commit by default if no transaction was requested. */
if (ref_transaction_commit(transaction, &err))
die("%s", err.buf);
+ ref_transaction_for_each_rejected_update(transaction,
+ print_rejected_refs, NULL);
ref_transaction_free(transaction);
break;
case UPDATE_REFS_STARTED:
@@ -726,7 +781,9 @@ int cmd_update_ref(int argc,
const char *refname, *oldval;
struct object_id oid, oldoid;
int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;
- int create_reflog = 0;
+ int create_reflog = 0, allow_partial = 0;
+ unsigned int flags = 0;
+
struct option options[] = {
OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
OPT_BOOL('d', NULL, &delete, N_("delete the reference")),
@@ -735,6 +792,8 @@ int cmd_update_ref(int argc,
OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
OPT_BOOL( 0 , "create-reflog", &create_reflog, N_("create a reflog")),
+ OPT_BIT('0', "allow-partial", &flags, N_("allow partial transactions"),
+ REF_TRANSACTION_ALLOW_PARTIAL),
OPT_END(),
};
@@ -756,9 +815,10 @@ int cmd_update_ref(int argc,
usage_with_options(git_update_ref_usage, options);
if (end_null)
line_termination = '\0';
- update_refs_stdin();
+ update_refs_stdin(flags);
return 0;
- }
+ } else if (allow_partial)
+ die("--allow-partial can only be used with --stdin");
if (end_null)
usage_with_options(git_update_ref_usage, options);
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 29045aad43..fb9442982e 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -2066,6 +2066,222 @@ do
grep "$(git rev-parse $a) $(git rev-parse $a)" actual
'
+ test_expect_success "stdin $type allow-partial" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit commit &&
+ head=$(git rev-parse HEAD) &&
+
+ format_command $type "update refs/heads/ref1" "$head" "$Z" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head" "$Z" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with invalid new_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$(test_oid 001)" "$head" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "invalid new value provided" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with non-commit new_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ head_tree=$(git rev-parse HEAD^{tree}) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head_tree" "$head" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "invalid new value provided" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with non-existent ref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ test_must_fail git rev-parse refs/heads/ref2 &&
+ test_grep -q "reference does not exist" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with dangling symref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git symbolic-ref refs/heads/ref2 refs/heads/nonexistent &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
+ git update-ref $type --no-deref --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ test_must_fail git rev-parse refs/heads/ref2 &&
+ test_grep -q "reference does not exist" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with regular ref as symref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "symref-update refs/heads/ref2" "$old_head" "ref" "refs/heads/nonexistent" >>stdin &&
+ git update-ref $type --no-deref --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "expected symref but found regular ref" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with invalid old_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$Z" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "reference already exists" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with incorrect old oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head" "$old_head" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "incorrect old value provided" stdout
+ )
+ '
+
+ # F/D conflicts on the files backend are resolved on an individual
+ # update level since refs are stored as files. On the reftable backend
+ # this check is batched to optimize for performance, so failures cannot
+ # be isolated to a single update.
+ test_expect_success REFFILES "stdin $type allow-partial refname conflict" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref/foo $head &&
+
+ format_command $type "update refs/heads/ref/foo" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref/foo >actual &&
+ test_cmp expect actual &&
+ test_grep -q "refname conflict" stdout
+ )
+ '
done
test_expect_success 'update-ref should also create reflog for HEAD' '
--
2.47.2
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH v2 6/7] refs: implement partial reference transaction support
2025-02-25 9:29 ` [PATCH v2 6/7] refs: implement partial reference transaction support Karthik Nayak
@ 2025-02-25 11:07 ` Patrick Steinhardt
2025-03-03 20:17 ` Karthik Nayak
2025-02-25 14:57 ` Phillip Wood
1 sibling, 1 reply; 143+ messages in thread
From: Patrick Steinhardt @ 2025-02-25 11:07 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler, phillip.wood123
On Tue, Feb 25, 2025 at 10:29:09AM +0100, Karthik Nayak wrote:
> diff --git a/refs.c b/refs.c
> index f989a46a5a..243c09c368 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -2726,6 +2736,27 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
> }
> }
>
> +void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
> + ref_transaction_for_each_rejected_update_fn cb,
> + void *cb_data)
> +{
> + if (!(transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL))
> + return;
> +
> + for (size_t i = 0; i < transaction->nr; i++) {
> + struct ref_update *update = transaction->updates[i];
> +
> + if (!update->rejection_err)
> + continue;
This kind of proves my point that `TRANSACTION_OK` is pointless and
leads to a mixture of using and not using the enum :)
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index 3b0adf8bb2..d0a53c9ace 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -2851,8 +2851,18 @@ static int files_transaction_prepare(struct ref_store *ref_store,
> ret = lock_ref_for_update(refs, update, transaction,
> head_ref, &refnames_to_check,
> err);
> - if (ret)
> + if (ret) {
> + if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL &&
Hm. If the error values were defined as a bitfield we could refactor
this to not be a flag, but have a `transaction->accepted_rejections`
instead that allows the caller to ask for only a subset of rejections to
be accepted.
I'm not quite sure whether it is a good idea, but the logic to handle
all of that could be self-contained in `ref_transaction_set_rejected()`:
if it returned an error code itself, it would swallow any errors in case
the transaction allows a given error, and bubble up the error again in
case the error is not allowed. The function could use a rename in that
case though, e.g. `ref_transaction_maybe_set_rejected()`.
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v2 7/7] update-ref: add --allow-partial flag for stdin mode
2025-02-25 9:29 ` [PATCH v2 7/7] update-ref: add --allow-partial flag for stdin mode Karthik Nayak
@ 2025-02-25 11:08 ` Patrick Steinhardt
2025-03-03 20:22 ` Karthik Nayak
2025-02-25 14:59 ` Phillip Wood
1 sibling, 1 reply; 143+ messages in thread
From: Patrick Steinhardt @ 2025-02-25 11:08 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler, phillip.wood123
On Tue, Feb 25, 2025 at 10:29:10AM +0100, Karthik Nayak wrote:
> diff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc
> index 9e6935d38d..fc73f1d8aa 100644
> --- a/Documentation/git-update-ref.adoc
> +++ b/Documentation/git-update-ref.adoc
> @@ -7,8 +7,10 @@ git-update-ref - Update the object name stored in a ref safely
>
> SYNOPSIS
> --------
> -[verse]
> -'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z])
> +[synopsis]
> +git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
> + [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
> + [-m <reason>] [--no-deref] --stdin [-z] [--allow-partial]
>
> DESCRIPTION
> -----------
> @@ -57,6 +59,17 @@ performs all modifications together. Specify commands of the form:
> With `--create-reflog`, update-ref will create a reflog for each ref
> even if one would not ordinarily be created.
>
> +With `--allow-partial`, update-ref continues executing the transaction even if
> +some updates fail due to invalid or incorrect user input, applying only the
> +successful updates. Errors resulting from user-provided input are treated as
> +non-system-related and do not cause the entire transaction to be aborted.
> +However, system-related errors—such as I/O failures or memory issues—will still
> +result in a full failure. Additionally, errors like F/D conflicts are batched
> +for performance optimization and will also cause a full failure. Any failed
> +updates will be reported in the following format:
Shouldn't it be possible to detect F/D conflicts though and not abort
the transaction? If we want to make use of partial transactions in the
context of git-fetch(1) and/or git-receive-pack(1) we would have to
handle them.
> diff --git a/builtin/update-ref.c b/builtin/update-ref.c
> index 1d541e13ad..b03b40eacb 100644
> --- a/builtin/update-ref.c
> +++ b/builtin/update-ref.c
> @@ -565,6 +566,54 @@ static void parse_cmd_abort(struct ref_transaction *transaction,
> report_ok("abort");
> }
>
> +static void print_rejected_refs(const char *refname,
> + const struct object_id *old_oid,
> + const struct object_id *new_oid,
> + const char *old_target,
> + const char *new_target,
> + enum transaction_error err,
> + void *cb_data UNUSED)
> +{
> + struct strbuf sb = STRBUF_INIT;
> + char space = ' ';
> + const char *reason = "";
> +
> + switch (err) {
> + case TRANSACTION_NAME_CONFLICT:
> + reason = _("refname conflict");
> + break;
> + case TRANSACTION_CREATE_EXISTS:
> + reason = _("reference already exists");
> + break;
> + case TRANSACTION_NONEXISTENT_REF:
> + reason = _("reference does not exist");
> + break;
> + case TRANSACTION_INCORRECT_OLD_VALUE:
> + reason = _("incorrect old value provided");
> + break;
> + case TRANSACTION_INVALID_NEW_VALUE:
> + reason = _("invalid new value provided");
> + break;
> + case TRANSACTION_EXPECTED_SYMREF:
> + reason = _("expected symref but found regular ref");
> + break;
> + default:
> + reason = _("unkown failure");
> + }
As git-update-ref(1) is part of plumbing we don't want to translate
those messages.
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v2 5/7] refs: introduce enum-based transaction error types
2025-02-25 9:29 ` [PATCH v2 5/7] refs: introduce enum-based transaction error types Karthik Nayak
@ 2025-02-25 11:08 ` Patrick Steinhardt
2025-03-03 20:12 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Patrick Steinhardt @ 2025-02-25 11:08 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler, phillip.wood123
On Tue, Feb 25, 2025 at 10:29:08AM +0100, Karthik Nayak wrote:
> diff --git a/refs.h b/refs.h
> index b14ba1f9ff..8e9ead174c 100644
> --- a/refs.h
> +++ b/refs.h
> @@ -16,6 +16,31 @@ struct worktree;
> enum ref_storage_format ref_storage_format_by_name(const char *name);
> const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);
>
> +/*
> + * enum transaction_error represents the following return codes:
> + * TRANSACTION_OK: success code.
> + * TRANSACTION_GENERIC_ERROR error_code: default error code.
> + * TRANSACTION_NAME_CONFLICT error_code: ref name conflict like A vs A/B.
> + * TRANSACTION_CREATE_EXISTS error_code: ref to be created already exists.
> + * TRANSACTION_NONEXISTENT_REF error_code: ref expected but doesn't exist.
> + * TRANSACTION_INCORRECT_OLD_VALUE error_code: provided old_oid or old_target of
> + * reference doesn't match actual.
> + * TRANSACTION_INVALID_NEW_VALUE error_code: provided new_oid or new_target is
> + * invalid.
> + * TRANSACTION_EXPECTED_SYMREF error_code: expected ref to be symref, but is a
> + * regular ref.
> + */
> +enum transaction_error {
> + TRANSACTION_OK = 0,
> + TRANSACTION_GENERIC_ERROR = -1,
> + TRANSACTION_NAME_CONFLICT = -2,
> + TRANSACTION_CREATE_EXISTS = -3,
> + TRANSACTION_NONEXISTENT_REF = -4,
> + TRANSACTION_INCORRECT_OLD_VALUE = -5,
> + TRANSACTION_INVALID_NEW_VALUE = -6,
> + TRANSACTION_EXPECTED_SYMREF = -7,
> +};
Nit: how about we name this `ref_transaction_error` and adapt the the
enum values accordingly? We may eventually also introduce similar errors
for the object database, so it may make sense to have the errors be
specific. Doing both the enum and changing the name might be a bit hard
to review, so we could also rename in a preparatory commit. Or we just
punt on it for now and do it once it becomes necessary, that would also
be fine with me.
I also wonder whether we really want to introduce `TRANSACTION_OK`. It's
always a bit of a mouthful, and in many cases one ends up with a mixture
of `ret < 0`, `ret != TRANSACTION_OK` and `ret != 0`, which may lead to
confusion. Continuing to use `0` for the successful case should be fine.
> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
> index 3247871574..75e1ebf67d 100644
> --- a/refs/packed-backend.c
> +++ b/refs/packed-backend.c
> @@ -1672,7 +1676,8 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
> data->own_lock = 1;
> }
>
> - if (write_with_updates(refs, &transaction->refnames, err))
> + ret = write_with_updates(refs, &transaction->refnames, err);
> + if (ret)
> goto failure;
>
> transaction->state = REF_TRANSACTION_PREPARED;
Do we also want to change the local variable declaration of `int ret` to
use the new type?
> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
> index 2c1e2995de..e1fd9c2de2 100644
> --- a/refs/reftable-backend.c
> +++ b/refs/reftable-backend.c
> @@ -1255,11 +1255,12 @@ static int prepare_single_update(struct reftable_ref_store *refs,
> "but is a regular ref"),
> ref_update_original_update_refname(u),
> u->old_target);
> - return -1;
> + return TRANSACTION_EXPECTED_SYMREF;
> }
>
> - if (ref_update_check_old_target(referent->buf, u, err)) {
> - return -1;
> + ret = ref_update_check_old_target(referent->buf, u, err);
> + if (ret) {
> + return ret;
> }
> } else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
> if (is_null_oid(&u->old_oid)) {
Nit: superfluous braces that we could remove while at it.
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v2 6/7] refs: implement partial reference transaction support
2025-02-25 9:29 ` [PATCH v2 6/7] refs: implement partial reference transaction support Karthik Nayak
2025-02-25 11:07 ` Patrick Steinhardt
@ 2025-02-25 14:57 ` Phillip Wood
2025-03-03 20:21 ` Karthik Nayak
1 sibling, 1 reply; 143+ messages in thread
From: Phillip Wood @ 2025-02-25 14:57 UTC (permalink / raw)
To: Karthik Nayak, git; +Cc: ps, jltobler, phillip.wood123
Hi Karthik
On 25/02/2025 09:29, Karthik Nayak wrote:
> Git's reference transactions are all-or-nothing: either all updates
> succeed, or none do. While this atomic behavior is generally desirable,
> it can be suboptimal especially when using the reftable backend, where
> batching multiple reference updates into a single transaction is more
> efficient than performing them sequentially.
>
> Introduce partial transaction support with a new flag,
> 'REF_TRANSACTION_ALLOW_PARTIAL'. When enabled, this flag allows
> individual reference updates that would typically cause the entire
> transaction to fail due to non-system-related errors to be marked as
> rejected while permitting other updates to proceed. Non-system-related
> errors include issues caused by user-provided input values, whereas
> system-related errors, such as I/O failures or memory issues, continue
> to result in a full transaction failure. This approach enhances
> flexibility while preserving transactional integrity where necessary.
>
> The implementation introduces several key components:
>
> - Add 'rejection_err' field to struct `ref_update` to track failed
> updates with failure reason.
>
> - Modify reference backends (files, packed, reftable) to handle
> partial transactions by using `ref_transaction_set_rejected()`
> instead of failing the entire transaction when
> `REF_TRANSACTION_ALLOW_PARTIAL` is set.
>
> - Add `ref_transaction_for_each_rejected_update()` to let callers
> examine which updates were rejected and why.
I think this is a much better design. I wonder if we want to signal to
the caller of ref_transaction_commit() that there were ignored errors
rather than forcing them to call ref_transaction_for_each_rejected() to
find that out. Another possibility would be to call the callback from
ref_transaction_commit() but that would mean changing the signature of
ref_transaction_begin() to take the callback and user data when
REF_TRANSACTION_ALLOW_PARTIAL is passed.
Best Wishes
Phillip
> This foundational change enables partial transaction support throughout
> the reference subsystem. The next commit will expose this capability to
> users by adding a `--allow-partial` flag to 'git-update-ref(1)',
> providing both a user-facing feature and a testable implementation.
>
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
> ---
> refs.c | 31 +++++++++++++++++++++++++++++++
> refs.h | 22 ++++++++++++++++++++++
> refs/files-backend.c | 12 +++++++++++-
> refs/packed-backend.c | 30 ++++++++++++++++++++++++++++--
> refs/refs-internal.h | 13 +++++++++++++
> refs/reftable-backend.c | 12 +++++++++++-
> 6 files changed, 116 insertions(+), 4 deletions(-)
>
> diff --git a/refs.c b/refs.c
> index f989a46a5a..243c09c368 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1211,6 +1211,15 @@ void ref_transaction_free(struct ref_transaction *transaction)
> free(transaction);
> }
>
> +void ref_transaction_set_rejected(struct ref_transaction *transaction,
> + size_t update_idx,
> + enum transaction_error err)
> +{
> + if (update_idx >= transaction->nr)
> + BUG("trying to set rejection on invalid update index");
> + transaction->updates[update_idx]->rejection_err = err;
> +}
> +
> struct ref_update *ref_transaction_add_update(
> struct ref_transaction *transaction,
> const char *refname, unsigned int flags,
> @@ -1236,6 +1245,7 @@ struct ref_update *ref_transaction_add_update(
> transaction->updates[transaction->nr++] = update;
>
> update->flags = flags;
> + update->rejection_err = TRANSACTION_OK;
>
> update->new_target = xstrdup_or_null(new_target);
> update->old_target = xstrdup_or_null(old_target);
> @@ -2726,6 +2736,27 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
> }
> }
>
> +void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
> + ref_transaction_for_each_rejected_update_fn cb,
> + void *cb_data)
> +{
> + if (!(transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL))
> + return;
> +
> + for (size_t i = 0; i < transaction->nr; i++) {
> + struct ref_update *update = transaction->updates[i];
> +
> + if (!update->rejection_err)
> + continue;
> +
> + cb(update->refname,
> + (update->flags & REF_HAVE_OLD) ? &update->old_oid : NULL,
> + (update->flags & REF_HAVE_NEW) ? &update->new_oid : NULL,
> + update->old_target, update->new_target,
> + update->rejection_err, cb_data);
> + }
> +}
> +
> int refs_delete_refs(struct ref_store *refs, const char *logmsg,
> struct string_list *refnames, unsigned int flags)
> {
> diff --git a/refs.h b/refs.h
> index 8e9ead174c..e4a6a8218f 100644
> --- a/refs.h
> +++ b/refs.h
> @@ -675,6 +675,13 @@ enum ref_transaction_flag {
> * either be absent or null_oid.
> */
> REF_TRANSACTION_FLAG_INITIAL = (1 << 0),
> +
> + /*
> + * The transaction mechanism by default fails all updates if any conflict
> + * is detected. This flag allows transactions to partially apply updates
> + * while rejecting updates which do not match the expected state.
> + */
> + REF_TRANSACTION_ALLOW_PARTIAL = (1 << 1),
> };
>
> /*
> @@ -905,6 +912,21 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
> ref_transaction_for_each_queued_update_fn cb,
> void *cb_data);
>
> +/*
> + * Execute the given callback function for each of the reference updates which
> + * have been rejected in the given transaction.
> + */
> +typedef void ref_transaction_for_each_rejected_update_fn(const char *refname,
> + const struct object_id *old_oid,
> + const struct object_id *new_oid,
> + const char *old_target,
> + const char *new_target,
> + enum transaction_error err,
> + void *cb_data);
> +void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
> + ref_transaction_for_each_rejected_update_fn cb,
> + void *cb_data);
> +
> /*
> * Free `*transaction` and all associated data.
> */
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index 3b0adf8bb2..d0a53c9ace 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -2851,8 +2851,18 @@ static int files_transaction_prepare(struct ref_store *ref_store,
> ret = lock_ref_for_update(refs, update, transaction,
> head_ref, &refnames_to_check,
> err);
> - if (ret)
> + if (ret) {
> + if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL &&
> + ret != TRANSACTION_GENERIC_ERROR) {
> + ref_transaction_set_rejected(transaction, i, ret);
> +
> + strbuf_setlen(err, 0);
> + ret = TRANSACTION_OK;
> +
> + continue;
> + }
> goto cleanup;
> + }
>
> if (update->flags & REF_DELETING &&
> !(update->flags & REF_LOG_ONLY) &&
> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
> index 75e1ebf67d..0857204213 100644
> --- a/refs/packed-backend.c
> +++ b/refs/packed-backend.c
> @@ -1324,10 +1324,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
> * remain locked when it is done.
> */
> static enum transaction_error write_with_updates(struct packed_ref_store *refs,
> - struct string_list *updates,
> + struct ref_transaction *transaction,
> struct strbuf *err)
> {
> enum transaction_error ret = TRANSACTION_GENERIC_ERROR;
> + struct string_list *updates = &transaction->refnames;
> struct ref_iterator *iter = NULL;
> size_t i;
> int ok;
> @@ -1408,6 +1409,14 @@ static enum transaction_error write_with_updates(struct packed_ref_store *refs,
> "reference already exists",
> update->refname);
> ret = TRANSACTION_CREATE_EXISTS;
> +
> + if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
> + ref_transaction_set_rejected(transaction, i, ret);
> + strbuf_setlen(err, 0);
> + ret = 0;
> + continue;
> + }
> +
> goto error;
> } else if (!oideq(&update->old_oid, iter->oid)) {
> strbuf_addf(err, "cannot update ref '%s': "
> @@ -1416,6 +1425,14 @@ static enum transaction_error write_with_updates(struct packed_ref_store *refs,
> oid_to_hex(iter->oid),
> oid_to_hex(&update->old_oid));
> ret = TRANSACTION_INCORRECT_OLD_VALUE;
> +
> + if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
> + ref_transaction_set_rejected(transaction, i, ret);
> + strbuf_setlen(err, 0);
> + ret = 0;
> + continue;
> + }
> +
> goto error;
> }
> }
> @@ -1453,6 +1470,14 @@ static enum transaction_error write_with_updates(struct packed_ref_store *refs,
> update->refname,
> oid_to_hex(&update->old_oid));
> return TRANSACTION_NONEXISTENT_REF;
> +
> + if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
> + ref_transaction_set_rejected(transaction, i, ret);
> + strbuf_setlen(err, 0);
> + ret = 0;
> + continue;
> + }
> +
> goto error;
> }
> }
> @@ -1518,6 +1543,7 @@ static enum transaction_error write_with_updates(struct packed_ref_store *refs,
> write_error:
> strbuf_addf(err, "error writing to %s: %s",
> get_tempfile_path(refs->tempfile), strerror(errno));
> + ret = TRANSACTION_GENERIC_ERROR;
>
> error:
> ref_iterator_free(iter);
> @@ -1676,7 +1702,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
> data->own_lock = 1;
> }
>
> - ret = write_with_updates(refs, &transaction->refnames, err);
> + ret = write_with_updates(refs, transaction, err);
> if (ret)
> goto failure;
>
> diff --git a/refs/refs-internal.h b/refs/refs-internal.h
> index c97045fbed..7196f2d880 100644
> --- a/refs/refs-internal.h
> +++ b/refs/refs-internal.h
> @@ -123,6 +123,11 @@ struct ref_update {
> */
> uint64_t index;
>
> + /*
> + * Used in partial transactions to mark if a given update was rejected.
> + */
> + enum transaction_error rejection_err;
> +
> /*
> * If this ref_update was split off of a symref update via
> * split_symref_update(), then this member points at that
> @@ -142,6 +147,14 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
> struct object_id *oid, struct strbuf *referent,
> unsigned int *type, int *failure_errno);
>
> +/*
> + * Mark a given update as rejected with a given reason. To be used in conjuction
> + * with the `REF_TRANSACTION_ALLOW_PARTIAL` flag to allow partial transactions.
> + */
> +void ref_transaction_set_rejected(struct ref_transaction *transaction,
> + size_t update_idx,
> + enum transaction_error err);
> +
> /*
> * Add a ref_update with the specified properties to transaction, and
> * return a pointer to the new object. This function does not verify
> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
> index e1fd9c2de2..83cf8d582b 100644
> --- a/refs/reftable-backend.c
> +++ b/refs/reftable-backend.c
> @@ -1374,8 +1374,18 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
> transaction->updates[i],
> &refnames_to_check, head_type,
> &head_referent, &referent, err);
> - if (ret)
> + if (ret) {
> + if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL &&
> + ret != TRANSACTION_GENERIC_ERROR) {
> + ref_transaction_set_rejected(transaction, i, ret);
> +
> + strbuf_setlen(err, 0);
> + ret = TRANSACTION_OK;
> +
> + continue;
> + }
> goto done;
> + }
> }
>
> string_list_sort(&refnames_to_check);
>
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v2 7/7] update-ref: add --allow-partial flag for stdin mode
2025-02-25 9:29 ` [PATCH v2 7/7] update-ref: add --allow-partial flag for stdin mode Karthik Nayak
2025-02-25 11:08 ` Patrick Steinhardt
@ 2025-02-25 14:59 ` Phillip Wood
2025-03-03 20:34 ` Karthik Nayak
1 sibling, 1 reply; 143+ messages in thread
From: Phillip Wood @ 2025-02-25 14:59 UTC (permalink / raw)
To: Karthik Nayak, git; +Cc: ps, jltobler, phillip.wood123
Hi Karthik
On 25/02/2025 09:29, Karthik Nayak wrote:
> When updating multiple references through stdin, Git's update-ref
> command normally aborts the entire transaction if any single update
> fails. While this atomic behavior prevents partial updates by default,
> there are cases where applying successful updates while reporting
> failures is desirable.
>
> Add a new `--allow-partial` flag that allows the transaction to continue
> even when individual reference updates fail. This flag can only be used
> in `--stdin` mode and builds upon the partial transaction support added
> to the refs subsystem.
As '--stdin' allows a single instance of "git update-ref" to create more
than one transaction perhaps we should instead allow the caller to
specify which transactions they want to allow to fail by passing an
argument to "start", similar to how we support "no-deref" with "update"
> following format:
>
> rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
>
> or with `-z`:
>
> rejected NUL (<old-oid> | <old-target>) NUL (<new-oid> | <new-target>) NUL <rejection-reason> NUL
What's the reason for the different output with '-z'? In the list of
options '-z' is documented as only applying to the input stream. Looking
at the code the existing messages generated by report_ok() are all
printed to stdout with a LF terminator.
> +static void print_rejected_refs(const char *refname,
> + const struct object_id *old_oid,
> + const struct object_id *new_oid,
> + const char *old_target,
> + const char *new_target,
> + enum transaction_error err,
> + void *cb_data UNUSED)
> +{
> + struct strbuf sb = STRBUF_INIT;
> + char space = ' ';
> + const char *reason = "";
> +
> + switch (err) {
> + case TRANSACTION_NAME_CONFLICT:
> + reason = _("refname conflict");
> + break;
> + case TRANSACTION_CREATE_EXISTS:
> + reason = _("reference already exists");
> + break;
> + case TRANSACTION_NONEXISTENT_REF:
> + reason = _("reference does not exist");
> + break;
> + case TRANSACTION_INCORRECT_OLD_VALUE:
> + reason = _("incorrect old value provided");
> + break;
> + case TRANSACTION_INVALID_NEW_VALUE:
> + reason = _("invalid new value provided");
> + break;
> + case TRANSACTION_EXPECTED_SYMREF:
> + reason = _("expected symref but found regular ref");
> + break;
> + default:
> + reason = _("unkown failure");
> + }
I agree with Patrick that these messages should not be translated.
> + if (!line_termination)
> + space = line_termination;
> +
> + strbuf_addf(&sb, "rejected%c%s%c%s%c%c%s%c%s%c", space,
> + refname, space, new_oid ? oid_to_hex(new_oid) : new_target,
> + space, space, old_oid ? oid_to_hex(old_oid) : old_target,
> + space, reason, line_termination);
> +
> + fwrite(sb.buf, sb.len, 1, stdout);
> + strbuf_release(&sb);
> + fflush(stdout);
There is no need to flush after each line, we'll flush all the error
messages when we call report_ok() in parse_cmd_commit() or when the
program exits. The caller has no way to know how many error messages
there are to read so flushing each one individually does not help the
reader avoid deadlocks.
> +}
> +
> static void parse_cmd_commit(struct ref_transaction *transaction,
> const char *next, const char *end UNUSED)
> {
> @@ -573,6 +622,10 @@ static void parse_cmd_commit(struct ref_transaction *transaction,
> die("commit: extra input: %s", next);
> if (ref_transaction_commit(transaction, &error))
> die("commit: %s", error.buf);
> +
> + ref_transaction_for_each_rejected_update(transaction,
> + print_rejected_refs, NULL);
> +
> report_ok("commit");
This is good, the caller knows to stop reading when they see "commit: ok"
Best Wishes
Phillip
> ref_transaction_free(transaction);
> }
> @@ -609,7 +662,7 @@ static const struct parse_cmd {
> { "commit", parse_cmd_commit, 0, UPDATE_REFS_CLOSED },
> };
>
> -static void update_refs_stdin(void)
> +static void update_refs_stdin(unsigned int flags)
> {
> struct strbuf input = STRBUF_INIT, err = STRBUF_INIT;
> enum update_refs_state state = UPDATE_REFS_OPEN;
> @@ -617,7 +670,7 @@ static void update_refs_stdin(void)
> int i, j;
>
> transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
> - 0, &err);
> + flags, &err);
> if (!transaction)
> die("%s", err.buf);
>
> @@ -685,7 +738,7 @@ static void update_refs_stdin(void)
> */
> state = cmd->state;
> transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
> - 0, &err);
> + flags, &err);
> if (!transaction)
> die("%s", err.buf);
>
> @@ -701,6 +754,8 @@ static void update_refs_stdin(void)
> /* Commit by default if no transaction was requested. */
> if (ref_transaction_commit(transaction, &err))
> die("%s", err.buf);
> + ref_transaction_for_each_rejected_update(transaction,
> + print_rejected_refs, NULL);
> ref_transaction_free(transaction);
> break;
> case UPDATE_REFS_STARTED:
> @@ -726,7 +781,9 @@ int cmd_update_ref(int argc,
> const char *refname, *oldval;
> struct object_id oid, oldoid;
> int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;
> - int create_reflog = 0;
> + int create_reflog = 0, allow_partial = 0;
> + unsigned int flags = 0;
> +
> struct option options[] = {
> OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
> OPT_BOOL('d', NULL, &delete, N_("delete the reference")),
> @@ -735,6 +792,8 @@ int cmd_update_ref(int argc,
> OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
> OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
> OPT_BOOL( 0 , "create-reflog", &create_reflog, N_("create a reflog")),
> + OPT_BIT('0', "allow-partial", &flags, N_("allow partial transactions"),
> + REF_TRANSACTION_ALLOW_PARTIAL),
> OPT_END(),
> };
>
> @@ -756,9 +815,10 @@ int cmd_update_ref(int argc,
> usage_with_options(git_update_ref_usage, options);
> if (end_null)
> line_termination = '\0';
> - update_refs_stdin();
> + update_refs_stdin(flags);
> return 0;
> - }
> + } else if (allow_partial)
> + die("--allow-partial can only be used with --stdin");
>
> if (end_null)
> usage_with_options(git_update_ref_usage, options);
> diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
> index 29045aad43..fb9442982e 100755
> --- a/t/t1400-update-ref.sh
> +++ b/t/t1400-update-ref.sh
> @@ -2066,6 +2066,222 @@ do
> grep "$(git rev-parse $a) $(git rev-parse $a)" actual
> '
>
> + test_expect_success "stdin $type allow-partial" '
> + git init repo &&
> + test_when_finished "rm -fr repo" &&
> + (
> + cd repo &&
> + test_commit commit &&
> + head=$(git rev-parse HEAD) &&
> +
> + format_command $type "update refs/heads/ref1" "$head" "$Z" >stdin &&
> + format_command $type "update refs/heads/ref2" "$head" "$Z" >>stdin &&
> + git update-ref $type --stdin --allow-partial <stdin &&
> + echo $head >expect &&
> + git rev-parse refs/heads/ref1 >actual &&
> + test_cmp expect actual &&
> + git rev-parse refs/heads/ref2 >actual &&
> + test_cmp expect actual
> + )
> + '
> +
> + test_expect_success "stdin $type allow-partial with invalid new_oid" '
> + git init repo &&
> + test_when_finished "rm -fr repo" &&
> + (
> + cd repo &&
> + test_commit one &&
> + old_head=$(git rev-parse HEAD) &&
> + test_commit two &&
> + head=$(git rev-parse HEAD) &&
> + git update-ref refs/heads/ref1 $head &&
> + git update-ref refs/heads/ref2 $head &&
> +
> + format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
> + format_command $type "update refs/heads/ref2" "$(test_oid 001)" "$head" >>stdin &&
> + git update-ref $type --stdin --allow-partial <stdin >stdout &&
> + echo $old_head >expect &&
> + git rev-parse refs/heads/ref1 >actual &&
> + test_cmp expect actual &&
> + echo $head >expect &&
> + git rev-parse refs/heads/ref2 >actual &&
> + test_cmp expect actual &&
> + test_grep -q "invalid new value provided" stdout
> + )
> + '
> +
> + test_expect_success "stdin $type allow-partial with non-commit new_oid" '
> + git init repo &&
> + test_when_finished "rm -fr repo" &&
> + (
> + cd repo &&
> + test_commit one &&
> + old_head=$(git rev-parse HEAD) &&
> + test_commit two &&
> + head=$(git rev-parse HEAD) &&
> + head_tree=$(git rev-parse HEAD^{tree}) &&
> + git update-ref refs/heads/ref1 $head &&
> + git update-ref refs/heads/ref2 $head &&
> +
> + format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
> + format_command $type "update refs/heads/ref2" "$head_tree" "$head" >>stdin &&
> + git update-ref $type --stdin --allow-partial <stdin >stdout &&
> + echo $old_head >expect &&
> + git rev-parse refs/heads/ref1 >actual &&
> + test_cmp expect actual &&
> + echo $head >expect &&
> + git rev-parse refs/heads/ref2 >actual &&
> + test_cmp expect actual &&
> + test_grep -q "invalid new value provided" stdout
> + )
> + '
> +
> + test_expect_success "stdin $type allow-partial with non-existent ref" '
> + git init repo &&
> + test_when_finished "rm -fr repo" &&
> + (
> + cd repo &&
> + test_commit one &&
> + old_head=$(git rev-parse HEAD) &&
> + test_commit two &&
> + head=$(git rev-parse HEAD) &&
> + git update-ref refs/heads/ref1 $head &&
> +
> + format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
> + format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
> + git update-ref $type --stdin --allow-partial <stdin >stdout &&
> + echo $old_head >expect &&
> + git rev-parse refs/heads/ref1 >actual &&
> + test_cmp expect actual &&
> + test_must_fail git rev-parse refs/heads/ref2 &&
> + test_grep -q "reference does not exist" stdout
> + )
> + '
> +
> + test_expect_success "stdin $type allow-partial with dangling symref" '
> + git init repo &&
> + test_when_finished "rm -fr repo" &&
> + (
> + cd repo &&
> + test_commit one &&
> + old_head=$(git rev-parse HEAD) &&
> + test_commit two &&
> + head=$(git rev-parse HEAD) &&
> + git update-ref refs/heads/ref1 $head &&
> + git symbolic-ref refs/heads/ref2 refs/heads/nonexistent &&
> +
> + format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
> + format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
> + git update-ref $type --no-deref --stdin --allow-partial <stdin >stdout &&
> + echo $old_head >expect &&
> + git rev-parse refs/heads/ref1 >actual &&
> + test_cmp expect actual &&
> + echo $head >expect &&
> + test_must_fail git rev-parse refs/heads/ref2 &&
> + test_grep -q "reference does not exist" stdout
> + )
> + '
> +
> + test_expect_success "stdin $type allow-partial with regular ref as symref" '
> + git init repo &&
> + test_when_finished "rm -fr repo" &&
> + (
> + cd repo &&
> + test_commit one &&
> + old_head=$(git rev-parse HEAD) &&
> + test_commit two &&
> + head=$(git rev-parse HEAD) &&
> + git update-ref refs/heads/ref1 $head &&
> + git update-ref refs/heads/ref2 $head &&
> +
> + format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
> + format_command $type "symref-update refs/heads/ref2" "$old_head" "ref" "refs/heads/nonexistent" >>stdin &&
> + git update-ref $type --no-deref --stdin --allow-partial <stdin >stdout &&
> + echo $old_head >expect &&
> + git rev-parse refs/heads/ref1 >actual &&
> + test_cmp expect actual &&
> + echo $head >expect &&
> + echo $head >expect &&
> + git rev-parse refs/heads/ref2 >actual &&
> + test_cmp expect actual &&
> + test_grep -q "expected symref but found regular ref" stdout
> + )
> + '
> +
> + test_expect_success "stdin $type allow-partial with invalid old_oid" '
> + git init repo &&
> + test_when_finished "rm -fr repo" &&
> + (
> + cd repo &&
> + test_commit one &&
> + old_head=$(git rev-parse HEAD) &&
> + test_commit two &&
> + head=$(git rev-parse HEAD) &&
> + git update-ref refs/heads/ref1 $head &&
> + git update-ref refs/heads/ref2 $head &&
> +
> + format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
> + format_command $type "update refs/heads/ref2" "$old_head" "$Z" >>stdin &&
> + git update-ref $type --stdin --allow-partial <stdin >stdout &&
> + echo $old_head >expect &&
> + git rev-parse refs/heads/ref1 >actual &&
> + test_cmp expect actual &&
> + echo $head >expect &&
> + git rev-parse refs/heads/ref2 >actual &&
> + test_cmp expect actual &&
> + test_grep -q "reference already exists" stdout
> + )
> + '
> +
> + test_expect_success "stdin $type allow-partial with incorrect old oid" '
> + git init repo &&
> + test_when_finished "rm -fr repo" &&
> + (
> + cd repo &&
> + test_commit one &&
> + old_head=$(git rev-parse HEAD) &&
> + test_commit two &&
> + head=$(git rev-parse HEAD) &&
> + git update-ref refs/heads/ref1 $head &&
> + git update-ref refs/heads/ref2 $head &&
> +
> + format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
> + format_command $type "update refs/heads/ref2" "$head" "$old_head" >>stdin &&
> + git update-ref $type --stdin --allow-partial <stdin >stdout &&
> + echo $old_head >expect &&
> + git rev-parse refs/heads/ref1 >actual &&
> + test_cmp expect actual &&
> + echo $head >expect &&
> + git rev-parse refs/heads/ref2 >actual &&
> + test_cmp expect actual &&
> + test_grep -q "incorrect old value provided" stdout
> + )
> + '
> +
> + # F/D conflicts on the files backend are resolved on an individual
> + # update level since refs are stored as files. On the reftable backend
> + # this check is batched to optimize for performance, so failures cannot
> + # be isolated to a single update.
> + test_expect_success REFFILES "stdin $type allow-partial refname conflict" '
> + git init repo &&
> + test_when_finished "rm -fr repo" &&
> + (
> + cd repo &&
> + test_commit one &&
> + old_head=$(git rev-parse HEAD) &&
> + test_commit two &&
> + head=$(git rev-parse HEAD) &&
> + git update-ref refs/heads/ref/foo $head &&
> +
> + format_command $type "update refs/heads/ref/foo" "$old_head" "$head" >stdin &&
> + format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
> + git update-ref $type --stdin --allow-partial <stdin >stdout &&
> + echo $old_head >expect &&
> + git rev-parse refs/heads/ref/foo >actual &&
> + test_cmp expect actual &&
> + test_grep -q "refname conflict" stdout
> + )
> + '
> done
>
> test_expect_success 'update-ref should also create reflog for HEAD' '
>
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v2 5/7] refs: introduce enum-based transaction error types
2025-02-25 11:08 ` Patrick Steinhardt
@ 2025-03-03 20:12 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-03 20:12 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 3859 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> On Tue, Feb 25, 2025 at 10:29:08AM +0100, Karthik Nayak wrote:
>> diff --git a/refs.h b/refs.h
>> index b14ba1f9ff..8e9ead174c 100644
>> --- a/refs.h
>> +++ b/refs.h
>> @@ -16,6 +16,31 @@ struct worktree;
>> enum ref_storage_format ref_storage_format_by_name(const char *name);
>> const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);
>>
>> +/*
>> + * enum transaction_error represents the following return codes:
>> + * TRANSACTION_OK: success code.
>> + * TRANSACTION_GENERIC_ERROR error_code: default error code.
>> + * TRANSACTION_NAME_CONFLICT error_code: ref name conflict like A vs A/B.
>> + * TRANSACTION_CREATE_EXISTS error_code: ref to be created already exists.
>> + * TRANSACTION_NONEXISTENT_REF error_code: ref expected but doesn't exist.
>> + * TRANSACTION_INCORRECT_OLD_VALUE error_code: provided old_oid or old_target of
>> + * reference doesn't match actual.
>> + * TRANSACTION_INVALID_NEW_VALUE error_code: provided new_oid or new_target is
>> + * invalid.
>> + * TRANSACTION_EXPECTED_SYMREF error_code: expected ref to be symref, but is a
>> + * regular ref.
>> + */
>> +enum transaction_error {
>> + TRANSACTION_OK = 0,
>> + TRANSACTION_GENERIC_ERROR = -1,
>> + TRANSACTION_NAME_CONFLICT = -2,
>> + TRANSACTION_CREATE_EXISTS = -3,
>> + TRANSACTION_NONEXISTENT_REF = -4,
>> + TRANSACTION_INCORRECT_OLD_VALUE = -5,
>> + TRANSACTION_INVALID_NEW_VALUE = -6,
>> + TRANSACTION_EXPECTED_SYMREF = -7,
>> +};
>
> Nit: how about we name this `ref_transaction_error` and adapt the the
> enum values accordingly? We may eventually also introduce similar errors
> for the object database, so it may make sense to have the errors be
> specific. Doing both the enum and changing the name might be a bit hard
> to review, so we could also rename in a preparatory commit. Or we just
> punt on it for now and do it once it becomes necessary, that would also
> be fine with me.
>
I'm happy to rename it, should be easier now. We can always change later
if needed.
> I also wonder whether we really want to introduce `TRANSACTION_OK`. It's
> always a bit of a mouthful, and in many cases one ends up with a mixture
> of `ret < 0`, `ret != TRANSACTION_OK` and `ret != 0`, which may lead to
> confusion. Continuing to use `0` for the successful case should be fine.
>
Fair enough, let me remove it!
>> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
>> index 3247871574..75e1ebf67d 100644
>> --- a/refs/packed-backend.c
>> +++ b/refs/packed-backend.c
>> @@ -1672,7 +1676,8 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
>> data->own_lock = 1;
>> }
>>
>> - if (write_with_updates(refs, &transaction->refnames, err))
>> + ret = write_with_updates(refs, &transaction->refnames, err);
>> + if (ret)
>> goto failure;
>>
>> transaction->state = REF_TRANSACTION_PREPARED;
>
> Do we also want to change the local variable declaration of `int ret` to
> use the new type?
>
Yes! Good catch.
>> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
>> index 2c1e2995de..e1fd9c2de2 100644
>> --- a/refs/reftable-backend.c
>> +++ b/refs/reftable-backend.c
>> @@ -1255,11 +1255,12 @@ static int prepare_single_update(struct reftable_ref_store *refs,
>> "but is a regular ref"),
>> ref_update_original_update_refname(u),
>> u->old_target);
>> - return -1;
>> + return TRANSACTION_EXPECTED_SYMREF;
>> }
>>
>> - if (ref_update_check_old_target(referent->buf, u, err)) {
>> - return -1;
>> + ret = ref_update_check_old_target(referent->buf, u, err);
>> + if (ret) {
>> + return ret;
>> }
>> } else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
>> if (is_null_oid(&u->old_oid)) {
>
> Nit: superfluous braces that we could remove while at it.
>
Indeed, will fix.
> Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v2 6/7] refs: implement partial reference transaction support
2025-02-25 11:07 ` Patrick Steinhardt
@ 2025-03-03 20:17 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-03 20:17 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 2374 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> On Tue, Feb 25, 2025 at 10:29:09AM +0100, Karthik Nayak wrote:
>> diff --git a/refs.c b/refs.c
>> index f989a46a5a..243c09c368 100644
>> --- a/refs.c
>> +++ b/refs.c
>> @@ -2726,6 +2736,27 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
>> }
>> }
>>
>> +void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
>> + ref_transaction_for_each_rejected_update_fn cb,
>> + void *cb_data)
>> +{
>> + if (!(transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL))
>> + return;
>> +
>> + for (size_t i = 0; i < transaction->nr; i++) {
>> + struct ref_update *update = transaction->updates[i];
>> +
>> + if (!update->rejection_err)
>> + continue;
>
> This kind of proves my point that `TRANSACTION_OK` is pointless and
> leads to a mixture of using and not using the enum :)
>
>> diff --git a/refs/files-backend.c b/refs/files-backend.c
>> index 3b0adf8bb2..d0a53c9ace 100644
>> --- a/refs/files-backend.c
>> +++ b/refs/files-backend.c
>> @@ -2851,8 +2851,18 @@ static int files_transaction_prepare(struct ref_store *ref_store,
>> ret = lock_ref_for_update(refs, update, transaction,
>> head_ref, &refnames_to_check,
>> err);
>> - if (ret)
>> + if (ret) {
>> + if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL &&
>
> Hm. If the error values were defined as a bitfield we could refactor
> this to not be a flag, but have a `transaction->accepted_rejections`
> instead that allows the caller to ask for only a subset of rejections to
> be accepted.
>
I did consider making them bitfield and even tried playing with that
idea. There are a bunch of side affects of doing that in other
subsystems which check call use the reference API and expect '< 0'
errors.
> I'm not quite sure whether it is a good idea, but the logic to handle
> all of that could be self-contained in `ref_transaction_set_rejected()`:
> if it returned an error code itself, it would swallow any errors in case
> the transaction allows a given error, and bubble up the error again in
> case the error is not allowed. The function could use a rename in that
> case though, e.g. `ref_transaction_maybe_set_rejected()`.
>
> Patrick
I think this is great idea, it does cleanup a bunch of the code. I will
add this in the next version with the rename. Thanks!
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v2 6/7] refs: implement partial reference transaction support
2025-02-25 14:57 ` Phillip Wood
@ 2025-03-03 20:21 ` Karthik Nayak
2025-03-04 10:31 ` Phillip Wood
0 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-03-03 20:21 UTC (permalink / raw)
To: phillip.wood, git; +Cc: ps, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 2613 bytes --]
Phillip Wood <phillip.wood123@gmail.com> writes:
Hello Phillip!
> Hi Karthik
>
> On 25/02/2025 09:29, Karthik Nayak wrote:
>> Git's reference transactions are all-or-nothing: either all updates
>> succeed, or none do. While this atomic behavior is generally desirable,
>> it can be suboptimal especially when using the reftable backend, where
>> batching multiple reference updates into a single transaction is more
>> efficient than performing them sequentially.
>>
>> Introduce partial transaction support with a new flag,
>> 'REF_TRANSACTION_ALLOW_PARTIAL'. When enabled, this flag allows
>> individual reference updates that would typically cause the entire
>> transaction to fail due to non-system-related errors to be marked as
>> rejected while permitting other updates to proceed. Non-system-related
>> errors include issues caused by user-provided input values, whereas
>> system-related errors, such as I/O failures or memory issues, continue
>> to result in a full transaction failure. This approach enhances
>> flexibility while preserving transactional integrity where necessary.
>>
>> The implementation introduces several key components:
>>
>> - Add 'rejection_err' field to struct `ref_update` to track failed
>> updates with failure reason.
>>
>> - Modify reference backends (files, packed, reftable) to handle
>> partial transactions by using `ref_transaction_set_rejected()`
>> instead of failing the entire transaction when
>> `REF_TRANSACTION_ALLOW_PARTIAL` is set.
>>
>> - Add `ref_transaction_for_each_rejected_update()` to let callers
>> examine which updates were rejected and why.
>
> I think this is a much better design. I wonder if we want to signal to
> the caller of ref_transaction_commit() that there were ignored errors
> rather than forcing them to call ref_transaction_for_each_rejected() to
> find that out. Another possibility would be to call the callback from
> ref_transaction_commit() but that would mean changing the signature of
> ref_transaction_begin() to take the callback and user data when
> REF_TRANSACTION_ALLOW_PARTIAL is passed.
>
Yes, I did toy around modifying `ref_transaction_*` at first, but I
think the current implementation is slightly better. Users of the ref
API do not have to worry about complexity of partial transactions unless
they really need to. So in that case, for most users, the API remains
simple and clean, and for specific users who do want partial transaction
support, they can activate it via the flag and use the iterator to
collect the rejections at the end.
> Best Wishes
>
> Phillip
>
[snip]
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v2 7/7] update-ref: add --allow-partial flag for stdin mode
2025-02-25 11:08 ` Patrick Steinhardt
@ 2025-03-03 20:22 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-03 20:22 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 3686 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> On Tue, Feb 25, 2025 at 10:29:10AM +0100, Karthik Nayak wrote:
>> diff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc
>> index 9e6935d38d..fc73f1d8aa 100644
>> --- a/Documentation/git-update-ref.adoc
>> +++ b/Documentation/git-update-ref.adoc
>> @@ -7,8 +7,10 @@ git-update-ref - Update the object name stored in a ref safely
>>
>> SYNOPSIS
>> --------
>> -[verse]
>> -'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z])
>> +[synopsis]
>> +git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
>> + [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
>> + [-m <reason>] [--no-deref] --stdin [-z] [--allow-partial]
>>
>> DESCRIPTION
>> -----------
>> @@ -57,6 +59,17 @@ performs all modifications together. Specify commands of the form:
>> With `--create-reflog`, update-ref will create a reflog for each ref
>> even if one would not ordinarily be created.
>>
>> +With `--allow-partial`, update-ref continues executing the transaction even if
>> +some updates fail due to invalid or incorrect user input, applying only the
>> +successful updates. Errors resulting from user-provided input are treated as
>> +non-system-related and do not cause the entire transaction to be aborted.
>> +However, system-related errors—such as I/O failures or memory issues—will still
>> +result in a full failure. Additionally, errors like F/D conflicts are batched
>> +for performance optimization and will also cause a full failure. Any failed
>> +updates will be reported in the following format:
>
> Shouldn't it be possible to detect F/D conflicts though and not abort
> the transaction? If we want to make use of partial transactions in the
> context of git-fetch(1) and/or git-receive-pack(1) we would have to
> handle them.
>
Yes, this was a miss in this version, mostly from me not thinking enough
about this series interacts with yours. This should be fixed in the next
version.
>> diff --git a/builtin/update-ref.c b/builtin/update-ref.c
>> index 1d541e13ad..b03b40eacb 100644
>> --- a/builtin/update-ref.c
>> +++ b/builtin/update-ref.c
>> @@ -565,6 +566,54 @@ static void parse_cmd_abort(struct ref_transaction *transaction,
>> report_ok("abort");
>> }
>>
>> +static void print_rejected_refs(const char *refname,
>> + const struct object_id *old_oid,
>> + const struct object_id *new_oid,
>> + const char *old_target,
>> + const char *new_target,
>> + enum transaction_error err,
>> + void *cb_data UNUSED)
>> +{
>> + struct strbuf sb = STRBUF_INIT;
>> + char space = ' ';
>> + const char *reason = "";
>> +
>> + switch (err) {
>> + case TRANSACTION_NAME_CONFLICT:
>> + reason = _("refname conflict");
>> + break;
>> + case TRANSACTION_CREATE_EXISTS:
>> + reason = _("reference already exists");
>> + break;
>> + case TRANSACTION_NONEXISTENT_REF:
>> + reason = _("reference does not exist");
>> + break;
>> + case TRANSACTION_INCORRECT_OLD_VALUE:
>> + reason = _("incorrect old value provided");
>> + break;
>> + case TRANSACTION_INVALID_NEW_VALUE:
>> + reason = _("invalid new value provided");
>> + break;
>> + case TRANSACTION_EXPECTED_SYMREF:
>> + reason = _("expected symref but found regular ref");
>> + break;
>> + default:
>> + reason = _("unkown failure");
>> + }
>
> As git-update-ref(1) is part of plumbing we don't want to translate
> those messages.
>
Makes sense, let me remove that.
> Patrick
>
Thanks for the review on the series!
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v2 7/7] update-ref: add --allow-partial flag for stdin mode
2025-02-25 14:59 ` Phillip Wood
@ 2025-03-03 20:34 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-03 20:34 UTC (permalink / raw)
To: phillip.wood, git; +Cc: ps, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 4628 bytes --]
Phillip Wood <phillip.wood123@gmail.com> writes:
> Hi Karthik
>
> On 25/02/2025 09:29, Karthik Nayak wrote:
>> When updating multiple references through stdin, Git's update-ref
>> command normally aborts the entire transaction if any single update
>> fails. While this atomic behavior prevents partial updates by default,
>> there are cases where applying successful updates while reporting
>> failures is desirable.
>>
>> Add a new `--allow-partial` flag that allows the transaction to continue
>> even when individual reference updates fail. This flag can only be used
>> in `--stdin` mode and builds upon the partial transaction support added
>> to the refs subsystem.
>
> As '--stdin' allows a single instance of "git update-ref" to create more
> than one transaction perhaps we should instead allow the caller to
> specify which transactions they want to allow to fail by passing an
> argument to "start", similar to how we support "no-deref" with "update"
>
I was considering adding a 'allow-rejection' flag similar to "no-deref"
with "update", but decided against it since that was an update level
config and we want a transaction level configuration.
But I like your idea better. I'm not bullish on either the current
implementation or your suggestion, since we can extend one from the
other. My main motivation to add this flag to update-ref(1) is to show
viability. The goal would be to follow up with changes to `git-fetch(1)`
and `git-receive-pack(1)` to use partial transactions.
>> following format:
>>
>> rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
>>
>> or with `-z`:
>>
>> rejected NUL (<old-oid> | <old-target>) NUL (<new-oid> | <new-target>) NUL <rejection-reason> NUL
>
> What's the reason for the different output with '-z'? In the list of
> options '-z' is documented as only applying to the input stream. Looking
> at the code the existing messages generated by report_ok() are all
> printed to stdout with a LF terminator.
>
That's a great point, there was no real reason and I think we can drop
this.
>> +static void print_rejected_refs(const char *refname,
>> + const struct object_id *old_oid,
>> + const struct object_id *new_oid,
>> + const char *old_target,
>> + const char *new_target,
>> + enum transaction_error err,
>> + void *cb_data UNUSED)
>> +{
>> + struct strbuf sb = STRBUF_INIT;
>> + char space = ' ';
>> + const char *reason = "";
>> +
>> + switch (err) {
>> + case TRANSACTION_NAME_CONFLICT:
>> + reason = _("refname conflict");
>> + break;
>> + case TRANSACTION_CREATE_EXISTS:
>> + reason = _("reference already exists");
>> + break;
>> + case TRANSACTION_NONEXISTENT_REF:
>> + reason = _("reference does not exist");
>> + break;
>> + case TRANSACTION_INCORRECT_OLD_VALUE:
>> + reason = _("incorrect old value provided");
>> + break;
>> + case TRANSACTION_INVALID_NEW_VALUE:
>> + reason = _("invalid new value provided");
>> + break;
>> + case TRANSACTION_EXPECTED_SYMREF:
>> + reason = _("expected symref but found regular ref");
>> + break;
>> + default:
>> + reason = _("unkown failure");
>> + }
>
> I agree with Patrick that these messages should not be translated.
>
>> + if (!line_termination)
>> + space = line_termination;
>> +
>> + strbuf_addf(&sb, "rejected%c%s%c%s%c%c%s%c%s%c", space,
>> + refname, space, new_oid ? oid_to_hex(new_oid) : new_target,
>> + space, space, old_oid ? oid_to_hex(old_oid) : old_target,
>> + space, reason, line_termination);
>> +
>> + fwrite(sb.buf, sb.len, 1, stdout);
>> + strbuf_release(&sb);
>> + fflush(stdout);
>
> There is no need to flush after each line, we'll flush all the error
> messages when we call report_ok() in parse_cmd_commit() or when the
> program exits. The caller has no way to know how many error messages
> there are to read so flushing each one individually does not help the
> reader avoid deadlocks.
>
That does make sense, I'll remove the flush here!
>> +}
>> +
>> static void parse_cmd_commit(struct ref_transaction *transaction,
>> const char *next, const char *end UNUSED)
>> {
>> @@ -573,6 +622,10 @@ static void parse_cmd_commit(struct ref_transaction *transaction,
>> die("commit: extra input: %s", next);
>> if (ref_transaction_commit(transaction, &error))
>> die("commit: %s", error.buf);
>> +
>> + ref_transaction_for_each_rejected_update(transaction,
>> + print_rejected_refs, NULL);
>> +
>> report_ok("commit");
>
> This is good, the caller knows to stop reading when they see "commit: ok"
>
>
> Best Wishes
>
> Phillip
Thanks for the review on the series.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v2 6/7] refs: implement partial reference transaction support
2025-03-03 20:21 ` Karthik Nayak
@ 2025-03-04 10:31 ` Phillip Wood
2025-03-05 14:20 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Phillip Wood @ 2025-03-04 10:31 UTC (permalink / raw)
To: Karthik Nayak, phillip.wood, git; +Cc: ps, jltobler
Hi Karthik
On 03/03/2025 20:21, Karthik Nayak wrote:
> Phillip Wood <phillip.wood123@gmail.com> writes:
>> On 25/02/2025 09:29, Karthik Nayak wrote:
>>> Git's reference transactions are all-or-nothing: either all updates
>>> succeed, or none do. While this atomic behavior is generally desirable,
>>> it can be suboptimal especially when using the reftable backend, where
>>> batching multiple reference updates into a single transaction is more
>>> efficient than performing them sequentially.
>>>
>>> Introduce partial transaction support with a new flag,
>>> 'REF_TRANSACTION_ALLOW_PARTIAL'. When enabled, this flag allows
>>> individual reference updates that would typically cause the entire
>>> transaction to fail due to non-system-related errors to be marked as
>>> rejected while permitting other updates to proceed. Non-system-related
>>> errors include issues caused by user-provided input values, whereas
>>> system-related errors, such as I/O failures or memory issues, continue
>>> to result in a full transaction failure. This approach enhances
>>> flexibility while preserving transactional integrity where necessary.
>>>
>>> The implementation introduces several key components:
>>>
>>> - Add 'rejection_err' field to struct `ref_update` to track failed
>>> updates with failure reason.
>>>
>>> - Modify reference backends (files, packed, reftable) to handle
>>> partial transactions by using `ref_transaction_set_rejected()`
>>> instead of failing the entire transaction when
>>> `REF_TRANSACTION_ALLOW_PARTIAL` is set.
>>>
>>> - Add `ref_transaction_for_each_rejected_update()` to let callers
>>> examine which updates were rejected and why.
>>
>> I think this is a much better design. I wonder if we want to signal to
>> the caller of ref_transaction_commit() that there were ignored errors
>> rather than forcing them to call ref_transaction_for_each_rejected() to
>> find that out. Another possibility would be to call the callback from
>> ref_transaction_commit() but that would mean changing the signature of
>> ref_transaction_begin() to take the callback and user data when
>> REF_TRANSACTION_ALLOW_PARTIAL is passed.
>>
>
> Yes, I did toy around modifying `ref_transaction_*` at first, but I
> think the current implementation is slightly better. Users of the ref
> API do not have to worry about complexity of partial transactions unless
> they really need to. So in that case, for most users, the API remains
> simple and clean, and for specific users who do want partial transaction
> support, they can activate it via the flag and use the iterator to
> collect the rejections at the end.
That makes sense. I have a slight concern that iterating through the
errors is O(number of ref updates) rather than O(number of errors). If
we expect most updates to succeed that it is a shame to have to check
them all just to see there were no errors. Maybe we could be store the
errors in a separate list of (update-index, error) pairs to avoid that.
Best Wishes
Phillip
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v2 6/7] refs: implement partial reference transaction support
2025-03-04 10:31 ` Phillip Wood
@ 2025-03-05 14:20 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-05 14:20 UTC (permalink / raw)
To: Phillip Wood, phillip.wood, git; +Cc: ps, jltobler
[-- Attachment #1: Type: text/plain, Size: 3272 bytes --]
Phillip Wood <phillip.wood123@gmail.com> writes:
> Hi Karthik
>
> On 03/03/2025 20:21, Karthik Nayak wrote:
>> Phillip Wood <phillip.wood123@gmail.com> writes:
>>> On 25/02/2025 09:29, Karthik Nayak wrote:
>>>> Git's reference transactions are all-or-nothing: either all updates
>>>> succeed, or none do. While this atomic behavior is generally desirable,
>>>> it can be suboptimal especially when using the reftable backend, where
>>>> batching multiple reference updates into a single transaction is more
>>>> efficient than performing them sequentially.
>>>>
>>>> Introduce partial transaction support with a new flag,
>>>> 'REF_TRANSACTION_ALLOW_PARTIAL'. When enabled, this flag allows
>>>> individual reference updates that would typically cause the entire
>>>> transaction to fail due to non-system-related errors to be marked as
>>>> rejected while permitting other updates to proceed. Non-system-related
>>>> errors include issues caused by user-provided input values, whereas
>>>> system-related errors, such as I/O failures or memory issues, continue
>>>> to result in a full transaction failure. This approach enhances
>>>> flexibility while preserving transactional integrity where necessary.
>>>>
>>>> The implementation introduces several key components:
>>>>
>>>> - Add 'rejection_err' field to struct `ref_update` to track failed
>>>> updates with failure reason.
>>>>
>>>> - Modify reference backends (files, packed, reftable) to handle
>>>> partial transactions by using `ref_transaction_set_rejected()`
>>>> instead of failing the entire transaction when
>>>> `REF_TRANSACTION_ALLOW_PARTIAL` is set.
>>>>
>>>> - Add `ref_transaction_for_each_rejected_update()` to let callers
>>>> examine which updates were rejected and why.
>>>
>>> I think this is a much better design. I wonder if we want to signal to
>>> the caller of ref_transaction_commit() that there were ignored errors
>>> rather than forcing them to call ref_transaction_for_each_rejected() to
>>> find that out. Another possibility would be to call the callback from
>>> ref_transaction_commit() but that would mean changing the signature of
>>> ref_transaction_begin() to take the callback and user data when
>>> REF_TRANSACTION_ALLOW_PARTIAL is passed.
>>>
>>
>> Yes, I did toy around modifying `ref_transaction_*` at first, but I
>> think the current implementation is slightly better. Users of the ref
>> API do not have to worry about complexity of partial transactions unless
>> they really need to. So in that case, for most users, the API remains
>> simple and clean, and for specific users who do want partial transaction
>> support, they can activate it via the flag and use the iterator to
>> collect the rejections at the end.
>
> That makes sense. I have a slight concern that iterating through the
> errors is O(number of ref updates) rather than O(number of errors). If
> we expect most updates to succeed that it is a shame to have to check
> them all just to see there were no errors. Maybe we could be store the
> errors in a separate list of (update-index, error) pairs to avoid that.
>
That is a good point, let me add in a array inside 'ref_transaction'
which will track rejected updates.
> Best Wishes
>
> Phillip
Thanks!
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH v3 0/8] refs: introduce support for partial reference transactions
2025-02-07 7:34 [PATCH 0/6] refs: introduce support for partial reference transactions Karthik Nayak
` (7 preceding siblings ...)
2025-02-25 9:29 ` [PATCH v2 0/7] " Karthik Nayak
@ 2025-03-05 17:38 ` Karthik Nayak
2025-03-05 17:38 ` [PATCH v3 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
` (8 more replies)
2025-03-20 11:43 ` [PATCH v4 0/8] refs: introduce support for batched reference updates Karthik Nayak
` (2 subsequent siblings)
11 siblings, 9 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-05 17:38 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Documentation/git-update-ref.adoc | 17 +-
builtin/fetch.c | 2 +-
builtin/update-ref.c | 67 ++++-
refs.c | 162 ++++++++++--
refs.h | 76 ++++--
refs/files-backend.c | 314 +++++++++++-------------
refs/packed-backend.c | 69 +++---
refs/refs-internal.h | 51 +++-
refs/reftable-backend.c | 502 +++++++++++++++++++-------------------
t/t1400-update-ref.sh | 233 ++++++++++++++++++
10 files changed, 971 insertions(+), 522 deletions(-)
Karthik Nayak (8):
refs/files: remove redundant check in split_symref_update()
refs: move duplicate refname update check to generic layer
refs/files: remove duplicate duplicates check
refs/reftable: extract code from the transaction preparation
refs: introduce enum-based transaction error types
refs: implement partial reference transaction support
refs: support partial update rejections during F/D checks
update-ref: add --allow-partial flag for stdin mode
Git's reference updates are traditionally all or nothing - when updating
multiple references in a transaction, either all updates succeed or none
do. While this behavior is generally desirable, it can be limiting in
certain scenarios, particularly with the reftable backend where batching
multiple reference updates is more efficient than performing them
sequentially.
This series introduces support for partial reference transactions,
allowing individual reference updates to fail while letting others
proceed. This capability is exposed through git-update-ref's
`--allow-partial` flag, which can be used in `--stdin` mode to batch
updates and handle failures gracefully.
The changes are structured to carefully build up this functionality:
First, we clean up and consolidate the reference update checking logic.
This includes removing duplicate checks in the files backend and moving
refname tracking to the generic layer, which simplifies the codebase and
prepares it for the new feature.
We then restructure the reftable backend's transaction preparation code,
extracting the update validation logic into a dedicated function. This
not only improves code organization but sets the stage for implementing
partial transaction support.
To ensure we only skip errors which are user-oriented, we introduce
typed errors for transactions with 'enum ref_transaction_error'. We
extend the existing errors to include other scenarios and use this new
errors throughout the refs code.
With this groundwork in place, we implement the core partial transaction
support in the refs subsystem. This adds the necessary infrastructure to
track and report rejected updates while allowing transactions to proceed.
All reference backends are modified to support this behavior when enabled.
Finally, we expose this functionality to users through
git-update-ref(1)'s `--allow-partial` flag, complete with test coverage
and documentation. The flag is specifically limited to `--stdin` mode
where batching multiple updates is most relevant.
This enhancement improves Git's flexibility in handling reference
updates while maintaining the safety of atomic transactions by default.
It's particularly valuable for tools and workflows that need to handle
reference update failures gracefully without abandoning the entire batch
of updates.
This series is based on top of b838bf1938 (Merge branch 'master' of
https://github.com/j6t/gitk, 2025-02-20) with Patrick's series 'refs:
batch refname availability checks' [1] merged in.
[1]: https://lore.kernel.org/all/20250217-pks-update-ref-optimization-v1-0-a2b6d87a24af@pks.im/
---
Changes in v3:
- Changed 'transaction_error' to 'ref_transaction_error' along with the
error names. Removed 'TRANSACTION_OK' since it can potentially be
missed instead of simply 'return 0'.
- Rename 'ref_transaction_set_rejected' to
'ref_transaction_maybe_set_rejected' and move logic around error
checks to within this function.
- Add a new struct 'ref_transaction_rejections' to track the rejections
within a transaction. This allows us to only iterate over rejected
updates.
- Add a new commit to also support partial transactions within the
batched F/D checks.
- Remove NUL delimited outputs in 'git-update-ref(1)'.
- Remove translations for plumbing outputs.
- Other small cleanups in the commit message and code.
Changes in v2:
- Introduce and use structured errors. This consolidates the errors
and their handling between the ref backends.
- In the previous version, we skipped over all failures. This include
system failures such as low memory or IO problems. Let's instead, only
skip user-oriented failures, such as invalid old OID and so on.
- Change the rejection function name to `ref_transaction_set_rejected()`.
- Modify the commit messages and documentation to be a little more
verbose.
- Link to v1: https://lore.kernel.org/r/20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com
Range-diff versus v2:
1: a7a5f8c752 = 1: 1bd0878fd7 refs/files: remove redundant check in split_symref_update()
2: 61ebc1e133 = 2: 92181469bf refs: move duplicate refname update check to generic layer
3: f54f3d7722 = 3: 6fb0b6b03d refs/files: remove duplicate duplicates check
4: 463e043cd2 = 4: 07788f97e9 refs/reftable: extract code from the transaction preparation
5: baa94ddfb6 ! 5: 2f872b650f refs: introduce enum-based transaction error types
@@ Commit message
refs: introduce enum-based transaction error types
Replace preprocessor-defined transaction errors with a strongly-typed
- enum `transaction_error`. This change:
+ enum `ref_transaction_error`. This change:
- Improves type safety and function signature clarity.
- Makes error handling more explicit and discoverable.
@@ Commit message
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
+ ## builtin/fetch.c ##
+@@ builtin/fetch.c: static int s_update_ref(const char *action,
+ switch (ref_transaction_commit(our_transaction, &err)) {
+ case 0:
+ break;
+- case TRANSACTION_NAME_CONFLICT:
++ case REF_TRANSACTION_ERROR_NAME_CONFLICT:
+ ret = STORE_REF_ERROR_DF_CONFLICT;
+ goto out;
+ default:
+
## refs.c ##
+@@ refs.c: int refs_update_symref_extended(struct ref_store *refs, const char *ref,
+ REF_NO_DEREF, logmsg, &err))
+ goto error_return;
+ prepret = ref_transaction_prepare(transaction, &err);
+- if (prepret && prepret != TRANSACTION_CREATE_EXISTS)
++ if (prepret && prepret != REF_TRANSACTION_ERROR_CREATE_EXISTS)
+ goto error_return;
+ } else {
+ if (ref_transaction_update(transaction, ref, NULL, NULL,
+@@ refs.c: int refs_update_symref_extended(struct ref_store *refs, const char *ref,
+ }
+ }
+
+- if (prepret == TRANSACTION_CREATE_EXISTS)
++ if (prepret == REF_TRANSACTION_ERROR_CREATE_EXISTS)
+ goto cleanup;
+
+ if (ref_transaction_commit(transaction, &err))
+@@ refs.c: int ref_transaction_prepare(struct ref_transaction *transaction,
+
+ string_list_sort(&transaction->refnames);
+ if (ref_update_reject_duplicates(&transaction->refnames, err))
+- return TRANSACTION_GENERIC_ERROR;
++ return REF_TRANSACTION_ERROR_GENERIC;
+
+ ret = refs->be->transaction_prepare(refs, transaction, err);
+ if (ret)
@@ refs.c: int ref_transaction_commit(struct ref_transaction *transaction,
return ret;
}
@@ refs.c: int ref_transaction_commit(struct ref_transaction *transaction,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err)
-+enum transaction_error refs_verify_refnames_available(struct ref_store *refs,
-+ const struct string_list *refnames,
-+ const struct string_list *extras,
-+ const struct string_list *skip,
-+ unsigned int initial_transaction,
-+ struct strbuf *err)
++enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
++ const struct string_list *refnames,
++ const struct string_list *extras,
++ const struct string_list *skip,
++ unsigned int initial_transaction,
++ struct strbuf *err)
{
struct strbuf dirname = STRBUF_INIT;
struct strbuf referent = STRBUF_INIT;
struct ref_iterator *iter = NULL;
struct strset dirnames;
- int ret = -1;
-+ int ret = TRANSACTION_NAME_CONFLICT;
++ int ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
/*
* For the sake of comments in this function, suppose that
@@ refs.c: int refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err)
-+enum transaction_error refs_verify_refname_available(struct ref_store *refs,
-+ const char *refname,
-+ const struct string_list *extras,
-+ const struct string_list *skip,
-+ unsigned int initial_transaction,
-+ struct strbuf *err)
++enum ref_transaction_error refs_verify_refname_available(
++ struct ref_store *refs,
++ const char *refname,
++ const struct string_list *extras,
++ const struct string_list *skip,
++ unsigned int initial_transaction,
++ struct strbuf *err)
{
struct string_list_item item = { .string = (char *) refname };
struct string_list refnames = {
@@ refs.c: int ref_update_has_null_new_value(struct ref_update *update)
-int ref_update_check_old_target(const char *referent, struct ref_update *update,
- struct strbuf *err)
-+enum transaction_error ref_update_check_old_target(const char *referent,
-+ struct ref_update *update,
-+ struct strbuf *err)
++enum ref_transaction_error ref_update_check_old_target(const char *referent,
++ struct ref_update *update,
++ struct strbuf *err)
{
if (!update->old_target)
BUG("called without old_target set");
-
+@@ refs.c: int ref_update_check_old_target(const char *referent, struct ref_update *update,
if (!strcmp(referent, update->old_target))
-- return 0;
-+ return TRANSACTION_OK;
+ return 0;
- if (!strcmp(referent, ""))
+ if (!strcmp(referent, "")) {
@@ refs.c: int ref_update_has_null_new_value(struct ref_update *update)
- else
- strbuf_addf(err, "verifying symref target: '%s': "
- "is at %s but expected %s",
-+ return TRANSACTION_NONEXISTENT_REF;
++ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ }
+
+ strbuf_addf(err, "verifying symref target: '%s': is at %s but expected %s",
ref_update_original_update_refname(update),
referent, update->old_target);
- return -1;
-+ return TRANSACTION_INCORRECT_OLD_VALUE;
++ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
}
struct migration_data {
@@ refs.h: struct worktree;
const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);
+/*
-+ * enum transaction_error represents the following return codes:
-+ * TRANSACTION_OK: success code.
-+ * TRANSACTION_GENERIC_ERROR error_code: default error code.
-+ * TRANSACTION_NAME_CONFLICT error_code: ref name conflict like A vs A/B.
-+ * TRANSACTION_CREATE_EXISTS error_code: ref to be created already exists.
-+ * TRANSACTION_NONEXISTENT_REF error_code: ref expected but doesn't exist.
-+ * TRANSACTION_INCORRECT_OLD_VALUE error_code: provided old_oid or old_target of
++ * enum ref_transaction_error represents the following return codes:
++ * REF_TRANSACTION_ERROR_GENERIC error_code: default error code.
++ * REF_TRANSACTION_ERROR_NAME_CONFLICT error_code: ref name conflict like A vs A/B.
++ * REF_TRANSACTION_ERROR_CREATE_EXISTS error_code: ref to be created already exists.
++ * REF_TRANSACTION_ERROR_NONEXISTENT_REF error_code: ref expected but doesn't exist.
++ * REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE error_code: provided old_oid or old_target of
+ * reference doesn't match actual.
-+ * TRANSACTION_INVALID_NEW_VALUE error_code: provided new_oid or new_target is
++ * REF_TRANSACTION_ERROR_INVALID_NEW_VALUE error_code: provided new_oid or new_target is
+ * invalid.
-+ * TRANSACTION_EXPECTED_SYMREF error_code: expected ref to be symref, but is a
++ * REF_TRANSACTION_ERROR_EXPECTED_SYMREF error_code: expected ref to be symref, but is a
+ * regular ref.
+ */
-+enum transaction_error {
-+ TRANSACTION_OK = 0,
-+ TRANSACTION_GENERIC_ERROR = -1,
-+ TRANSACTION_NAME_CONFLICT = -2,
-+ TRANSACTION_CREATE_EXISTS = -3,
-+ TRANSACTION_NONEXISTENT_REF = -4,
-+ TRANSACTION_INCORRECT_OLD_VALUE = -5,
-+ TRANSACTION_INVALID_NEW_VALUE = -6,
-+ TRANSACTION_EXPECTED_SYMREF = -7,
++enum ref_transaction_error {
++ REF_TRANSACTION_ERROR_GENERIC = -1,
++ REF_TRANSACTION_ERROR_NAME_CONFLICT = -2,
++ REF_TRANSACTION_ERROR_CREATE_EXISTS = -3,
++ REF_TRANSACTION_ERROR_NONEXISTENT_REF = -4,
++ REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE = -5,
++ REF_TRANSACTION_ERROR_INVALID_NEW_VALUE = -6,
++ REF_TRANSACTION_ERROR_EXPECTED_SYMREF = -7,
+};
+
/*
@@ refs.h: int refs_read_symbolic_ref(struct ref_store *ref_store, const char *refn
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
-+enum transaction_error refs_verify_refname_available(struct ref_store *refs,
-+ const char *refname,
-+ const struct string_list *extras,
-+ const struct string_list *skip,
-+ unsigned int initial_transaction,
-+ struct strbuf *err);
++enum ref_transaction_error refs_verify_refname_available(struct ref_store *refs,
++ const char *refname,
++ const struct string_list *extras,
++ const struct string_list *skip,
++ unsigned int initial_transaction,
++ struct strbuf *err);
/*
* Same as `refs_verify_refname_available()`, but checking for a list of
@@ refs.h: int refs_read_symbolic_ref(struct ref_store *ref_store, const char *refn
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
-+enum transaction_error refs_verify_refnames_available(struct ref_store *refs,
-+ const struct string_list *refnames,
-+ const struct string_list *extras,
-+ const struct string_list *skip,
-+ unsigned int initial_transaction,
-+ struct strbuf *err);
++enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
++ const struct string_list *refnames,
++ const struct string_list *extras,
++ const struct string_list *skip,
++ unsigned int initial_transaction,
++ struct strbuf *err);
int refs_ref_exists(struct ref_store *refs, const char *refname);
@@ refs.h: int ref_transaction_verify(struct ref_transaction *transaction,
* any needed locks, check preconditions, etc.; basically, do as much
## refs/files-backend.c ##
+@@ refs/files-backend.c: static void unlock_ref(struct ref_lock *lock)
+ * broken, lock the reference anyway but clear old_oid.
+ *
+ * Return 0 on success. On failure, write an error message to err and
+- * return TRANSACTION_NAME_CONFLICT or TRANSACTION_GENERIC_ERROR.
++ * return REF_TRANSACTION_ERROR_NAME_CONFLICT or REF_TRANSACTION_ERROR_GENERIC.
+ *
+ * Implementation note: This function is basically
+ *
@@ refs/files-backend.c: static void unlock_ref(struct ref_lock *lock)
* avoided, namely if we were successfully able to read the ref
* - Generate informative error messages in the case of failure
@@ refs/files-backend.c: static void unlock_ref(struct ref_lock *lock)
- struct strbuf *referent,
- unsigned int *type,
- struct strbuf *err)
-+static enum transaction_error lock_raw_ref(struct files_ref_store *refs,
-+ const char *refname, int mustexist,
-+ struct string_list *refnames_to_check,
-+ const struct string_list *extras,
-+ struct ref_lock **lock_p,
-+ struct strbuf *referent,
-+ unsigned int *type,
-+ struct strbuf *err)
- {
-+ enum transaction_error ret = TRANSACTION_GENERIC_ERROR;
+-{
++static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
++ const char *refname,
++ int mustexist,
++ struct string_list *refnames_to_check,
++ const struct string_list *extras,
++ struct ref_lock **lock_p,
++ struct strbuf *referent,
++ unsigned int *type,
++ struct strbuf *err)
++{
++ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
struct ref_lock *lock;
struct strbuf ref_file = STRBUF_INIT;
int attempts_remaining = 3;
@@ refs/files-backend.c: static int lock_raw_ref(struct files_ref_store *refs,
strbuf_reset(err);
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
-+ ret = TRANSACTION_NONEXISTENT_REF;
++ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
} else {
/*
* The error message set by
+ * refs_verify_refname_available() is
+ * OK.
+ */
+- ret = TRANSACTION_NAME_CONFLICT;
++ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
+ }
+ } else {
+ /*
@@ refs/files-backend.c: static int lock_raw_ref(struct files_ref_store *refs,
/* Garden variety missing reference. */
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
-+ ret = TRANSACTION_NONEXISTENT_REF;
++ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error_return;
} else {
/*
@@ refs/files-backend.c: static int lock_raw_ref(struct files_ref_store *refs,
/* Garden variety missing reference. */
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
-+ ret = TRANSACTION_NONEXISTENT_REF;
++ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error_return;
} else if (remove_dir_recursively(&ref_file,
REMOVE_DIR_EMPTY_ONLY)) {
@@ refs/files-backend.c: static int lock_raw_ref(struct files_ref_store *refs,
- string_list_insert(refnames_to_check, refname);
- }
-
-- ret = 0;
-+ ret = TRANSACTION_OK;
- goto out;
-
- error_return:
+ * The error message set by
+ * verify_refname_available() is OK.
+ */
+- ret = TRANSACTION_NAME_CONFLICT;
++ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
+ goto error_return;
+ } else {
+ /*
@@ refs/files-backend.c: static int rename_tmp_log(struct files_ref_store *refs, const char *newrefname)
return ret;
}
@@ refs/files-backend.c: static int rename_tmp_log(struct files_ref_store *refs, co
- struct ref_lock *lock,
- const struct object_id *oid,
- int skip_oid_verification, struct strbuf *err);
-+static enum transaction_error write_ref_to_lockfile(struct files_ref_store *refs,
-+ struct ref_lock *lock,
-+ const struct object_id *oid,
-+ int skip_oid_verification, struct strbuf *err);
++static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs,
++ struct ref_lock *lock,
++ const struct object_id *oid,
++ int skip_oid_verification,
++ struct strbuf *err);
static int commit_ref_update(struct files_ref_store *refs,
struct ref_lock *lock,
const struct object_id *oid, const char *logmsg,
@@ refs/files-backend.c: static int files_log_ref_write(struct files_ref_store *ref
- struct ref_lock *lock,
- const struct object_id *oid,
- int skip_oid_verification, struct strbuf *err)
-+static enum transaction_error write_ref_to_lockfile(struct files_ref_store *refs,
-+ struct ref_lock *lock,
-+ const struct object_id *oid,
-+ int skip_oid_verification,
-+ struct strbuf *err)
++static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs,
++ struct ref_lock *lock,
++ const struct object_id *oid,
++ int skip_oid_verification,
++ struct strbuf *err)
{
static char term = '\n';
struct object *o;
@@ refs/files-backend.c: static int write_ref_to_lockfile(struct files_ref_store *r
lock->ref_name, oid_to_hex(oid));
unlock_ref(lock);
- return -1;
-+ return TRANSACTION_INVALID_NEW_VALUE;
++ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
if (o->type != OBJ_COMMIT && is_branch(lock->ref_name)) {
strbuf_addf(
@@ refs/files-backend.c: static int write_ref_to_lockfile(struct files_ref_store *r
oid_to_hex(oid), lock->ref_name);
unlock_ref(lock);
- return -1;
-+ return TRANSACTION_INVALID_NEW_VALUE;
++ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
}
fd = get_lock_file_fd(&lock->lk);
@@ refs/files-backend.c: static int write_ref_to_lockfile(struct files_ref_store *r
"couldn't write '%s'", get_lock_file_path(&lock->lk));
unlock_ref(lock);
- return -1;
-+ return TRANSACTION_GENERIC_ERROR;
++ return REF_TRANSACTION_ERROR_GENERIC;
}
-- return 0;
-+ return TRANSACTION_OK;
+ return 0;
}
-
- /*
@@ refs/files-backend.c: static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st
* If update is a direct update of head_ref (the reference pointed to
* by HEAD), then add an extra REF_LOG_ONLY update for HEAD.
@@ refs/files-backend.c: static struct ref_iterator *files_reflog_iterator_begin(st
-static int split_head_update(struct ref_update *update,
- struct ref_transaction *transaction,
- const char *head_ref, struct strbuf *err)
-+static enum transaction_error split_head_update(struct ref_update *update,
-+ struct ref_transaction *transaction,
-+ const char *head_ref,
-+ struct strbuf *err)
++static enum ref_transaction_error split_head_update(struct ref_update *update,
++ struct ref_transaction *transaction,
++ const char *head_ref,
++ struct strbuf *err)
{
struct ref_update *new_update;
@@ refs/files-backend.c: static int split_head_update(struct ref_update *update,
- (update->flags & REF_SKIP_CREATE_REFLOG) ||
- (update->flags & REF_IS_PRUNING) ||
- (update->flags & REF_UPDATE_VIA_HEAD))
-- return 0;
-+ return TRANSACTION_OK;
-
- if (strcmp(update->refname, head_ref))
-- return 0;
-+ return TRANSACTION_OK;
-
- /*
- * First make sure that HEAD is not already in the
-@@ refs/files-backend.c: static int split_head_update(struct ref_update *update,
- if (strcmp(new_update->refname, "HEAD"))
- BUG("%s unexpectedly not 'HEAD'", new_update->refname);
-
-- return 0;
-+ return TRANSACTION_OK;
- }
+ "multiple updates for 'HEAD' (including one "
+ "via its referent '%s') are not allowed",
+ update->refname);
+- return TRANSACTION_NAME_CONFLICT;
++ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
+ }
- /*
+ new_update = ref_transaction_add_update(
@@ refs/files-backend.c: static int split_head_update(struct ref_update *update,
* Note that the new update will itself be subject to splitting when
* the iteration gets to it.
@@ refs/files-backend.c: static int split_head_update(struct ref_update *update,
- const char *referent,
- struct ref_transaction *transaction,
- struct strbuf *err)
-+static enum transaction_error split_symref_update(struct ref_update *update,
-+ const char *referent,
-+ struct ref_transaction *transaction,
-+ struct strbuf *err)
++static enum ref_transaction_error split_symref_update(struct ref_update *update,
++ const char *referent,
++ struct ref_transaction *transaction,
++ struct strbuf *err)
{
struct ref_update *new_update;
unsigned int new_flags;
@@ refs/files-backend.c: static int split_symref_update(struct ref_update *update,
- update->flags |= REF_LOG_ONLY | REF_NO_DEREF;
- update->flags &= ~REF_HAVE_OLD;
-
-- return 0;
-+ return TRANSACTION_OK;
- }
+ "multiple updates for '%s' (including one "
+ "via symref '%s') are not allowed",
+ referent, update->refname);
+- return TRANSACTION_NAME_CONFLICT;
++ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
+ }
- /*
+ new_flags = update->flags;
@@ refs/files-backend.c: static int split_symref_update(struct ref_update *update,
* everything is OK, return 0; otherwise, write an error message to
* err and return -1.
*/
-static int check_old_oid(struct ref_update *update, struct object_id *oid,
- struct strbuf *err)
-+static enum transaction_error check_old_oid(struct ref_update *update,
-+ struct object_id *oid,
-+ struct strbuf *err)
++static enum ref_transaction_error check_old_oid(struct ref_update *update,
++ struct object_id *oid,
++ struct strbuf *err)
{
- int ret = TRANSACTION_GENERIC_ERROR;
-
if (!(update->flags & REF_HAVE_OLD) ||
oideq(oid, &update->old_oid))
-- return 0;
-+ return TRANSACTION_OK;
-
- if (is_null_oid(&update->old_oid)) {
+ return 0;
+@@ refs/files-backend.c: static int check_old_oid(struct ref_update *update, struct object_id *oid,
strbuf_addf(err, "cannot lock ref '%s': "
"reference already exists",
ref_update_original_update_refname(update));
- ret = TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(oid))
-+ return TRANSACTION_CREATE_EXISTS;
++ return REF_TRANSACTION_ERROR_CREATE_EXISTS;
+ } else if (is_null_oid(oid)) {
strbuf_addf(err, "cannot lock ref '%s': "
"reference is missing but expected %s",
@@ refs/files-backend.c: static int split_symref_update(struct ref_update *update,
- ref_update_original_update_refname(update),
- oid_to_hex(oid),
- oid_to_hex(&update->old_oid));
-+ return TRANSACTION_NONEXISTENT_REF;
++ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ }
- return ret;
@@ refs/files-backend.c: static int split_symref_update(struct ref_update *update,
+ ref_update_original_update_refname(update), oid_to_hex(oid),
+ oid_to_hex(&update->old_oid));
+
-+ return TRANSACTION_INCORRECT_OLD_VALUE;
++ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
}
struct files_transaction_backend_data {
@@ refs/files-backend.c: struct files_transaction_backend_data {
- const char *head_ref,
- struct string_list *refnames_to_check,
- struct strbuf *err)
-+static enum transaction_error lock_ref_for_update(struct files_ref_store *refs,
-+ struct ref_update *update,
-+ struct ref_transaction *transaction,
-+ const char *head_ref,
-+ struct string_list *refnames_to_check,
-+ struct strbuf *err)
++static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *refs,
++ struct ref_update *update,
++ struct ref_transaction *transaction,
++ const char *head_ref,
++ struct string_list *refnames_to_check,
++ struct strbuf *err)
{
struct strbuf referent = STRBUF_INIT;
int mustexist = ref_update_expects_existing_old_ref(update);
struct files_transaction_backend_data *backend_data;
- int ret = 0;
-+ enum transaction_error ret = TRANSACTION_OK;
++ enum ref_transaction_error ret = 0;
struct ref_lock *lock;
files_assert_main_repository(refs, "lock_ref_for_update");
@@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
+ strbuf_addf(err, "cannot lock ref '%s': "
+ "error reading reference",
+ ref_update_original_update_refname(update));
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto out;
}
}
@@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
- if (ret) {
- goto out;
- }
-+ if (ret) {
+- }
++ if (ret)
+ goto out;
- }
} else {
/*
+ * Create a new update for the reference this
@@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
"but is a regular ref"),
ref_update_original_update_refname(update),
update->old_target);
- ret = TRANSACTION_GENERIC_ERROR;
-+ ret = TRANSACTION_EXPECTED_SYMREF;
++ ret = REF_TRANSACTION_ERROR_EXPECTED_SYMREF;
goto out;
} else {
ret = check_old_oid(update, &lock->old_oid, err);
+@@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
+
+ if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
+ if (create_symref_lock(lock, update->new_target, err)) {
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto out;
+ }
+
+ if (close_ref_gently(lock)) {
+ strbuf_addf(err, "couldn't close '%s.lock'",
+ update->refname);
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto out;
+ }
+
@@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
* The reference already has the desired
* value, so we don't need to write it.
@@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
}
}
if (!(update->flags & REF_NEEDS_COMMIT)) {
+@@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
+ if (close_ref_gently(lock)) {
+ strbuf_addf(err, "couldn't close '%s.lock'",
+ update->refname);
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto out;
+ }
+ }
+@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref_store,
+ refs->packed_ref_store,
+ transaction->flags, err);
+ if (!packed_transaction) {
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto cleanup;
+ }
+
+@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref_store,
+ */
+ if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
+ &transaction->refnames, NULL, 0, err)) {
+- ret = TRANSACTION_NAME_CONFLICT;
++ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
+ goto cleanup;
+ }
+
+ if (packed_transaction) {
+ if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto cleanup;
+ }
+ backend_data->packed_refs_locked = 1;
+@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref_store,
+ */
+ backend_data->packed_transaction = NULL;
+ if (ref_transaction_abort(packed_transaction, err)) {
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto cleanup;
+ }
+ }
+@@ refs/files-backend.c: static int files_transaction_finish_initial(struct files_ref_store *refs,
+ packed_transaction = ref_store_transaction_begin(refs->packed_ref_store,
+ transaction->flags, err);
+ if (!packed_transaction) {
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto cleanup;
+ }
+
+@@ refs/files-backend.c: static int files_transaction_finish_initial(struct files_ref_store *refs,
+ if (!loose_transaction) {
+ loose_transaction = ref_store_transaction_begin(&refs->base, 0, err);
+ if (!loose_transaction) {
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto cleanup;
+ }
+ }
+@@ refs/files-backend.c: static int files_transaction_finish_initial(struct files_ref_store *refs,
+ }
+
+ if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto cleanup;
+ }
+
+ if (refs_verify_refnames_available(&refs->base, &refnames_to_check,
+ &affected_refnames, NULL, 1, err)) {
+ packed_refs_unlock(refs->packed_ref_store);
+- ret = TRANSACTION_NAME_CONFLICT;
++ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
+ goto cleanup;
+ }
+
+ if (ref_transaction_commit(packed_transaction, err)) {
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto cleanup;
+ }
+ packed_refs_unlock(refs->packed_ref_store);
+@@ refs/files-backend.c: static int files_transaction_finish_initial(struct files_ref_store *refs,
+ if (loose_transaction) {
+ if (ref_transaction_prepare(loose_transaction, err) ||
+ ref_transaction_commit(loose_transaction, err)) {
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto cleanup;
+ }
+ }
+@@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_store,
+ if (update->flags & REF_NEEDS_COMMIT ||
+ update->flags & REF_LOG_ONLY) {
+ if (parse_and_write_reflog(refs, update, lock, err)) {
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto cleanup;
+ }
+ }
+@@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_store,
+ strbuf_addf(err, "couldn't set '%s'", lock->ref_name);
+ unlock_ref(lock);
+ update->backend_data = NULL;
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto cleanup;
+ }
+ }
+@@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_store,
+ strbuf_reset(&sb);
+ files_ref_path(refs, &sb, lock->ref_name);
+ if (unlink_or_msg(sb.buf, err)) {
+- ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
+ goto cleanup;
+ }
+ }
## refs/packed-backend.c ##
@@ refs/packed-backend.c: static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
@@ refs/packed-backend.c: static int packed_ref_store_remove_on_disk(struct ref_sto
-static int write_with_updates(struct packed_ref_store *refs,
- struct string_list *updates,
- struct strbuf *err)
-+static enum transaction_error write_with_updates(struct packed_ref_store *refs,
-+ struct string_list *updates,
-+ struct strbuf *err)
++static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,
++ struct string_list *updates,
++ struct strbuf *err)
{
-+ enum transaction_error ret = TRANSACTION_GENERIC_ERROR;
++ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
struct ref_iterator *iter = NULL;
size_t i;
int ok;
@@ refs/packed-backend.c: static int write_with_updates(struct packed_ref_store *re
sb.buf, strerror(errno));
strbuf_release(&sb);
- return -1;
-+ return TRANSACTION_GENERIC_ERROR;
++ return REF_TRANSACTION_ERROR_GENERIC;
}
strbuf_release(&sb);
@@ refs/packed-backend.c: static int write_with_updates(struct packed_ref_store *re
strbuf_addf(err, "cannot update ref '%s': "
"reference already exists",
update->refname);
-+ ret = TRANSACTION_CREATE_EXISTS;
++ ret = REF_TRANSACTION_ERROR_CREATE_EXISTS;
goto error;
} else if (!oideq(&update->old_oid, iter->oid)) {
strbuf_addf(err, "cannot update ref '%s': "
@@ refs/packed-backend.c: static int write_with_updates(struct packed_ref_store *re
update->refname,
oid_to_hex(iter->oid),
oid_to_hex(&update->old_oid));
-+ ret = TRANSACTION_INCORRECT_OLD_VALUE;
++ ret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
goto error;
}
}
@@ refs/packed-backend.c: static int write_with_updates(struct packed_ref_store *re
"reference is missing but expected %s",
update->refname,
oid_to_hex(&update->old_oid));
-+ return TRANSACTION_NONEXISTENT_REF;
++ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error;
}
}
@@ refs/packed-backend.c: static int write_with_updates(struct packed_ref_store *re
strbuf_release(&sb);
delete_tempfile(&refs->tempfile);
- return -1;
-+ return TRANSACTION_GENERIC_ERROR;
++ return REF_TRANSACTION_ERROR_GENERIC;
}
-- return 0;
-+ return TRANSACTION_OK;
-
- write_error:
- strbuf_addf(err, "error writing to %s: %s",
+ return 0;
@@ refs/packed-backend.c: static int write_with_updates(struct packed_ref_store *refs,
error:
ref_iterator_free(iter);
@@ refs/packed-backend.c: static int write_with_updates(struct packed_ref_store *re
}
int is_packed_transaction_needed(struct ref_store *ref_store,
+@@ refs/packed-backend.c: static int packed_transaction_prepare(struct ref_store *ref_store,
+ REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
+ "ref_transaction_prepare");
+ struct packed_transaction_backend_data *data;
+- int ret = TRANSACTION_GENERIC_ERROR;
++ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
+
+ /*
+ * Note that we *don't* skip transactions with zero updates,
@@ refs/packed-backend.c: static int packed_transaction_prepare(struct ref_store *ref_store,
data->own_lock = 1;
}
@@ refs/packed-backend.c: static int packed_transaction_prepare(struct ref_store *r
goto failure;
transaction->state = REF_TRANSACTION_PREPARED;
+@@ refs/packed-backend.c: static int packed_transaction_finish(struct ref_store *ref_store,
+ ref_store,
+ REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
+ "ref_transaction_finish");
+- int ret = TRANSACTION_GENERIC_ERROR;
++ int ret = REF_TRANSACTION_ERROR_GENERIC;
+ char *packed_refs_path;
+
+ clear_snapshot(refs);
## refs/refs-internal.h ##
@@ refs/refs-internal.h: int ref_update_has_null_new_value(struct ref_update *update);
@@ refs/refs-internal.h: int ref_update_has_null_new_value(struct ref_update *updat
*/
-int ref_update_check_old_target(const char *referent, struct ref_update *update,
- struct strbuf *err);
-+enum transaction_error ref_update_check_old_target(const char *referent,
-+ struct ref_update *update,
-+ struct strbuf *err);
++enum ref_transaction_error ref_update_check_old_target(const char *referent,
++ struct ref_update *update,
++ struct strbuf *err);
/*
* Check if the ref must exist, this means that the old_oid or
@@ refs/reftable-backend.c: static int queue_transaction_update(struct reftable_ref
- struct strbuf *head_referent,
- struct strbuf *referent,
- struct strbuf *err)
-+static enum transaction_error prepare_single_update(struct reftable_ref_store *refs,
-+ struct reftable_transaction_data *tx_data,
-+ struct ref_transaction *transaction,
-+ struct reftable_backend *be,
-+ struct ref_update *u,
-+ struct string_list *refnames_to_check,
-+ unsigned int head_type,
-+ struct strbuf *head_referent,
-+ struct strbuf *referent,
-+ struct strbuf *err)
++static enum ref_transaction_error prepare_single_update(struct reftable_ref_store *refs,
++ struct reftable_transaction_data *tx_data,
++ struct ref_transaction *transaction,
++ struct reftable_backend *be,
++ struct ref_update *u,
++ struct string_list *refnames_to_check,
++ unsigned int head_type,
++ struct strbuf *head_referent,
++ struct strbuf *referent,
++ struct strbuf *err)
{
-+ enum transaction_error ret = TRANSACTION_OK;
++ enum ref_transaction_error ret = 0;
struct object_id current_oid = {0};
const char *rewritten_ref;
- int ret = 0;
@@ refs/reftable-backend.c: static int prepare_single_update(struct reftable_ref_st
ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
if (ret)
- return ret;
-+ return TRANSACTION_GENERIC_ERROR;
++ return REF_TRANSACTION_ERROR_GENERIC;
/* Verify that the new object ID is valid. */
if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
@@ refs/reftable-backend.c: static int prepare_single_update(struct reftable_ref_st
_("trying to write ref '%s' with nonexistent object %s"),
u->refname, oid_to_hex(&u->new_oid));
- return -1;
-+ return TRANSACTION_INVALID_NEW_VALUE;
++ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
oid_to_hex(&u->new_oid), u->refname);
- return -1;
-+ return TRANSACTION_INVALID_NEW_VALUE;
++ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
}
+@@ refs/reftable-backend.c: static int prepare_single_update(struct reftable_ref_store *refs,
+ _("multiple updates for 'HEAD' (including one "
+ "via its referent '%s') are not allowed"),
+ u->refname);
+- return TRANSACTION_NAME_CONFLICT;
++ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
+ }
+
+ ref_transaction_add_update(
@@ refs/reftable-backend.c: static int prepare_single_update(struct reftable_ref_store *refs,
ret = reftable_backend_read_ref(be, rewritten_ref,
¤t_oid, referent, &u->type);
if (ret < 0)
- return ret;
-+ return TRANSACTION_GENERIC_ERROR;
++ return REF_TRANSACTION_ERROR_GENERIC;
if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
/*
* The reference does not exist, and we either have no
@@ refs/reftable-backend.c: static int prepare_single_update(struct reftable_ref_st
¤t_oid, err);
if (ret)
- return ret;
-+ return TRANSACTION_GENERIC_ERROR;
++ return REF_TRANSACTION_ERROR_GENERIC;
}
return 0;
@@ refs/reftable-backend.c: static int prepare_single_update(struct reftable_ref_st
"unable to resolve reference '%s'"),
ref_update_original_update_refname(u), u->refname);
- return -1;
-+ return TRANSACTION_NONEXISTENT_REF;
++ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
}
if (u->type & REF_ISSYMREF) {
@@ refs/reftable-backend.c: static int prepare_single_update(struct reftable_ref_st
strbuf_addf(err, _("cannot lock ref '%s': "
"error reading reference"), u->refname);
- return -1;
-+ return TRANSACTION_GENERIC_ERROR;
++ return REF_TRANSACTION_ERROR_GENERIC;
}
} else {
struct ref_update *new_update;
+@@ refs/reftable-backend.c: static int prepare_single_update(struct reftable_ref_store *refs,
+ _("multiple updates for '%s' (including one "
+ "via symref '%s') are not allowed"),
+ referent->buf, u->refname);
+- return TRANSACTION_NAME_CONFLICT;
++ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
+ }
+
+ /*
@@ refs/reftable-backend.c: static int prepare_single_update(struct reftable_ref_store *refs,
"but is a regular ref"),
ref_update_original_update_refname(u),
u->old_target);
- return -1;
-+ return TRANSACTION_EXPECTED_SYMREF;
++ return REF_TRANSACTION_ERROR_EXPECTED_SYMREF;
}
- if (ref_update_check_old_target(referent->buf, u, err)) {
- return -1;
+- }
+ ret = ref_update_check_old_target(referent->buf, u, err);
-+ if (ret) {
++ if (ret)
+ return ret;
- }
} else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
if (is_null_oid(&u->old_oid)) {
-@@ refs/reftable-backend.c: static int prepare_single_update(struct reftable_ref_store *refs,
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "reference already exists"),
ref_update_original_update_refname(u));
- return TRANSACTION_CREATE_EXISTS;
- }
+- return TRANSACTION_CREATE_EXISTS;
+- }
- else if (is_null_oid(¤t_oid))
-+ else if (is_null_oid(¤t_oid)) {
++ return REF_TRANSACTION_ERROR_CREATE_EXISTS;
++ } else if (is_null_oid(¤t_oid)) {
strbuf_addf(err, _("cannot lock ref '%s': "
"reference is missing but expected %s"),
ref_update_original_update_refname(u),
oid_to_hex(&u->old_oid));
- else
-+ return TRANSACTION_NONEXISTENT_REF;
-+
++ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ } else {
strbuf_addf(err, _("cannot lock ref '%s': "
"is at %s but expected %s"),
@@ refs/reftable-backend.c: static int prepare_single_update(struct reftable_ref_st
oid_to_hex(¤t_oid),
oid_to_hex(&u->old_oid));
- return TRANSACTION_NAME_CONFLICT;
-+ return TRANSACTION_INCORRECT_OLD_VALUE;
++ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
+ }
}
@@ refs/reftable-backend.c: static int prepare_single_update(struct reftable_ref_st
- return queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
+ if (queue_transaction_update(refs, tx_data, u, ¤t_oid, err))
-+ return TRANSACTION_GENERIC_ERROR;
++ return REF_TRANSACTION_ERROR_GENERIC;
-- return 0;
-+ return TRANSACTION_OK;
+ return 0;
}
-
- static int reftable_be_transaction_prepare(struct ref_store *ref_store,
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->state = REF_TRANSACTION_PREPARED;
6: 49a0e65427 ! 6: 73f8970cb9 refs: implement partial reference transaction support
@@ Commit message
'REF_TRANSACTION_ALLOW_PARTIAL'. When enabled, this flag allows
individual reference updates that would typically cause the entire
transaction to fail due to non-system-related errors to be marked as
- rejected while permitting other updates to proceed. Non-system-related
- errors include issues caused by user-provided input values, whereas
- system-related errors, such as I/O failures or memory issues, continue
- to result in a full transaction failure. This approach enhances
- flexibility while preserving transactional integrity where necessary.
+ rejected while permitting other updates to proceed. System errors
+ referred by 'REF_TRANSACTION_ERROR_GENERIC' continue to result in the
+ entire transaction failing. This approach enhances flexibility while
+ preserving transactional integrity where necessary.
The implementation introduces several key components:
- Add 'rejection_err' field to struct `ref_update` to track failed
updates with failure reason.
+ - Add a new struct `ref_transaction_rejections` and a field within
+ `ref_transaction` to this struct to allow quick iteration over
+ rejected updates.
+
- Modify reference backends (files, packed, reftable) to handle
partial transactions by using `ref_transaction_set_rejected()`
instead of failing the entire transaction when
@@ Commit message
examine which updates were rejected and why.
This foundational change enables partial transaction support throughout
- the reference subsystem. The next commit will expose this capability to
- users by adding a `--allow-partial` flag to 'git-update-ref(1)',
+ the reference subsystem. A following commit will expose this capability
+ to users by adding a `--allow-partial` flag to 'git-update-ref(1)',
providing both a user-facing feature and a testable implementation.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
## refs.c ##
+@@ refs.c: struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,
+ tr->ref_store = refs;
+ tr->flags = flags;
+ string_list_init_dup(&tr->refnames);
++
++ if (flags & REF_TRANSACTION_ALLOW_PARTIAL)
++ CALLOC_ARRAY(tr->rejections, 1);
++
+ return tr;
+ }
+
@@ refs.c: void ref_transaction_free(struct ref_transaction *transaction)
+ free((char *)transaction->updates[i]->old_target);
+ free(transaction->updates[i]);
+ }
++
++ if (transaction->rejections)
++ free(transaction->rejections->update_indices);
++ free(transaction->rejections);
++
+ string_list_clear(&transaction->refnames, 0);
+ free(transaction->updates);
free(transaction);
}
-+void ref_transaction_set_rejected(struct ref_transaction *transaction,
-+ size_t update_idx,
-+ enum transaction_error err)
++int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,
++ size_t update_idx,
++ enum ref_transaction_error err)
+{
+ if (update_idx >= transaction->nr)
+ BUG("trying to set rejection on invalid update index");
++
++ if (!(transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL))
++ return 0;
++
++ if (!transaction->rejections)
++ BUG("transaction not inititalized with partial support");
++
++ /*
++ * Don't accept generic errors, since these errors are not user
++ * input related.
++ */
++ if (err == REF_TRANSACTION_ERROR_GENERIC)
++ return 0;
++
+ transaction->updates[update_idx]->rejection_err = err;
++ ALLOC_GROW(transaction->rejections->update_indices,
++ transaction->rejections->nr + 1,
++ transaction->rejections->alloc);
++ transaction->rejections->update_indices[transaction->rejections->nr++] = update_idx;
++
++ return 1;
+}
+
struct ref_update *ref_transaction_add_update(
@@ refs.c: struct ref_update *ref_transaction_add_update(
transaction->updates[transaction->nr++] = update;
update->flags = flags;
-+ update->rejection_err = TRANSACTION_OK;
++ update->rejection_err = 0;
update->new_target = xstrdup_or_null(new_target);
update->old_target = xstrdup_or_null(old_target);
@@ refs.c: void ref_transaction_for_each_queued_update(struct ref_transaction *tran
+ ref_transaction_for_each_rejected_update_fn cb,
+ void *cb_data)
+{
-+ if (!(transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL))
++ if (!transaction->rejections)
+ return;
+
-+ for (size_t i = 0; i < transaction->nr; i++) {
-+ struct ref_update *update = transaction->updates[i];
++ for (size_t i = 0; i < transaction->rejections->nr; i++) {
++ size_t update_index = transaction->rejections->update_indices[i];
++ struct ref_update *update = transaction->updates[update_index];
+
+ if (!update->rejection_err)
+ continue;
@@ refs.h: void ref_transaction_for_each_queued_update(struct ref_transaction *tran
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
-+ enum transaction_error err,
++ enum ref_transaction_error err,
+ void *cb_data);
+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
+ ref_transaction_for_each_rejected_update_fn cb,
@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref
err);
- if (ret)
+ if (ret) {
-+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL &&
-+ ret != TRANSACTION_GENERIC_ERROR) {
-+ ref_transaction_set_rejected(transaction, i, ret);
-+
++ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
-+ ret = TRANSACTION_OK;
++ ret = 0;
+
+ continue;
+ }
@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref
if (update->flags & REF_DELETING &&
!(update->flags & REF_LOG_ONLY) &&
+@@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_store,
+ struct ref_update *update = transaction->updates[i];
+ struct ref_lock *lock = update->backend_data;
+
++ if (update->rejection_err)
++ continue;
++
+ if (update->flags & REF_NEEDS_COMMIT ||
+ update->flags & REF_LOG_ONLY) {
+ if (parse_and_write_reflog(refs, update, lock, err)) {
## refs/packed-backend.c ##
@@ refs/packed-backend.c: static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
* remain locked when it is done.
*/
- static enum transaction_error write_with_updates(struct packed_ref_store *refs,
-- struct string_list *updates,
-+ struct ref_transaction *transaction,
- struct strbuf *err)
+ static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,
+- struct string_list *updates,
++ struct ref_transaction *transaction,
+ struct strbuf *err)
{
- enum transaction_error ret = TRANSACTION_GENERIC_ERROR;
+ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
+ struct string_list *updates = &transaction->refnames;
struct ref_iterator *iter = NULL;
size_t i;
int ok;
-@@ refs/packed-backend.c: static enum transaction_error write_with_updates(struct packed_ref_store *refs,
+@@ refs/packed-backend.c: static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
"reference already exists",
update->refname);
- ret = TRANSACTION_CREATE_EXISTS;
+ ret = REF_TRANSACTION_ERROR_CREATE_EXISTS;
+
-+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
-+ ref_transaction_set_rejected(transaction, i, ret);
++ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
+ ret = 0;
+ continue;
@@ refs/packed-backend.c: static enum transaction_error write_with_updates(struct p
goto error;
} else if (!oideq(&update->old_oid, iter->oid)) {
strbuf_addf(err, "cannot update ref '%s': "
-@@ refs/packed-backend.c: static enum transaction_error write_with_updates(struct packed_ref_store *refs,
+@@ refs/packed-backend.c: static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
oid_to_hex(iter->oid),
oid_to_hex(&update->old_oid));
- ret = TRANSACTION_INCORRECT_OLD_VALUE;
+ ret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
+
-+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
-+ ref_transaction_set_rejected(transaction, i, ret);
++ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
+ ret = 0;
+ continue;
@@ refs/packed-backend.c: static enum transaction_error write_with_updates(struct p
goto error;
}
}
-@@ refs/packed-backend.c: static enum transaction_error write_with_updates(struct packed_ref_store *refs,
+@@ refs/packed-backend.c: static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
update->refname,
oid_to_hex(&update->old_oid));
- return TRANSACTION_NONEXISTENT_REF;
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+
-+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL) {
-+ ref_transaction_set_rejected(transaction, i, ret);
++ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
+ ret = 0;
+ continue;
@@ refs/packed-backend.c: static enum transaction_error write_with_updates(struct p
goto error;
}
}
-@@ refs/packed-backend.c: static enum transaction_error write_with_updates(struct packed_ref_store *refs,
+@@ refs/packed-backend.c: static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
write_error:
strbuf_addf(err, "error writing to %s: %s",
get_tempfile_path(refs->tempfile), strerror(errno));
-+ ret = TRANSACTION_GENERIC_ERROR;
++ ret = REF_TRANSACTION_ERROR_GENERIC;
error:
ref_iterator_free(iter);
@@ refs/refs-internal.h: struct ref_update {
+ /*
+ * Used in partial transactions to mark if a given update was rejected.
+ */
-+ enum transaction_error rejection_err;
++ enum ref_transaction_error rejection_err;
+
/*
* If this ref_update was split off of a symref update via
@@ refs/refs-internal.h: int refs_read_raw_ref(struct ref_store *ref_store, const c
unsigned int *type, int *failure_errno);
+/*
-+ * Mark a given update as rejected with a given reason. To be used in conjuction
-+ * with the `REF_TRANSACTION_ALLOW_PARTIAL` flag to allow partial transactions.
++ * Mark a given update as rejected with a given reason.
+ */
-+void ref_transaction_set_rejected(struct ref_transaction *transaction,
-+ size_t update_idx,
-+ enum transaction_error err);
++int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,
++ size_t update_idx,
++ enum ref_transaction_error err);
+
/*
* Add a ref_update with the specified properties to transaction, and
* return a pointer to the new object. This function does not verify
+@@ refs/refs-internal.h: enum ref_transaction_state {
+ REF_TRANSACTION_CLOSED = 2
+ };
+
++/*
++ * Data structure to hold indices of updates which were rejected, when
++ * partial transactions where enabled. While the updates themselves hold
++ * the rejection error, this structure allows a transaction to iterate
++ * only over the rejected updates.
++ */
++struct ref_transaction_rejections {
++ size_t *update_indices;
++ size_t alloc;
++ size_t nr;
++};
++
+ /*
+ * Data structure for holding a reference transaction, which can
+ * consist of checks and updates to multiple references, carried out
+@@ refs/refs-internal.h: struct ref_transaction {
+ size_t alloc;
+ size_t nr;
+ enum ref_transaction_state state;
++ struct ref_transaction_rejections *rejections;
+ void *backend_data;
+ unsigned int flags;
+ uint64_t max_index;
## refs/reftable-backend.c ##
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_store *ref_store,
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
&head_referent, &referent, err);
- if (ret)
+ if (ret) {
-+ if (transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL &&
-+ ret != TRANSACTION_GENERIC_ERROR) {
-+ ref_transaction_set_rejected(transaction, i, ret);
-+
++ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
-+ ret = TRANSACTION_OK;
++ ret = 0;
+
+ continue;
+ }
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
}
string_list_sort(&refnames_to_check);
+@@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writer *writer, void *cb_data
+ struct reftable_transaction_update *tx_update = &arg->updates[i];
+ struct ref_update *u = tx_update->update;
+
++ if (u->rejection_err)
++ continue;
++
+ /*
+ * Write a reflog entry when updating a ref to point to
+ * something new in either of the following cases:
-: ---------- > 7: f0284388ce refs: support partial update rejections during F/D checks
7: 0dc37f87a7 ! 8: f0e7c44eb7 update-ref: add --allow-partial flag for stdin mode
@@ Commit message
rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
- or with `-z`:
-
- rejected NUL (<old-oid> | <old-target>) NUL (<new-oid> | <new-target>) NUL <rejection-reason> NUL
-
Update the documentation to reflect this change and also tests to cover
different scenarios where an update could be rejected.
@@ Documentation/git-update-ref.adoc: performs all modifications together. Specify
Quote fields containing whitespace as if they were strings in C source
code; i.e., surrounded by double-quotes and with backslash escapes.
Use 40 "0" characters or the empty string to specify a zero value. To
-@@ Documentation/git-update-ref.adoc: quoting:
- In this format, use 40 "0" to specify a zero value, and use the empty
- string to specify a missing value.
-
-+With `-z`, `--allow-partial` will print rejections in the following form:
-+
-+ rejected NUL (<old-oid> | <old-target>) NUL (<new-oid> | <new-target>) NUL <rejection-reason> NUL
-+
- In either format, values can be specified in any form that Git
- recognizes as an object name. Commands in any other format or a
- repeated <ref> produce an error. Command meanings are:
## builtin/update-ref.c ##
@@
@@ builtin/update-ref.c: static void parse_cmd_abort(struct ref_transaction *transa
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
-+ enum transaction_error err,
++ enum ref_transaction_error err,
+ void *cb_data UNUSED)
+{
+ struct strbuf sb = STRBUF_INIT;
-+ char space = ' ';
+ const char *reason = "";
+
+ switch (err) {
-+ case TRANSACTION_NAME_CONFLICT:
-+ reason = _("refname conflict");
++ case REF_TRANSACTION_ERROR_NAME_CONFLICT:
++ reason = "refname conflict";
+ break;
-+ case TRANSACTION_CREATE_EXISTS:
-+ reason = _("reference already exists");
++ case REF_TRANSACTION_ERROR_CREATE_EXISTS:
++ reason = "reference already exists";
+ break;
-+ case TRANSACTION_NONEXISTENT_REF:
-+ reason = _("reference does not exist");
++ case REF_TRANSACTION_ERROR_NONEXISTENT_REF:
++ reason = "reference does not exist";
+ break;
-+ case TRANSACTION_INCORRECT_OLD_VALUE:
-+ reason = _("incorrect old value provided");
++ case REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE:
++ reason = "incorrect old value provided";
+ break;
-+ case TRANSACTION_INVALID_NEW_VALUE:
-+ reason = _("invalid new value provided");
++ case REF_TRANSACTION_ERROR_INVALID_NEW_VALUE:
++ reason = "invalid new value provided";
+ break;
-+ case TRANSACTION_EXPECTED_SYMREF:
-+ reason = _("expected symref but found regular ref");
++ case REF_TRANSACTION_ERROR_EXPECTED_SYMREF:
++ reason = "expected symref but found regular ref";
+ break;
+ default:
-+ reason = _("unkown failure");
++ reason = "unkown failure";
+ }
+
-+ if (!line_termination)
-+ space = line_termination;
-+
-+ strbuf_addf(&sb, "rejected%c%s%c%s%c%c%s%c%s%c", space,
-+ refname, space, new_oid ? oid_to_hex(new_oid) : new_target,
-+ space, space, old_oid ? oid_to_hex(old_oid) : old_target,
-+ space, reason, line_termination);
++ strbuf_addf(&sb, "rejected %s %s %s %s\n", refname,
++ new_oid ? oid_to_hex(new_oid) : new_target,
++ old_oid ? oid_to_hex(old_oid) : old_target,
++ reason);
+
+ fwrite(sb.buf, sb.len, 1, stdout);
+ strbuf_release(&sb);
-+ fflush(stdout);
+}
+
static void parse_cmd_commit(struct ref_transaction *transaction,
@@ builtin/update-ref.c: static void update_refs_stdin(void)
break;
case UPDATE_REFS_STARTED:
@@ builtin/update-ref.c: int cmd_update_ref(int argc,
- const char *refname, *oldval;
struct object_id oid, oldoid;
int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;
-- int create_reflog = 0;
-+ int create_reflog = 0, allow_partial = 0;
+ int create_reflog = 0;
+ unsigned int flags = 0;
+
struct option options[] = {
@@ builtin/update-ref.c: int cmd_update_ref(int argc,
+ update_refs_stdin(flags);
return 0;
- }
-+ } else if (allow_partial)
++ } else if (flags & REF_TRANSACTION_ALLOW_PARTIAL)
+ die("--allow-partial can only be used with --stdin");
if (end_null)
@@ t/t1400-update-ref.sh: do
+ )
+ '
+
-+ # F/D conflicts on the files backend are resolved on an individual
-+ # update level since refs are stored as files. On the reftable backend
-+ # this check is batched to optimize for performance, so failures cannot
-+ # be isolated to a single update.
-+ test_expect_success REFFILES "stdin $type allow-partial refname conflict" '
++ test_expect_success "stdin $type allow-partial refname conflict" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
@@ t/t1400-update-ref.sh: do
+ test_cmp expect actual &&
+ test_grep -q "refname conflict" stdout
+ )
++ '
++
++ test_expect_success "stdin $type allow-partial refname conflict new ref" '
++ git init repo &&
++ test_when_finished "rm -fr repo" &&
++ (
++ cd repo &&
++ test_commit one &&
++ old_head=$(git rev-parse HEAD) &&
++ test_commit two &&
++ head=$(git rev-parse HEAD) &&
++ git update-ref refs/heads/ref/foo $head &&
++
++ format_command $type "update refs/heads/foo" "$old_head" "" >stdin &&
++ format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
++ git update-ref $type --stdin --allow-partial <stdin >stdout &&
++ echo $old_head >expect &&
++ git rev-parse refs/heads/foo >actual &&
++ test_cmp expect actual &&
++ test_grep -q "refname conflict" stdout
++ )
+ '
done
base-commit: f032e4cb6777d229cc1e662e142e99ef71741eb4
change-id: 20241206-245-partially-atomic-ref-updates-9fe8b080345c
Thanks
- Karthik
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH v3 1/8] refs/files: remove redundant check in split_symref_update()
2025-03-05 17:38 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Karthik Nayak
@ 2025-03-05 17:38 ` Karthik Nayak
2025-03-05 21:20 ` Junio C Hamano
2025-03-05 17:38 ` [PATCH v3 2/8] refs: move duplicate refname update check to generic layer Karthik Nayak
` (7 subsequent siblings)
8 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-03-05 17:38 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
In `split_symref_update()`, there were two checks for duplicate
refnames:
- At the start, `string_list_has_string()` ensures the refname is not
already in `affected_refnames`, preventing duplicates from being
added.
- After adding the refname, another check verifies whether the newly
inserted item has a `util` value.
The second check is unnecessary because the first one guarantees that
`string_list_insert()` will never encounter a preexisting entry.
Since `item->util` is only used in this context, remove the assignment and
simplify the surrounding code.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs/files-backend.c | 20 +++-----------------
1 file changed, 3 insertions(+), 17 deletions(-)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 4e1c50fead..6c7df30738 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2382,7 +2382,6 @@ static int split_head_update(struct ref_update *update,
struct string_list *affected_refnames,
struct strbuf *err)
{
- struct string_list_item *item;
struct ref_update *new_update;
if ((update->flags & REF_LOG_ONLY) ||
@@ -2421,8 +2420,7 @@ static int split_head_update(struct ref_update *update,
*/
if (strcmp(new_update->refname, "HEAD"))
BUG("%s unexpectedly not 'HEAD'", new_update->refname);
- item = string_list_insert(affected_refnames, new_update->refname);
- item->util = new_update;
+ string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2441,7 +2439,6 @@ static int split_symref_update(struct ref_update *update,
struct string_list *affected_refnames,
struct strbuf *err)
{
- struct string_list_item *item;
struct ref_update *new_update;
unsigned int new_flags;
@@ -2496,11 +2493,7 @@ static int split_symref_update(struct ref_update *update,
* be valid as long as affected_refnames is in use, and NOT
* referent, which might soon be freed by our caller.
*/
- item = string_list_insert(affected_refnames, new_update->refname);
- if (item->util)
- BUG("%s unexpectedly found in affected_refnames",
- new_update->refname);
- item->util = new_update;
+ string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2834,7 +2827,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
*/
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
- struct string_list_item *item;
if ((update->flags & REF_IS_PRUNING) &&
!(update->flags & REF_NO_DEREF))
@@ -2843,13 +2835,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
if (update->flags & REF_LOG_ONLY)
continue;
- item = string_list_append(&affected_refnames, update->refname);
- /*
- * We store a pointer to update in item->util, but at
- * the moment we never use the value of this field
- * except to check whether it is non-NULL.
- */
- item->util = update;
+ string_list_append(&affected_refnames, update->refname);
}
string_list_sort(&affected_refnames);
if (ref_update_reject_duplicates(&affected_refnames, err)) {
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v3 2/8] refs: move duplicate refname update check to generic layer
2025-03-05 17:38 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Karthik Nayak
2025-03-05 17:38 ` [PATCH v3 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
@ 2025-03-05 17:38 ` Karthik Nayak
2025-03-05 21:56 ` Junio C Hamano
2025-03-05 17:38 ` [PATCH v3 3/8] refs/files: remove duplicate duplicates check Karthik Nayak
` (6 subsequent siblings)
8 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-03-05 17:38 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Move the tracking of refnames in `affected_refnames` from individual
backends into the generic layer in 'refs.c'. This centralizes the
duplicate refname detection that was previously handled separately by
each backend.
Make some changes to accommodate this move:
- Add a `string_list` field `refnames` to `ref_transaction` to contain
all the references in a transaction. This field is updated whenever
a new update is added via `ref_transaction_add_update`, so manual
additions in reference backends are dropped.
- Modify the backends to use this field internally as needed. The
backends need to check if an update for refname already exists when
splitting symrefs or adding an update for 'HEAD'.
- In the reftable backend, within `reftable_be_transaction_prepare()`,
move the `string_list_has_string()` check above
`ref_transaction_add_update()`. Since `ref_transaction_add_update()`
automatically adds the refname to `transaction->refnames`,
performing the check after will always return true, so we perform
the check before adding the update.
This helps reduce duplication of functionality between the backends and
makes it easier to make changes in a more centralized manner.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 17 +++++++++++++
refs/files-backend.c | 67 +++++++++++--------------------------------------
refs/packed-backend.c | 25 +-----------------
refs/refs-internal.h | 2 ++
refs/reftable-backend.c | 54 +++++++++++++--------------------------
5 files changed, 51 insertions(+), 114 deletions(-)
diff --git a/refs.c b/refs.c
index 54fd5ce21e..ab69746947 100644
--- a/refs.c
+++ b/refs.c
@@ -1175,6 +1175,7 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,
CALLOC_ARRAY(tr, 1);
tr->ref_store = refs;
tr->flags = flags;
+ string_list_init_dup(&tr->refnames);
return tr;
}
@@ -1205,6 +1206,7 @@ void ref_transaction_free(struct ref_transaction *transaction)
free((char *)transaction->updates[i]->old_target);
free(transaction->updates[i]);
}
+ string_list_clear(&transaction->refnames, 0);
free(transaction->updates);
free(transaction);
}
@@ -1218,6 +1220,7 @@ struct ref_update *ref_transaction_add_update(
const char *committer_info,
const char *msg)
{
+ struct string_list_item *item;
struct ref_update *update;
if (transaction->state != REF_TRANSACTION_OPEN)
@@ -1245,6 +1248,16 @@ struct ref_update *ref_transaction_add_update(
update->msg = normalize_reflog_message(msg);
}
+ /*
+ * This list is generally used by the backends to avoid duplicates.
+ * But we do support multiple log updates for a given refname within
+ * a single transaction.
+ */
+ if (!(update->flags & REF_LOG_ONLY)) {
+ item = string_list_append(&transaction->refnames, refname);
+ item->util = update;
+ }
+
return update;
}
@@ -2405,6 +2418,10 @@ int ref_transaction_prepare(struct ref_transaction *transaction,
return -1;
}
+ string_list_sort(&transaction->refnames);
+ if (ref_update_reject_duplicates(&transaction->refnames, err))
+ return TRANSACTION_GENERIC_ERROR;
+
ret = refs->be->transaction_prepare(refs, transaction, err);
if (ret)
return ret;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 6c7df30738..85ed85ad87 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2378,9 +2378,7 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st
*/
static int split_head_update(struct ref_update *update,
struct ref_transaction *transaction,
- const char *head_ref,
- struct string_list *affected_refnames,
- struct strbuf *err)
+ const char *head_ref, struct strbuf *err)
{
struct ref_update *new_update;
@@ -2398,7 +2396,7 @@ static int split_head_update(struct ref_update *update,
* transaction. This check is O(lg N) in the transaction
* size, but it happens at most once per transaction.
*/
- if (string_list_has_string(affected_refnames, "HEAD")) {
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
/* An entry already existed */
strbuf_addf(err,
"multiple updates for 'HEAD' (including one "
@@ -2420,7 +2418,6 @@ static int split_head_update(struct ref_update *update,
*/
if (strcmp(new_update->refname, "HEAD"))
BUG("%s unexpectedly not 'HEAD'", new_update->refname);
- string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2436,7 +2433,6 @@ static int split_head_update(struct ref_update *update,
static int split_symref_update(struct ref_update *update,
const char *referent,
struct ref_transaction *transaction,
- struct string_list *affected_refnames,
struct strbuf *err)
{
struct ref_update *new_update;
@@ -2448,7 +2444,7 @@ static int split_symref_update(struct ref_update *update,
* size, but it happens at most once per symref in a
* transaction.
*/
- if (string_list_has_string(affected_refnames, referent)) {
+ if (string_list_has_string(&transaction->refnames, referent)) {
/* An entry already exists */
strbuf_addf(err,
"multiple updates for '%s' (including one "
@@ -2486,15 +2482,6 @@ static int split_symref_update(struct ref_update *update,
update->flags |= REF_LOG_ONLY | REF_NO_DEREF;
update->flags &= ~REF_HAVE_OLD;
- /*
- * Add the referent. This insertion is O(N) in the transaction
- * size, but it happens at most once per symref in a
- * transaction. Make sure to add new_update->refname, which will
- * be valid as long as affected_refnames is in use, and NOT
- * referent, which might soon be freed by our caller.
- */
- string_list_insert(affected_refnames, new_update->refname);
-
return 0;
}
@@ -2558,7 +2545,6 @@ static int lock_ref_for_update(struct files_ref_store *refs,
struct ref_transaction *transaction,
const char *head_ref,
struct string_list *refnames_to_check,
- struct string_list *affected_refnames,
struct strbuf *err)
{
struct strbuf referent = STRBUF_INIT;
@@ -2575,8 +2561,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
update->flags |= REF_DELETING;
if (head_ref) {
- ret = split_head_update(update, transaction, head_ref,
- affected_refnames, err);
+ ret = split_head_update(update, transaction, head_ref, err);
if (ret)
goto out;
}
@@ -2586,9 +2571,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,
lock->count++;
} else {
ret = lock_raw_ref(refs, update->refname, mustexist,
- refnames_to_check, affected_refnames,
- &lock, &referent,
- &update->type, err);
+ refnames_to_check, &transaction->refnames,
+ &lock, &referent, &update->type, err);
if (ret) {
char *reason;
@@ -2642,9 +2626,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,
* of processing the split-off update, so we
* don't have to do it here.
*/
- ret = split_symref_update(update,
- referent.buf, transaction,
- affected_refnames, err);
+ ret = split_symref_update(update, referent.buf,
+ transaction, err);
if (ret)
goto out;
}
@@ -2799,7 +2782,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
"ref_transaction_prepare");
size_t i;
int ret = 0;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
struct string_list refnames_to_check = STRING_LIST_INIT_NODUP;
char *head_ref = NULL;
int head_type;
@@ -2818,12 +2800,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
transaction->backend_data = backend_data;
/*
- * Fail if a refname appears more than once in the
- * transaction. (If we end up splitting up any updates using
- * split_symref_update() or split_head_update(), those
- * functions will check that the new updates don't have the
- * same refname as any existing ones.) Also fail if any of the
- * updates use REF_IS_PRUNING without REF_NO_DEREF.
+ * Fail if any of the updates use REF_IS_PRUNING without REF_NO_DEREF.
*/
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
@@ -2831,16 +2808,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
if ((update->flags & REF_IS_PRUNING) &&
!(update->flags & REF_NO_DEREF))
BUG("REF_IS_PRUNING set without REF_NO_DEREF");
-
- if (update->flags & REF_LOG_ONLY)
- continue;
-
- string_list_append(&affected_refnames, update->refname);
- }
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
}
/*
@@ -2882,7 +2849,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
ret = lock_ref_for_update(refs, update, transaction,
head_ref, &refnames_to_check,
- &affected_refnames, err);
+ err);
if (ret)
goto cleanup;
@@ -2929,7 +2896,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
* So instead, we accept the race for now.
*/
if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
- &affected_refnames, NULL, 0, err)) {
+ &transaction->refnames, NULL, 0, err)) {
ret = TRANSACTION_NAME_CONFLICT;
goto cleanup;
}
@@ -2975,7 +2942,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
cleanup:
free(head_ref);
- string_list_clear(&affected_refnames, 0);
string_list_clear(&refnames_to_check, 0);
if (ret)
@@ -3050,13 +3016,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (transaction->state != REF_TRANSACTION_PREPARED)
BUG("commit called for transaction that is not prepared");
- /* Fail if a refname appears more than once in the transaction: */
- for (i = 0; i < transaction->nr; i++)
- if (!(transaction->updates[i]->flags & REF_LOG_ONLY))
- string_list_append(&affected_refnames,
- transaction->updates[i]->refname);
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
+ string_list_sort(&transaction->refnames);
+ if (ref_update_reject_duplicates(&transaction->refnames, err)) {
ret = TRANSACTION_GENERIC_ERROR;
goto cleanup;
}
@@ -3074,7 +3035,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
* that we are creating already exists.
*/
if (refs_for_each_rawref(&refs->base, ref_present,
- &affected_refnames))
+ &transaction->refnames))
BUG("initial ref transaction called with existing refs");
packed_transaction = ref_store_transaction_begin(refs->packed_ref_store,
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index f4c82ba2c7..19220d2e99 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1622,8 +1622,6 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
struct packed_transaction_backend_data {
/* True iff the transaction owns the packed-refs lock. */
int own_lock;
-
- struct string_list updates;
};
static void packed_transaction_cleanup(struct packed_ref_store *refs,
@@ -1632,8 +1630,6 @@ static void packed_transaction_cleanup(struct packed_ref_store *refs,
struct packed_transaction_backend_data *data = transaction->backend_data;
if (data) {
- string_list_clear(&data->updates, 0);
-
if (is_tempfile_active(refs->tempfile))
delete_tempfile(&refs->tempfile);
@@ -1658,7 +1654,6 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
"ref_transaction_prepare");
struct packed_transaction_backend_data *data;
- size_t i;
int ret = TRANSACTION_GENERIC_ERROR;
/*
@@ -1671,34 +1666,16 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
*/
CALLOC_ARRAY(data, 1);
- string_list_init_nodup(&data->updates);
transaction->backend_data = data;
- /*
- * Stick the updates in a string list by refname so that we
- * can sort them:
- */
- for (i = 0; i < transaction->nr; i++) {
- struct ref_update *update = transaction->updates[i];
- struct string_list_item *item =
- string_list_append(&data->updates, update->refname);
-
- /* Store a pointer to update in item->util: */
- item->util = update;
- }
- string_list_sort(&data->updates);
-
- if (ref_update_reject_duplicates(&data->updates, err))
- goto failure;
-
if (!is_lock_file_locked(&refs->lock)) {
if (packed_refs_lock(ref_store, 0, err))
goto failure;
data->own_lock = 1;
}
- if (write_with_updates(refs, &data->updates, err))
+ if (write_with_updates(refs, &transaction->refnames, err))
goto failure;
transaction->state = REF_TRANSACTION_PREPARED;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index e5862757a7..92db793026 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -3,6 +3,7 @@
#include "refs.h"
#include "iterator.h"
+#include "string-list.h"
struct fsck_options;
struct ref_transaction;
@@ -198,6 +199,7 @@ enum ref_transaction_state {
struct ref_transaction {
struct ref_store *ref_store;
struct ref_update **updates;
+ struct string_list refnames;
size_t alloc;
size_t nr;
enum ref_transaction_state state;
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 441b8c69c1..f616d9aabe 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1076,7 +1076,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
struct reftable_ref_store *refs =
reftable_be_downcast(ref_store, REF_STORE_WRITE|REF_STORE_MAIN, "ref_transaction_prepare");
struct strbuf referent = STRBUF_INIT, head_referent = STRBUF_INIT;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
struct string_list refnames_to_check = STRING_LIST_INIT_NODUP;
struct reftable_transaction_data *tx_data = NULL;
struct reftable_backend *be;
@@ -1101,10 +1100,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->updates[i], err);
if (ret)
goto done;
-
- if (!(transaction->updates[i]->flags & REF_LOG_ONLY))
- string_list_append(&affected_refnames,
- transaction->updates[i]->refname);
}
/*
@@ -1116,17 +1111,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
tx_data->args[i].updates_alloc = tx_data->args[i].updates_expected;
}
- /*
- * Fail if a refname appears more than once in the transaction.
- * This code is taken from the files backend and is a good candidate to
- * be moved into the generic layer.
- */
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto done;
- }
-
/*
* TODO: it's dubious whether we should reload the stack that "HEAD"
* belongs to or not. In theory, it may happen that we only modify
@@ -1194,14 +1178,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
!(u->flags & REF_LOG_ONLY) &&
!(u->flags & REF_UPDATE_VIA_HEAD) &&
!strcmp(rewritten_ref, head_referent.buf)) {
- struct ref_update *new_update;
-
/*
* First make sure that HEAD is not already in the
* transaction. This check is O(lg N) in the transaction
* size, but it happens at most once per transaction.
*/
- if (string_list_has_string(&affected_refnames, "HEAD")) {
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
/* An entry already existed */
strbuf_addf(err,
_("multiple updates for 'HEAD' (including one "
@@ -1211,12 +1193,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
goto done;
}
- new_update = ref_transaction_add_update(
- transaction, "HEAD",
- u->flags | REF_LOG_ONLY | REF_NO_DEREF,
- &u->new_oid, &u->old_oid, NULL, NULL, NULL,
- u->msg);
- string_list_insert(&affected_refnames, new_update->refname);
+ ref_transaction_add_update(
+ transaction, "HEAD",
+ u->flags | REF_LOG_ONLY | REF_NO_DEREF,
+ &u->new_oid, &u->old_oid, NULL, NULL, NULL,
+ u->msg);
}
ret = reftable_backend_read_ref(be, rewritten_ref,
@@ -1281,6 +1262,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
if (!strcmp(rewritten_ref, "HEAD"))
new_flags |= REF_UPDATE_VIA_HEAD;
+ if (string_list_has_string(&transaction->refnames, referent.buf)) {
+ strbuf_addf(err,
+ _("multiple updates for '%s' (including one "
+ "via symref '%s') are not allowed"),
+ referent.buf, u->refname);
+ ret = TRANSACTION_NAME_CONFLICT;
+ goto done;
+ }
+
/*
* If we are updating a symref (eg. HEAD), we should also
* update the branch that the symref points to.
@@ -1305,16 +1295,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
*/
u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
u->flags &= ~REF_HAVE_OLD;
-
- if (string_list_has_string(&affected_refnames, new_update->refname)) {
- strbuf_addf(err,
- _("multiple updates for '%s' (including one "
- "via symref '%s') are not allowed"),
- referent.buf, u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
- string_list_insert(&affected_refnames, new_update->refname);
}
}
@@ -1384,7 +1364,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
}
string_list_sort(&refnames_to_check);
- ret = refs_verify_refnames_available(ref_store, &refnames_to_check, &affected_refnames, NULL,
+ ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
+ &transaction->refnames, NULL,
transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
err);
if (ret < 0)
@@ -1402,7 +1383,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
strbuf_addf(err, _("reftable: transaction prepare: %s"),
reftable_error_str(ret));
}
- string_list_clear(&affected_refnames, 0);
strbuf_release(&referent);
strbuf_release(&head_referent);
string_list_clear(&refnames_to_check, 0);
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v3 3/8] refs/files: remove duplicate duplicates check
2025-03-05 17:38 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Karthik Nayak
2025-03-05 17:38 ` [PATCH v3 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
2025-03-05 17:38 ` [PATCH v3 2/8] refs: move duplicate refname update check to generic layer Karthik Nayak
@ 2025-03-05 17:38 ` Karthik Nayak
2025-03-05 17:38 ` [PATCH v3 4/8] refs/reftable: extract code from the transaction preparation Karthik Nayak
` (5 subsequent siblings)
8 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-05 17:38 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Within the files reference backend's transaction's 'finish' phase, a
verification step is currently performed wherein the refnames list is
sorted and examined for multiple updates targeting the same refname.
It has been observed that this verification is redundant, as an
identical check is already executed during the transaction's 'prepare'
stage. Since the refnames list remains unmodified following the
'prepare' stage, this secondary verification can be safely eliminated.
The duplicate check has been removed accordingly, and the
`ref_update_reject_duplicates()` function has been marked as static, as
its usage is now confined to 'refs.c'.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 9 +++++++--
refs/files-backend.c | 6 ------
refs/refs-internal.h | 8 --------
3 files changed, 7 insertions(+), 16 deletions(-)
diff --git a/refs.c b/refs.c
index ab69746947..69f385f344 100644
--- a/refs.c
+++ b/refs.c
@@ -2303,8 +2303,13 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
return ret;
}
-int ref_update_reject_duplicates(struct string_list *refnames,
- struct strbuf *err)
+/*
+ * Write an error to `err` and return a nonzero value iff the same
+ * refname appears multiple times in `refnames`. `refnames` must be
+ * sorted on entry to this function.
+ */
+static int ref_update_reject_duplicates(struct string_list *refnames,
+ struct strbuf *err)
{
size_t i, n = refnames->nr;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 85ed85ad87..7c6a0b3478 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3016,12 +3016,6 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (transaction->state != REF_TRANSACTION_PREPARED)
BUG("commit called for transaction that is not prepared");
- string_list_sort(&transaction->refnames);
- if (ref_update_reject_duplicates(&transaction->refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
- }
-
/*
* It's really undefined to call this function in an active
* repository or when there are existing references: we are
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 92db793026..6d3770d0cc 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -142,14 +142,6 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
struct object_id *oid, struct strbuf *referent,
unsigned int *type, int *failure_errno);
-/*
- * Write an error to `err` and return a nonzero value iff the same
- * refname appears multiple times in `refnames`. `refnames` must be
- * sorted on entry to this function.
- */
-int ref_update_reject_duplicates(struct string_list *refnames,
- struct strbuf *err);
-
/*
* Add a ref_update with the specified properties to transaction, and
* return a pointer to the new object. This function does not verify
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v3 4/8] refs/reftable: extract code from the transaction preparation
2025-03-05 17:38 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Karthik Nayak
` (2 preceding siblings ...)
2025-03-05 17:38 ` [PATCH v3 3/8] refs/files: remove duplicate duplicates check Karthik Nayak
@ 2025-03-05 17:38 ` Karthik Nayak
2025-03-05 17:39 ` [PATCH v3 5/8] refs: introduce enum-based transaction error types Karthik Nayak
` (4 subsequent siblings)
8 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-05 17:38 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Extract the core logic for preparing individual reference updates from
`reftable_be_transaction_prepare()` into `prepare_single_update()`. This
dedicated function now handles all validation and preparation steps for
each reference update in the transaction, including object ID
verification, HEAD reference handling, and symref processing.
The refactoring consolidates all reference update validation into a
single logical block, which improves code maintainability and
readability. More importantly, this restructuring lays the groundwork
for implementing partial transaction support in the reftable backend,
which will be introduced in the following commit.
No functional changes are included in this commit - it is purely a code
reorganization to support future enhancements.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs/reftable-backend.c | 463 +++++++++++++++++++++++++-----------------------
1 file changed, 237 insertions(+), 226 deletions(-)
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index f616d9aabe..2c1e2995de 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1069,6 +1069,239 @@ static int queue_transaction_update(struct reftable_ref_store *refs,
return 0;
}
+static int prepare_single_update(struct reftable_ref_store *refs,
+ struct reftable_transaction_data *tx_data,
+ struct ref_transaction *transaction,
+ struct reftable_backend *be,
+ struct ref_update *u,
+ struct string_list *refnames_to_check,
+ unsigned int head_type,
+ struct strbuf *head_referent,
+ struct strbuf *referent,
+ struct strbuf *err)
+{
+ struct object_id current_oid = {0};
+ const char *rewritten_ref;
+ int ret = 0;
+
+ /*
+ * There is no need to reload the respective backends here as
+ * we have already reloaded them when preparing the transaction
+ * update. And given that the stacks have been locked there
+ * shouldn't have been any concurrent modifications of the
+ * stack.
+ */
+ ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
+ if (ret)
+ return ret;
+
+ /* Verify that the new object ID is valid. */
+ if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
+ !(u->flags & REF_SKIP_OID_VERIFICATION) &&
+ !(u->flags & REF_LOG_ONLY)) {
+ struct object *o = parse_object(refs->base.repo, &u->new_oid);
+ if (!o) {
+ strbuf_addf(err,
+ _("trying to write ref '%s' with nonexistent object %s"),
+ u->refname, oid_to_hex(&u->new_oid));
+ return -1;
+ }
+
+ if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
+ strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
+ oid_to_hex(&u->new_oid), u->refname);
+ return -1;
+ }
+ }
+
+ /*
+ * When we update the reference that HEAD points to we enqueue
+ * a second log-only update for HEAD so that its reflog is
+ * updated accordingly.
+ */
+ if (head_type == REF_ISSYMREF &&
+ !(u->flags & REF_LOG_ONLY) &&
+ !(u->flags & REF_UPDATE_VIA_HEAD) &&
+ !strcmp(rewritten_ref, head_referent->buf)) {
+ /*
+ * First make sure that HEAD is not already in the
+ * transaction. This check is O(lg N) in the transaction
+ * size, but it happens at most once per transaction.
+ */
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
+ /* An entry already existed */
+ strbuf_addf(err,
+ _("multiple updates for 'HEAD' (including one "
+ "via its referent '%s') are not allowed"),
+ u->refname);
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ ref_transaction_add_update(
+ transaction, "HEAD",
+ u->flags | REF_LOG_ONLY | REF_NO_DEREF,
+ &u->new_oid, &u->old_oid, NULL, NULL, NULL,
+ u->msg);
+ }
+
+ ret = reftable_backend_read_ref(be, rewritten_ref,
+ ¤t_oid, referent, &u->type);
+ if (ret < 0)
+ return ret;
+ if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
+ /*
+ * The reference does not exist, and we either have no
+ * old object ID or expect the reference to not exist.
+ * We can thus skip below safety checks as well as the
+ * symref splitting. But we do want to verify that
+ * there is no conflicting reference here so that we
+ * can output a proper error message instead of failing
+ * at a later point.
+ */
+ string_list_append(refnames_to_check, u->refname);
+
+ /*
+ * There is no need to write the reference deletion
+ * when the reference in question doesn't exist.
+ */
+ if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
+ ret = queue_transaction_update(refs, tx_data, u,
+ ¤t_oid, err);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+ }
+ if (ret > 0) {
+ /* The reference does not exist, but we expected it to. */
+ strbuf_addf(err, _("cannot lock ref '%s': "
+
+
+ "unable to resolve reference '%s'"),
+ ref_update_original_update_refname(u), u->refname);
+ return -1;
+ }
+
+ if (u->type & REF_ISSYMREF) {
+ /*
+ * The reftable stack is locked at this point already,
+ * so it is safe to call `refs_resolve_ref_unsafe()`
+ * here without causing races.
+ */
+ const char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,
+ ¤t_oid, NULL);
+
+ if (u->flags & REF_NO_DEREF) {
+ if (u->flags & REF_HAVE_OLD && !resolved) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "error reading reference"), u->refname);
+ return -1;
+ }
+ } else {
+ struct ref_update *new_update;
+ int new_flags;
+
+ new_flags = u->flags;
+ if (!strcmp(rewritten_ref, "HEAD"))
+ new_flags |= REF_UPDATE_VIA_HEAD;
+
+ if (string_list_has_string(&transaction->refnames, referent->buf)) {
+ strbuf_addf(err,
+ _("multiple updates for '%s' (including one "
+ "via symref '%s') are not allowed"),
+ referent->buf, u->refname);
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ /*
+ * If we are updating a symref (eg. HEAD), we should also
+ * update the branch that the symref points to.
+ *
+ * This is generic functionality, and would be better
+ * done in refs.c, but the current implementation is
+ * intertwined with the locking in files-backend.c.
+ */
+ new_update = ref_transaction_add_update(
+ transaction, referent->buf, new_flags,
+ u->new_target ? NULL : &u->new_oid,
+ u->old_target ? NULL : &u->old_oid,
+ u->new_target, u->old_target,
+ u->committer_info, u->msg);
+
+ new_update->parent_update = u;
+
+ /*
+ * Change the symbolic ref update to log only. Also, it
+ * doesn't need to check its old OID value, as that will be
+ * done when new_update is processed.
+ */
+ u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
+ u->flags &= ~REF_HAVE_OLD;
+ }
+ }
+
+ /*
+ * Verify that the old object matches our expectations. Note
+ * that the error messages here do not make a lot of sense in
+ * the context of the reftable backend as we never lock
+ * individual refs. But the error messages match what the files
+ * backend returns, which keeps our tests happy.
+ */
+ if (u->old_target) {
+ if (!(u->type & REF_ISSYMREF)) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "expected symref with target '%s': "
+ "but is a regular ref"),
+ ref_update_original_update_refname(u),
+ u->old_target);
+ return -1;
+ }
+
+ if (ref_update_check_old_target(referent->buf, u, err)) {
+ return -1;
+ }
+ } else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
+ if (is_null_oid(&u->old_oid)) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "reference already exists"),
+ ref_update_original_update_refname(u));
+ return TRANSACTION_CREATE_EXISTS;
+ }
+ else if (is_null_oid(¤t_oid))
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "reference is missing but expected %s"),
+ ref_update_original_update_refname(u),
+ oid_to_hex(&u->old_oid));
+ else
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "is at %s but expected %s"),
+ ref_update_original_update_refname(u),
+ oid_to_hex(¤t_oid),
+ oid_to_hex(&u->old_oid));
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ /*
+ * If all of the following conditions are true:
+ *
+ * - We're not about to write a symref.
+ * - We're not about to write a log-only entry.
+ * - Old and new object ID are different.
+ *
+ * Then we're essentially doing a no-op update that can be
+ * skipped. This is not only for the sake of efficiency, but
+ * also skips writing unneeded reflog entries.
+ */
+ if ((u->type & REF_ISSYMREF) ||
+ (u->flags & REF_LOG_ONLY) ||
+ (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))
+ return queue_transaction_update(refs, tx_data, u,
+ ¤t_oid, err);
+
+ return 0;
+}
+
static int reftable_be_transaction_prepare(struct ref_store *ref_store,
struct ref_transaction *transaction,
struct strbuf *err)
@@ -1133,234 +1366,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
ret = 0;
for (i = 0; i < transaction->nr; i++) {
- struct ref_update *u = transaction->updates[i];
- struct object_id current_oid = {0};
- const char *rewritten_ref;
-
- /*
- * There is no need to reload the respective backends here as
- * we have already reloaded them when preparing the transaction
- * update. And given that the stacks have been locked there
- * shouldn't have been any concurrent modifications of the
- * stack.
- */
- ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
+ ret = prepare_single_update(refs, tx_data, transaction, be,
+ transaction->updates[i],
+ &refnames_to_check, head_type,
+ &head_referent, &referent, err);
if (ret)
goto done;
-
- /* Verify that the new object ID is valid. */
- if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
- !(u->flags & REF_SKIP_OID_VERIFICATION) &&
- !(u->flags & REF_LOG_ONLY)) {
- struct object *o = parse_object(refs->base.repo, &u->new_oid);
- if (!o) {
- strbuf_addf(err,
- _("trying to write ref '%s' with nonexistent object %s"),
- u->refname, oid_to_hex(&u->new_oid));
- ret = -1;
- goto done;
- }
-
- if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
- strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
- oid_to_hex(&u->new_oid), u->refname);
- ret = -1;
- goto done;
- }
- }
-
- /*
- * When we update the reference that HEAD points to we enqueue
- * a second log-only update for HEAD so that its reflog is
- * updated accordingly.
- */
- if (head_type == REF_ISSYMREF &&
- !(u->flags & REF_LOG_ONLY) &&
- !(u->flags & REF_UPDATE_VIA_HEAD) &&
- !strcmp(rewritten_ref, head_referent.buf)) {
- /*
- * First make sure that HEAD is not already in the
- * transaction. This check is O(lg N) in the transaction
- * size, but it happens at most once per transaction.
- */
- if (string_list_has_string(&transaction->refnames, "HEAD")) {
- /* An entry already existed */
- strbuf_addf(err,
- _("multiple updates for 'HEAD' (including one "
- "via its referent '%s') are not allowed"),
- u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
-
- ref_transaction_add_update(
- transaction, "HEAD",
- u->flags | REF_LOG_ONLY | REF_NO_DEREF,
- &u->new_oid, &u->old_oid, NULL, NULL, NULL,
- u->msg);
- }
-
- ret = reftable_backend_read_ref(be, rewritten_ref,
- ¤t_oid, &referent, &u->type);
- if (ret < 0)
- goto done;
- if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
- /*
- * The reference does not exist, and we either have no
- * old object ID or expect the reference to not exist.
- * We can thus skip below safety checks as well as the
- * symref splitting. But we do want to verify that
- * there is no conflicting reference here so that we
- * can output a proper error message instead of failing
- * at a later point.
- */
- string_list_append(&refnames_to_check, u->refname);
-
- /*
- * There is no need to write the reference deletion
- * when the reference in question doesn't exist.
- */
- if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
- ret = queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
- if (ret)
- goto done;
- }
-
- continue;
- }
- if (ret > 0) {
- /* The reference does not exist, but we expected it to. */
- strbuf_addf(err, _("cannot lock ref '%s': "
- "unable to resolve reference '%s'"),
- ref_update_original_update_refname(u), u->refname);
- ret = -1;
- goto done;
- }
-
- if (u->type & REF_ISSYMREF) {
- /*
- * The reftable stack is locked at this point already,
- * so it is safe to call `refs_resolve_ref_unsafe()`
- * here without causing races.
- */
- const char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,
- ¤t_oid, NULL);
-
- if (u->flags & REF_NO_DEREF) {
- if (u->flags & REF_HAVE_OLD && !resolved) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "error reading reference"), u->refname);
- ret = -1;
- goto done;
- }
- } else {
- struct ref_update *new_update;
- int new_flags;
-
- new_flags = u->flags;
- if (!strcmp(rewritten_ref, "HEAD"))
- new_flags |= REF_UPDATE_VIA_HEAD;
-
- if (string_list_has_string(&transaction->refnames, referent.buf)) {
- strbuf_addf(err,
- _("multiple updates for '%s' (including one "
- "via symref '%s') are not allowed"),
- referent.buf, u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
-
- /*
- * If we are updating a symref (eg. HEAD), we should also
- * update the branch that the symref points to.
- *
- * This is generic functionality, and would be better
- * done in refs.c, but the current implementation is
- * intertwined with the locking in files-backend.c.
- */
- new_update = ref_transaction_add_update(
- transaction, referent.buf, new_flags,
- u->new_target ? NULL : &u->new_oid,
- u->old_target ? NULL : &u->old_oid,
- u->new_target, u->old_target,
- u->committer_info, u->msg);
-
- new_update->parent_update = u;
-
- /*
- * Change the symbolic ref update to log only. Also, it
- * doesn't need to check its old OID value, as that will be
- * done when new_update is processed.
- */
- u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
- u->flags &= ~REF_HAVE_OLD;
- }
- }
-
- /*
- * Verify that the old object matches our expectations. Note
- * that the error messages here do not make a lot of sense in
- * the context of the reftable backend as we never lock
- * individual refs. But the error messages match what the files
- * backend returns, which keeps our tests happy.
- */
- if (u->old_target) {
- if (!(u->type & REF_ISSYMREF)) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "expected symref with target '%s': "
- "but is a regular ref"),
- ref_update_original_update_refname(u),
- u->old_target);
- ret = -1;
- goto done;
- }
-
- if (ref_update_check_old_target(referent.buf, u, err)) {
- ret = -1;
- goto done;
- }
- } else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
- ret = TRANSACTION_NAME_CONFLICT;
- if (is_null_oid(&u->old_oid)) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "reference already exists"),
- ref_update_original_update_refname(u));
- ret = TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(¤t_oid))
- strbuf_addf(err, _("cannot lock ref '%s': "
- "reference is missing but expected %s"),
- ref_update_original_update_refname(u),
- oid_to_hex(&u->old_oid));
- else
- strbuf_addf(err, _("cannot lock ref '%s': "
- "is at %s but expected %s"),
- ref_update_original_update_refname(u),
- oid_to_hex(¤t_oid),
- oid_to_hex(&u->old_oid));
- goto done;
- }
-
- /*
- * If all of the following conditions are true:
- *
- * - We're not about to write a symref.
- * - We're not about to write a log-only entry.
- * - Old and new object ID are different.
- *
- * Then we're essentially doing a no-op update that can be
- * skipped. This is not only for the sake of efficiency, but
- * also skips writing unneeded reflog entries.
- */
- if ((u->type & REF_ISSYMREF) ||
- (u->flags & REF_LOG_ONLY) ||
- (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid))) {
- ret = queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
- if (ret)
- goto done;
- }
}
string_list_sort(&refnames_to_check);
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v3 5/8] refs: introduce enum-based transaction error types
2025-03-05 17:38 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Karthik Nayak
` (3 preceding siblings ...)
2025-03-05 17:38 ` [PATCH v3 4/8] refs/reftable: extract code from the transaction preparation Karthik Nayak
@ 2025-03-05 17:39 ` Karthik Nayak
2025-03-05 17:39 ` [PATCH v3 6/8] refs: implement partial reference transaction support Karthik Nayak
` (3 subsequent siblings)
8 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-05 17:39 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Replace preprocessor-defined transaction errors with a strongly-typed
enum `ref_transaction_error`. This change:
- Improves type safety and function signature clarity.
- Makes error handling more explicit and discoverable.
- Maintains existing error cases, while adding new error cases for
common scenarios.
This refactoring paves the way for more comprehensive error handling
which we will utilize in the upcoming commits to add partial transaction
support.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
builtin/fetch.c | 2 +-
refs.c | 49 ++++++------
refs.h | 54 ++++++++-----
refs/files-backend.c | 202 ++++++++++++++++++++++++------------------------
refs/packed-backend.c | 23 +++---
refs/refs-internal.h | 5 +-
refs/reftable-backend.c | 64 +++++++--------
7 files changed, 213 insertions(+), 186 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 1c740d5aac..52c913d28a 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -687,7 +687,7 @@ static int s_update_ref(const char *action,
switch (ref_transaction_commit(our_transaction, &err)) {
case 0:
break;
- case TRANSACTION_NAME_CONFLICT:
+ case REF_TRANSACTION_ERROR_NAME_CONFLICT:
ret = STORE_REF_ERROR_DF_CONFLICT;
goto out;
default:
diff --git a/refs.c b/refs.c
index 69f385f344..63b8050ce2 100644
--- a/refs.c
+++ b/refs.c
@@ -2271,7 +2271,7 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
REF_NO_DEREF, logmsg, &err))
goto error_return;
prepret = ref_transaction_prepare(transaction, &err);
- if (prepret && prepret != TRANSACTION_CREATE_EXISTS)
+ if (prepret && prepret != REF_TRANSACTION_ERROR_CREATE_EXISTS)
goto error_return;
} else {
if (ref_transaction_update(transaction, ref, NULL, NULL,
@@ -2289,7 +2289,7 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
}
}
- if (prepret == TRANSACTION_CREATE_EXISTS)
+ if (prepret == REF_TRANSACTION_ERROR_CREATE_EXISTS)
goto cleanup;
if (ref_transaction_commit(transaction, &err))
@@ -2425,7 +2425,7 @@ int ref_transaction_prepare(struct ref_transaction *transaction,
string_list_sort(&transaction->refnames);
if (ref_update_reject_duplicates(&transaction->refnames, err))
- return TRANSACTION_GENERIC_ERROR;
+ return REF_TRANSACTION_ERROR_GENERIC;
ret = refs->be->transaction_prepare(refs, transaction, err);
if (ret)
@@ -2497,18 +2497,18 @@ int ref_transaction_commit(struct ref_transaction *transaction,
return ret;
}
-int refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *refnames,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err)
+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err)
{
struct strbuf dirname = STRBUF_INIT;
struct strbuf referent = STRBUF_INIT;
struct ref_iterator *iter = NULL;
struct strset dirnames;
- int ret = -1;
+ int ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
/*
* For the sake of comments in this function, suppose that
@@ -2624,12 +2624,13 @@ int refs_verify_refnames_available(struct ref_store *refs,
return ret;
}
-int refs_verify_refname_available(struct ref_store *refs,
- const char *refname,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err)
+enum ref_transaction_error refs_verify_refname_available(
+ struct ref_store *refs,
+ const char *refname,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err)
{
struct string_list_item item = { .string = (char *) refname };
struct string_list refnames = {
@@ -2817,8 +2818,9 @@ int ref_update_has_null_new_value(struct ref_update *update)
return !update->new_target && is_null_oid(&update->new_oid);
}
-int ref_update_check_old_target(const char *referent, struct ref_update *update,
- struct strbuf *err)
+enum ref_transaction_error ref_update_check_old_target(const char *referent,
+ struct ref_update *update,
+ struct strbuf *err)
{
if (!update->old_target)
BUG("called without old_target set");
@@ -2826,17 +2828,18 @@ int ref_update_check_old_target(const char *referent, struct ref_update *update,
if (!strcmp(referent, update->old_target))
return 0;
- if (!strcmp(referent, ""))
+ if (!strcmp(referent, "")) {
strbuf_addf(err, "verifying symref target: '%s': "
"reference is missing but expected %s",
ref_update_original_update_refname(update),
update->old_target);
- else
- strbuf_addf(err, "verifying symref target: '%s': "
- "is at %s but expected %s",
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ }
+
+ strbuf_addf(err, "verifying symref target: '%s': is at %s but expected %s",
ref_update_original_update_refname(update),
referent, update->old_target);
- return -1;
+ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
}
struct migration_data {
diff --git a/refs.h b/refs.h
index b14ba1f9ff..1b9213f9ce 100644
--- a/refs.h
+++ b/refs.h
@@ -16,6 +16,29 @@ struct worktree;
enum ref_storage_format ref_storage_format_by_name(const char *name);
const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);
+/*
+ * enum ref_transaction_error represents the following return codes:
+ * REF_TRANSACTION_ERROR_GENERIC error_code: default error code.
+ * REF_TRANSACTION_ERROR_NAME_CONFLICT error_code: ref name conflict like A vs A/B.
+ * REF_TRANSACTION_ERROR_CREATE_EXISTS error_code: ref to be created already exists.
+ * REF_TRANSACTION_ERROR_NONEXISTENT_REF error_code: ref expected but doesn't exist.
+ * REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE error_code: provided old_oid or old_target of
+ * reference doesn't match actual.
+ * REF_TRANSACTION_ERROR_INVALID_NEW_VALUE error_code: provided new_oid or new_target is
+ * invalid.
+ * REF_TRANSACTION_ERROR_EXPECTED_SYMREF error_code: expected ref to be symref, but is a
+ * regular ref.
+ */
+enum ref_transaction_error {
+ REF_TRANSACTION_ERROR_GENERIC = -1,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT = -2,
+ REF_TRANSACTION_ERROR_CREATE_EXISTS = -3,
+ REF_TRANSACTION_ERROR_NONEXISTENT_REF = -4,
+ REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE = -5,
+ REF_TRANSACTION_ERROR_INVALID_NEW_VALUE = -6,
+ REF_TRANSACTION_ERROR_EXPECTED_SYMREF = -7,
+};
+
/*
* Resolve a reference, recursively following symbolic references.
*
@@ -117,24 +140,24 @@ int refs_read_symbolic_ref(struct ref_store *ref_store, const char *refname,
*
* extras and skip must be sorted.
*/
-int refs_verify_refname_available(struct ref_store *refs,
- const char *refname,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
+enum ref_transaction_error refs_verify_refname_available(struct ref_store *refs,
+ const char *refname,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err);
/*
* Same as `refs_verify_refname_available()`, but checking for a list of
* refnames instead of only a single item. This is more efficient in the case
* where one needs to check multiple refnames.
*/
-int refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *refnames,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err);
int refs_ref_exists(struct ref_store *refs, const char *refname);
@@ -830,13 +853,6 @@ int ref_transaction_verify(struct ref_transaction *transaction,
unsigned int flags,
struct strbuf *err);
-/* Naming conflict (for example, the ref names A and A/B conflict). */
-#define TRANSACTION_NAME_CONFLICT -1
-/* When only creation was requested, but the ref already exists. */
-#define TRANSACTION_CREATE_EXISTS -2
-/* All other errors. */
-#define TRANSACTION_GENERIC_ERROR -3
-
/*
* Perform the preparatory stages of committing `transaction`. Acquire
* any needed locks, check preconditions, etc.; basically, do as much
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 7c6a0b3478..1e1663f44b 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -663,7 +663,7 @@ static void unlock_ref(struct ref_lock *lock)
* broken, lock the reference anyway but clear old_oid.
*
* Return 0 on success. On failure, write an error message to err and
- * return TRANSACTION_NAME_CONFLICT or TRANSACTION_GENERIC_ERROR.
+ * return REF_TRANSACTION_ERROR_NAME_CONFLICT or REF_TRANSACTION_ERROR_GENERIC.
*
* Implementation note: This function is basically
*
@@ -676,19 +676,20 @@ static void unlock_ref(struct ref_lock *lock)
* avoided, namely if we were successfully able to read the ref
* - Generate informative error messages in the case of failure
*/
-static int lock_raw_ref(struct files_ref_store *refs,
- const char *refname, int mustexist,
- struct string_list *refnames_to_check,
- const struct string_list *extras,
- struct ref_lock **lock_p,
- struct strbuf *referent,
- unsigned int *type,
- struct strbuf *err)
-{
+static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
+ const char *refname,
+ int mustexist,
+ struct string_list *refnames_to_check,
+ const struct string_list *extras,
+ struct ref_lock **lock_p,
+ struct strbuf *referent,
+ unsigned int *type,
+ struct strbuf *err)
+{
+ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
struct ref_lock *lock;
struct strbuf ref_file = STRBUF_INIT;
int attempts_remaining = 3;
- int ret = TRANSACTION_GENERIC_ERROR;
int failure_errno;
assert(err);
@@ -728,13 +729,14 @@ static int lock_raw_ref(struct files_ref_store *refs,
strbuf_reset(err);
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
} else {
/*
* The error message set by
* refs_verify_refname_available() is
* OK.
*/
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
} else {
/*
@@ -788,6 +790,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
/* Garden variety missing reference. */
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error_return;
} else {
/*
@@ -820,6 +823,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
/* Garden variety missing reference. */
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error_return;
} else if (remove_dir_recursively(&ref_file,
REMOVE_DIR_EMPTY_ONLY)) {
@@ -830,7 +834,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
* The error message set by
* verify_refname_available() is OK.
*/
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto error_return;
} else {
/*
@@ -1517,10 +1521,11 @@ static int rename_tmp_log(struct files_ref_store *refs, const char *newrefname)
return ret;
}
-static int write_ref_to_lockfile(struct files_ref_store *refs,
- struct ref_lock *lock,
- const struct object_id *oid,
- int skip_oid_verification, struct strbuf *err);
+static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs,
+ struct ref_lock *lock,
+ const struct object_id *oid,
+ int skip_oid_verification,
+ struct strbuf *err);
static int commit_ref_update(struct files_ref_store *refs,
struct ref_lock *lock,
const struct object_id *oid, const char *logmsg,
@@ -1926,10 +1931,11 @@ static int files_log_ref_write(struct files_ref_store *refs,
* Write oid into the open lockfile, then close the lockfile. On
* errors, rollback the lockfile, fill in *err and return -1.
*/
-static int write_ref_to_lockfile(struct files_ref_store *refs,
- struct ref_lock *lock,
- const struct object_id *oid,
- int skip_oid_verification, struct strbuf *err)
+static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs,
+ struct ref_lock *lock,
+ const struct object_id *oid,
+ int skip_oid_verification,
+ struct strbuf *err)
{
static char term = '\n';
struct object *o;
@@ -1943,7 +1949,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
"trying to write ref '%s' with nonexistent object %s",
lock->ref_name, oid_to_hex(oid));
unlock_ref(lock);
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
if (o->type != OBJ_COMMIT && is_branch(lock->ref_name)) {
strbuf_addf(
@@ -1951,7 +1957,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
"trying to write non-commit object %s to branch '%s'",
oid_to_hex(oid), lock->ref_name);
unlock_ref(lock);
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
}
fd = get_lock_file_fd(&lock->lk);
@@ -1962,7 +1968,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
strbuf_addf(err,
"couldn't write '%s'", get_lock_file_path(&lock->lk));
unlock_ref(lock);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
return 0;
}
@@ -2376,9 +2382,10 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st
* If update is a direct update of head_ref (the reference pointed to
* by HEAD), then add an extra REF_LOG_ONLY update for HEAD.
*/
-static int split_head_update(struct ref_update *update,
- struct ref_transaction *transaction,
- const char *head_ref, struct strbuf *err)
+static enum ref_transaction_error split_head_update(struct ref_update *update,
+ struct ref_transaction *transaction,
+ const char *head_ref,
+ struct strbuf *err)
{
struct ref_update *new_update;
@@ -2402,7 +2409,7 @@ static int split_head_update(struct ref_update *update,
"multiple updates for 'HEAD' (including one "
"via its referent '%s') are not allowed",
update->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
new_update = ref_transaction_add_update(
@@ -2430,10 +2437,10 @@ static int split_head_update(struct ref_update *update,
* Note that the new update will itself be subject to splitting when
* the iteration gets to it.
*/
-static int split_symref_update(struct ref_update *update,
- const char *referent,
- struct ref_transaction *transaction,
- struct strbuf *err)
+static enum ref_transaction_error split_symref_update(struct ref_update *update,
+ const char *referent,
+ struct ref_transaction *transaction,
+ struct strbuf *err)
{
struct ref_update *new_update;
unsigned int new_flags;
@@ -2450,7 +2457,7 @@ static int split_symref_update(struct ref_update *update,
"multiple updates for '%s' (including one "
"via symref '%s') are not allowed",
referent, update->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
new_flags = update->flags;
@@ -2491,11 +2498,10 @@ static int split_symref_update(struct ref_update *update,
* everything is OK, return 0; otherwise, write an error message to
* err and return -1.
*/
-static int check_old_oid(struct ref_update *update, struct object_id *oid,
- struct strbuf *err)
+static enum ref_transaction_error check_old_oid(struct ref_update *update,
+ struct object_id *oid,
+ struct strbuf *err)
{
- int ret = TRANSACTION_GENERIC_ERROR;
-
if (!(update->flags & REF_HAVE_OLD) ||
oideq(oid, &update->old_oid))
return 0;
@@ -2504,21 +2510,20 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid,
strbuf_addf(err, "cannot lock ref '%s': "
"reference already exists",
ref_update_original_update_refname(update));
- ret = TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(oid))
+ return REF_TRANSACTION_ERROR_CREATE_EXISTS;
+ } else if (is_null_oid(oid)) {
strbuf_addf(err, "cannot lock ref '%s': "
"reference is missing but expected %s",
ref_update_original_update_refname(update),
oid_to_hex(&update->old_oid));
- else
- strbuf_addf(err, "cannot lock ref '%s': "
- "is at %s but expected %s",
- ref_update_original_update_refname(update),
- oid_to_hex(oid),
- oid_to_hex(&update->old_oid));
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ }
- return ret;
+ strbuf_addf(err, "cannot lock ref '%s': is at %s but expected %s",
+ ref_update_original_update_refname(update), oid_to_hex(oid),
+ oid_to_hex(&update->old_oid));
+
+ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
}
struct files_transaction_backend_data {
@@ -2540,17 +2545,17 @@ struct files_transaction_backend_data {
* - If it is an update of head_ref, add a corresponding REF_LOG_ONLY
* update of HEAD.
*/
-static int lock_ref_for_update(struct files_ref_store *refs,
- struct ref_update *update,
- struct ref_transaction *transaction,
- const char *head_ref,
- struct string_list *refnames_to_check,
- struct strbuf *err)
+static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *refs,
+ struct ref_update *update,
+ struct ref_transaction *transaction,
+ const char *head_ref,
+ struct string_list *refnames_to_check,
+ struct strbuf *err)
{
struct strbuf referent = STRBUF_INIT;
int mustexist = ref_update_expects_existing_old_ref(update);
struct files_transaction_backend_data *backend_data;
- int ret = 0;
+ enum ref_transaction_error ret = 0;
struct ref_lock *lock;
files_assert_main_repository(refs, "lock_ref_for_update");
@@ -2602,22 +2607,17 @@ static int lock_ref_for_update(struct files_ref_store *refs,
strbuf_addf(err, "cannot lock ref '%s': "
"error reading reference",
ref_update_original_update_refname(update));
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
}
- if (update->old_target) {
- if (ref_update_check_old_target(referent.buf, update, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto out;
- }
- } else {
+ if (update->old_target)
+ ret = ref_update_check_old_target(referent.buf, update, err);
+ else
ret = check_old_oid(update, &lock->old_oid, err);
- if (ret) {
- goto out;
- }
- }
+ if (ret)
+ goto out;
} else {
/*
* Create a new update for the reference this
@@ -2644,7 +2644,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
"but is a regular ref"),
ref_update_original_update_refname(update),
update->old_target);
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_EXPECTED_SYMREF;
goto out;
} else {
ret = check_old_oid(update, &lock->old_oid, err);
@@ -2668,14 +2668,14 @@ static int lock_ref_for_update(struct files_ref_store *refs,
if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
if (create_symref_lock(lock, update->new_target, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
if (close_ref_gently(lock)) {
strbuf_addf(err, "couldn't close '%s.lock'",
update->refname);
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
@@ -2693,25 +2693,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
* The reference already has the desired
* value, so we don't need to write it.
*/
- } else if (write_ref_to_lockfile(
- refs, lock, &update->new_oid,
- update->flags & REF_SKIP_OID_VERIFICATION,
- err)) {
- char *write_err = strbuf_detach(err, NULL);
-
- /*
- * The lock was freed upon failure of
- * write_ref_to_lockfile():
- */
- update->backend_data = NULL;
- strbuf_addf(err,
- "cannot update ref '%s': %s",
- update->refname, write_err);
- free(write_err);
- ret = TRANSACTION_GENERIC_ERROR;
- goto out;
} else {
- update->flags |= REF_NEEDS_COMMIT;
+ ret = write_ref_to_lockfile(
+ refs, lock, &update->new_oid,
+ update->flags & REF_SKIP_OID_VERIFICATION,
+ err);
+ if (ret) {
+ char *write_err = strbuf_detach(err, NULL);
+
+ /*
+ * The lock was freed upon failure of
+ * write_ref_to_lockfile():
+ */
+ update->backend_data = NULL;
+ strbuf_addf(err,
+ "cannot update ref '%s': %s",
+ update->refname, write_err);
+ free(write_err);
+ goto out;
+ } else {
+ update->flags |= REF_NEEDS_COMMIT;
+ }
}
}
if (!(update->flags & REF_NEEDS_COMMIT)) {
@@ -2723,7 +2725,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
if (close_ref_gently(lock)) {
strbuf_addf(err, "couldn't close '%s.lock'",
update->refname);
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
}
@@ -2865,7 +2867,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
refs->packed_ref_store,
transaction->flags, err);
if (!packed_transaction) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
@@ -2897,13 +2899,13 @@ static int files_transaction_prepare(struct ref_store *ref_store,
*/
if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
&transaction->refnames, NULL, 0, err)) {
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
}
if (packed_transaction) {
if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
backend_data->packed_refs_locked = 1;
@@ -2934,7 +2936,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
*/
backend_data->packed_transaction = NULL;
if (ref_transaction_abort(packed_transaction, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3035,7 +3037,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
packed_transaction = ref_store_transaction_begin(refs->packed_ref_store,
transaction->flags, err);
if (!packed_transaction) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
@@ -3058,7 +3060,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (!loose_transaction) {
loose_transaction = ref_store_transaction_begin(&refs->base, 0, err);
if (!loose_transaction) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3083,19 +3085,19 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
}
if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
if (refs_verify_refnames_available(&refs->base, &refnames_to_check,
&affected_refnames, NULL, 1, err)) {
packed_refs_unlock(refs->packed_ref_store);
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
}
if (ref_transaction_commit(packed_transaction, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
packed_refs_unlock(refs->packed_ref_store);
@@ -3103,7 +3105,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (loose_transaction) {
if (ref_transaction_prepare(loose_transaction, err) ||
ref_transaction_commit(loose_transaction, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3152,7 +3154,7 @@ static int files_transaction_finish(struct ref_store *ref_store,
if (update->flags & REF_NEEDS_COMMIT ||
update->flags & REF_LOG_ONLY) {
if (parse_and_write_reflog(refs, update, lock, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3171,7 +3173,7 @@ static int files_transaction_finish(struct ref_store *ref_store,
strbuf_addf(err, "couldn't set '%s'", lock->ref_name);
unlock_ref(lock);
update->backend_data = NULL;
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3227,7 +3229,7 @@ static int files_transaction_finish(struct ref_store *ref_store,
strbuf_reset(&sb);
files_ref_path(refs, &sb, lock->ref_name);
if (unlink_or_msg(sb.buf, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 19220d2e99..5458952624 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1326,10 +1326,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
* The packfile must be locked before calling this function and will
* remain locked when it is done.
*/
-static int write_with_updates(struct packed_ref_store *refs,
- struct string_list *updates,
- struct strbuf *err)
+static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,
+ struct string_list *updates,
+ struct strbuf *err)
{
+ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
struct ref_iterator *iter = NULL;
size_t i;
int ok;
@@ -1353,7 +1354,7 @@ static int write_with_updates(struct packed_ref_store *refs,
strbuf_addf(err, "unable to create file %s: %s",
sb.buf, strerror(errno));
strbuf_release(&sb);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
strbuf_release(&sb);
@@ -1409,6 +1410,7 @@ static int write_with_updates(struct packed_ref_store *refs,
strbuf_addf(err, "cannot update ref '%s': "
"reference already exists",
update->refname);
+ ret = REF_TRANSACTION_ERROR_CREATE_EXISTS;
goto error;
} else if (!oideq(&update->old_oid, iter->oid)) {
strbuf_addf(err, "cannot update ref '%s': "
@@ -1416,6 +1418,7 @@ static int write_with_updates(struct packed_ref_store *refs,
update->refname,
oid_to_hex(iter->oid),
oid_to_hex(&update->old_oid));
+ ret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
goto error;
}
}
@@ -1452,6 +1455,7 @@ static int write_with_updates(struct packed_ref_store *refs,
"reference is missing but expected %s",
update->refname,
oid_to_hex(&update->old_oid));
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error;
}
}
@@ -1509,7 +1513,7 @@ static int write_with_updates(struct packed_ref_store *refs,
strerror(errno));
strbuf_release(&sb);
delete_tempfile(&refs->tempfile);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
return 0;
@@ -1521,7 +1525,7 @@ static int write_with_updates(struct packed_ref_store *refs,
error:
ref_iterator_free(iter);
delete_tempfile(&refs->tempfile);
- return -1;
+ return ret;
}
int is_packed_transaction_needed(struct ref_store *ref_store,
@@ -1654,7 +1658,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
"ref_transaction_prepare");
struct packed_transaction_backend_data *data;
- int ret = TRANSACTION_GENERIC_ERROR;
+ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
/*
* Note that we *don't* skip transactions with zero updates,
@@ -1675,7 +1679,8 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
data->own_lock = 1;
}
- if (write_with_updates(refs, &transaction->refnames, err))
+ ret = write_with_updates(refs, &transaction->refnames, err);
+ if (ret)
goto failure;
transaction->state = REF_TRANSACTION_PREPARED;
@@ -1707,7 +1712,7 @@ static int packed_transaction_finish(struct ref_store *ref_store,
ref_store,
REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
"ref_transaction_finish");
- int ret = TRANSACTION_GENERIC_ERROR;
+ int ret = REF_TRANSACTION_ERROR_GENERIC;
char *packed_refs_path;
clear_snapshot(refs);
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 6d3770d0cc..3f1d19abd9 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -770,8 +770,9 @@ int ref_update_has_null_new_value(struct ref_update *update);
* If everything is OK, return 0; otherwise, write an error message to
* err and return -1.
*/
-int ref_update_check_old_target(const char *referent, struct ref_update *update,
- struct strbuf *err);
+enum ref_transaction_error ref_update_check_old_target(const char *referent,
+ struct ref_update *update,
+ struct strbuf *err);
/*
* Check if the ref must exist, this means that the old_oid or
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 2c1e2995de..0132b8b06a 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1069,20 +1069,20 @@ static int queue_transaction_update(struct reftable_ref_store *refs,
return 0;
}
-static int prepare_single_update(struct reftable_ref_store *refs,
- struct reftable_transaction_data *tx_data,
- struct ref_transaction *transaction,
- struct reftable_backend *be,
- struct ref_update *u,
- struct string_list *refnames_to_check,
- unsigned int head_type,
- struct strbuf *head_referent,
- struct strbuf *referent,
- struct strbuf *err)
+static enum ref_transaction_error prepare_single_update(struct reftable_ref_store *refs,
+ struct reftable_transaction_data *tx_data,
+ struct ref_transaction *transaction,
+ struct reftable_backend *be,
+ struct ref_update *u,
+ struct string_list *refnames_to_check,
+ unsigned int head_type,
+ struct strbuf *head_referent,
+ struct strbuf *referent,
+ struct strbuf *err)
{
+ enum ref_transaction_error ret = 0;
struct object_id current_oid = {0};
const char *rewritten_ref;
- int ret = 0;
/*
* There is no need to reload the respective backends here as
@@ -1093,7 +1093,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
*/
ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
if (ret)
- return ret;
+ return REF_TRANSACTION_ERROR_GENERIC;
/* Verify that the new object ID is valid. */
if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
@@ -1104,13 +1104,13 @@ static int prepare_single_update(struct reftable_ref_store *refs,
strbuf_addf(err,
_("trying to write ref '%s' with nonexistent object %s"),
u->refname, oid_to_hex(&u->new_oid));
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
oid_to_hex(&u->new_oid), u->refname);
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
}
@@ -1134,7 +1134,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
_("multiple updates for 'HEAD' (including one "
"via its referent '%s') are not allowed"),
u->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
ref_transaction_add_update(
@@ -1147,7 +1147,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
ret = reftable_backend_read_ref(be, rewritten_ref,
¤t_oid, referent, &u->type);
if (ret < 0)
- return ret;
+ return REF_TRANSACTION_ERROR_GENERIC;
if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
/*
* The reference does not exist, and we either have no
@@ -1168,7 +1168,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
ret = queue_transaction_update(refs, tx_data, u,
¤t_oid, err);
if (ret)
- return ret;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
return 0;
@@ -1180,7 +1180,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
"unable to resolve reference '%s'"),
ref_update_original_update_refname(u), u->refname);
- return -1;
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
}
if (u->type & REF_ISSYMREF) {
@@ -1196,7 +1196,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
if (u->flags & REF_HAVE_OLD && !resolved) {
strbuf_addf(err, _("cannot lock ref '%s': "
"error reading reference"), u->refname);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
} else {
struct ref_update *new_update;
@@ -1211,7 +1211,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
_("multiple updates for '%s' (including one "
"via symref '%s') are not allowed"),
referent->buf, u->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
/*
@@ -1255,31 +1255,32 @@ static int prepare_single_update(struct reftable_ref_store *refs,
"but is a regular ref"),
ref_update_original_update_refname(u),
u->old_target);
- return -1;
+ return REF_TRANSACTION_ERROR_EXPECTED_SYMREF;
}
- if (ref_update_check_old_target(referent->buf, u, err)) {
- return -1;
- }
+ ret = ref_update_check_old_target(referent->buf, u, err);
+ if (ret)
+ return ret;
} else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
if (is_null_oid(&u->old_oid)) {
strbuf_addf(err, _("cannot lock ref '%s': "
"reference already exists"),
ref_update_original_update_refname(u));
- return TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(¤t_oid))
+ return REF_TRANSACTION_ERROR_CREATE_EXISTS;
+ } else if (is_null_oid(¤t_oid)) {
strbuf_addf(err, _("cannot lock ref '%s': "
"reference is missing but expected %s"),
ref_update_original_update_refname(u),
oid_to_hex(&u->old_oid));
- else
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ } else {
strbuf_addf(err, _("cannot lock ref '%s': "
"is at %s but expected %s"),
ref_update_original_update_refname(u),
oid_to_hex(¤t_oid),
oid_to_hex(&u->old_oid));
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
+ }
}
/*
@@ -1296,8 +1297,8 @@ static int prepare_single_update(struct reftable_ref_store *refs,
if ((u->type & REF_ISSYMREF) ||
(u->flags & REF_LOG_ONLY) ||
(u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))
- return queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
+ if (queue_transaction_update(refs, tx_data, u, ¤t_oid, err))
+ return REF_TRANSACTION_ERROR_GENERIC;
return 0;
}
@@ -1386,7 +1387,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->state = REF_TRANSACTION_PREPARED;
done:
- assert(ret != REFTABLE_API_ERROR);
if (ret < 0) {
free_transaction_data(tx_data);
transaction->state = REF_TRANSACTION_CLOSED;
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v3 6/8] refs: implement partial reference transaction support
2025-03-05 17:38 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Karthik Nayak
` (4 preceding siblings ...)
2025-03-05 17:39 ` [PATCH v3 5/8] refs: introduce enum-based transaction error types Karthik Nayak
@ 2025-03-05 17:39 ` Karthik Nayak
2025-03-07 19:50 ` Jeff King
2025-03-07 19:57 ` Jeff King
2025-03-05 17:39 ` [PATCH v3 7/8] refs: support partial update rejections during F/D checks Karthik Nayak
` (2 subsequent siblings)
8 siblings, 2 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-05 17:39 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Git's reference transactions are all-or-nothing: either all updates
succeed, or none do. While this atomic behavior is generally desirable,
it can be suboptimal especially when using the reftable backend, where
batching multiple reference updates into a single transaction is more
efficient than performing them sequentially.
Introduce partial transaction support with a new flag,
'REF_TRANSACTION_ALLOW_PARTIAL'. When enabled, this flag allows
individual reference updates that would typically cause the entire
transaction to fail due to non-system-related errors to be marked as
rejected while permitting other updates to proceed. System errors
referred by 'REF_TRANSACTION_ERROR_GENERIC' continue to result in the
entire transaction failing. This approach enhances flexibility while
preserving transactional integrity where necessary.
The implementation introduces several key components:
- Add 'rejection_err' field to struct `ref_update` to track failed
updates with failure reason.
- Add a new struct `ref_transaction_rejections` and a field within
`ref_transaction` to this struct to allow quick iteration over
rejected updates.
- Modify reference backends (files, packed, reftable) to handle
partial transactions by using `ref_transaction_set_rejected()`
instead of failing the entire transaction when
`REF_TRANSACTION_ALLOW_PARTIAL` is set.
- Add `ref_transaction_for_each_rejected_update()` to let callers
examine which updates were rejected and why.
This foundational change enables partial transaction support throughout
the reference subsystem. A following commit will expose this capability
to users by adding a `--allow-partial` flag to 'git-update-ref(1)',
providing both a user-facing feature and a testable implementation.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++
refs.h | 22 ++++++++++++++++++
refs/files-backend.c | 12 +++++++++-
refs/packed-backend.c | 27 ++++++++++++++++++++--
refs/refs-internal.h | 25 ++++++++++++++++++++
refs/reftable-backend.c | 12 +++++++++-
6 files changed, 155 insertions(+), 4 deletions(-)
diff --git a/refs.c b/refs.c
index 63b8050ce2..b735510c3b 100644
--- a/refs.c
+++ b/refs.c
@@ -1176,6 +1176,10 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,
tr->ref_store = refs;
tr->flags = flags;
string_list_init_dup(&tr->refnames);
+
+ if (flags & REF_TRANSACTION_ALLOW_PARTIAL)
+ CALLOC_ARRAY(tr->rejections, 1);
+
return tr;
}
@@ -1206,11 +1210,45 @@ void ref_transaction_free(struct ref_transaction *transaction)
free((char *)transaction->updates[i]->old_target);
free(transaction->updates[i]);
}
+
+ if (transaction->rejections)
+ free(transaction->rejections->update_indices);
+ free(transaction->rejections);
+
string_list_clear(&transaction->refnames, 0);
free(transaction->updates);
free(transaction);
}
+int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,
+ size_t update_idx,
+ enum ref_transaction_error err)
+{
+ if (update_idx >= transaction->nr)
+ BUG("trying to set rejection on invalid update index");
+
+ if (!(transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL))
+ return 0;
+
+ if (!transaction->rejections)
+ BUG("transaction not inititalized with partial support");
+
+ /*
+ * Don't accept generic errors, since these errors are not user
+ * input related.
+ */
+ if (err == REF_TRANSACTION_ERROR_GENERIC)
+ return 0;
+
+ transaction->updates[update_idx]->rejection_err = err;
+ ALLOC_GROW(transaction->rejections->update_indices,
+ transaction->rejections->nr + 1,
+ transaction->rejections->alloc);
+ transaction->rejections->update_indices[transaction->rejections->nr++] = update_idx;
+
+ return 1;
+}
+
struct ref_update *ref_transaction_add_update(
struct ref_transaction *transaction,
const char *refname, unsigned int flags,
@@ -1236,6 +1274,7 @@ struct ref_update *ref_transaction_add_update(
transaction->updates[transaction->nr++] = update;
update->flags = flags;
+ update->rejection_err = 0;
update->new_target = xstrdup_or_null(new_target);
update->old_target = xstrdup_or_null(old_target);
@@ -2727,6 +2766,28 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
}
}
+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
+ ref_transaction_for_each_rejected_update_fn cb,
+ void *cb_data)
+{
+ if (!transaction->rejections)
+ return;
+
+ for (size_t i = 0; i < transaction->rejections->nr; i++) {
+ size_t update_index = transaction->rejections->update_indices[i];
+ struct ref_update *update = transaction->updates[update_index];
+
+ if (!update->rejection_err)
+ continue;
+
+ cb(update->refname,
+ (update->flags & REF_HAVE_OLD) ? &update->old_oid : NULL,
+ (update->flags & REF_HAVE_NEW) ? &update->new_oid : NULL,
+ update->old_target, update->new_target,
+ update->rejection_err, cb_data);
+ }
+}
+
int refs_delete_refs(struct ref_store *refs, const char *logmsg,
struct string_list *refnames, unsigned int flags)
{
diff --git a/refs.h b/refs.h
index 1b9213f9ce..5e5ff9e57d 100644
--- a/refs.h
+++ b/refs.h
@@ -673,6 +673,13 @@ enum ref_transaction_flag {
* either be absent or null_oid.
*/
REF_TRANSACTION_FLAG_INITIAL = (1 << 0),
+
+ /*
+ * The transaction mechanism by default fails all updates if any conflict
+ * is detected. This flag allows transactions to partially apply updates
+ * while rejecting updates which do not match the expected state.
+ */
+ REF_TRANSACTION_ALLOW_PARTIAL = (1 << 1),
};
/*
@@ -903,6 +910,21 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
ref_transaction_for_each_queued_update_fn cb,
void *cb_data);
+/*
+ * Execute the given callback function for each of the reference updates which
+ * have been rejected in the given transaction.
+ */
+typedef void ref_transaction_for_each_rejected_update_fn(const char *refname,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
+ enum ref_transaction_error err,
+ void *cb_data);
+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
+ ref_transaction_for_each_rejected_update_fn cb,
+ void *cb_data);
+
/*
* Free `*transaction` and all associated data.
*/
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 1e1663f44b..c2fdee6013 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2852,8 +2852,15 @@ static int files_transaction_prepare(struct ref_store *ref_store,
ret = lock_ref_for_update(refs, update, transaction,
head_ref, &refnames_to_check,
err);
- if (ret)
+ if (ret) {
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
+ ret = 0;
+
+ continue;
+ }
goto cleanup;
+ }
if (update->flags & REF_DELETING &&
!(update->flags & REF_LOG_ONLY) &&
@@ -3151,6 +3158,9 @@ static int files_transaction_finish(struct ref_store *ref_store,
struct ref_update *update = transaction->updates[i];
struct ref_lock *lock = update->backend_data;
+ if (update->rejection_err)
+ continue;
+
if (update->flags & REF_NEEDS_COMMIT ||
update->flags & REF_LOG_ONLY) {
if (parse_and_write_reflog(refs, update, lock, err)) {
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 5458952624..bfc6135743 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1327,10 +1327,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
* remain locked when it is done.
*/
static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,
- struct string_list *updates,
+ struct ref_transaction *transaction,
struct strbuf *err)
{
enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
+ struct string_list *updates = &transaction->refnames;
struct ref_iterator *iter = NULL;
size_t i;
int ok;
@@ -1411,6 +1412,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
"reference already exists",
update->refname);
ret = REF_TRANSACTION_ERROR_CREATE_EXISTS;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
+ ret = 0;
+ continue;
+ }
+
goto error;
} else if (!oideq(&update->old_oid, iter->oid)) {
strbuf_addf(err, "cannot update ref '%s': "
@@ -1419,6 +1427,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
oid_to_hex(iter->oid),
oid_to_hex(&update->old_oid));
ret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
+ ret = 0;
+ continue;
+ }
+
goto error;
}
}
@@ -1456,6 +1471,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
update->refname,
oid_to_hex(&update->old_oid));
return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
+ ret = 0;
+ continue;
+ }
+
goto error;
}
}
@@ -1521,6 +1543,7 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
write_error:
strbuf_addf(err, "error writing to %s: %s",
get_tempfile_path(refs->tempfile), strerror(errno));
+ ret = REF_TRANSACTION_ERROR_GENERIC;
error:
ref_iterator_free(iter);
@@ -1679,7 +1702,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
data->own_lock = 1;
}
- ret = write_with_updates(refs, &transaction->refnames, err);
+ ret = write_with_updates(refs, transaction, err);
if (ret)
goto failure;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 3f1d19abd9..c417aec217 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -123,6 +123,11 @@ struct ref_update {
*/
uint64_t index;
+ /*
+ * Used in partial transactions to mark if a given update was rejected.
+ */
+ enum ref_transaction_error rejection_err;
+
/*
* If this ref_update was split off of a symref update via
* split_symref_update(), then this member points at that
@@ -142,6 +147,13 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
struct object_id *oid, struct strbuf *referent,
unsigned int *type, int *failure_errno);
+/*
+ * Mark a given update as rejected with a given reason.
+ */
+int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,
+ size_t update_idx,
+ enum ref_transaction_error err);
+
/*
* Add a ref_update with the specified properties to transaction, and
* return a pointer to the new object. This function does not verify
@@ -183,6 +195,18 @@ enum ref_transaction_state {
REF_TRANSACTION_CLOSED = 2
};
+/*
+ * Data structure to hold indices of updates which were rejected, when
+ * partial transactions where enabled. While the updates themselves hold
+ * the rejection error, this structure allows a transaction to iterate
+ * only over the rejected updates.
+ */
+struct ref_transaction_rejections {
+ size_t *update_indices;
+ size_t alloc;
+ size_t nr;
+};
+
/*
* Data structure for holding a reference transaction, which can
* consist of checks and updates to multiple references, carried out
@@ -195,6 +219,7 @@ struct ref_transaction {
size_t alloc;
size_t nr;
enum ref_transaction_state state;
+ struct ref_transaction_rejections *rejections;
void *backend_data;
unsigned int flags;
uint64_t max_index;
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 0132b8b06a..dd9912d637 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1371,8 +1371,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->updates[i],
&refnames_to_check, head_type,
&head_referent, &referent, err);
- if (ret)
+ if (ret) {
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
+ ret = 0;
+
+ continue;
+ }
goto done;
+ }
}
string_list_sort(&refnames_to_check);
@@ -1455,6 +1462,9 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
struct reftable_transaction_update *tx_update = &arg->updates[i];
struct ref_update *u = tx_update->update;
+ if (u->rejection_err)
+ continue;
+
/*
* Write a reflog entry when updating a ref to point to
* something new in either of the following cases:
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v3 7/8] refs: support partial update rejections during F/D checks
2025-03-05 17:38 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Karthik Nayak
` (5 preceding siblings ...)
2025-03-05 17:39 ` [PATCH v3 6/8] refs: implement partial reference transaction support Karthik Nayak
@ 2025-03-05 17:39 ` Karthik Nayak
2025-03-05 17:39 ` [PATCH v3 8/8] update-ref: add --allow-partial flag for stdin mode Karthik Nayak
2025-03-05 19:28 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Junio C Hamano
8 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-05 17:39 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
The `refs_verify_refnames_available()` is used to batch check refnames
for F/D conflicts. While this is the more performant alternative than
its individual version, it does not provide rejection capabilities on a
single update level. For partial transactions, this would mean a
rejection of the entire transaction whenever one reference has a F/D
conflict.
Modify the function to call `ref_transaction_maybe_set_rejected()` to
check if a single update can be rejected. Since this function is only
internally used within 'refs/' and we want to pass in a `struct
ref_transaction *` as a variable. We also move and mark
`refs_verify_refnames_available()` to 'refs-internal.h' to be an
internal function.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 28 +++++++++++++++++++++++++++-
refs.h | 12 ------------
refs/files-backend.c | 27 ++++++++++++++++++---------
refs/refs-internal.h | 17 +++++++++++++++++
refs/reftable-backend.c | 11 ++++++++---
5 files changed, 70 insertions(+), 25 deletions(-)
diff --git a/refs.c b/refs.c
index b735510c3b..c4dccf9d8b 100644
--- a/refs.c
+++ b/refs.c
@@ -2540,6 +2540,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
const struct string_list *refnames,
const struct string_list *extras,
const struct string_list *skip,
+ struct ref_transaction *transaction,
unsigned int initial_transaction,
struct strbuf *err)
{
@@ -2559,6 +2560,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
strset_init(&dirnames);
for (size_t i = 0; i < refnames->nr; i++) {
+ const size_t *update_idx = (size_t *)refnames->items[i].util;
const char *refname = refnames->items[i].string;
const char *extra_refname;
struct object_id oid;
@@ -2598,12 +2600,26 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
if (!initial_transaction &&
!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
&type, &ignore_errno)) {
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT)) {
+ strset_remove(&dirnames, dirname.buf);
+ continue;
+ }
+
strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
dirname.buf, refname);
goto cleanup;
}
if (extras && string_list_has_string(extras, dirname.buf)) {
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT)) {
+ strset_remove(&dirnames, dirname.buf);
+ continue;
+ }
+
strbuf_addf(err, _("cannot process '%s' and '%s' at the same time"),
refname, dirname.buf);
goto cleanup;
@@ -2636,6 +2652,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
string_list_has_string(skip, iter->refname))
continue;
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT))
+ continue;
+
strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
iter->refname, refname);
goto cleanup;
@@ -2647,6 +2668,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
extra_refname = find_descendant_ref(dirname.buf, extras, skip);
if (extra_refname) {
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT))
+ continue;
+
strbuf_addf(err, _("cannot process '%s' and '%s' at the same time"),
refname, extra_refname);
goto cleanup;
@@ -2678,7 +2704,7 @@ enum ref_transaction_error refs_verify_refname_available(
};
return refs_verify_refnames_available(refs, &refnames, extras, skip,
- initial_transaction, err);
+ NULL, initial_transaction, err);
}
struct do_for_each_reflog_help {
diff --git a/refs.h b/refs.h
index 5e5ff9e57d..938420bec4 100644
--- a/refs.h
+++ b/refs.h
@@ -147,18 +147,6 @@ enum ref_transaction_error refs_verify_refname_available(struct ref_store *refs,
unsigned int initial_transaction,
struct strbuf *err);
-/*
- * Same as `refs_verify_refname_available()`, but checking for a list of
- * refnames instead of only a single item. This is more efficient in the case
- * where one needs to check multiple refnames.
- */
-enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *refnames,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
-
int refs_ref_exists(struct ref_store *refs, const char *refname);
int should_autocreate_reflog(enum log_refs_config log_all_ref_updates,
diff --git a/refs/files-backend.c b/refs/files-backend.c
index c2fdee6013..7525bf75ab 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -677,16 +677,18 @@ static void unlock_ref(struct ref_lock *lock)
* - Generate informative error messages in the case of failure
*/
static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
- const char *refname,
+ struct ref_update *update,
+ size_t update_idx,
int mustexist,
struct string_list *refnames_to_check,
const struct string_list *extras,
struct ref_lock **lock_p,
struct strbuf *referent,
- unsigned int *type,
struct strbuf *err)
{
enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
+ const char *refname = update->refname;
+ unsigned int *type = &update->type;
struct ref_lock *lock;
struct strbuf ref_file = STRBUF_INIT;
int attempts_remaining = 3;
@@ -785,6 +787,8 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
type, &failure_errno)) {
+ struct string_list_item *item;
+
if (failure_errno == ENOENT) {
if (mustexist) {
/* Garden variety missing reference. */
@@ -864,7 +868,9 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
* make sure there is no existing packed ref that conflicts
* with refname. This check is deferred so that we can batch it.
*/
- string_list_insert(refnames_to_check, refname);
+ item = string_list_insert(refnames_to_check, refname);
+ item->util = xmalloc(sizeof(update_idx));
+ memcpy(item->util, &update_idx, sizeof(update_idx));
}
ret = 0;
@@ -2547,6 +2553,7 @@ struct files_transaction_backend_data {
*/
static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *refs,
struct ref_update *update,
+ size_t update_idx,
struct ref_transaction *transaction,
const char *head_ref,
struct string_list *refnames_to_check,
@@ -2575,9 +2582,9 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re
if (lock) {
lock->count++;
} else {
- ret = lock_raw_ref(refs, update->refname, mustexist,
+ ret = lock_raw_ref(refs, update, update_idx, mustexist,
refnames_to_check, &transaction->refnames,
- &lock, &referent, &update->type, err);
+ &lock, &referent, err);
if (ret) {
char *reason;
@@ -2849,7 +2856,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
- ret = lock_ref_for_update(refs, update, transaction,
+ ret = lock_ref_for_update(refs, update, i, transaction,
head_ref, &refnames_to_check,
err);
if (ret) {
@@ -2905,7 +2912,8 @@ static int files_transaction_prepare(struct ref_store *ref_store,
* So instead, we accept the race for now.
*/
if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
- &transaction->refnames, NULL, 0, err)) {
+ &transaction->refnames, NULL, transaction,
+ 0, err)) {
ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
}
@@ -2951,7 +2959,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
cleanup:
free(head_ref);
- string_list_clear(&refnames_to_check, 0);
+ string_list_clear(&refnames_to_check, 1);
if (ret)
files_transaction_cleanup(refs, transaction);
@@ -3097,7 +3105,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
}
if (refs_verify_refnames_available(&refs->base, &refnames_to_check,
- &affected_refnames, NULL, 1, err)) {
+ &affected_refnames, NULL, transaction,
+ 1, err)) {
packed_refs_unlock(refs->packed_ref_store);
ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index c417aec217..f0e958dc83 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -805,4 +805,21 @@ enum ref_transaction_error ref_update_check_old_target(const char *referent,
*/
int ref_update_expects_existing_old_ref(struct ref_update *update);
+/*
+ * Same as `refs_verify_refname_available()`, but checking for a list of
+ * refnames instead of only a single item. This is more efficient in the case
+ * where one needs to check multiple refnames.
+ *
+ * If a transaction is provided with partial support, then individual updates
+ * are marked rejected, reference backends are then in charge of not committing
+ * those updates.
+ */
+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ struct ref_transaction *transaction,
+ unsigned int initial_transaction,
+ struct strbuf *err);
+
#endif /* REFS_REFS_INTERNAL_H */
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index dd9912d637..a50e004d96 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1074,6 +1074,7 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor
struct ref_transaction *transaction,
struct reftable_backend *be,
struct ref_update *u,
+ size_t update_idx,
struct string_list *refnames_to_check,
unsigned int head_type,
struct strbuf *head_referent,
@@ -1149,6 +1150,7 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor
if (ret < 0)
return REF_TRANSACTION_ERROR_GENERIC;
if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
+ struct string_list_item *item;
/*
* The reference does not exist, and we either have no
* old object ID or expect the reference to not exist.
@@ -1158,7 +1160,9 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor
* can output a proper error message instead of failing
* at a later point.
*/
- string_list_append(refnames_to_check, u->refname);
+ item = string_list_append(refnames_to_check, u->refname);
+ item->util = xmalloc(sizeof(update_idx));
+ memcpy(item->util, &update_idx, sizeof(update_idx));
/*
* There is no need to write the reference deletion
@@ -1368,7 +1372,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
for (i = 0; i < transaction->nr; i++) {
ret = prepare_single_update(refs, tx_data, transaction, be,
- transaction->updates[i],
+ transaction->updates[i], i,
&refnames_to_check, head_type,
&head_referent, &referent, err);
if (ret) {
@@ -1385,6 +1389,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
string_list_sort(&refnames_to_check);
ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
&transaction->refnames, NULL,
+ transaction,
transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
err);
if (ret < 0)
@@ -1403,7 +1408,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
}
strbuf_release(&referent);
strbuf_release(&head_referent);
- string_list_clear(&refnames_to_check, 0);
+ string_list_clear(&refnames_to_check, 1);
return ret;
}
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v3 8/8] update-ref: add --allow-partial flag for stdin mode
2025-03-05 17:38 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Karthik Nayak
` (6 preceding siblings ...)
2025-03-05 17:39 ` [PATCH v3 7/8] refs: support partial update rejections during F/D checks Karthik Nayak
@ 2025-03-05 17:39 ` Karthik Nayak
2025-03-05 19:28 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Junio C Hamano
8 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-05 17:39 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
When updating multiple references through stdin, Git's update-ref
command normally aborts the entire transaction if any single update
fails. While this atomic behavior prevents partial updates by default,
there are cases where applying successful updates while reporting
failures is desirable.
Add a new `--allow-partial` flag that allows the transaction to continue
even when individual reference updates fail. This flag can only be used
in `--stdin` mode and builds upon the partial transaction support added
to the refs subsystem. When enabled, failed updates are reported in the
following format:
rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
Update the documentation to reflect this change and also tests to cover
different scenarios where an update could be rejected.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
Documentation/git-update-ref.adoc | 17 ++-
builtin/update-ref.c | 67 ++++++++++-
t/t1400-update-ref.sh | 233 ++++++++++++++++++++++++++++++++++++++
3 files changed, 309 insertions(+), 8 deletions(-)
diff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc
index 9e6935d38d..bcf38850a4 100644
--- a/Documentation/git-update-ref.adoc
+++ b/Documentation/git-update-ref.adoc
@@ -7,8 +7,10 @@ git-update-ref - Update the object name stored in a ref safely
SYNOPSIS
--------
-[verse]
-'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z])
+[synopsis]
+git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
+ [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
+ [-m <reason>] [--no-deref] --stdin [-z] [--allow-partial]
DESCRIPTION
-----------
@@ -57,6 +59,17 @@ performs all modifications together. Specify commands of the form:
With `--create-reflog`, update-ref will create a reflog for each ref
even if one would not ordinarily be created.
+With `--allow-partial`, update-ref continues executing the transaction even if
+some updates fail due to invalid or incorrect user input, applying only the
+successful updates. Errors resulting from user-provided input are treated as
+non-system-related and do not cause the entire transaction to be aborted.
+However, system-related errors—such as I/O failures or memory issues—will still
+result in a full failure. Additionally, errors like F/D conflicts are batched
+for performance optimization and will also cause a full failure. Any failed
+updates will be reported in the following format:
+
+ rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
+
Quote fields containing whitespace as if they were strings in C source
code; i.e., surrounded by double-quotes and with backslash escapes.
Use 40 "0" characters or the empty string to specify a zero value. To
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 1d541e13ad..66bd3cb44f 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -5,6 +5,7 @@
#include "config.h"
#include "gettext.h"
#include "hash.h"
+#include "hex.h"
#include "refs.h"
#include "object-name.h"
#include "parse-options.h"
@@ -13,7 +14,7 @@
static const char * const git_update_ref_usage[] = {
N_("git update-ref [<options>] -d <refname> [<old-oid>]"),
N_("git update-ref [<options>] <refname> <new-oid> [<old-oid>]"),
- N_("git update-ref [<options>] --stdin [-z]"),
+ N_("git update-ref [<options>] --stdin [-z] [--allow-partial]"),
NULL
};
@@ -565,6 +566,49 @@ static void parse_cmd_abort(struct ref_transaction *transaction,
report_ok("abort");
}
+static void print_rejected_refs(const char *refname,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
+ enum ref_transaction_error err,
+ void *cb_data UNUSED)
+{
+ struct strbuf sb = STRBUF_INIT;
+ const char *reason = "";
+
+ switch (err) {
+ case REF_TRANSACTION_ERROR_NAME_CONFLICT:
+ reason = "refname conflict";
+ break;
+ case REF_TRANSACTION_ERROR_CREATE_EXISTS:
+ reason = "reference already exists";
+ break;
+ case REF_TRANSACTION_ERROR_NONEXISTENT_REF:
+ reason = "reference does not exist";
+ break;
+ case REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE:
+ reason = "incorrect old value provided";
+ break;
+ case REF_TRANSACTION_ERROR_INVALID_NEW_VALUE:
+ reason = "invalid new value provided";
+ break;
+ case REF_TRANSACTION_ERROR_EXPECTED_SYMREF:
+ reason = "expected symref but found regular ref";
+ break;
+ default:
+ reason = "unkown failure";
+ }
+
+ strbuf_addf(&sb, "rejected %s %s %s %s\n", refname,
+ new_oid ? oid_to_hex(new_oid) : new_target,
+ old_oid ? oid_to_hex(old_oid) : old_target,
+ reason);
+
+ fwrite(sb.buf, sb.len, 1, stdout);
+ strbuf_release(&sb);
+}
+
static void parse_cmd_commit(struct ref_transaction *transaction,
const char *next, const char *end UNUSED)
{
@@ -573,6 +617,10 @@ static void parse_cmd_commit(struct ref_transaction *transaction,
die("commit: extra input: %s", next);
if (ref_transaction_commit(transaction, &error))
die("commit: %s", error.buf);
+
+ ref_transaction_for_each_rejected_update(transaction,
+ print_rejected_refs, NULL);
+
report_ok("commit");
ref_transaction_free(transaction);
}
@@ -609,7 +657,7 @@ static const struct parse_cmd {
{ "commit", parse_cmd_commit, 0, UPDATE_REFS_CLOSED },
};
-static void update_refs_stdin(void)
+static void update_refs_stdin(unsigned int flags)
{
struct strbuf input = STRBUF_INIT, err = STRBUF_INIT;
enum update_refs_state state = UPDATE_REFS_OPEN;
@@ -617,7 +665,7 @@ static void update_refs_stdin(void)
int i, j;
transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
- 0, &err);
+ flags, &err);
if (!transaction)
die("%s", err.buf);
@@ -685,7 +733,7 @@ static void update_refs_stdin(void)
*/
state = cmd->state;
transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
- 0, &err);
+ flags, &err);
if (!transaction)
die("%s", err.buf);
@@ -701,6 +749,8 @@ static void update_refs_stdin(void)
/* Commit by default if no transaction was requested. */
if (ref_transaction_commit(transaction, &err))
die("%s", err.buf);
+ ref_transaction_for_each_rejected_update(transaction,
+ print_rejected_refs, NULL);
ref_transaction_free(transaction);
break;
case UPDATE_REFS_STARTED:
@@ -727,6 +777,8 @@ int cmd_update_ref(int argc,
struct object_id oid, oldoid;
int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;
int create_reflog = 0;
+ unsigned int flags = 0;
+
struct option options[] = {
OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
OPT_BOOL('d', NULL, &delete, N_("delete the reference")),
@@ -735,6 +787,8 @@ int cmd_update_ref(int argc,
OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
OPT_BOOL( 0 , "create-reflog", &create_reflog, N_("create a reflog")),
+ OPT_BIT('0', "allow-partial", &flags, N_("allow partial transactions"),
+ REF_TRANSACTION_ALLOW_PARTIAL),
OPT_END(),
};
@@ -756,9 +810,10 @@ int cmd_update_ref(int argc,
usage_with_options(git_update_ref_usage, options);
if (end_null)
line_termination = '\0';
- update_refs_stdin();
+ update_refs_stdin(flags);
return 0;
- }
+ } else if (flags & REF_TRANSACTION_ALLOW_PARTIAL)
+ die("--allow-partial can only be used with --stdin");
if (end_null)
usage_with_options(git_update_ref_usage, options);
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 29045aad43..62a82f4af6 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -2066,6 +2066,239 @@ do
grep "$(git rev-parse $a) $(git rev-parse $a)" actual
'
+ test_expect_success "stdin $type allow-partial" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit commit &&
+ head=$(git rev-parse HEAD) &&
+
+ format_command $type "update refs/heads/ref1" "$head" "$Z" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head" "$Z" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with invalid new_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$(test_oid 001)" "$head" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "invalid new value provided" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with non-commit new_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ head_tree=$(git rev-parse HEAD^{tree}) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head_tree" "$head" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "invalid new value provided" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with non-existent ref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ test_must_fail git rev-parse refs/heads/ref2 &&
+ test_grep -q "reference does not exist" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with dangling symref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git symbolic-ref refs/heads/ref2 refs/heads/nonexistent &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
+ git update-ref $type --no-deref --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ test_must_fail git rev-parse refs/heads/ref2 &&
+ test_grep -q "reference does not exist" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with regular ref as symref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "symref-update refs/heads/ref2" "$old_head" "ref" "refs/heads/nonexistent" >>stdin &&
+ git update-ref $type --no-deref --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "expected symref but found regular ref" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with invalid old_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$Z" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "reference already exists" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial with incorrect old oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head" "$old_head" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "incorrect old value provided" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial refname conflict" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref/foo $head &&
+
+ format_command $type "update refs/heads/ref/foo" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref/foo >actual &&
+ test_cmp expect actual &&
+ test_grep -q "refname conflict" stdout
+ )
+ '
+
+ test_expect_success "stdin $type allow-partial refname conflict new ref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref/foo $head &&
+
+ format_command $type "update refs/heads/foo" "$old_head" "" >stdin &&
+ format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/foo >actual &&
+ test_cmp expect actual &&
+ test_grep -q "refname conflict" stdout
+ )
+ '
done
test_expect_success 'update-ref should also create reflog for HEAD' '
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH v3 0/8] refs: introduce support for partial reference transactions
2025-03-05 17:38 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Karthik Nayak
` (7 preceding siblings ...)
2025-03-05 17:39 ` [PATCH v3 8/8] update-ref: add --allow-partial flag for stdin mode Karthik Nayak
@ 2025-03-05 19:28 ` Junio C Hamano
2025-03-06 9:06 ` Karthik Nayak
8 siblings, 1 reply; 143+ messages in thread
From: Junio C Hamano @ 2025-03-05 19:28 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, ps, jltobler, phillip.wood123
Karthik Nayak <karthik.188@gmail.com> writes:
> Git's reference updates are traditionally all or nothing - when
> updating multiple references in a transaction, either all updates
> succeed or none do.
I am quite confused. In the beginning (traditionally), there was no
transaction to speak of. You try to update two refs at the same
time, we did best effort but that was never atomic. Later we
introduced transactions to optionally make the changes all-or-none.
So, if you want "I have these N updates, but I do not care if some
of them have to fail---just make your best effort to update as many
of them as you can", why are you still doing a transaction?
Perhaps it is merely the phrasing that makes this proposal
confusing. If presented as "non-transactional batched updates",
perhaps it may have been more palatable. I dunno, but "partial
transaction" does not quite sound like a transaction, at least to
me.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 1/8] refs/files: remove redundant check in split_symref_update()
2025-03-05 17:38 ` [PATCH v3 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
@ 2025-03-05 21:20 ` Junio C Hamano
2025-03-06 9:13 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Junio C Hamano @ 2025-03-05 21:20 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, ps, jltobler, phillip.wood123
Karthik Nayak <karthik.188@gmail.com> writes:
> In `split_symref_update()`, there were two checks for duplicate
> refnames:
>
> - At the start, `string_list_has_string()` ensures the refname is not
> already in `affected_refnames`, preventing duplicates from being
> added.
>
> - After adding the refname, another check verifies whether the newly
> inserted item has a `util` value.
>
> The second check is unnecessary because the first one guarantees that
> `string_list_insert()` will never encounter a preexisting entry.
>
> Since `item->util` is only used in this context, remove the assignment and
> simplify the surrounding code.
It was a bit unclear what "this context" refers to. We lost all
assignments to the .util member and that is a safe thing to do
because ...
> @@ -2843,13 +2835,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
> if (update->flags & REF_LOG_ONLY)
> continue;
>
> - item = string_list_append(&affected_refnames, update->refname);
> - /*
> - * We store a pointer to update in item->util, but at
> - * the moment we never use the value of this field
> - * except to check whether it is non-NULL.
> - */
> - item->util = update;
... of this comment, and the "except to check whether" used to
happen in this code ...
> * be valid as long as affected_refnames is in use, and NOT
> * referent, which might soon be freed by our caller.
> */
> - item = string_list_insert(affected_refnames, new_update->refname);
> - if (item->util)
> - BUG("%s unexpectedly found in affected_refnames",
> - new_update->refname);
> - item->util = new_update;
... which the patch removed.
OK. Makes perfect sense.
Thanks.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 2/8] refs: move duplicate refname update check to generic layer
2025-03-05 17:38 ` [PATCH v3 2/8] refs: move duplicate refname update check to generic layer Karthik Nayak
@ 2025-03-05 21:56 ` Junio C Hamano
2025-03-06 9:46 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Junio C Hamano @ 2025-03-05 21:56 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, ps, jltobler, phillip.wood123
Karthik Nayak <karthik.188@gmail.com> writes:
> Move the tracking of refnames in `affected_refnames` from individual
> backends into the generic layer in 'refs.c'. This centralizes the
> duplicate refname detection that was previously handled separately by
> each backend.
>
> Make some changes to accommodate this move:
>
> - Add a `string_list` field `refnames` to `ref_transaction` to contain
> all the references in a transaction. This field is updated whenever
> a new update is added via `ref_transaction_add_update`, so manual
> additions in reference backends are dropped.
The transaction object is the most logical place to keep track of
what is involved in the transaction. Nice.
> - Modify the backends to use this field internally as needed. The
> backends need to check if an update for refname already exists when
> splitting symrefs or adding an update for 'HEAD'.
The above reads to me as if you are saying that the files backend
needs to notice that it is updating "HEAD", notice that it is a
symbolic ref that points at "refs/heads/main", notice that "HEAD"
and "refs/heads/main" are the two things involved in the
transaction, and must check if an update is already queued.
But when an update changes a symbolic ref in the sense that the
underlying ref gets updated through it, the need to update both the
underlying ref and the symbolic ref is common across backends, isn't
it? IOW, shouldn't "splitting symrefs" (which I take to mean "ah,
we are updating HEAD so we need to update it and at the same time
update the underlying refs/heads/main, two updates in total") be
done also at the generic layer?
And if that happens at the generic layer, should .refname member
even be visible to backends?
> - In the reftable backend, within `reftable_be_transaction_prepare()`,
> move the `string_list_has_string()` check above
> `ref_transaction_add_update()`. Since `ref_transaction_add_update()`
> automatically adds the refname to `transaction->refnames`,
> performing the check after will always return true, so we perform
> the check before adding the update.
This change makes perfect tense. It is the most natural to check
and modify at the transaction layer the .refnames member, as it
belongs at the transaction layer after all.
> This helps reduce duplication of functionality between the backends and
> makes it easier to make changes in a more centralized manner.
Nice.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 0/8] refs: introduce support for partial reference transactions
2025-03-05 19:28 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Junio C Hamano
@ 2025-03-06 9:06 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-06 9:06 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, ps, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 1439 bytes --]
Junio C Hamano <gitster@pobox.com> writes:
> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> Git's reference updates are traditionally all or nothing - when
>> updating multiple references in a transaction, either all updates
>> succeed or none do.
>
> I am quite confused. In the beginning (traditionally), there was no
> transaction to speak of. You try to update two refs at the same
> time, we did best effort but that was never atomic. Later we
> introduced transactions to optionally make the changes all-or-none.
>
> So, if you want "I have these N updates, but I do not care if some
> of them have to fail---just make your best effort to update as many
> of them as you can", why are you still doing a transaction?
>
> Perhaps it is merely the phrasing that makes this proposal
> confusing. If presented as "non-transactional batched updates",
> perhaps it may have been more palatable. I dunno, but "partial
> transaction" does not quite sound like a transaction, at least to
> me.
That's fair. There was also some discussion earlier around this [1]. It
is in indeed batched updates which can allow failures, but it is built
on top of the transaction infrastructure in the refs subsystem.
Perhaps the best way would be to use the transaction interface under the
hood, but present this feature as 'batched updates' to users, so there
is no confusion between the two.
[1]: 4beb0359-763d-425d-b416-ac40bda59e2e@gmail.com
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 1/8] refs/files: remove redundant check in split_symref_update()
2025-03-05 21:20 ` Junio C Hamano
@ 2025-03-06 9:13 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-06 9:13 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, ps, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 2116 bytes --]
Junio C Hamano <gitster@pobox.com> writes:
> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> In `split_symref_update()`, there were two checks for duplicate
>> refnames:
>>
>> - At the start, `string_list_has_string()` ensures the refname is not
>> already in `affected_refnames`, preventing duplicates from being
>> added.
>>
>> - After adding the refname, another check verifies whether the newly
>> inserted item has a `util` value.
>>
>> The second check is unnecessary because the first one guarantees that
>> `string_list_insert()` will never encounter a preexisting entry.
>>
>> Since `item->util` is only used in this context, remove the assignment and
>> simplify the surrounding code.
>
> It was a bit unclear what "this context" refers to. We lost all
> assignments to the .util member and that is a safe thing to do
> because ...
>
Definitely could use some clarification. Will change to:
The `item->util` field is assigned to validate that a rename doesn't
already exist in the list. The validation is done after the first
check. As this check is removed, clean up the validation and the
assignment of this field.
>> @@ -2843,13 +2835,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
>> if (update->flags & REF_LOG_ONLY)
>> continue;
>>
>> - item = string_list_append(&affected_refnames, update->refname);
>> - /*
>> - * We store a pointer to update in item->util, but at
>> - * the moment we never use the value of this field
>> - * except to check whether it is non-NULL.
>> - */
>> - item->util = update;
>
> ... of this comment, and the "except to check whether" used to
> happen in this code ...
>
>> * be valid as long as affected_refnames is in use, and NOT
>> * referent, which might soon be freed by our caller.
>> */
>> - item = string_list_insert(affected_refnames, new_update->refname);
>> - if (item->util)
>> - BUG("%s unexpectedly found in affected_refnames",
>> - new_update->refname);
>> - item->util = new_update;
>
> ... which the patch removed.
>
> OK. Makes perfect sense.
>
> Thanks.
Thanks!
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 2/8] refs: move duplicate refname update check to generic layer
2025-03-05 21:56 ` Junio C Hamano
@ 2025-03-06 9:46 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-06 9:46 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, ps, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 3039 bytes --]
Junio C Hamano <gitster@pobox.com> writes:
> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> Move the tracking of refnames in `affected_refnames` from individual
>> backends into the generic layer in 'refs.c'. This centralizes the
>> duplicate refname detection that was previously handled separately by
>> each backend.
>>
>> Make some changes to accommodate this move:
>>
>> - Add a `string_list` field `refnames` to `ref_transaction` to contain
>> all the references in a transaction. This field is updated whenever
>> a new update is added via `ref_transaction_add_update`, so manual
>> additions in reference backends are dropped.
>
> The transaction object is the most logical place to keep track of
> what is involved in the transaction. Nice.
>
>> - Modify the backends to use this field internally as needed. The
>> backends need to check if an update for refname already exists when
>> splitting symrefs or adding an update for 'HEAD'.
>
> The above reads to me as if you are saying that the files backend
> needs to notice that it is updating "HEAD", notice that it is a
> symbolic ref that points at "refs/heads/main", notice that "HEAD"
> and "refs/heads/main" are the two things involved in the
> transaction, and must check if an update is already queued.
>
> But when an update changes a symbolic ref in the sense that the
> underlying ref gets updated through it, the need to update both the
> underlying ref and the symbolic ref is common across backends, isn't
> it? IOW, shouldn't "splitting symrefs" (which I take to mean "ah,
> we are updating HEAD so we need to update it and at the same time
> update the underlying refs/heads/main, two updates in total") be
> done also at the generic layer?
Yup that is correct, in the files backend, we do this via the
'split_symref_update()' function and in the reftable backend it is
directly handled in the 'reftable_be_transaction_prepare()' function.
I don't have a reason for why I didn't undertake that too in this
series. Mostly I think I didn't observe it. But it something that
can/should be done in the future.
>
> And if that happens at the generic layer, should .refname member
> even be visible to backends?
>
It shouldn't be necessary anymore with that change. I think this is good
step in that direction.
>> - In the reftable backend, within `reftable_be_transaction_prepare()`,
>> move the `string_list_has_string()` check above
>> `ref_transaction_add_update()`. Since `ref_transaction_add_update()`
>> automatically adds the refname to `transaction->refnames`,
>> performing the check after will always return true, so we perform
>> the check before adding the update.
>
> This change makes perfect tense. It is the most natural to check
> and modify at the transaction layer the .refnames member, as it
> belongs at the transaction layer after all.
>
>> This helps reduce duplication of functionality between the backends and
>> makes it easier to make changes in a more centralized manner.
>
> Nice.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 6/8] refs: implement partial reference transaction support
2025-03-05 17:39 ` [PATCH v3 6/8] refs: implement partial reference transaction support Karthik Nayak
@ 2025-03-07 19:50 ` Jeff King
2025-03-07 20:46 ` Junio C Hamano
2025-03-07 21:02 ` [PATCH v3 6/8] refs: implement partial reference transaction support Karthik Nayak
2025-03-07 19:57 ` Jeff King
1 sibling, 2 replies; 143+ messages in thread
From: Jeff King @ 2025-03-07 19:50 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, ps, jltobler, phillip.wood123
On Wed, Mar 05, 2025 at 06:39:01PM +0100, Karthik Nayak wrote:
> @@ -1456,6 +1471,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
> update->refname,
> oid_to_hex(&update->old_oid));
> return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
> +
> + if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
> + strbuf_setlen(err, 0);
> + ret = 0;
> + continue;
> + }
> +
> goto error;
> }
> }
This new code isn't reachable, since we return in the lines shown in the
diff context.
Should it have been "ret = REF_TRANSACTION_ERROR"... in the first place?
I think the "goto error" was already unreachable, so possibly the error
is in an earlier patch. (I didn't look; Coverity flagged this in the
final state in 'jch').
-Peff
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 6/8] refs: implement partial reference transaction support
2025-03-05 17:39 ` [PATCH v3 6/8] refs: implement partial reference transaction support Karthik Nayak
2025-03-07 19:50 ` Jeff King
@ 2025-03-07 19:57 ` Jeff King
2025-03-07 21:07 ` Karthik Nayak
1 sibling, 1 reply; 143+ messages in thread
From: Jeff King @ 2025-03-07 19:57 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, ps, jltobler, phillip.wood123
On Wed, Mar 05, 2025 at 06:39:01PM +0100, Karthik Nayak wrote:
> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
> index 0132b8b06a..dd9912d637 100644
> --- a/refs/reftable-backend.c
> +++ b/refs/reftable-backend.c
> @@ -1371,8 +1371,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
> transaction->updates[i],
> &refnames_to_check, head_type,
> &head_referent, &referent, err);
> - if (ret)
> + if (ret) {
> + if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
> + strbuf_setlen(err, 0);
> + ret = 0;
> +
> + continue;
> + }
> goto done;
> + }
> }
>
> string_list_sort(&refnames_to_check);
Coverity complains that this "ret = 0" is a dead store. I think it's
right, because either:
1. Our continue loops again, and we overwrite "ret" with the next call
to prepare_single_update().
2. We leave the loop (because this is the final entry in the
transaction update array), and then we overwrite "ret" with the
result of refs_verify_refnames_available().
But it may be better to leave it in place as a defensive measure against
the rest of the function changing.
-Peff
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 6/8] refs: implement partial reference transaction support
2025-03-07 19:50 ` Jeff King
@ 2025-03-07 20:46 ` Junio C Hamano
2025-03-07 20:48 ` Junio C Hamano
` (2 more replies)
2025-03-07 21:02 ` [PATCH v3 6/8] refs: implement partial reference transaction support Karthik Nayak
1 sibling, 3 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-07 20:46 UTC (permalink / raw)
To: Jeff King; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
Jeff King <peff@peff.net> writes:
> On Wed, Mar 05, 2025 at 06:39:01PM +0100, Karthik Nayak wrote:
>
>> @@ -1456,6 +1471,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
>> update->refname,
>> oid_to_hex(&update->old_oid));
>> return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
>> +
>> + if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
>> + strbuf_setlen(err, 0);
>> + ret = 0;
>> + continue;
>> + }
>> +
>> goto error;
>> }
>> }
>
> This new code isn't reachable, since we return in the lines shown in the
> diff context.
>
> Should it have been "ret = REF_TRANSACTION_ERROR"... in the first place?
> I think the "goto error" was already unreachable, so possibly the error
> is in an earlier patch. (I didn't look; Coverity flagged this in the
> final state in 'jch').
Sorry about that. It shows that I lack the bandwidth necessary to
go through fine toothed comb on all the topics I queue. Perhaps I
should be more selective and queue only the ones I personally had
enough bandwidth to look over (or have seen clear "I looked each and
every line of this series with fine toothed comb, put reviewed-by:
me" messages sent by trusted reviewers) while ignoring others?
I dunno.
Thanks.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 6/8] refs: implement partial reference transaction support
2025-03-07 20:46 ` Junio C Hamano
@ 2025-03-07 20:48 ` Junio C Hamano
2025-03-07 21:05 ` Karthik Nayak
2025-03-07 22:54 ` [PATCH] config.mak.dev: enable -Wunreachable-code Jeff King
2 siblings, 0 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-07 20:48 UTC (permalink / raw)
To: Jeff King; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
Junio C Hamano <gitster@pobox.com> writes:
> Sorry about that. It shows that I lack the bandwidth necessary to
> go through fine toothed comb on all the topics I queue. Perhaps I
> should be more selective and queue only the ones I personally had
> enough bandwidth to look over (or have seen clear "I looked each and
> every line of this series with fine toothed comb, put reviewed-by:
> me" messages sent by trusted reviewers) while ignoring others?
I forgot a third category. I should be able to queue series by
those who have track record of being meticulous and not have made
silly mistakes without reading each and every line.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 6/8] refs: implement partial reference transaction support
2025-03-07 19:50 ` Jeff King
2025-03-07 20:46 ` Junio C Hamano
@ 2025-03-07 21:02 ` Karthik Nayak
1 sibling, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-07 21:02 UTC (permalink / raw)
To: Jeff King; +Cc: git, ps, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 1021 bytes --]
Jeff King <peff@peff.net> writes:
> On Wed, Mar 05, 2025 at 06:39:01PM +0100, Karthik Nayak wrote:
>
>> @@ -1456,6 +1471,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
>> update->refname,
>> oid_to_hex(&update->old_oid));
>> return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
>> +
>> + if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
>> + strbuf_setlen(err, 0);
>> + ret = 0;
>> + continue;
>> + }
>> +
>> goto error;
>> }
>> }
>
> This new code isn't reachable, since we return in the lines shown in the
> diff context.
>
> Should it have been "ret = REF_TRANSACTION_ERROR"... in the first place?
> I think the "goto error" was already unreachable, so possibly the error
> is in an earlier patch. (I didn't look; Coverity flagged this in the
> final state in 'jch').
>
> -Peff
It should have bee `ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF` and it
should have been in the previous commit!
Thanks for reporting!
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 6/8] refs: implement partial reference transaction support
2025-03-07 20:46 ` Junio C Hamano
2025-03-07 20:48 ` Junio C Hamano
@ 2025-03-07 21:05 ` Karthik Nayak
2025-03-07 22:54 ` [PATCH] config.mak.dev: enable -Wunreachable-code Jeff King
2 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-07 21:05 UTC (permalink / raw)
To: Junio C Hamano, Jeff King; +Cc: git, ps, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 1558 bytes --]
Junio C Hamano <gitster@pobox.com> writes:
> Jeff King <peff@peff.net> writes:
>
>> On Wed, Mar 05, 2025 at 06:39:01PM +0100, Karthik Nayak wrote:
>>
>>> @@ -1456,6 +1471,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
>>> update->refname,
>>> oid_to_hex(&update->old_oid));
>>> return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
>>> +
>>> + if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
>>> + strbuf_setlen(err, 0);
>>> + ret = 0;
>>> + continue;
>>> + }
>>> +
>>> goto error;
>>> }
>>> }
>>
>> This new code isn't reachable, since we return in the lines shown in the
>> diff context.
>>
>> Should it have been "ret = REF_TRANSACTION_ERROR"... in the first place?
>> I think the "goto error" was already unreachable, so possibly the error
>> is in an earlier patch. (I didn't look; Coverity flagged this in the
>> final state in 'jch').
>
> Sorry about that. It shows that I lack the bandwidth necessary to
> go through fine toothed comb on all the topics I queue. Perhaps I
> should be more selective and queue only the ones I personally had
> enough bandwidth to look over (or have seen clear "I looked each and
> every line of this series with fine toothed comb, put reviewed-by:
> me" messages sent by trusted reviewers) while ignoring others?
>
> I dunno.
>
> Thanks.
Apologies, I see that this was also present in the previous version.
Definitely a miss on my side. I'll see how it was missed in the tests
and add one if necessary!
Thanks!
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 6/8] refs: implement partial reference transaction support
2025-03-07 19:57 ` Jeff King
@ 2025-03-07 21:07 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-07 21:07 UTC (permalink / raw)
To: Jeff King; +Cc: git, ps, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 1386 bytes --]
Jeff King <peff@peff.net> writes:
> On Wed, Mar 05, 2025 at 06:39:01PM +0100, Karthik Nayak wrote:
>
>> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
>> index 0132b8b06a..dd9912d637 100644
>> --- a/refs/reftable-backend.c
>> +++ b/refs/reftable-backend.c
>> @@ -1371,8 +1371,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
>> transaction->updates[i],
>> &refnames_to_check, head_type,
>> &head_referent, &referent, err);
>> - if (ret)
>> + if (ret) {
>> + if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
>> + strbuf_setlen(err, 0);
>> + ret = 0;
>> +
>> + continue;
>> + }
>> goto done;
>> + }
>> }
>>
>> string_list_sort(&refnames_to_check);
>
> Coverity complains that this "ret = 0" is a dead store. I think it's
> right, because either:
>
> 1. Our continue loops again, and we overwrite "ret" with the next call
> to prepare_single_update().
>
> 2. We leave the loop (because this is the final entry in the
> transaction update array), and then we overwrite "ret" with the
> result of refs_verify_refnames_available().
>
> But it may be better to leave it in place as a defensive measure against
> the rest of the function changing.
>
Yes agreed with your analysis, and also your inference. So I'll let this stay.
Thanks for reporting!
> -Peff
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH] config.mak.dev: enable -Wunreachable-code
2025-03-07 20:46 ` Junio C Hamano
2025-03-07 20:48 ` Junio C Hamano
2025-03-07 21:05 ` Karthik Nayak
@ 2025-03-07 22:54 ` Jeff King
2025-03-07 23:28 ` Junio C Hamano
` (2 more replies)
2 siblings, 3 replies; 143+ messages in thread
From: Jeff King @ 2025-03-07 22:54 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
On Fri, Mar 07, 2025 at 12:46:27PM -0800, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > On Wed, Mar 05, 2025 at 06:39:01PM +0100, Karthik Nayak wrote:
> >
> >> @@ -1456,6 +1471,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
> >> update->refname,
> >> oid_to_hex(&update->old_oid));
> >> return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
> >> +
> >> + if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
> >> + strbuf_setlen(err, 0);
> >> + ret = 0;
> >> + continue;
> >> + }
> >> +
> >> goto error;
> >> }
> >> }
> >
> > This new code isn't reachable, since we return in the lines shown in the
> > diff context.
> >
> > Should it have been "ret = REF_TRANSACTION_ERROR"... in the first place?
> > I think the "goto error" was already unreachable, so possibly the error
> > is in an earlier patch. (I didn't look; Coverity flagged this in the
> > final state in 'jch').
>
> Sorry about that. It shows that I lack the bandwidth necessary to
> go through fine toothed comb on all the topics I queue. Perhaps I
> should be more selective and queue only the ones I personally had
> enough bandwidth to look over (or have seen clear "I looked each and
> every line of this series with fine toothed comb, put reviewed-by:
> me" messages sent by trusted reviewers) while ignoring others?
Eh, I would not worry about it too much. Things get missed, and that is
why we have many layers of reviews, static analysis, and ultimately
users to help us find bugs. ;)
I was disappointed that the compiler didn't complain, though. Maybe we
should do this:
-- >8 --
Subject: [PATCH] config.mak.dev: enable -Wunreachable-code
Having the compiler point out unreachable code can help avoid bugs, like
the one discussed in:
https://lore.kernel.org/git/20250307195057.GA3675279@coredump.intra.peff.net/
In that case it was found by Coverity, but finding it earlier saves
everybody time and effort.
We can use -Wunreachable-code to get some help from the compiler here.
Interestingly, this is a noop in gcc. It was a real warning up until gcc
4.x, when it was removed for being too flaky, but they left the
command-line option to avoid breaking users. See:
https://stackoverflow.com/questions/17249934/why-does-gcc-not-warn-for-unreachable-code
However, clang does implement this option, and it finds the case
mentioned above (and no other cases within the code base). And since we
run clang in several of our CI jobs, that's enough to get an early
warning of breakage.
We could enable it only for clang, but since gcc is happy to ignore it,
it's simpler to just turn it on for all developer builds.
Signed-off-by: Jeff King <peff@peff.net>
---
You can see it in action (merged into 'jch') here:
https://github.com/peff/git/actions/runs/13729842188
where all of the clang jobs fail.
config.mak.dev | 1 +
1 file changed, 1 insertion(+)
diff --git a/config.mak.dev b/config.mak.dev
index 0fd8cc4d35..95b7bc46ae 100644
--- a/config.mak.dev
+++ b/config.mak.dev
@@ -39,6 +39,7 @@ DEVELOPER_CFLAGS += -Wunused
DEVELOPER_CFLAGS += -Wvla
DEVELOPER_CFLAGS += -Wwrite-strings
DEVELOPER_CFLAGS += -fno-common
+DEVELOPER_CFLAGS += -Wunreachable-code
ifneq ($(filter clang4,$(COMPILER_FEATURES)),)
DEVELOPER_CFLAGS += -Wtautological-constant-out-of-range-compare
--
2.49.0.rc1.380.g53e738dd21
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH] config.mak.dev: enable -Wunreachable-code
2025-03-07 22:54 ` [PATCH] config.mak.dev: enable -Wunreachable-code Jeff King
@ 2025-03-07 23:28 ` Junio C Hamano
2025-03-08 3:23 ` Jeff King
2025-03-14 21:09 ` [PATCH v2 0/3] -Wunreachable-code Junio C Hamano
2 siblings, 0 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-07 23:28 UTC (permalink / raw)
To: Jeff King; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
Jeff King <peff@peff.net> writes:
> I was disappointed that the compiler didn't complain, though. Maybe we
> should do this:
Indeed. It would have helped us if it were already there in place.
> -- >8 --
> Subject: [PATCH] config.mak.dev: enable -Wunreachable-code
>
> Having the compiler point out unreachable code can help avoid bugs, like
> the one discussed in:
>
> https://lore.kernel.org/git/20250307195057.GA3675279@coredump.intra.peff.net/
>
> In that case it was found by Coverity, but finding it earlier saves
> everybody time and effort.
>
> We can use -Wunreachable-code to get some help from the compiler here.
> Interestingly, this is a noop in gcc. It was a real warning up until gcc
> 4.x, when it was removed for being too flaky, but they left the
> command-line option to avoid breaking users. See:
>
> https://stackoverflow.com/questions/17249934/why-does-gcc-not-warn-for-unreachable-code
Wow, now they leave their users confused, making them wondering why
their command line option does not do anything useful ;-)
> However, clang does implement this option, and it finds the case
> mentioned above (and no other cases within the code base). And since we
> run clang in several of our CI jobs, that's enough to get an early
> warning of breakage.
Yes, this is great.
Thanks.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH] config.mak.dev: enable -Wunreachable-code
2025-03-07 22:54 ` [PATCH] config.mak.dev: enable -Wunreachable-code Jeff King
2025-03-07 23:28 ` Junio C Hamano
@ 2025-03-08 3:23 ` Jeff King
2025-03-10 15:40 ` Junio C Hamano
2025-03-14 21:09 ` [PATCH v2 0/3] -Wunreachable-code Junio C Hamano
2 siblings, 1 reply; 143+ messages in thread
From: Jeff King @ 2025-03-08 3:23 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
On Fri, Mar 07, 2025 at 05:54:45PM -0500, Jeff King wrote:
> However, clang does implement this option, and it finds the case
> mentioned above (and no other cases within the code base). And since we
> run clang in several of our CI jobs, that's enough to get an early
> warning of breakage.
Hmph, this might be more trouble than it is worth.
After correcting the problem in the refs code, the osx CI builds (and
only those) now fail with:
run-command.c:519:3: error: code will never be executed [-Werror,-Wunreachable-code]
die_errno("sigfillset");
^~~~~~~~~
The code in question is just:
if (sigfillset(&all))
die_errno("sigfillset");
So I have to imagine that the issue is that sigfillset() on that
platform is an inline or macro that will never return an error, and the
compiler can see that. But since POSIX says this can fail (though I'd
imagine it's unlikely on most platforms), we should check in the general
case.
So I don't see how to solve it short of:
#ifdef SIGFILLSET_CANNOT_FAIL
sigfillset(&all);
#else
if (sigfillset(&all))
die_errno("sigfillset");
#endif
which is rather ugly. It's only used in one spot, so the damage doesn't
go too far, but I don't love the idea of getting surprised by the
compiler over-analyzing system functions (and having to add Makefile
knobs to support it).
I guess a knob-less version is:
errno = 0;
sigfillset(&all); /* don't check return value! only errno */
if (errno)
die_errno("sigfillset");
which is subtle, to say the least.
-Peff
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH] config.mak.dev: enable -Wunreachable-code
2025-03-08 3:23 ` Jeff King
@ 2025-03-10 15:40 ` Junio C Hamano
2025-03-10 16:04 ` Jeff King
0 siblings, 1 reply; 143+ messages in thread
From: Junio C Hamano @ 2025-03-10 15:40 UTC (permalink / raw)
To: Jeff King; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
Jeff King <peff@peff.net> writes:
> On Fri, Mar 07, 2025 at 05:54:45PM -0500, Jeff King wrote:
>
>> However, clang does implement this option, and it finds the case
>> mentioned above (and no other cases within the code base). And since we
>> run clang in several of our CI jobs, that's enough to get an early
>> warning of breakage.
>
> Hmph, this might be more trouble than it is worth.
>
> After correcting the problem in the refs code, the osx CI builds (and
> only those) now fail with:
>
> run-command.c:519:3: error: code will never be executed [-Werror,-Wunreachable-code]
> die_errno("sigfillset");
> ^~~~~~~~~
> ...
> I guess a knob-less version is:
>
> errno = 0;
> sigfillset(&all); /* don't check return value! only errno */
> if (errno)
> die_errno("sigfillset");
>
> which is subtle, to say the least.
Bah. This is just as horrible as some other warnings that are not
enabled by default. I guess we should just be more vigilant X-<.
Thanks.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH] config.mak.dev: enable -Wunreachable-code
2025-03-10 15:40 ` Junio C Hamano
@ 2025-03-10 16:04 ` Jeff King
2025-03-10 18:50 ` Junio C Hamano
0 siblings, 1 reply; 143+ messages in thread
From: Jeff King @ 2025-03-10 16:04 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
On Mon, Mar 10, 2025 at 08:40:46AM -0700, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > On Fri, Mar 07, 2025 at 05:54:45PM -0500, Jeff King wrote:
> >
> >> However, clang does implement this option, and it finds the case
> >> mentioned above (and no other cases within the code base). And since we
> >> run clang in several of our CI jobs, that's enough to get an early
> >> warning of breakage.
> >
> > Hmph, this might be more trouble than it is worth.
> >
> > After correcting the problem in the refs code, the osx CI builds (and
> > only those) now fail with:
> >
> > run-command.c:519:3: error: code will never be executed [-Werror,-Wunreachable-code]
> > die_errno("sigfillset");
> > ^~~~~~~~~
> > ...
> > I guess a knob-less version is:
> >
> > errno = 0;
> > sigfillset(&all); /* don't check return value! only errno */
> > if (errno)
> > die_errno("sigfillset");
> >
> > which is subtle, to say the least.
>
> Bah. This is just as horrible as some other warnings that are not
> enabled by default. I guess we should just be more vigilant X-<.
Yeah. We could perhaps live with hacking around this one specific spot.
But there's an open question of how often these kinds of false positives
will come up.
Maybe not often, if there is only one instance in the current code base.
Or maybe a lot, but we wouldn't know because we haven't had the warning
enabled.
I guess another option is to enable it in _one_ CI job that uses clang
on Linux (maybe linux-sha256?) and see how often it is helpful or
harmful.
-Peff
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH] config.mak.dev: enable -Wunreachable-code
2025-03-10 16:04 ` Jeff King
@ 2025-03-10 18:50 ` Junio C Hamano
2025-03-14 16:10 ` Jeff King
0 siblings, 1 reply; 143+ messages in thread
From: Junio C Hamano @ 2025-03-10 18:50 UTC (permalink / raw)
To: Jeff King; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
Jeff King <peff@peff.net> writes:
> Maybe not often, if there is only one instance in the current code base.
> Or maybe a lot, but we wouldn't know because we haven't had the warning
> enabled.
>
> I guess another option is to enable it in _one_ CI job that uses clang
> on Linux (maybe linux-sha256?) and see how often it is helpful or
> harmful.
The reason why you said Linux rather than macOS is because the
single instance we know about would not have to be worked around if
we did it that way?
I am OK with that.
Thanks.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH] config.mak.dev: enable -Wunreachable-code
2025-03-10 18:50 ` Junio C Hamano
@ 2025-03-14 16:10 ` Jeff King
2025-03-14 16:13 ` Jeff King
2025-03-14 17:15 ` Junio C Hamano
0 siblings, 2 replies; 143+ messages in thread
From: Jeff King @ 2025-03-14 16:10 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
On Mon, Mar 10, 2025 at 11:50:20AM -0700, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > Maybe not often, if there is only one instance in the current code base.
> > Or maybe a lot, but we wouldn't know because we haven't had the warning
> > enabled.
> >
> > I guess another option is to enable it in _one_ CI job that uses clang
> > on Linux (maybe linux-sha256?) and see how often it is helpful or
> > harmful.
>
> The reason why you said Linux rather than macOS is because the
> single instance we know about would not have to be worked around if
> we did it that way?
>
> I am OK with that.
Yes, exactly. I started to prepare a patch for that, but then I realized
I'd probably be adding support in config.mak.dev. So we could also just
handle it automatically there, skipping the flag on macOS.
That would use the flag in more situations (blocking the known-bad case,
rather than enabling it in a known-good one). It might hit more false
positives, but I'd rather experiment in that direction and see if
anybody setting DEVELOPER=1 complains. After all, in either case it is
still a big question of whether this is the only false positive we'll
see, or if this is opening up a can of worms. So I consider it all
kind-of exploratory.
So that patch could look like this (on top of what you've queued already
in jk/use-wunreachable-code-for-devs).
-- >8 --
Subject: [PATCH] config.mak.dev: disable -Wunreachable-code on macOS
We've seen false positives here related to calling sigfillset(); even
though POSIX specifies that it may return an error, it transparently (to
the compiler) always returns success on macOS. As a result, the compiler
flags the error path in something like:
if (sigfillset(&set))
die(...);
as unreachable (which it is on this platform, but not in the general
case). We could work around it, but let's just disable the warning on
this platform. There are plenty of CI jobs that will still trigger it
(e.g., all of the linux+clang jobs).
Signed-off-by: Jeff King <peff@peff.net>
---
It's possible FreeBSD might share the same problem, but their manpage
does not seem to have the same "it always returns 0" language. But we
might need to expand this list if people report more problems.
config.mak.dev | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/config.mak.dev b/config.mak.dev
index 95b7bc46ae..30dcd0c175 100644
--- a/config.mak.dev
+++ b/config.mak.dev
@@ -39,7 +39,12 @@ DEVELOPER_CFLAGS += -Wunused
DEVELOPER_CFLAGS += -Wvla
DEVELOPER_CFLAGS += -Wwrite-strings
DEVELOPER_CFLAGS += -fno-common
+
+# There are false positives for unreachable code related to system
+# functions on macOS.
+ifneq ($(uname_S),Darwin)
DEVELOPER_CFLAGS += -Wunreachable-code
+endif
ifneq ($(filter clang4,$(COMPILER_FEATURES)),)
DEVELOPER_CFLAGS += -Wtautological-constant-out-of-range-compare
--
2.49.0.rc2.384.gf2d6285ccb
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH] config.mak.dev: enable -Wunreachable-code
2025-03-14 16:10 ` Jeff King
@ 2025-03-14 16:13 ` Jeff King
2025-03-14 17:27 ` Junio C Hamano
2025-03-14 17:15 ` Junio C Hamano
1 sibling, 1 reply; 143+ messages in thread
From: Jeff King @ 2025-03-14 16:13 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
On Fri, Mar 14, 2025 at 12:10:10PM -0400, Jeff King wrote:
> So that patch could look like this (on top of what you've queued already
> in jk/use-wunreachable-code-for-devs).
>
> -- >8 --
> Subject: [PATCH] config.mak.dev: disable -Wunreachable-code on macOS
> [...]
> ---
> It's possible FreeBSD might share the same problem, but their manpage
> does not seem to have the same "it always returns 0" language. But we
> might need to expand this list if people report more problems.
And I'm still a bit tempted to instead actually silence this one false
positive, and keep the warning enabled everywhere. That would help
FreeBSD (if it indeed does have the same issue) and let macOS benefit
from the warning (most code paths would be covered on Linux anyway, but
there could be platform-specific ones).
And that patch would look like this (again, on top of what you've
already queued, and replacing the patch I'm replying to):
-- >8 --
Subject: [PATCH] run-command: use errno to check for sigfillset() error
Since enabling -Wunreachable-code, builds with clang on macOS now fail,
complaining that the die_errno() call in:
if (sigfillset(&all))
die_errno("sigfillset");
is unreachable. On that platform the manpage documents that sigfillset()
always returns success, and presumably the implementation is a macro or
inline function that does so in a way that is transparent to the
compiler.
But we should continue to check on other platforms, since POSIX says it
may return an error.
We could solve this with a compile-time knob to split the two cases
(assuming success on macOS and checking for the error elsewhere). But we
can also work around it more directly by relying on errno to check the
outcome (since POSIX dictates that errno will be set on error). And that
works around the compiler's cleverness, since it doesn't know the
semantics of errno (though I suppose if sigfillset() is simple enough,
it could perhaps realize that no writes to errno are possible; however
this does seem to work in practice).
Signed-off-by: Jeff King <peff@peff.net>
---
run-command.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/run-command.c b/run-command.c
index 402138b8b5..d527c46175 100644
--- a/run-command.c
+++ b/run-command.c
@@ -515,7 +515,15 @@ static void atfork_prepare(struct atfork_state *as)
{
sigset_t all;
- if (sigfillset(&all))
+ /*
+ * Do not use the return value of sigfillset(). It is transparently 0
+ * on some platforms, meaning a clever compiler may complain that
+ * the conditional body is dead code. Instead, check for error via
+ * errno, which outsmarts the compiler.
+ */
+ errno = 0;
+ sigfillset(&all);
+ if (errno)
die_errno("sigfillset");
#ifdef NO_PTHREADS
if (sigprocmask(SIG_SETMASK, &all, &as->old))
--
2.49.0.rc2.384.gf2d6285ccb
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH] config.mak.dev: enable -Wunreachable-code
2025-03-14 16:10 ` Jeff King
2025-03-14 16:13 ` Jeff King
@ 2025-03-14 17:15 ` Junio C Hamano
1 sibling, 0 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-14 17:15 UTC (permalink / raw)
To: Jeff King; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
Jeff King <peff@peff.net> writes:
> That would use the flag in more situations (blocking the known-bad case,
> rather than enabling it in a known-good one). It might hit more false
> positives, but I'd rather experiment in that direction and see if
> anybody setting DEVELOPER=1 complains.
Good.
> After all, in either case it is
> still a big question of whether this is the only false positive we'll
> see, or if this is opening up a can of worms. So I consider it all
> kind-of exploratory.
Again, good.
> So that patch could look like this (on top of what you've queued already
> in jk/use-wunreachable-code-for-devs).
> +
> +# There are false positives for unreachable code related to system
> +# functions on macOS.
> +ifneq ($(uname_S),Darwin)
> DEVELOPER_CFLAGS += -Wunreachable-code
> +endif
One possible downside of this is that we would not know when their
compiler stops giving the "false positive" and becomes as usuable as
other platforms (oh, it came out unintendedly harsh---it could be
that the situation is that their compiler is doing the right thing,
and the right thing is a bit inconvenient for this codebase).
Unless diligent volunteers with macOS step up to do trial builds
with the option when they notice that their toolchain or OS header
files got upgraded, that is.
But other than that, I am fine with this. Let's have this for some
time to see how much problems (false positives) our newly added code
would get to judge if it is worth our time to deal with them.
Thanks.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH] config.mak.dev: enable -Wunreachable-code
2025-03-14 16:13 ` Jeff King
@ 2025-03-14 17:27 ` Junio C Hamano
2025-03-14 17:40 ` Junio C Hamano
0 siblings, 1 reply; 143+ messages in thread
From: Junio C Hamano @ 2025-03-14 17:27 UTC (permalink / raw)
To: Jeff King; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
Jeff King <peff@peff.net> writes:
> -- >8 --
> Subject: [PATCH] run-command: use errno to check for sigfillset() error
>
> Since enabling -Wunreachable-code, builds with clang on macOS now fail,
> complaining that the die_errno() call in:
>
> if (sigfillset(&all))
> die_errno("sigfillset");
>
> is unreachable. On that platform the manpage documents that sigfillset()
> always returns success, and presumably the implementation is a macro or
> inline function that does so in a way that is transparent to the
> compiler.
Would it work to instead do this here
if (sigfillset(&all) || false_but_compiler_does_not_know_it)
die_error("sigfillset");
with
extern int false_but_compiler_does_not_know_it;
in <git-compat-util.h>? And a standalone .c file with its
definition
#include <git-compat-util.h>
int false_but_compiler_does_not_know_it;
and nothing else, linked into libgit.a?
I am hoping that such a false-positive would come from conditionals
that are known to be compiler to be always taken (or never taken),
so eventually we can mark such an expression with a macro, e.g.
if (CAN_BE_TAKEN(sigfilllset(&all))
die_error("sigfillset");
Because in this particular case we _can_ rely on errno, so the patch
we see here is perfectly fine by me, but a more generic approach
like the above would make it unnecessary to
- have a 4-line comment
- come up with workaround
suitable for each such places we need to work around compiler
smarta^hness.
> But we should continue to check on other platforms, since POSIX says it
> may return an error.
>
> We could solve this with a compile-time knob to split the two cases
> (assuming success on macOS and checking for the error elsewhere). But we
> can also work around it more directly by relying on errno to check the
> outcome (since POSIX dictates that errno will be set on error). And that
> works around the compiler's cleverness, since it doesn't know the
> semantics of errno (though I suppose if sigfillset() is simple enough,
> it could perhaps realize that no writes to errno are possible; however
> this does seem to work in practice).
>
> Signed-off-by: Jeff King <peff@peff.net>
> ---
> run-command.c | 10 +++++++++-
> 1 file changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/run-command.c b/run-command.c
> index 402138b8b5..d527c46175 100644
> --- a/run-command.c
> +++ b/run-command.c
> @@ -515,7 +515,15 @@ static void atfork_prepare(struct atfork_state *as)
> {
> sigset_t all;
>
> - if (sigfillset(&all))
> + /*
> + * Do not use the return value of sigfillset(). It is transparently 0
> + * on some platforms, meaning a clever compiler may complain that
> + * the conditional body is dead code. Instead, check for error via
> + * errno, which outsmarts the compiler.
> + */
> + errno = 0;
> + sigfillset(&all);
> + if (errno)
> die_errno("sigfillset");
> #ifdef NO_PTHREADS
> if (sigprocmask(SIG_SETMASK, &all, &as->old))
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH] config.mak.dev: enable -Wunreachable-code
2025-03-14 17:27 ` Junio C Hamano
@ 2025-03-14 17:40 ` Junio C Hamano
2025-03-14 17:43 ` Patrick Steinhardt
2025-03-14 18:53 ` Jeff King
0 siblings, 2 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-14 17:40 UTC (permalink / raw)
To: Jeff King; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
Junio C Hamano <gitster@pobox.com> writes:
> Jeff King <peff@peff.net> writes:
>
>> -- >8 --
>> Subject: [PATCH] run-command: use errno to check for sigfillset() error
>>
>> Since enabling -Wunreachable-code, builds with clang on macOS now fail,
>> complaining that the die_errno() call in:
>>
>> if (sigfillset(&all))
>> die_errno("sigfillset");
>>
>> is unreachable. On that platform the manpage documents that sigfillset()
>> always returns success, and presumably the implementation is a macro or
>> inline function that does so in a way that is transparent to the
>> compiler.
>
> Would it work to instead do this here
> ...
I forgot to say a more important thing. Between the "let's excempt
developers on macOS" and the "let's see how far we can go with the
warning turned on everywhere and wack-a-mole this particular one
with errno check" patches, I prefer the latter at least for a short
term.
Thanks.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH] config.mak.dev: enable -Wunreachable-code
2025-03-14 17:40 ` Junio C Hamano
@ 2025-03-14 17:43 ` Patrick Steinhardt
2025-03-14 18:53 ` Jeff King
1 sibling, 0 replies; 143+ messages in thread
From: Patrick Steinhardt @ 2025-03-14 17:43 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jeff King, Karthik Nayak, git, jltobler, phillip.wood123
On Fri, Mar 14, 2025 at 10:40:24AM -0700, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
> > Jeff King <peff@peff.net> writes:
> >
> >> -- >8 --
> >> Subject: [PATCH] run-command: use errno to check for sigfillset() error
> >>
> >> Since enabling -Wunreachable-code, builds with clang on macOS now fail,
> >> complaining that the die_errno() call in:
> >>
> >> if (sigfillset(&all))
> >> die_errno("sigfillset");
> >>
> >> is unreachable. On that platform the manpage documents that sigfillset()
> >> always returns success, and presumably the implementation is a macro or
> >> inline function that does so in a way that is transparent to the
> >> compiler.
> >
> > Would it work to instead do this here
> > ...
>
> I forgot to say a more important thing. Between the "let's excempt
> developers on macOS" and the "let's see how far we can go with the
> warning turned on everywhere and wack-a-mole this particular one
> with errno check" patches, I prefer the latter at least for a short
> term.
Yeah, I'm also in favor of generally enabling the warning and seeing
whether it will end up being a pain or not. This particular edge case
here is ugly, but it's manageable and may protect us from mistakes in
other places going forward.
If we do so, could we please also include the following patch for Meson?
Thanks!
Patrick
diff --git a/meson.build b/meson.build
index efe2871c9d..a0a602864a 100644
--- a/meson.build
+++ b/meson.build
@@ -721,6 +721,7 @@ if get_option('warning_level') in ['2','3', 'everything'] and compiler.get_argum
'-Woverflow',
'-Wpointer-arith',
'-Wstrict-prototypes',
+ '-Wunreachable-code',
'-Wunused',
'-Wvla',
'-Wwrite-strings',
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH] config.mak.dev: enable -Wunreachable-code
2025-03-14 17:40 ` Junio C Hamano
2025-03-14 17:43 ` Patrick Steinhardt
@ 2025-03-14 18:53 ` Jeff King
2025-03-14 19:50 ` Junio C Hamano
1 sibling, 1 reply; 143+ messages in thread
From: Jeff King @ 2025-03-14 18:53 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
On Fri, Mar 14, 2025 at 10:40:24AM -0700, Junio C Hamano wrote:
> >> -- >8 --
> >> Subject: [PATCH] run-command: use errno to check for sigfillset() error
> >>
> >> Since enabling -Wunreachable-code, builds with clang on macOS now fail,
> >> complaining that the die_errno() call in:
> >>
> >> if (sigfillset(&all))
> >> die_errno("sigfillset");
> >>
> >> is unreachable. On that platform the manpage documents that sigfillset()
> >> always returns success, and presumably the implementation is a macro or
> >> inline function that does so in a way that is transparent to the
> >> compiler.
> >
> > Would it work to instead do this here
> > ...
>
> I forgot to say a more important thing. Between the "let's excempt
> developers on macOS" and the "let's see how far we can go with the
> warning turned on everywhere and wack-a-mole this particular one
> with errno check" patches, I prefer the latter at least for a short
> term.
That's my gut feeling, too. I wasn't sure how people would feel about
actually touching the code (whereas the other patches were purely
turning compiler knobs). It may turn into wack-a-mole, but finding out
is part of the experiment.
Your CAN_BE_TAKEN() approach is certainly less subtle, and can be
applied in a more general way. If this is the only spot needed it may be
overkill, but the readability improvement alone probably makes it
worthwhile.
Do you want to turn that into a patch?
-Peff
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH] config.mak.dev: enable -Wunreachable-code
2025-03-14 18:53 ` Jeff King
@ 2025-03-14 19:50 ` Junio C Hamano
0 siblings, 0 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-14 19:50 UTC (permalink / raw)
To: Jeff King; +Cc: Karthik Nayak, git, ps, jltobler, phillip.wood123
Jeff King <peff@peff.net> writes:
> Your CAN_BE_TAKEN() approach is certainly less subtle, and can be
> applied in a more general way. If this is the only spot needed it may be
> overkill, but the readability improvement alone probably makes it
> worthwhile.
>
> Do you want to turn that into a patch?
Yes, but after I come up with a better name. CAN_BE_TAKEN may be OK
for if/while but not good enough for switch() for example. "Do not
opmimize out because, despite your beliefs, this expression is ..."
is what we want to convey.
Makefile | 1 +
git-compat-util.h | 9 +++++++++
meson.build | 1 +
run-command.c | 12 +++++-------
4 files changed, 16 insertions(+), 7 deletions(-)
diff --git c/Makefile w/Makefile
index 97e8385b66..2158bf6916 100644
--- c/Makefile
+++ w/Makefile
@@ -1018,6 +1018,7 @@ LIB_OBJS += ewah/ewah_bitmap.o
LIB_OBJS += ewah/ewah_io.o
LIB_OBJS += ewah/ewah_rlw.o
LIB_OBJS += exec-cmd.o
+LIB_OBJS += fbtcdnki.o
LIB_OBJS += fetch-negotiator.o
LIB_OBJS += fetch-pack.o
LIB_OBJS += fmt-merge-msg.o
diff --git c/git-compat-util.h w/git-compat-util.h
index e283c46c6f..63a3ef6b70 100644
--- c/git-compat-util.h
+++ w/git-compat-util.h
@@ -1593,4 +1593,13 @@ static inline void *container_of_or_null_offset(void *ptr, size_t offset)
((uintptr_t)&(ptr)->member - (uintptr_t)(ptr))
#endif /* !__GNUC__ */
+/*
+ * Prevent an overly clever compiler from optimizing an expression
+ * out, triggering a false positive when building with the
+ * -Wunreachable-code option. false_but_the_compiler_does_not_know_it_
+ * is defined in a compilation unit separate from where the macro is
+ * used, initialized to 0, and never modified.
+ */
+#define NOT_A_CONST(expr) ((expr) || false_but_the_compiler_does_not_know_it_)
+extern int false_but_the_compiler_does_not_know_it_;
#endif
diff --git c/meson.build w/meson.build
index f60f3f49e4..ce642dcf65 100644
--- c/meson.build
+++ w/meson.build
@@ -282,6 +282,7 @@ libgit_sources = [
'ewah/ewah_io.c',
'ewah/ewah_rlw.c',
'exec-cmd.c',
+ 'fbtcdnki.c',
'fetch-negotiator.c',
'fetch-pack.c',
'fmt-merge-msg.c',
diff --git c/run-command.c w/run-command.c
index d527c46175..535c73a059 100644
--- c/run-command.c
+++ w/run-command.c
@@ -516,14 +516,12 @@ static void atfork_prepare(struct atfork_state *as)
sigset_t all;
/*
- * Do not use the return value of sigfillset(). It is transparently 0
- * on some platforms, meaning a clever compiler may complain that
- * the conditional body is dead code. Instead, check for error via
- * errno, which outsmarts the compiler.
+ * POSIX says sitfillset() can fail, but an overly clever
+ * compiler can see through the header files and decide
+ * it cannot fail on a particular platform it is compiling for,
+ * triggering -Wunreachable-code false positive.
*/
- errno = 0;
- sigfillset(&all);
- if (errno)
+ if (NOT_A_CONST(sigfillset(&all)))
die_errno("sigfillset");
#ifdef NO_PTHREADS
if (sigprocmask(SIG_SETMASK, &all, &as->old))
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v2 0/3] -Wunreachable-code
2025-03-07 22:54 ` [PATCH] config.mak.dev: enable -Wunreachable-code Jeff King
2025-03-07 23:28 ` Junio C Hamano
2025-03-08 3:23 ` Jeff King
@ 2025-03-14 21:09 ` Junio C Hamano
2025-03-14 21:09 ` [PATCH v2 1/3] config.mak.dev: enable -Wunreachable-code Junio C Hamano
` (3 more replies)
2 siblings, 4 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-14 21:09 UTC (permalink / raw)
To: git; +Cc: Jeff King, Patrick Steinhardt
So here is a recap. The first one has meson.build change from
Patrick squashed in, the second "errno" based one was what made
me write the last one, and is kept as-is. The third one introduces
NOT_A_CONST() marking to an expression to tell the compiler not to
be overly aggressive to optimize it out.
Jeff King (2):
config.mak.dev: enable -Wunreachable-code
run-command: use errno to check for sigfillset() error
Junio C Hamano (1):
git-compat-util: add NOT_A_CONST macro and use it in atfork_prepare()
Makefile | 1 +
config.mak.dev | 1 +
git-compat-util.h | 9 +++++++++
meson.build | 2 ++
run-command.c | 8 +++++++-
5 files changed, 20 insertions(+), 1 deletion(-)
--
2.49.0-188-g35fcca2323
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH v2 1/3] config.mak.dev: enable -Wunreachable-code
2025-03-14 21:09 ` [PATCH v2 0/3] -Wunreachable-code Junio C Hamano
@ 2025-03-14 21:09 ` Junio C Hamano
2025-03-14 21:09 ` [PATCH v2 2/3] run-command: use errno to check for sigfillset() error Junio C Hamano
` (2 subsequent siblings)
3 siblings, 0 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-14 21:09 UTC (permalink / raw)
To: git; +Cc: Jeff King, Patrick Steinhardt
From: Jeff King <peff@peff.net>
Having the compiler point out unreachable code can help avoid bugs, like
the one discussed in:
https://lore.kernel.org/git/20250307195057.GA3675279@coredump.intra.peff.net/
In that case it was found by Coverity, but finding it earlier saves
everybody time and effort.
We can use -Wunreachable-code to get some help from the compiler here.
Interestingly, this is a noop in gcc. It was a real warning up until gcc
4.x, when it was removed for being too flaky, but they left the
command-line option to avoid breaking users. See:
https://stackoverflow.com/questions/17249934/why-does-gcc-not-warn-for-unreachable-code
However, clang does implement this option, and it finds the case
mentioned above (and no other cases within the code base). And since we
run clang in several of our CI jobs, that's enough to get an early
warning of breakage.
We could enable it only for clang, but since gcc is happy to ignore it,
it's simpler to just turn it on for all developer builds.
Signed-off-by: Jeff King <peff@peff.net>
[jc: squashed meson.build change sent by Patrick]
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
config.mak.dev | 1 +
meson.build | 1 +
2 files changed, 2 insertions(+)
diff --git a/config.mak.dev b/config.mak.dev
index 0fd8cc4d35..95b7bc46ae 100644
--- a/config.mak.dev
+++ b/config.mak.dev
@@ -39,6 +39,7 @@ DEVELOPER_CFLAGS += -Wunused
DEVELOPER_CFLAGS += -Wvla
DEVELOPER_CFLAGS += -Wwrite-strings
DEVELOPER_CFLAGS += -fno-common
+DEVELOPER_CFLAGS += -Wunreachable-code
ifneq ($(filter clang4,$(COMPILER_FEATURES)),)
DEVELOPER_CFLAGS += -Wtautological-constant-out-of-range-compare
diff --git a/meson.build b/meson.build
index 0064eb64f5..f60f3f49e4 100644
--- a/meson.build
+++ b/meson.build
@@ -697,6 +697,7 @@ if get_option('warning_level') in ['2','3', 'everything'] and compiler.get_argum
'-Woverflow',
'-Wpointer-arith',
'-Wstrict-prototypes',
+ '-Wunreachable-code',
'-Wunused',
'-Wvla',
'-Wwrite-strings',
--
2.49.0-188-g35fcca2323
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v2 2/3] run-command: use errno to check for sigfillset() error
2025-03-14 21:09 ` [PATCH v2 0/3] -Wunreachable-code Junio C Hamano
2025-03-14 21:09 ` [PATCH v2 1/3] config.mak.dev: enable -Wunreachable-code Junio C Hamano
@ 2025-03-14 21:09 ` Junio C Hamano
2025-03-17 21:30 ` Taylor Blau
2025-03-14 21:09 ` [PATCH v2 3/3] git-compat-util: add NOT_A_CONST macro and use it in atfork_prepare() Junio C Hamano
2025-03-17 23:53 ` [PATCH v3 0/3] -Wunreachable-code Junio C Hamano
3 siblings, 1 reply; 143+ messages in thread
From: Junio C Hamano @ 2025-03-14 21:09 UTC (permalink / raw)
To: git; +Cc: Jeff King
From: Jeff King <peff@peff.net>
Since enabling -Wunreachable-code, builds with clang on macOS now fail,
complaining that the die_errno() call in:
if (sigfillset(&all))
die_errno("sigfillset");
is unreachable. On that platform the manpage documents that sigfillset()
always returns success, and presumably the implementation is a macro or
inline function that does so in a way that is transparent to the
compiler.
But we should continue to check on other platforms, since POSIX says it
may return an error.
We could solve this with a compile-time knob to split the two cases
(assuming success on macOS and checking for the error elsewhere). But we
can also work around it more directly by relying on errno to check the
outcome (since POSIX dictates that errno will be set on error). And that
works around the compiler's cleverness, since it doesn't know the
semantics of errno (though I suppose if sigfillset() is simple enough,
it could perhaps realize that no writes to errno are possible; however
this does seem to work in practice).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
run-command.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/run-command.c b/run-command.c
index 402138b8b5..d527c46175 100644
--- a/run-command.c
+++ b/run-command.c
@@ -515,7 +515,15 @@ static void atfork_prepare(struct atfork_state *as)
{
sigset_t all;
- if (sigfillset(&all))
+ /*
+ * Do not use the return value of sigfillset(). It is transparently 0
+ * on some platforms, meaning a clever compiler may complain that
+ * the conditional body is dead code. Instead, check for error via
+ * errno, which outsmarts the compiler.
+ */
+ errno = 0;
+ sigfillset(&all);
+ if (errno)
die_errno("sigfillset");
#ifdef NO_PTHREADS
if (sigprocmask(SIG_SETMASK, &all, &as->old))
--
2.49.0-188-g35fcca2323
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v2 3/3] git-compat-util: add NOT_A_CONST macro and use it in atfork_prepare()
2025-03-14 21:09 ` [PATCH v2 0/3] -Wunreachable-code Junio C Hamano
2025-03-14 21:09 ` [PATCH v2 1/3] config.mak.dev: enable -Wunreachable-code Junio C Hamano
2025-03-14 21:09 ` [PATCH v2 2/3] run-command: use errno to check for sigfillset() error Junio C Hamano
@ 2025-03-14 21:09 ` Junio C Hamano
2025-03-14 22:29 ` Junio C Hamano
2025-03-17 23:53 ` [PATCH v3 0/3] -Wunreachable-code Junio C Hamano
3 siblings, 1 reply; 143+ messages in thread
From: Junio C Hamano @ 2025-03-14 21:09 UTC (permalink / raw)
To: git; +Cc: Jeff King, Patrick Steinhardt
Our hope is that the number of code paths that falsely trigger
warnings with the -Wunreachable-code compilation option are small,
and they can be worked around case-by-case basis, like we just did
in the previous commit. If we need such a workaround a bit more
often, however, we may benefit from a more generic and descriptive
facility that helps document the cases we need such workarounds.
Side note: if we need the workaround all over the place, it
simply means -Wunreachable-code is not a good tool for us to
save engineering effort to catch mistakes. We are still
exploring if it helps us, so let's assume that it is not the
case.
Introduce NOT_A_CONST() macro, with which, the developer can tell
the compiler:
Do not optimize this expression out, because, despite whatever
you are told by the system headers, this expression should *not*
be treated as a constant.
and use it as a replacement for the workaround we used that was
somewhat specific to the sigfillset case. If the compiler already
knows that the call to sigfillset() cannot fail on a particular
platform it is compiling for and declares that the if() condition
would not hold, it is plausible that the next version of the
compiler may learn that sigfillset() that never fails would not
touch errno and decide that in this sequence:
errno = 0;
sigfillset(&all)
if (errno)
die_errno("sigfillset");
the if() statement will never trigger. Marking that the value
returned by sigfillset() cannot be a constant would document our
intention better and would not break with such a new version of
compiler that is even more "clever". With the marco, the above
sequence can be rewritten:
if (NOT_A_CONST(sigfillset(&all)))
die_errno("sigfillset");
which looks almost like other innocuous annotations we have,
e.g. UNUSED.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
Makefile | 1 +
git-compat-util.h | 9 +++++++++
meson.build | 1 +
run-command.c | 12 +++++-------
4 files changed, 16 insertions(+), 7 deletions(-)
diff --git a/Makefile b/Makefile
index 97e8385b66..2158bf6916 100644
--- a/Makefile
+++ b/Makefile
@@ -1018,6 +1018,7 @@ LIB_OBJS += ewah/ewah_bitmap.o
LIB_OBJS += ewah/ewah_io.o
LIB_OBJS += ewah/ewah_rlw.o
LIB_OBJS += exec-cmd.o
+LIB_OBJS += fbtcdnki.o
LIB_OBJS += fetch-negotiator.o
LIB_OBJS += fetch-pack.o
LIB_OBJS += fmt-merge-msg.o
diff --git a/git-compat-util.h b/git-compat-util.h
index e283c46c6f..63a3ef6b70 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -1593,4 +1593,13 @@ static inline void *container_of_or_null_offset(void *ptr, size_t offset)
((uintptr_t)&(ptr)->member - (uintptr_t)(ptr))
#endif /* !__GNUC__ */
+/*
+ * Prevent an overly clever compiler from optimizing an expression
+ * out, triggering a false positive when building with the
+ * -Wunreachable-code option. false_but_the_compiler_does_not_know_it_
+ * is defined in a compilation unit separate from where the macro is
+ * used, initialized to 0, and never modified.
+ */
+#define NOT_A_CONST(expr) ((expr) || false_but_the_compiler_does_not_know_it_)
+extern int false_but_the_compiler_does_not_know_it_;
#endif
diff --git a/meson.build b/meson.build
index f60f3f49e4..ce642dcf65 100644
--- a/meson.build
+++ b/meson.build
@@ -282,6 +282,7 @@ libgit_sources = [
'ewah/ewah_io.c',
'ewah/ewah_rlw.c',
'exec-cmd.c',
+ 'fbtcdnki.c',
'fetch-negotiator.c',
'fetch-pack.c',
'fmt-merge-msg.c',
diff --git a/run-command.c b/run-command.c
index d527c46175..535c73a059 100644
--- a/run-command.c
+++ b/run-command.c
@@ -516,14 +516,12 @@ static void atfork_prepare(struct atfork_state *as)
sigset_t all;
/*
- * Do not use the return value of sigfillset(). It is transparently 0
- * on some platforms, meaning a clever compiler may complain that
- * the conditional body is dead code. Instead, check for error via
- * errno, which outsmarts the compiler.
+ * POSIX says sitfillset() can fail, but an overly clever
+ * compiler can see through the header files and decide
+ * it cannot fail on a particular platform it is compiling for,
+ * triggering -Wunreachable-code false positive.
*/
- errno = 0;
- sigfillset(&all);
- if (errno)
+ if (NOT_A_CONST(sigfillset(&all)))
die_errno("sigfillset");
#ifdef NO_PTHREADS
if (sigprocmask(SIG_SETMASK, &all, &as->old))
--
2.49.0-188-g35fcca2323
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH v2 3/3] git-compat-util: add NOT_A_CONST macro and use it in atfork_prepare()
2025-03-14 21:09 ` [PATCH v2 3/3] git-compat-util: add NOT_A_CONST macro and use it in atfork_prepare() Junio C Hamano
@ 2025-03-14 22:29 ` Junio C Hamano
2025-03-17 18:00 ` Jeff King
0 siblings, 1 reply; 143+ messages in thread
From: Junio C Hamano @ 2025-03-14 22:29 UTC (permalink / raw)
To: git; +Cc: Jeff King, Patrick Steinhardt
Sorry, one new file was left out of the patch. Here is a quick fix
(I am not rerolling the earlier 2 steps).
---- >8 ----
Our hope is that the number of code paths that falsely trigger
warnings with the -Wunreachable-code compilation option are small,
and they can be worked around case-by-case basis, like we just did
in the previous commit. If we need such a workaround a bit more
often, however, we may benefit from a more generic and descriptive
facility that helps document the cases we need such workarounds.
Side note: if we need the workaround all over the place, it
simply means -Wunreachable-code is not a good tool for us to
save engineering effort to catch mistakes. We are still
exploring if it helps us, so let's assume that it is not the
case.
Introduce NOT_A_CONST() macro, with which, the developer can tell
the compiler:
Do not optimize this expression out, because, despite whatever
you are told by the system headers, this expression should *not*
be treated as a constant.
and use it as a replacement for the workaround we used that was
somewhat specific to the sigfillset case. If the compiler already
knows that the call to sigfillset() cannot fail on a particular
platform it is compiling for and declares that the if() condition
would not hold, it is plausible that the next version of the
compiler may learn that sigfillset() that never fails would not
touch errno and decide that in this sequence:
errno = 0;
sigfillset(&all)
if (errno)
die_errno("sigfillset");
the if() statement will never trigger. Marking that the value
returned by sigfillset() cannot be a constant would document our
intention better and would not break with such a new version of
compiler that is even more "clever". With the marco, the above
sequence can be rewritten:
if (NOT_A_CONST(sigfillset(&all)))
die_errno("sigfillset");
which looks almost like other innocuous annotations we have,
e.g. UNUSED.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
Makefile | 1 +
fbtcdnki.c | 2 ++
git-compat-util.h | 9 +++++++++
meson.build | 1 +
run-command.c | 12 +++++-------
5 files changed, 18 insertions(+), 7 deletions(-)
create mode 100644 fbtcdnki.c
diff --git a/Makefile b/Makefile
index 97e8385b66..2158bf6916 100644
--- a/Makefile
+++ b/Makefile
@@ -1018,6 +1018,7 @@ LIB_OBJS += ewah/ewah_bitmap.o
LIB_OBJS += ewah/ewah_io.o
LIB_OBJS += ewah/ewah_rlw.o
LIB_OBJS += exec-cmd.o
+LIB_OBJS += fbtcdnki.o
LIB_OBJS += fetch-negotiator.o
LIB_OBJS += fetch-pack.o
LIB_OBJS += fmt-merge-msg.o
diff --git a/fbtcdnki.c b/fbtcdnki.c
new file mode 100644
index 0000000000..1da3ffc2f5
--- /dev/null
+++ b/fbtcdnki.c
@@ -0,0 +1,2 @@
+#include <git-compat-util.h>
+int false_but_the_compiler_does_not_know_it_;
diff --git a/git-compat-util.h b/git-compat-util.h
index e283c46c6f..63a3ef6b70 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -1593,4 +1593,13 @@ static inline void *container_of_or_null_offset(void *ptr, size_t offset)
((uintptr_t)&(ptr)->member - (uintptr_t)(ptr))
#endif /* !__GNUC__ */
+/*
+ * Prevent an overly clever compiler from optimizing an expression
+ * out, triggering a false positive when building with the
+ * -Wunreachable-code option. false_but_the_compiler_does_not_know_it_
+ * is defined in a compilation unit separate from where the macro is
+ * used, initialized to 0, and never modified.
+ */
+#define NOT_A_CONST(expr) ((expr) || false_but_the_compiler_does_not_know_it_)
+extern int false_but_the_compiler_does_not_know_it_;
#endif
diff --git a/meson.build b/meson.build
index f60f3f49e4..ce642dcf65 100644
--- a/meson.build
+++ b/meson.build
@@ -282,6 +282,7 @@ libgit_sources = [
'ewah/ewah_io.c',
'ewah/ewah_rlw.c',
'exec-cmd.c',
+ 'fbtcdnki.c',
'fetch-negotiator.c',
'fetch-pack.c',
'fmt-merge-msg.c',
diff --git a/run-command.c b/run-command.c
index d527c46175..535c73a059 100644
--- a/run-command.c
+++ b/run-command.c
@@ -516,14 +516,12 @@ static void atfork_prepare(struct atfork_state *as)
sigset_t all;
/*
- * Do not use the return value of sigfillset(). It is transparently 0
- * on some platforms, meaning a clever compiler may complain that
- * the conditional body is dead code. Instead, check for error via
- * errno, which outsmarts the compiler.
+ * POSIX says sitfillset() can fail, but an overly clever
+ * compiler can see through the header files and decide
+ * it cannot fail on a particular platform it is compiling for,
+ * triggering -Wunreachable-code false positive.
*/
- errno = 0;
- sigfillset(&all);
- if (errno)
+ if (NOT_A_CONST(sigfillset(&all)))
die_errno("sigfillset");
#ifdef NO_PTHREADS
if (sigprocmask(SIG_SETMASK, &all, &as->old))
--
2.49.0-188-g35fcca2323
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH v2 3/3] git-compat-util: add NOT_A_CONST macro and use it in atfork_prepare()
2025-03-14 22:29 ` Junio C Hamano
@ 2025-03-17 18:00 ` Jeff King
0 siblings, 0 replies; 143+ messages in thread
From: Jeff King @ 2025-03-17 18:00 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Patrick Steinhardt
On Fri, Mar 14, 2025 at 03:29:54PM -0700, Junio C Hamano wrote:
> ---- >8 ----
> Our hope is that the number of code paths that falsely trigger
> warnings with the -Wunreachable-code compilation option are small,
> and they can be worked around case-by-case basis, like we just did
> in the previous commit. If we need such a workaround a bit more
> often, however, we may benefit from a more generic and descriptive
> facility that helps document the cases we need such workarounds.
>
> Side note: if we need the workaround all over the place, it
> simply means -Wunreachable-code is not a good tool for us to
> save engineering effort to catch mistakes. We are still
> exploring if it helps us, so let's assume that it is not the
> case.
Yup, I very much agree with this, especially the side note. (I'd
probably have just dropped patch 2 and gone straight here, but I don't
mind leaving it in as documentation of that other direction).
> Introduce NOT_A_CONST() macro, with which, the developer can tell
> the compiler:
>
> Do not optimize this expression out, because, despite whatever
> you are told by the system headers, this expression should *not*
> be treated as a constant.
This is definitely better than the other name. I might spell it out
as "NOT_A_CONSTANT", just because "const" to me is a variable annotation
(for something that _could_ change, but we are not allowed to). Whereas
"constant" is something defined to a single value in the program. Maybe
splitting hairs, but as somebody who read NOT_A_CONST(foo) I might
expect it to be casting away "const" or something.
> --- a/Makefile
> +++ b/Makefile
> @@ -1018,6 +1018,7 @@ LIB_OBJS += ewah/ewah_bitmap.o
> LIB_OBJS += ewah/ewah_io.o
> LIB_OBJS += ewah/ewah_rlw.o
> LIB_OBJS += exec-cmd.o
> +LIB_OBJS += fbtcdnki.o
That name is a mouthful, for sure. The long name is really an
implementation detail. Would calling it not-constant.c or something be
more descriptive? (Yes, the macro itself does not appear in the file,
but hopefully it links the two semantically in the reader's head).
I almost want to suggest a name like "compiler-tricks.c", but part of
the point of this particular trick is that there's nothing else in its
translation unit. So later when somebody adds another trick, it cannot
use this macro. ;)
> +/*
> + * Prevent an overly clever compiler from optimizing an expression
> + * out, triggering a false positive when building with the
> + * -Wunreachable-code option. false_but_the_compiler_does_not_know_it_
> + * is defined in a compilation unit separate from where the macro is
> + * used, initialized to 0, and never modified.
> + */
> +#define NOT_A_CONST(expr) ((expr) || false_but_the_compiler_does_not_know_it_)
> +extern int false_but_the_compiler_does_not_know_it_;
Good explanation. I do wonder if we'd eventually see a compiler that
reaches across translation units to optimize, but I'd hope we probably
bought ourselves a decade or two.
> diff --git a/run-command.c b/run-command.c
> index d527c46175..535c73a059 100644
> --- a/run-command.c
> +++ b/run-command.c
> @@ -516,14 +516,12 @@ static void atfork_prepare(struct atfork_state *as)
> sigset_t all;
>
> /*
> - * Do not use the return value of sigfillset(). It is transparently 0
> - * on some platforms, meaning a clever compiler may complain that
> - * the conditional body is dead code. Instead, check for error via
> - * errno, which outsmarts the compiler.
> + * POSIX says sitfillset() can fail, but an overly clever
> + * compiler can see through the header files and decide
> + * it cannot fail on a particular platform it is compiling for,
> + * triggering -Wunreachable-code false positive.
> */
> - errno = 0;
> - sigfillset(&all);
> - if (errno)
> + if (NOT_A_CONST(sigfillset(&all)))
> die_errno("sigfillset");
And this looks much nicer and more descriptive. You could probably even
get away without the comment, but I certainly do not mind it.
s/sitfillset/sigfillset/ in your comment text, though.
-Peff
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v2 2/3] run-command: use errno to check for sigfillset() error
2025-03-14 21:09 ` [PATCH v2 2/3] run-command: use errno to check for sigfillset() error Junio C Hamano
@ 2025-03-17 21:30 ` Taylor Blau
2025-03-17 23:12 ` Junio C Hamano
0 siblings, 1 reply; 143+ messages in thread
From: Taylor Blau @ 2025-03-17 21:30 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Jeff King
On Fri, Mar 14, 2025 at 02:09:08PM -0700, Junio C Hamano wrote:
> From: Jeff King <peff@peff.net>
>
> Since enabling -Wunreachable-code, builds with clang on macOS now fail,
> complaining that the die_errno() call in:
>
> if (sigfillset(&all))
> die_errno("sigfillset");
Hmm. Would it have made sense to swap the order of this and the first
patch so we don't have a DEVELOPER=1 breakage (for macOS with Clang) in
history?
I think it's too late now since this topic is already on 'next', but it
occurred to me idly while reading this patch.
Thanks,
Taylor
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v2 2/3] run-command: use errno to check for sigfillset() error
2025-03-17 21:30 ` Taylor Blau
@ 2025-03-17 23:12 ` Junio C Hamano
2025-03-18 0:36 ` Junio C Hamano
0 siblings, 1 reply; 143+ messages in thread
From: Junio C Hamano @ 2025-03-17 23:12 UTC (permalink / raw)
To: Taylor Blau; +Cc: git, Jeff King
Taylor Blau <me@ttaylorr.com> writes:
> On Fri, Mar 14, 2025 at 02:09:08PM -0700, Junio C Hamano wrote:
>> From: Jeff King <peff@peff.net>
>>
>> Since enabling -Wunreachable-code, builds with clang on macOS now fail,
>> complaining that the die_errno() call in:
>>
>> if (sigfillset(&all))
>> die_errno("sigfillset");
>
> Hmm. Would it have made sense to swap the order of this and the first
> patch so we don't have a DEVELOPER=1 breakage (for macOS with Clang) in
> history?
>
> I think it's too late now since this topic is already on 'next', but it
> occurred to me idly while reading this patch.
I thought db1d1f5d (config.mak.dev: enable -Wunreachable-code,
2025-03-14) aka jk/use-wunreachable-code-for-devs~2 is still out of
'next'?
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH v3 0/3] -Wunreachable-code
2025-03-14 21:09 ` [PATCH v2 0/3] -Wunreachable-code Junio C Hamano
` (2 preceding siblings ...)
2025-03-14 21:09 ` [PATCH v2 3/3] git-compat-util: add NOT_A_CONST macro and use it in atfork_prepare() Junio C Hamano
@ 2025-03-17 23:53 ` Junio C Hamano
2025-03-17 23:53 ` [PATCH v3 1/3] run-command: use errno to check for sigfillset() error Junio C Hamano
` (3 more replies)
3 siblings, 4 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-17 23:53 UTC (permalink / raw)
To: git; +Cc: Jeff King, Taylor Blau
As Taylor noticed, we can still help macOS users by first dealing
with the false positive in the code, and then flip the warning
option for developers on.
[1/3] run-command: use errno to check for sigfillset() error
This was our first "workaround" that is very specific to the code
that gets falsely flagged by the compiler.
[2/3] git-compat-util: add NOT_CONSTANT macro and use it in atfork_prepare()
This adds a more generic way to work around a false positive from
-Wunreachable-code to prevent compilers from optimize away
expressions that are used in conditionals, and rewrite the earlier
workaround with it.
[3/3] config.mak.dev: enable -Wunreachable-code
Now we worked around known false positive of -Wunreachable-code,
we force it upon our developers, including macOS ones.
This is totally offtopic, but I often find the short-log (list of
commits, grouped by author) in the cover letter very awkward to work
with. Between v2 and v3, aside from the NOT_CONSTANT() improvements
in the patch [2/3] that used to be [3/3], one large change is the
reordering of the patches but that is not seen in the shortlog (I
ran "git log --oneline -reverse" to prepare the list of commits in
the order they are applied to describe them in the above list).
Jeff King (2):
run-command: use errno to check for sigfillset() error
config.mak.dev: enable -Wunreachable-code
Junio C Hamano (1):
git-compat-util: add NOT_CONSTANT macro and use it in atfork_prepare()
Makefile | 1 +
compiler-tricks/not-a-constant.c | 2 ++
config.mak.dev | 1 +
git-compat-util.h | 9 +++++++++
meson.build | 2 ++
run-command.c | 8 +++++++-
6 files changed, 22 insertions(+), 1 deletion(-)
create mode 100644 compiler-tricks/not-a-constant.c
--
2.49.0-207-gc8924421c3
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH v3 1/3] run-command: use errno to check for sigfillset() error
2025-03-17 23:53 ` [PATCH v3 0/3] -Wunreachable-code Junio C Hamano
@ 2025-03-17 23:53 ` Junio C Hamano
2025-03-17 23:53 ` [PATCH v3 2/3] git-compat-util: add NOT_CONSTANT macro and use it in atfork_prepare() Junio C Hamano
` (2 subsequent siblings)
3 siblings, 0 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-17 23:53 UTC (permalink / raw)
To: git; +Cc: Jeff King
From: Jeff King <peff@peff.net>
Since enabling -Wunreachable-code, builds with clang on macOS now fail,
complaining that the die_errno() call in:
if (sigfillset(&all))
die_errno("sigfillset");
is unreachable. On that platform the manpage documents that sigfillset()
always returns success, and presumably the implementation is a macro or
inline function that does so in a way that is transparent to the
compiler.
But we should continue to check on other platforms, since POSIX says it
may return an error.
We could solve this with a compile-time knob to split the two cases
(assuming success on macOS and checking for the error elsewhere). But we
can also work around it more directly by relying on errno to check the
outcome (since POSIX dictates that errno will be set on error). And that
works around the compiler's cleverness, since it doesn't know the
semantics of errno (though I suppose if sigfillset() is simple enough,
it could perhaps realize that no writes to errno are possible; however
this does seem to work in practice).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
run-command.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/run-command.c b/run-command.c
index 402138b8b5..d527c46175 100644
--- a/run-command.c
+++ b/run-command.c
@@ -515,7 +515,15 @@ static void atfork_prepare(struct atfork_state *as)
{
sigset_t all;
- if (sigfillset(&all))
+ /*
+ * Do not use the return value of sigfillset(). It is transparently 0
+ * on some platforms, meaning a clever compiler may complain that
+ * the conditional body is dead code. Instead, check for error via
+ * errno, which outsmarts the compiler.
+ */
+ errno = 0;
+ sigfillset(&all);
+ if (errno)
die_errno("sigfillset");
#ifdef NO_PTHREADS
if (sigprocmask(SIG_SETMASK, &all, &as->old))
--
2.49.0-207-gc8924421c3
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v3 2/3] git-compat-util: add NOT_CONSTANT macro and use it in atfork_prepare()
2025-03-17 23:53 ` [PATCH v3 0/3] -Wunreachable-code Junio C Hamano
2025-03-17 23:53 ` [PATCH v3 1/3] run-command: use errno to check for sigfillset() error Junio C Hamano
@ 2025-03-17 23:53 ` Junio C Hamano
2025-03-18 0:20 ` Jeff King
2025-03-18 22:04 ` Calvin Wan
2025-03-17 23:53 ` [PATCH v3 3/3] config.mak.dev: enable -Wunreachable-code Junio C Hamano
2025-03-18 0:18 ` [PATCH v3 0/3] -Wunreachable-code Jeff King
3 siblings, 2 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-17 23:53 UTC (permalink / raw)
To: git
Our hope is that the number of code paths that falsely trigger
warnings with the -Wunreachable-code compilation option are small,
and they can be worked around case-by-case basis, like we just did
in the previous commit. If we need such a workaround a bit more
often, however, we may benefit from a more generic and descriptive
facility that helps document the cases we need such workarounds.
Side note: if we need the workaround all over the place, it
simply means -Wunreachable-code is not a good tool for us to
save engineering effort to catch mistakes. We are still
exploring if it helps us, so let's assume that it is not the
case.
Introduce NOT_CONSTANT() macro, with which, the developer can tell
the compiler:
Do not optimize this expression out, because, despite whatever
you are told by the system headers, this expression should *not*
be treated as a constant.
and use it as a replacement for the workaround we used that was
somewhat specific to the sigfillset case. If the compiler already
knows that the call to sigfillset() cannot fail on a particular
platform it is compiling for and declares that the if() condition
would not hold, it is plausible that the next version of the
compiler may learn that sigfillset() that never fails would not
touch errno and decide that in this sequence:
errno = 0;
sigfillset(&all)
if (errno)
die_errno("sigfillset");
the if() statement will never trigger. Marking that the value
returned by sigfillset() cannot be a constant would document our
intention better and would not break with such a new version of
compiler that is even more "clever". With the marco, the above
sequence can be rewritten:
if (NOT_CONSTANT(sigfillset(&all)))
die_errno("sigfillset");
which looks almost like other innocuous annotations we have,
e.g. UNUSED.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
Makefile | 1 +
compiler-tricks/not-a-constant.c | 2 ++
git-compat-util.h | 9 +++++++++
meson.build | 1 +
run-command.c | 12 +++++-------
5 files changed, 18 insertions(+), 7 deletions(-)
create mode 100644 compiler-tricks/not-a-constant.c
diff --git a/Makefile b/Makefile
index 97e8385b66..605e2d7f61 100644
--- a/Makefile
+++ b/Makefile
@@ -985,6 +985,7 @@ LIB_OBJS += compat/nonblock.o
LIB_OBJS += compat/obstack.o
LIB_OBJS += compat/terminal.o
LIB_OBJS += compat/zlib-uncompress2.o
+LIB_OBJS += compiler-tricks/not-a-constant.o
LIB_OBJS += config.o
LIB_OBJS += connect.o
LIB_OBJS += connected.o
diff --git a/compiler-tricks/not-a-constant.c b/compiler-tricks/not-a-constant.c
new file mode 100644
index 0000000000..1da3ffc2f5
--- /dev/null
+++ b/compiler-tricks/not-a-constant.c
@@ -0,0 +1,2 @@
+#include <git-compat-util.h>
+int false_but_the_compiler_does_not_know_it_;
diff --git a/git-compat-util.h b/git-compat-util.h
index e283c46c6f..f6a149827b 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -1593,4 +1593,13 @@ static inline void *container_of_or_null_offset(void *ptr, size_t offset)
((uintptr_t)&(ptr)->member - (uintptr_t)(ptr))
#endif /* !__GNUC__ */
+/*
+ * Prevent an overly clever compiler from optimizing an expression
+ * out, triggering a false positive when building with the
+ * -Wunreachable-code option. false_but_the_compiler_does_not_know_it_
+ * is defined in a compilation unit separate from where the macro is
+ * used, initialized to 0, and never modified.
+ */
+#define NOT_CONSTANT(expr) ((expr) || false_but_the_compiler_does_not_know_it_)
+extern int false_but_the_compiler_does_not_know_it_;
#endif
diff --git a/meson.build b/meson.build
index 0064eb64f5..373524dad2 100644
--- a/meson.build
+++ b/meson.build
@@ -249,6 +249,7 @@ libgit_sources = [
'compat/obstack.c',
'compat/terminal.c',
'compat/zlib-uncompress2.c',
+ 'compiler-tricks/not-a-constant.c',
'config.c',
'connect.c',
'connected.c',
diff --git a/run-command.c b/run-command.c
index d527c46175..b74fd08056 100644
--- a/run-command.c
+++ b/run-command.c
@@ -516,14 +516,12 @@ static void atfork_prepare(struct atfork_state *as)
sigset_t all;
/*
- * Do not use the return value of sigfillset(). It is transparently 0
- * on some platforms, meaning a clever compiler may complain that
- * the conditional body is dead code. Instead, check for error via
- * errno, which outsmarts the compiler.
+ * POSIX says sigfillset() can fail, but an overly clever
+ * compiler can see through the header files and decide
+ * it cannot fail on a particular platform it is compiling for,
+ * triggering -Wunreachable-code false positive.
*/
- errno = 0;
- sigfillset(&all);
- if (errno)
+ if (NOT_CONSTANT(sigfillset(&all)))
die_errno("sigfillset");
#ifdef NO_PTHREADS
if (sigprocmask(SIG_SETMASK, &all, &as->old))
--
2.49.0-207-gc8924421c3
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v3 3/3] config.mak.dev: enable -Wunreachable-code
2025-03-17 23:53 ` [PATCH v3 0/3] -Wunreachable-code Junio C Hamano
2025-03-17 23:53 ` [PATCH v3 1/3] run-command: use errno to check for sigfillset() error Junio C Hamano
2025-03-17 23:53 ` [PATCH v3 2/3] git-compat-util: add NOT_CONSTANT macro and use it in atfork_prepare() Junio C Hamano
@ 2025-03-17 23:53 ` Junio C Hamano
2025-03-18 0:18 ` [PATCH v3 0/3] -Wunreachable-code Jeff King
3 siblings, 0 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-17 23:53 UTC (permalink / raw)
To: git; +Cc: Jeff King
From: Jeff King <peff@peff.net>
Having the compiler point out unreachable code can help avoid bugs, like
the one discussed in:
https://lore.kernel.org/git/20250307195057.GA3675279@coredump.intra.peff.net/
In that case it was found by Coverity, but finding it earlier saves
everybody time and effort.
We can use -Wunreachable-code to get some help from the compiler here.
Interestingly, this is a noop in gcc. It was a real warning up until gcc
4.x, when it was removed for being too flaky, but they left the
command-line option to avoid breaking users. See:
https://stackoverflow.com/questions/17249934/why-does-gcc-not-warn-for-unreachable-code
However, clang does implement this option, and it finds the case
mentioned above (and no other cases within the code base). And since we
run clang in several of our CI jobs, that's enough to get an early
warning of breakage.
We could enable it only for clang, but since gcc is happy to ignore it,
it's simpler to just turn it on for all developer builds.
Signed-off-by: Jeff King <peff@peff.net>
[jc: squashed meson.build change sent by Patrick]
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
config.mak.dev | 1 +
meson.build | 1 +
2 files changed, 2 insertions(+)
diff --git a/config.mak.dev b/config.mak.dev
index 0fd8cc4d35..95b7bc46ae 100644
--- a/config.mak.dev
+++ b/config.mak.dev
@@ -39,6 +39,7 @@ DEVELOPER_CFLAGS += -Wunused
DEVELOPER_CFLAGS += -Wvla
DEVELOPER_CFLAGS += -Wwrite-strings
DEVELOPER_CFLAGS += -fno-common
+DEVELOPER_CFLAGS += -Wunreachable-code
ifneq ($(filter clang4,$(COMPILER_FEATURES)),)
DEVELOPER_CFLAGS += -Wtautological-constant-out-of-range-compare
diff --git a/meson.build b/meson.build
index 373524dad2..fdccc59945 100644
--- a/meson.build
+++ b/meson.build
@@ -698,6 +698,7 @@ if get_option('warning_level') in ['2','3', 'everything'] and compiler.get_argum
'-Woverflow',
'-Wpointer-arith',
'-Wstrict-prototypes',
+ '-Wunreachable-code',
'-Wunused',
'-Wvla',
'-Wwrite-strings',
--
2.49.0-207-gc8924421c3
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH v3 0/3] -Wunreachable-code
2025-03-17 23:53 ` [PATCH v3 0/3] -Wunreachable-code Junio C Hamano
` (2 preceding siblings ...)
2025-03-17 23:53 ` [PATCH v3 3/3] config.mak.dev: enable -Wunreachable-code Junio C Hamano
@ 2025-03-18 0:18 ` Jeff King
3 siblings, 0 replies; 143+ messages in thread
From: Jeff King @ 2025-03-18 0:18 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Taylor Blau
On Mon, Mar 17, 2025 at 04:53:26PM -0700, Junio C Hamano wrote:
> As Taylor noticed, we can still help macOS users by first dealing
> with the false positive in the code, and then flip the warning
> option for developers on.
Yeah, this is worth doing.
> This is totally offtopic, but I often find the short-log (list of
> commits, grouped by author) in the cover letter very awkward to work
> with. Between v2 and v3, aside from the NOT_CONSTANT() improvements
> in the patch [2/3] that used to be [3/3], one large change is the
> reordering of the patches but that is not seen in the shortlog (I
> ran "git log --oneline -reverse" to prepare the list of commits in
> the order they are applied to describe them in the above list).
>
> Jeff King (2):
> run-command: use errno to check for sigfillset() error
> config.mak.dev: enable -Wunreachable-code
>
> Junio C Hamano (1):
> git-compat-util: add NOT_CONSTANT macro and use it in atfork_prepare()
The re-ordering does appear in the range-diff, if you provide one. But I
agree that the organized-by-name shortlog does not make much sense for
most series. As a reviewer, I care most about the patches, not the
authors.
I make my cover letters with something like this (part of a larger
script):
git format-patch --stdout origin..$topic |
perl -lne '
if (/^Subject: (.*)/) {
$subject = $1;
}
elsif ($subject && /^\s+(.*)/) {
$subject .= " $1";
}
elsif ($subject) {
print $subject;
$subject = undef;
}
' |
sed -e 's/\[PATCH /[/' \
-e 's/]/]:/' \
-e 's/^/ /'
which yields something like (for the older version of this series):
[1/3]: config.mak.dev: enable -Wunreachable-code
[2/3]: run-command: use errno to check for sigfillset() error
[3/3]: git-compat-util: add NOT_A_CONST macro and use it in atfork_prepare()
Having the correct order and the matching numbering next to each one
makes it much easier if you're going to comment on them inline.
The perl in the script above is required to handle rfc822 wrapping /
line continuation. I never bothered to implement rfc2047 unquoting. I
don't tend to use non-ascii chars in my subject lines. ;)
It would be nice if we had a format-patch option to avoid quoting and
wrapping in order to make text processing like this easier.
-Peff
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 2/3] git-compat-util: add NOT_CONSTANT macro and use it in atfork_prepare()
2025-03-17 23:53 ` [PATCH v3 2/3] git-compat-util: add NOT_CONSTANT macro and use it in atfork_prepare() Junio C Hamano
@ 2025-03-18 0:20 ` Jeff King
2025-03-18 0:28 ` Junio C Hamano
2025-03-18 22:04 ` Calvin Wan
1 sibling, 1 reply; 143+ messages in thread
From: Jeff King @ 2025-03-18 0:20 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
On Mon, Mar 17, 2025 at 04:53:28PM -0700, Junio C Hamano wrote:
> Introduce NOT_CONSTANT() macro, with which, the developer can tell
> the compiler:
This name looks great to me.
> compiler-tricks/not-a-constant.c | 2 ++
And this is much better, too. ;) I see you dropped the "a" in the macro
name; I don't know if it matters much to do it here, too.
-Peff
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 2/3] git-compat-util: add NOT_CONSTANT macro and use it in atfork_prepare()
2025-03-18 0:20 ` Jeff King
@ 2025-03-18 0:28 ` Junio C Hamano
0 siblings, 0 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-18 0:28 UTC (permalink / raw)
To: Jeff King; +Cc: git
Jeff King <peff@peff.net> writes:
> On Mon, Mar 17, 2025 at 04:53:28PM -0700, Junio C Hamano wrote:
>
>> Introduce NOT_CONSTANT() macro, with which, the developer can tell
>> the compiler:
>
> This name looks great to me.
>
>> compiler-tricks/not-a-constant.c | 2 ++
>
> And this is much better, too. ;) I see you dropped the "a" in the macro
> name; I don't know if it matters much to do it here, too.
Good eyes.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v2 2/3] run-command: use errno to check for sigfillset() error
2025-03-17 23:12 ` Junio C Hamano
@ 2025-03-18 0:36 ` Junio C Hamano
0 siblings, 0 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-18 0:36 UTC (permalink / raw)
To: Taylor Blau; +Cc: git, Jeff King
Junio C Hamano <gitster@pobox.com> writes:
> Taylor Blau <me@ttaylorr.com> writes:
>
>> On Fri, Mar 14, 2025 at 02:09:08PM -0700, Junio C Hamano wrote:
>>> From: Jeff King <peff@peff.net>
>>>
>>> Since enabling -Wunreachable-code, builds with clang on macOS now fail,
>>> complaining that the die_errno() call in:
>>>
>>> if (sigfillset(&all))
>>> die_errno("sigfillset");
>>
>> Hmm. Would it have made sense to swap the order of this and the first
>> patch so we don't have a DEVELOPER=1 breakage (for macOS with Clang) in
>> history?
>>
>> I think it's too late now since this topic is already on 'next', but it
>> occurred to me idly while reading this patch.
>
> I thought db1d1f5d (config.mak.dev: enable -Wunreachable-code,
> 2025-03-14) aka jk/use-wunreachable-code-for-devs~2 is still out of
> 'next'?
Ah, I did revert an earlier one-commit topic out of 'next'. Perhaps
I didn't tell What's cooking about it.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 2/3] git-compat-util: add NOT_CONSTANT macro and use it in atfork_prepare()
2025-03-17 23:53 ` [PATCH v3 2/3] git-compat-util: add NOT_CONSTANT macro and use it in atfork_prepare() Junio C Hamano
2025-03-18 0:20 ` Jeff King
@ 2025-03-18 22:04 ` Calvin Wan
2025-03-18 22:26 ` Calvin Wan
1 sibling, 1 reply; 143+ messages in thread
From: Calvin Wan @ 2025-03-18 22:04 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Calvin Wan, git
Junio C Hamano <gitster@pobox.com> writes:
> Our hope is that the number of code paths that falsely trigger
> warnings with the -Wunreachable-code compilation option are small,
> and they can be worked around case-by-case basis, like we just did
> in the previous commit. If we need such a workaround a bit more
> often, however, we may benefit from a more generic and descriptive
> facility that helps document the cases we need such workarounds.
>
> Side note: if we need the workaround all over the place, it
> simply means -Wunreachable-code is not a good tool for us to
> save engineering effort to catch mistakes. We are still
> exploring if it helps us, so let's assume that it is not the
> case.
>
> Introduce NOT_CONSTANT() macro, with which, the developer can tell
> the compiler:
>
> Do not optimize this expression out, because, despite whatever
> you are told by the system headers, this expression should *not*
> be treated as a constant.
>
> and use it as a replacement for the workaround we used that was
> somewhat specific to the sigfillset case. If the compiler already
> knows that the call to sigfillset() cannot fail on a particular
> platform it is compiling for and declares that the if() condition
> would not hold, it is plausible that the next version of the
> compiler may learn that sigfillset() that never fails would not
> touch errno and decide that in this sequence:
>
> errno = 0;
> sigfillset(&all)
> if (errno)
> die_errno("sigfillset");
>
> the if() statement will never trigger. Marking that the value
> returned by sigfillset() cannot be a constant would document our
> intention better and would not break with such a new version of
> compiler that is even more "clever". With the marco, the above
> sequence can be rewritten:
>
> if (NOT_CONSTANT(sigfillset(&all)))
> die_errno("sigfillset");
>
> which looks almost like other innocuous annotations we have,
> e.g. UNUSED.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
> Makefile | 1 +
> compiler-tricks/not-a-constant.c | 2 ++
> git-compat-util.h | 9 +++++++++
> meson.build | 1 +
> run-command.c | 12 +++++-------
> 5 files changed, 18 insertions(+), 7 deletions(-)
> create mode 100644 compiler-tricks/not-a-constant.c
>
> diff --git a/Makefile b/Makefile
> index 97e8385b66..605e2d7f61 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -985,6 +985,7 @@ LIB_OBJS += compat/nonblock.o
> LIB_OBJS += compat/obstack.o
> LIB_OBJS += compat/terminal.o
> LIB_OBJS += compat/zlib-uncompress2.o
> +LIB_OBJS += compiler-tricks/not-a-constant.o
The name is correctly added here, but in `next,` this name is set to
`compiler-tricks/not-constant.o`.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 2/3] git-compat-util: add NOT_CONSTANT macro and use it in atfork_prepare()
2025-03-18 22:04 ` Calvin Wan
@ 2025-03-18 22:26 ` Calvin Wan
2025-03-18 23:55 ` Junio C Hamano
0 siblings, 1 reply; 143+ messages in thread
From: Calvin Wan @ 2025-03-18 22:26 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
On Tue, Mar 18, 2025 at 3:05 PM Calvin Wan <calvinwan@google.com> wrote:
>
> Junio C Hamano <gitster@pobox.com> writes:
> > Our hope is that the number of code paths that falsely trigger
> > @@ -985,6 +985,7 @@ LIB_OBJS += compat/nonblock.o
> > LIB_OBJS += compat/obstack.o
> > LIB_OBJS += compat/terminal.o
> > LIB_OBJS += compat/zlib-uncompress2.o
> > +LIB_OBJS += compiler-tricks/not-a-constant.o
>
> The name is correctly added here, but in `next,` this name is set to
> `compiler-tricks/not-constant.o`.
Apologies you can ignore this -- we needed to add a reference to the new folder
internally so this was a red herring for our broken build.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v3 2/3] git-compat-util: add NOT_CONSTANT macro and use it in atfork_prepare()
2025-03-18 22:26 ` Calvin Wan
@ 2025-03-18 23:55 ` Junio C Hamano
0 siblings, 0 replies; 143+ messages in thread
From: Junio C Hamano @ 2025-03-18 23:55 UTC (permalink / raw)
To: Calvin Wan; +Cc: git
Calvin Wan <calvinwan@google.com> writes:
> On Tue, Mar 18, 2025 at 3:05 PM Calvin Wan <calvinwan@google.com> wrote:
>>
>> Junio C Hamano <gitster@pobox.com> writes:
>> > Our hope is that the number of code paths that falsely trigger
>> > @@ -985,6 +985,7 @@ LIB_OBJS += compat/nonblock.o
>> > LIB_OBJS += compat/obstack.o
>> > LIB_OBJS += compat/terminal.o
>> > LIB_OBJS += compat/zlib-uncompress2.o
>> > +LIB_OBJS += compiler-tricks/not-a-constant.o
>>
>> The name is correctly added here, but in `next,` this name is set to
>> `compiler-tricks/not-constant.o`.
>
> Apologies you can ignore this -- we needed to add a reference to the new folder
> internally so this was a red herring for our broken build.
Sorry, I may not have sent a reroll to the list for the version that
went into 'next'. It should have lost "a" from not-constant
consistently everywhere.
Thanks for being eagle-eyed.
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH v4 0/8] refs: introduce support for batched reference updates
2025-02-07 7:34 [PATCH 0/6] refs: introduce support for partial reference transactions Karthik Nayak
` (8 preceding siblings ...)
2025-03-05 17:38 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Karthik Nayak
@ 2025-03-20 11:43 ` Karthik Nayak
2025-03-20 11:43 ` [PATCH v4 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
` (7 more replies)
2025-03-27 11:13 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 " Karthik Nayak
11 siblings, 8 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-20 11:43 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Git supports making reference updates with or without transactions.
Updates with transactions are generally better optimized. But
transactions are all or nothing. This means, if a user wants to batch
updates to take advantage of the optimizations without the hard
requirement that all updates must succeed, there is no way currently to
do so. Particularly with the reftable backend where batching multiple
reference updates is more efficient than performing them sequentially.
This series introduces support for batched reference updates without
transactions allowing individual reference updates to fail while letting
others proceed. This capability is exposed through git-update-ref's
`--allow-partial` flag, which can be used in `--stdin` mode to batch
updates and handle failures gracefully. Under the hood, these batched
updates still use the transactions infrastructure, while modifying
sections to allow partial failures.
The changes are structured to carefully build up this functionality:
First, we clean up and consolidate the reference update checking logic.
This includes removing duplicate checks in the files backend and moving
refname tracking to the generic layer, which simplifies the codebase and
prepares it for the new feature.
We then restructure the reftable backend's transaction preparation code,
extracting the update validation logic into a dedicated function. This
not only improves code organization but sets the stage for implementing
partial transaction support.
To ensure we only skip errors which are user-oriented, we introduce
typed errors for transactions with 'enum ref_transaction_error'. We
extend the existing errors to include other scenarios and use this new
errors throughout the refs code.
With this groundwork in place, we implement the core batch update
support in the refs subsystem. This adds the necessary infrastructure to
track and report rejected updates while allowing transactions to
proceed. All reference backends are modified to support this behavior
when enabled.
Finally, we expose this functionality to users through
git-update-ref(1)'s `--allow-partial` flag, complete with test coverage
and documentation. The flag is specifically limited to `--stdin` mode
where batching multiple updates is most relevant.
This enhancement improves Git's flexibility in handling reference
updates while maintaining the safety of atomic transactions by default.
It's particularly valuable for tools and workflows that need to handle
reference update failures gracefully without abandoning the entire batch
of updates.
This series is based on top of 683c54c999 (Git 2.49, 2025-03-14) with
Patrick's series 'refs: batch refname availability checks' [1] merged
in.
[1]: https://lore.kernel.org/all/20250217-pks-update-ref-optimization-v1-0-a2b6d87a24af@pks.im/
---
Changes in v4:
- Rebased on top of 2.49 since there was a long time between the
previous iteration and we have a new release.
- Changed the naming to say 'batched' updates instead of 'partial
transactions'. While we still use the transaction infrastructure
underneath, the new naming causes less ambiguity.
- Clean up some of the commit messages.
- Raise BUG for invalid update index while setting rejections.
- Fix an incorrect early return.
- Link to v3: https://lore.kernel.org/r/20250305-245-partially-atomic-ref-updates-v3-0-0c64e3052354@gmail.com
Changes in v3:
- Changed 'transaction_error' to 'ref_transaction_error' along with the
error names. Removed 'TRANSACTION_OK' since it can potentially be
missed instead of simply 'return 0'.
- Rename 'ref_transaction_set_rejected' to
'ref_transaction_maybe_set_rejected' and move logic around error
checks to within this function.
- Add a new struct 'ref_transaction_rejections' to track the rejections
within a transaction. This allows us to only iterate over rejected
updates.
- Add a new commit to also support partial transactions within the
batched F/D checks.
- Remove NUL delimited outputs in 'git-update-ref(1)'.
- Remove translations for plumbing outputs.
- Other small cleanups in the commit message and code.
Changes in v2:
- Introduce and use structured errors. This consolidates the errors
and their handling between the ref backends.
- In the previous version, we skipped over all failures. This include
system failures such as low memory or IO problems. Let's instead, only
skip user-oriented failures, such as invalid old OID and so on.
- Change the rejection function name to `ref_transaction_set_rejected()`.
- Modify the commit messages and documentation to be a little more
verbose.
- Link to v1: https://lore.kernel.org/r/20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com
Documentation/git-update-ref.adoc | 14 +-
builtin/fetch.c | 2 +-
builtin/update-ref.c | 67 ++++-
refs.c | 162 ++++++++++--
refs.h | 76 ++++--
refs/files-backend.c | 314 +++++++++++-------------
refs/packed-backend.c | 69 +++---
refs/refs-internal.h | 51 +++-
refs/reftable-backend.c | 502 +++++++++++++++++++-------------------
t/t1400-update-ref.sh | 233 ++++++++++++++++++
10 files changed, 968 insertions(+), 522 deletions(-)
Karthik Nayak (8):
refs/files: remove redundant check in split_symref_update()
refs: move duplicate refname update check to generic layer
refs/files: remove duplicate duplicates check
refs/reftable: extract code from the transaction preparation
refs: introduce enum-based transaction error types
refs: implement batch reference update support
refs: support rejection in batch updates during F/D checks
update-ref: add --batch-updates flag for stdin mode
Range-diff versus v3:
1: c54471c108 ! 1: 0ddde0cb83 refs/files: remove redundant check in split_symref_update()
@@ Commit message
The second check is unnecessary because the first one guarantees that
`string_list_insert()` will never encounter a preexisting entry.
- Since `item->util` is only used in this context, remove the assignment and
- simplify the surrounding code.
+ The `item->util` field is assigned to validate that a rename doesn't
+ already exist in the list. The validation is done after the first check.
+ As this check is removed, clean up the validation and the assignment of
+ this field in `split_head_update()` and `files_transaction_prepare()`.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
2: 438ed0b49a ! 2: 4eba9153ff refs: move duplicate refname update check to generic layer
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
}
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_store *ref_store,
+ }
}
- string_list_sort(&refnames_to_check);
- ret = refs_verify_refnames_available(ref_store, &refnames_to_check, &affected_refnames, NULL,
+ ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
+ &transaction->refnames, NULL,
3: acdaa636d9 = 3: a06733298d refs/files: remove duplicate duplicates check
4: fc61deb9c6 ! 4: 707afdb039 refs/reftable: extract code from the transaction preparation
@@ Commit message
The refactoring consolidates all reference update validation into a
single logical block, which improves code maintainability and
readability. More importantly, this restructuring lays the groundwork
- for implementing partial transaction support in the reftable backend,
- which will be introduced in the following commit.
+ for implementing batched reference update support in the reftable
+ backend, which will be introduced in a followup commit.
No functional changes are included in this commit - it is purely a code
reorganization to support future enhancements.
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
- }
}
- string_list_sort(&refnames_to_check);
+ ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
5: c494ff7e99 ! 5: b93c3f3f9f refs: introduce enum-based transaction error types
@@ Commit message
common scenarios.
This refactoring paves the way for more comprehensive error handling
- which we will utilize in the upcoming commits to add partial transaction
- support.
+ which we will utilize in the upcoming commits to add batch reference
+ update support.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
@@ refs.c: int ref_transaction_commit(struct ref_transaction *transaction,
{
struct strbuf dirname = STRBUF_INIT;
struct strbuf referent = STRBUF_INIT;
+ struct string_list_item *item;
struct ref_iterator *iter = NULL;
struct strset dirnames;
- int ret = -1;
@@ refs/packed-backend.c: static int write_with_updates(struct packed_ref_store *re
"reference is missing but expected %s",
update->refname,
oid_to_hex(&update->old_oid));
-+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
++ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error;
}
}
6: b051e51162 ! 6: feacd50666 refs: implement partial reference transaction support
@@ Metadata
Author: Karthik Nayak <karthik.188@gmail.com>
## Commit message ##
- refs: implement partial reference transaction support
+ refs: implement batch reference update support
- Git's reference transactions are all-or-nothing: either all updates
- succeed, or none do. While this atomic behavior is generally desirable,
- it can be suboptimal especially when using the reftable backend, where
- batching multiple reference updates into a single transaction is more
- efficient than performing them sequentially.
+ Git supports making reference updates with or without transactions.
+ Updates with transactions are generally better optimized. But
+ transactions are all or nothing. This means, if a user wants to batch
+ updates to take advantage of the optimizations without the hard
+ requirement that all updates must succeed, there is no way currently to
+ do so. Particularly with the reftable backend where batching multiple
+ reference updates is more efficient than performing them sequentially.
- Introduce partial transaction support with a new flag,
- 'REF_TRANSACTION_ALLOW_PARTIAL'. When enabled, this flag allows
- individual reference updates that would typically cause the entire
- transaction to fail due to non-system-related errors to be marked as
- rejected while permitting other updates to proceed. System errors
- referred by 'REF_TRANSACTION_ERROR_GENERIC' continue to result in the
- entire transaction failing. This approach enhances flexibility while
- preserving transactional integrity where necessary.
+ Introduce batched update support with a new flag,
+ 'REF_TRANSACTION_ALLOW_FAILURE'. Batched updates while different from
+ transactions, use the transaction infrastructure under the hood. When
+ enabled, this flag allows individual reference updates that would
+ typically cause the entire transaction to fail due to non-system-related
+ errors to be marked as rejected while permitting other updates to
+ proceed. System errors referred by 'REF_TRANSACTION_ERROR_GENERIC'
+ continue to result in the entire transaction failing. This approach
+ enhances flexibility while preserving transactional integrity where
+ necessary.
The implementation introduces several key components:
@@ Commit message
- Modify reference backends (files, packed, reftable) to handle
partial transactions by using `ref_transaction_set_rejected()`
instead of failing the entire transaction when
- `REF_TRANSACTION_ALLOW_PARTIAL` is set.
+ `REF_TRANSACTION_ALLOW_FAILURE` is set.
- Add `ref_transaction_for_each_rejected_update()` to let callers
examine which updates were rejected and why.
- This foundational change enables partial transaction support throughout
- the reference subsystem. A following commit will expose this capability
- to users by adding a `--allow-partial` flag to 'git-update-ref(1)',
+ This foundational change enables batched update support throughout the
+ reference subsystem. A following commit will expose this capability to
+ users by adding a `--batch-updates` flag to 'git-update-ref(1)',
providing both a user-facing feature and a testable implementation.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
@@ refs.c: struct ref_transaction *ref_store_transaction_begin(struct ref_store *re
tr->flags = flags;
string_list_init_dup(&tr->refnames);
+
-+ if (flags & REF_TRANSACTION_ALLOW_PARTIAL)
++ if (flags & REF_TRANSACTION_ALLOW_FAILURE)
+ CALLOC_ARRAY(tr->rejections, 1);
+
return tr;
@@ refs.c: void ref_transaction_free(struct ref_transaction *transaction)
+ if (update_idx >= transaction->nr)
+ BUG("trying to set rejection on invalid update index");
+
-+ if (!(transaction->flags & REF_TRANSACTION_ALLOW_PARTIAL))
++ if (!(transaction->flags & REF_TRANSACTION_ALLOW_FAILURE))
+ return 0;
+
+ if (!transaction->rejections)
-+ BUG("transaction not inititalized with partial support");
++ BUG("transaction not inititalized with failure support");
+
+ /*
+ * Don't accept generic errors, since these errors are not user
@@ refs.h: enum ref_transaction_flag {
+ * is detected. This flag allows transactions to partially apply updates
+ * while rejecting updates which do not match the expected state.
+ */
-+ REF_TRANSACTION_ALLOW_PARTIAL = (1 << 1),
++ REF_TRANSACTION_ALLOW_FAILURE = (1 << 1),
};
/*
@@ refs/packed-backend.c: static enum ref_transaction_error write_with_updates(stru
@@ refs/packed-backend.c: static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
update->refname,
oid_to_hex(&update->old_oid));
- return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
@@ refs/refs-internal.h: struct ref_update {
uint64_t index;
+ /*
-+ * Used in partial transactions to mark if a given update was rejected.
++ * Used in batched reference updates to mark if a given update
++ * was rejected.
+ */
+ enum ref_transaction_error rejection_err;
+
@@ refs/refs-internal.h: enum ref_transaction_state {
};
+/*
-+ * Data structure to hold indices of updates which were rejected, when
-+ * partial transactions where enabled. While the updates themselves hold
-+ * the rejection error, this structure allows a transaction to iterate
-+ * only over the rejected updates.
++ * Data structure to hold indices of updates which were rejected, for batched
++ * reference updates. While the updates themselves hold the rejection error,
++ * this structure allows a transaction to iterate only over the rejected
++ * updates.
+ */
+struct ref_transaction_rejections {
+ size_t *update_indices;
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
+ }
}
- string_list_sort(&refnames_to_check);
+ ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
@@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writer *writer, void *cb_data
struct reftable_transaction_update *tx_update = &arg->updates[i];
struct ref_update *u = tx_update->update;
7: 47d7a64cb4 ! 7: f20877267d refs: support partial update rejections during F/D checks
@@ Metadata
Author: Karthik Nayak <karthik.188@gmail.com>
## Commit message ##
- refs: support partial update rejections during F/D checks
+ refs: support rejection in batch updates during F/D checks
The `refs_verify_refnames_available()` is used to batch check refnames
for F/D conflicts. While this is the more performant alternative than
its individual version, it does not provide rejection capabilities on a
- single update level. For partial transactions, this would mean a
- rejection of the entire transaction whenever one reference has a F/D
- conflict.
+ single update level. For batched updates, this would mean a rejection of
+ the entire transaction whenever one reference has a F/D conflict.
Modify the function to call `ref_transaction_maybe_set_rejected()` to
check if a single update can be rejected. Since this function is only
@@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_sto
@@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
strset_init(&dirnames);
- for (size_t i = 0; i < refnames->nr; i++) {
-+ const size_t *update_idx = (size_t *)refnames->items[i].util;
- const char *refname = refnames->items[i].string;
+ for_each_string_list_item(item, refnames) {
++ const size_t *update_idx = (size_t *)item->util;
+ const char *refname = item->string;
const char *extra_refname;
struct object_id oid;
@@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
@@ refs/files-backend.c: static enum ref_transaction_error lock_raw_ref(struct file
* make sure there is no existing packed ref that conflicts
* with refname. This check is deferred so that we can batch it.
*/
-- string_list_insert(refnames_to_check, refname);
-+ item = string_list_insert(refnames_to_check, refname);
+- string_list_append(refnames_to_check, refname);
++ item = string_list_append(refnames_to_check, refname);
+ item->util = xmalloc(sizeof(update_idx));
+ memcpy(item->util, &update_idx, sizeof(update_idx));
}
@@ refs/refs-internal.h: enum ref_transaction_error ref_update_check_old_target(con
+ * refnames instead of only a single item. This is more efficient in the case
+ * where one needs to check multiple refnames.
+ *
-+ * If a transaction is provided with partial support, then individual updates
-+ * are marked rejected, reference backends are then in charge of not committing
-+ * those updates.
++ * If using batched updates, then individual updates are marked rejected,
++ * reference backends are then in charge of not committing those updates.
+ */
+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
&head_referent, &referent, err);
if (ret) {
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_store *ref_store,
- string_list_sort(&refnames_to_check);
+
ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
&transaction->refnames, NULL,
+ transaction,
8: 2237086c65 ! 8: d6f43d5dce update-ref: add --allow-partial flag for stdin mode
@@ Metadata
Author: Karthik Nayak <karthik.188@gmail.com>
## Commit message ##
- update-ref: add --allow-partial flag for stdin mode
+ update-ref: add --batch-updates flag for stdin mode
When updating multiple references through stdin, Git's update-ref
command normally aborts the entire transaction if any single update
- fails. While this atomic behavior prevents partial updates by default,
- there are cases where applying successful updates while reporting
- failures is desirable.
+ fails. This atomic behavior prevents partial updates. Introduce a new
+ batch update system, where the updates the performed together similar
+ but individual updates are allowed to fail.
- Add a new `--allow-partial` flag that allows the transaction to continue
+ Add a new `--batch-updates` flag that allows the transaction to continue
even when individual reference updates fail. This flag can only be used
- in `--stdin` mode and builds upon the partial transaction support added
- to the refs subsystem. When enabled, failed updates are reported in the
- following format:
+ in `--stdin` mode and builds upon the batch update support added to the
+ refs subsystem in the previous commits. When enabled, failed updates are
+ reported in the following format:
rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
@@ Documentation/git-update-ref.adoc: git-update-ref - Update the object name store
+[synopsis]
+git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
+ [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
-+ [-m <reason>] [--no-deref] --stdin [-z] [--allow-partial]
++ [-m <reason>] [--no-deref] --stdin [-z] [--batch-updates]
DESCRIPTION
-----------
@@ Documentation/git-update-ref.adoc: performs all modifications together. Specify
With `--create-reflog`, update-ref will create a reflog for each ref
even if one would not ordinarily be created.
-+With `--allow-partial`, update-ref continues executing the transaction even if
-+some updates fail due to invalid or incorrect user input, applying only the
-+successful updates. Errors resulting from user-provided input are treated as
-+non-system-related and do not cause the entire transaction to be aborted.
-+However, system-related errors—such as I/O failures or memory issues—will still
-+result in a full failure. Additionally, errors like F/D conflicts are batched
-+for performance optimization and will also cause a full failure. Any failed
++With `--batch-updates`, update-ref executes the updates in a batch but allows
++individual updates to fail due to invalid or incorrect user input, applying only
++the successful updates. However, system-related errors—such as I/O failures or
++memory issues—will result in a full failure of all batched updates. Any failed
+updates will be reported in the following format:
+
+ rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
@@ builtin/update-ref.c
N_("git update-ref [<options>] -d <refname> [<old-oid>]"),
N_("git update-ref [<options>] <refname> <new-oid> [<old-oid>]"),
- N_("git update-ref [<options>] --stdin [-z]"),
-+ N_("git update-ref [<options>] --stdin [-z] [--allow-partial]"),
++ N_("git update-ref [<options>] --stdin [-z] [--batch-updates]"),
NULL
};
@@ builtin/update-ref.c: int cmd_update_ref(int argc,
OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
OPT_BOOL( 0 , "create-reflog", &create_reflog, N_("create a reflog")),
-+ OPT_BIT('0', "allow-partial", &flags, N_("allow partial transactions"),
-+ REF_TRANSACTION_ALLOW_PARTIAL),
++ OPT_BIT('0', "batch-updates", &flags, N_("batch reference updates"),
++ REF_TRANSACTION_ALLOW_FAILURE),
OPT_END(),
};
@@ builtin/update-ref.c: int cmd_update_ref(int argc,
+ update_refs_stdin(flags);
return 0;
- }
-+ } else if (flags & REF_TRANSACTION_ALLOW_PARTIAL)
-+ die("--allow-partial can only be used with --stdin");
++ } else if (flags & REF_TRANSACTION_ALLOW_FAILURE)
++ die("--batch-updates can only be used with --stdin");
if (end_null)
usage_with_options(git_update_ref_usage, options);
@@ t/t1400-update-ref.sh: do
grep "$(git rev-parse $a) $(git rev-parse $a)" actual
'
-+ test_expect_success "stdin $type allow-partial" '
++ test_expect_success "stdin $type batch-updates" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
@@ t/t1400-update-ref.sh: do
+
+ format_command $type "update refs/heads/ref1" "$head" "$Z" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head" "$Z" >>stdin &&
-+ git update-ref $type --stdin --allow-partial <stdin &&
++ git update-ref $type --stdin --batch-updates <stdin &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
@@ t/t1400-update-ref.sh: do
+ )
+ '
+
-+ test_expect_success "stdin $type allow-partial with invalid new_oid" '
++ test_expect_success "stdin $type batch-updates with invalid new_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
@@ t/t1400-update-ref.sh: do
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$(test_oid 001)" "$head" >>stdin &&
-+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
++ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
@@ t/t1400-update-ref.sh: do
+ )
+ '
+
-+ test_expect_success "stdin $type allow-partial with non-commit new_oid" '
++ test_expect_success "stdin $type batch-updates with non-commit new_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
@@ t/t1400-update-ref.sh: do
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head_tree" "$head" >>stdin &&
-+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
++ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
@@ t/t1400-update-ref.sh: do
+ )
+ '
+
-+ test_expect_success "stdin $type allow-partial with non-existent ref" '
++ test_expect_success "stdin $type batch-updates with non-existent ref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
@@ t/t1400-update-ref.sh: do
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
-+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
++ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
@@ t/t1400-update-ref.sh: do
+ )
+ '
+
-+ test_expect_success "stdin $type allow-partial with dangling symref" '
++ test_expect_success "stdin $type batch-updates with dangling symref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
@@ t/t1400-update-ref.sh: do
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
-+ git update-ref $type --no-deref --stdin --allow-partial <stdin >stdout &&
++ git update-ref $type --no-deref --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
@@ t/t1400-update-ref.sh: do
+ )
+ '
+
-+ test_expect_success "stdin $type allow-partial with regular ref as symref" '
++ test_expect_success "stdin $type batch-updates with regular ref as symref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
@@ t/t1400-update-ref.sh: do
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "symref-update refs/heads/ref2" "$old_head" "ref" "refs/heads/nonexistent" >>stdin &&
-+ git update-ref $type --no-deref --stdin --allow-partial <stdin >stdout &&
++ git update-ref $type --no-deref --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
@@ t/t1400-update-ref.sh: do
+ )
+ '
+
-+ test_expect_success "stdin $type allow-partial with invalid old_oid" '
++ test_expect_success "stdin $type batch-updates with invalid old_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
@@ t/t1400-update-ref.sh: do
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$Z" >>stdin &&
-+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
++ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
@@ t/t1400-update-ref.sh: do
+ )
+ '
+
-+ test_expect_success "stdin $type allow-partial with incorrect old oid" '
++ test_expect_success "stdin $type batch-updates with incorrect old oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
@@ t/t1400-update-ref.sh: do
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head" "$old_head" >>stdin &&
-+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
++ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
@@ t/t1400-update-ref.sh: do
+ )
+ '
+
-+ test_expect_success "stdin $type allow-partial refname conflict" '
++ test_expect_success "stdin $type batch-updates refname conflict" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
@@ t/t1400-update-ref.sh: do
+
+ format_command $type "update refs/heads/ref/foo" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
-+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
++ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref/foo >actual &&
+ test_cmp expect actual &&
@@ t/t1400-update-ref.sh: do
+ )
+ '
+
-+ test_expect_success "stdin $type allow-partial refname conflict new ref" '
++ test_expect_success "stdin $type batch-updates refname conflict new ref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
@@ t/t1400-update-ref.sh: do
+
+ format_command $type "update refs/heads/foo" "$old_head" "" >stdin &&
+ format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
-+ git update-ref $type --stdin --allow-partial <stdin >stdout &&
++ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/foo >actual &&
+ test_cmp expect actual &&
base-commit: 679c868f5fffadd1f7e8e49d4d87d745ee36ffb7
change-id: 20241206-245-partially-atomic-ref-updates-9fe8b080345c
Thanks
- Karthik
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH v4 1/8] refs/files: remove redundant check in split_symref_update()
2025-03-20 11:43 ` [PATCH v4 0/8] refs: introduce support for batched reference updates Karthik Nayak
@ 2025-03-20 11:43 ` Karthik Nayak
2025-03-20 11:43 ` [PATCH v4 2/8] refs: move duplicate refname update check to generic layer Karthik Nayak
` (6 subsequent siblings)
7 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-20 11:43 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
In `split_symref_update()`, there were two checks for duplicate
refnames:
- At the start, `string_list_has_string()` ensures the refname is not
already in `affected_refnames`, preventing duplicates from being
added.
- After adding the refname, another check verifies whether the newly
inserted item has a `util` value.
The second check is unnecessary because the first one guarantees that
`string_list_insert()` will never encounter a preexisting entry.
The `item->util` field is assigned to validate that a rename doesn't
already exist in the list. The validation is done after the first check.
As this check is removed, clean up the validation and the assignment of
this field in `split_head_update()` and `files_transaction_prepare()`.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs/files-backend.c | 20 +++-----------------
1 file changed, 3 insertions(+), 17 deletions(-)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index ff54a4bb7e..15559a09c5 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2382,7 +2382,6 @@ static int split_head_update(struct ref_update *update,
struct string_list *affected_refnames,
struct strbuf *err)
{
- struct string_list_item *item;
struct ref_update *new_update;
if ((update->flags & REF_LOG_ONLY) ||
@@ -2421,8 +2420,7 @@ static int split_head_update(struct ref_update *update,
*/
if (strcmp(new_update->refname, "HEAD"))
BUG("%s unexpectedly not 'HEAD'", new_update->refname);
- item = string_list_insert(affected_refnames, new_update->refname);
- item->util = new_update;
+ string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2441,7 +2439,6 @@ static int split_symref_update(struct ref_update *update,
struct string_list *affected_refnames,
struct strbuf *err)
{
- struct string_list_item *item;
struct ref_update *new_update;
unsigned int new_flags;
@@ -2496,11 +2493,7 @@ static int split_symref_update(struct ref_update *update,
* be valid as long as affected_refnames is in use, and NOT
* referent, which might soon be freed by our caller.
*/
- item = string_list_insert(affected_refnames, new_update->refname);
- if (item->util)
- BUG("%s unexpectedly found in affected_refnames",
- new_update->refname);
- item->util = new_update;
+ string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2834,7 +2827,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
*/
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
- struct string_list_item *item;
if ((update->flags & REF_IS_PRUNING) &&
!(update->flags & REF_NO_DEREF))
@@ -2843,13 +2835,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
if (update->flags & REF_LOG_ONLY)
continue;
- item = string_list_append(&affected_refnames, update->refname);
- /*
- * We store a pointer to update in item->util, but at
- * the moment we never use the value of this field
- * except to check whether it is non-NULL.
- */
- item->util = update;
+ string_list_append(&affected_refnames, update->refname);
}
string_list_sort(&affected_refnames);
if (ref_update_reject_duplicates(&affected_refnames, err)) {
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v4 2/8] refs: move duplicate refname update check to generic layer
2025-03-20 11:43 ` [PATCH v4 0/8] refs: introduce support for batched reference updates Karthik Nayak
2025-03-20 11:43 ` [PATCH v4 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
@ 2025-03-20 11:43 ` Karthik Nayak
2025-03-20 11:43 ` [PATCH v4 3/8] refs/files: remove duplicate duplicates check Karthik Nayak
` (5 subsequent siblings)
7 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-20 11:43 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Move the tracking of refnames in `affected_refnames` from individual
backends into the generic layer in 'refs.c'. This centralizes the
duplicate refname detection that was previously handled separately by
each backend.
Make some changes to accommodate this move:
- Add a `string_list` field `refnames` to `ref_transaction` to contain
all the references in a transaction. This field is updated whenever
a new update is added via `ref_transaction_add_update`, so manual
additions in reference backends are dropped.
- Modify the backends to use this field internally as needed. The
backends need to check if an update for refname already exists when
splitting symrefs or adding an update for 'HEAD'.
- In the reftable backend, within `reftable_be_transaction_prepare()`,
move the `string_list_has_string()` check above
`ref_transaction_add_update()`. Since `ref_transaction_add_update()`
automatically adds the refname to `transaction->refnames`,
performing the check after will always return true, so we perform
the check before adding the update.
This helps reduce duplication of functionality between the backends and
makes it easier to make changes in a more centralized manner.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 17 +++++++++++++
refs/files-backend.c | 67 +++++++++++--------------------------------------
refs/packed-backend.c | 25 +-----------------
refs/refs-internal.h | 2 ++
refs/reftable-backend.c | 54 +++++++++++++--------------------------
5 files changed, 51 insertions(+), 114 deletions(-)
diff --git a/refs.c b/refs.c
index 2ac9d8ebd0..504bf2063e 100644
--- a/refs.c
+++ b/refs.c
@@ -1175,6 +1175,7 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,
CALLOC_ARRAY(tr, 1);
tr->ref_store = refs;
tr->flags = flags;
+ string_list_init_dup(&tr->refnames);
return tr;
}
@@ -1205,6 +1206,7 @@ void ref_transaction_free(struct ref_transaction *transaction)
free((char *)transaction->updates[i]->old_target);
free(transaction->updates[i]);
}
+ string_list_clear(&transaction->refnames, 0);
free(transaction->updates);
free(transaction);
}
@@ -1218,6 +1220,7 @@ struct ref_update *ref_transaction_add_update(
const char *committer_info,
const char *msg)
{
+ struct string_list_item *item;
struct ref_update *update;
if (transaction->state != REF_TRANSACTION_OPEN)
@@ -1245,6 +1248,16 @@ struct ref_update *ref_transaction_add_update(
update->msg = normalize_reflog_message(msg);
}
+ /*
+ * This list is generally used by the backends to avoid duplicates.
+ * But we do support multiple log updates for a given refname within
+ * a single transaction.
+ */
+ if (!(update->flags & REF_LOG_ONLY)) {
+ item = string_list_append(&transaction->refnames, refname);
+ item->util = update;
+ }
+
return update;
}
@@ -2405,6 +2418,10 @@ int ref_transaction_prepare(struct ref_transaction *transaction,
return -1;
}
+ string_list_sort(&transaction->refnames);
+ if (ref_update_reject_duplicates(&transaction->refnames, err))
+ return TRANSACTION_GENERIC_ERROR;
+
ret = refs->be->transaction_prepare(refs, transaction, err);
if (ret)
return ret;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 15559a09c5..58f62ea8a3 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2378,9 +2378,7 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st
*/
static int split_head_update(struct ref_update *update,
struct ref_transaction *transaction,
- const char *head_ref,
- struct string_list *affected_refnames,
- struct strbuf *err)
+ const char *head_ref, struct strbuf *err)
{
struct ref_update *new_update;
@@ -2398,7 +2396,7 @@ static int split_head_update(struct ref_update *update,
* transaction. This check is O(lg N) in the transaction
* size, but it happens at most once per transaction.
*/
- if (string_list_has_string(affected_refnames, "HEAD")) {
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
/* An entry already existed */
strbuf_addf(err,
"multiple updates for 'HEAD' (including one "
@@ -2420,7 +2418,6 @@ static int split_head_update(struct ref_update *update,
*/
if (strcmp(new_update->refname, "HEAD"))
BUG("%s unexpectedly not 'HEAD'", new_update->refname);
- string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2436,7 +2433,6 @@ static int split_head_update(struct ref_update *update,
static int split_symref_update(struct ref_update *update,
const char *referent,
struct ref_transaction *transaction,
- struct string_list *affected_refnames,
struct strbuf *err)
{
struct ref_update *new_update;
@@ -2448,7 +2444,7 @@ static int split_symref_update(struct ref_update *update,
* size, but it happens at most once per symref in a
* transaction.
*/
- if (string_list_has_string(affected_refnames, referent)) {
+ if (string_list_has_string(&transaction->refnames, referent)) {
/* An entry already exists */
strbuf_addf(err,
"multiple updates for '%s' (including one "
@@ -2486,15 +2482,6 @@ static int split_symref_update(struct ref_update *update,
update->flags |= REF_LOG_ONLY | REF_NO_DEREF;
update->flags &= ~REF_HAVE_OLD;
- /*
- * Add the referent. This insertion is O(N) in the transaction
- * size, but it happens at most once per symref in a
- * transaction. Make sure to add new_update->refname, which will
- * be valid as long as affected_refnames is in use, and NOT
- * referent, which might soon be freed by our caller.
- */
- string_list_insert(affected_refnames, new_update->refname);
-
return 0;
}
@@ -2558,7 +2545,6 @@ static int lock_ref_for_update(struct files_ref_store *refs,
struct ref_transaction *transaction,
const char *head_ref,
struct string_list *refnames_to_check,
- struct string_list *affected_refnames,
struct strbuf *err)
{
struct strbuf referent = STRBUF_INIT;
@@ -2575,8 +2561,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
update->flags |= REF_DELETING;
if (head_ref) {
- ret = split_head_update(update, transaction, head_ref,
- affected_refnames, err);
+ ret = split_head_update(update, transaction, head_ref, err);
if (ret)
goto out;
}
@@ -2586,9 +2571,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,
lock->count++;
} else {
ret = lock_raw_ref(refs, update->refname, mustexist,
- refnames_to_check, affected_refnames,
- &lock, &referent,
- &update->type, err);
+ refnames_to_check, &transaction->refnames,
+ &lock, &referent, &update->type, err);
if (ret) {
char *reason;
@@ -2642,9 +2626,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,
* of processing the split-off update, so we
* don't have to do it here.
*/
- ret = split_symref_update(update,
- referent.buf, transaction,
- affected_refnames, err);
+ ret = split_symref_update(update, referent.buf,
+ transaction, err);
if (ret)
goto out;
}
@@ -2799,7 +2782,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
"ref_transaction_prepare");
size_t i;
int ret = 0;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
struct string_list refnames_to_check = STRING_LIST_INIT_NODUP;
char *head_ref = NULL;
int head_type;
@@ -2818,12 +2800,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
transaction->backend_data = backend_data;
/*
- * Fail if a refname appears more than once in the
- * transaction. (If we end up splitting up any updates using
- * split_symref_update() or split_head_update(), those
- * functions will check that the new updates don't have the
- * same refname as any existing ones.) Also fail if any of the
- * updates use REF_IS_PRUNING without REF_NO_DEREF.
+ * Fail if any of the updates use REF_IS_PRUNING without REF_NO_DEREF.
*/
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
@@ -2831,16 +2808,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
if ((update->flags & REF_IS_PRUNING) &&
!(update->flags & REF_NO_DEREF))
BUG("REF_IS_PRUNING set without REF_NO_DEREF");
-
- if (update->flags & REF_LOG_ONLY)
- continue;
-
- string_list_append(&affected_refnames, update->refname);
- }
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
}
/*
@@ -2882,7 +2849,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
ret = lock_ref_for_update(refs, update, transaction,
head_ref, &refnames_to_check,
- &affected_refnames, err);
+ err);
if (ret)
goto cleanup;
@@ -2929,7 +2896,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
* So instead, we accept the race for now.
*/
if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
- &affected_refnames, NULL, 0, err)) {
+ &transaction->refnames, NULL, 0, err)) {
ret = TRANSACTION_NAME_CONFLICT;
goto cleanup;
}
@@ -2975,7 +2942,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
cleanup:
free(head_ref);
- string_list_clear(&affected_refnames, 0);
string_list_clear(&refnames_to_check, 0);
if (ret)
@@ -3050,13 +3016,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (transaction->state != REF_TRANSACTION_PREPARED)
BUG("commit called for transaction that is not prepared");
- /* Fail if a refname appears more than once in the transaction: */
- for (i = 0; i < transaction->nr; i++)
- if (!(transaction->updates[i]->flags & REF_LOG_ONLY))
- string_list_append(&affected_refnames,
- transaction->updates[i]->refname);
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
+ string_list_sort(&transaction->refnames);
+ if (ref_update_reject_duplicates(&transaction->refnames, err)) {
ret = TRANSACTION_GENERIC_ERROR;
goto cleanup;
}
@@ -3074,7 +3035,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
* that we are creating already exists.
*/
if (refs_for_each_rawref(&refs->base, ref_present,
- &affected_refnames))
+ &transaction->refnames))
BUG("initial ref transaction called with existing refs");
packed_transaction = ref_store_transaction_begin(refs->packed_ref_store,
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index f4c82ba2c7..19220d2e99 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1622,8 +1622,6 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
struct packed_transaction_backend_data {
/* True iff the transaction owns the packed-refs lock. */
int own_lock;
-
- struct string_list updates;
};
static void packed_transaction_cleanup(struct packed_ref_store *refs,
@@ -1632,8 +1630,6 @@ static void packed_transaction_cleanup(struct packed_ref_store *refs,
struct packed_transaction_backend_data *data = transaction->backend_data;
if (data) {
- string_list_clear(&data->updates, 0);
-
if (is_tempfile_active(refs->tempfile))
delete_tempfile(&refs->tempfile);
@@ -1658,7 +1654,6 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
"ref_transaction_prepare");
struct packed_transaction_backend_data *data;
- size_t i;
int ret = TRANSACTION_GENERIC_ERROR;
/*
@@ -1671,34 +1666,16 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
*/
CALLOC_ARRAY(data, 1);
- string_list_init_nodup(&data->updates);
transaction->backend_data = data;
- /*
- * Stick the updates in a string list by refname so that we
- * can sort them:
- */
- for (i = 0; i < transaction->nr; i++) {
- struct ref_update *update = transaction->updates[i];
- struct string_list_item *item =
- string_list_append(&data->updates, update->refname);
-
- /* Store a pointer to update in item->util: */
- item->util = update;
- }
- string_list_sort(&data->updates);
-
- if (ref_update_reject_duplicates(&data->updates, err))
- goto failure;
-
if (!is_lock_file_locked(&refs->lock)) {
if (packed_refs_lock(ref_store, 0, err))
goto failure;
data->own_lock = 1;
}
- if (write_with_updates(refs, &data->updates, err))
+ if (write_with_updates(refs, &transaction->refnames, err))
goto failure;
transaction->state = REF_TRANSACTION_PREPARED;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index e5862757a7..92db793026 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -3,6 +3,7 @@
#include "refs.h"
#include "iterator.h"
+#include "string-list.h"
struct fsck_options;
struct ref_transaction;
@@ -198,6 +199,7 @@ enum ref_transaction_state {
struct ref_transaction {
struct ref_store *ref_store;
struct ref_update **updates;
+ struct string_list refnames;
size_t alloc;
size_t nr;
enum ref_transaction_state state;
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index ae434cd248..a92c9a2f4f 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1076,7 +1076,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
struct reftable_ref_store *refs =
reftable_be_downcast(ref_store, REF_STORE_WRITE|REF_STORE_MAIN, "ref_transaction_prepare");
struct strbuf referent = STRBUF_INIT, head_referent = STRBUF_INIT;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
struct string_list refnames_to_check = STRING_LIST_INIT_NODUP;
struct reftable_transaction_data *tx_data = NULL;
struct reftable_backend *be;
@@ -1101,10 +1100,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->updates[i], err);
if (ret)
goto done;
-
- if (!(transaction->updates[i]->flags & REF_LOG_ONLY))
- string_list_append(&affected_refnames,
- transaction->updates[i]->refname);
}
/*
@@ -1116,17 +1111,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
tx_data->args[i].updates_alloc = tx_data->args[i].updates_expected;
}
- /*
- * Fail if a refname appears more than once in the transaction.
- * This code is taken from the files backend and is a good candidate to
- * be moved into the generic layer.
- */
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto done;
- }
-
/*
* TODO: it's dubious whether we should reload the stack that "HEAD"
* belongs to or not. In theory, it may happen that we only modify
@@ -1194,14 +1178,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
!(u->flags & REF_LOG_ONLY) &&
!(u->flags & REF_UPDATE_VIA_HEAD) &&
!strcmp(rewritten_ref, head_referent.buf)) {
- struct ref_update *new_update;
-
/*
* First make sure that HEAD is not already in the
* transaction. This check is O(lg N) in the transaction
* size, but it happens at most once per transaction.
*/
- if (string_list_has_string(&affected_refnames, "HEAD")) {
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
/* An entry already existed */
strbuf_addf(err,
_("multiple updates for 'HEAD' (including one "
@@ -1211,12 +1193,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
goto done;
}
- new_update = ref_transaction_add_update(
- transaction, "HEAD",
- u->flags | REF_LOG_ONLY | REF_NO_DEREF,
- &u->new_oid, &u->old_oid, NULL, NULL, NULL,
- u->msg);
- string_list_insert(&affected_refnames, new_update->refname);
+ ref_transaction_add_update(
+ transaction, "HEAD",
+ u->flags | REF_LOG_ONLY | REF_NO_DEREF,
+ &u->new_oid, &u->old_oid, NULL, NULL, NULL,
+ u->msg);
}
ret = reftable_backend_read_ref(be, rewritten_ref,
@@ -1281,6 +1262,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
if (!strcmp(rewritten_ref, "HEAD"))
new_flags |= REF_UPDATE_VIA_HEAD;
+ if (string_list_has_string(&transaction->refnames, referent.buf)) {
+ strbuf_addf(err,
+ _("multiple updates for '%s' (including one "
+ "via symref '%s') are not allowed"),
+ referent.buf, u->refname);
+ ret = TRANSACTION_NAME_CONFLICT;
+ goto done;
+ }
+
/*
* If we are updating a symref (eg. HEAD), we should also
* update the branch that the symref points to.
@@ -1305,16 +1295,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
*/
u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
u->flags &= ~REF_HAVE_OLD;
-
- if (string_list_has_string(&affected_refnames, new_update->refname)) {
- strbuf_addf(err,
- _("multiple updates for '%s' (including one "
- "via symref '%s') are not allowed"),
- referent.buf, u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
- string_list_insert(&affected_refnames, new_update->refname);
}
}
@@ -1383,7 +1363,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
}
}
- ret = refs_verify_refnames_available(ref_store, &refnames_to_check, &affected_refnames, NULL,
+ ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
+ &transaction->refnames, NULL,
transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
err);
if (ret < 0)
@@ -1401,7 +1382,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
strbuf_addf(err, _("reftable: transaction prepare: %s"),
reftable_error_str(ret));
}
- string_list_clear(&affected_refnames, 0);
strbuf_release(&referent);
strbuf_release(&head_referent);
string_list_clear(&refnames_to_check, 0);
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v4 3/8] refs/files: remove duplicate duplicates check
2025-03-20 11:43 ` [PATCH v4 0/8] refs: introduce support for batched reference updates Karthik Nayak
2025-03-20 11:43 ` [PATCH v4 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
2025-03-20 11:43 ` [PATCH v4 2/8] refs: move duplicate refname update check to generic layer Karthik Nayak
@ 2025-03-20 11:43 ` Karthik Nayak
2025-03-20 11:43 ` [PATCH v4 4/8] refs/reftable: extract code from the transaction preparation Karthik Nayak
` (4 subsequent siblings)
7 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-20 11:43 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Within the files reference backend's transaction's 'finish' phase, a
verification step is currently performed wherein the refnames list is
sorted and examined for multiple updates targeting the same refname.
It has been observed that this verification is redundant, as an
identical check is already executed during the transaction's 'prepare'
stage. Since the refnames list remains unmodified following the
'prepare' stage, this secondary verification can be safely eliminated.
The duplicate check has been removed accordingly, and the
`ref_update_reject_duplicates()` function has been marked as static, as
its usage is now confined to 'refs.c'.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 9 +++++++--
refs/files-backend.c | 6 ------
refs/refs-internal.h | 8 --------
3 files changed, 7 insertions(+), 16 deletions(-)
diff --git a/refs.c b/refs.c
index 504bf2063e..61bed9672a 100644
--- a/refs.c
+++ b/refs.c
@@ -2303,8 +2303,13 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
return ret;
}
-int ref_update_reject_duplicates(struct string_list *refnames,
- struct strbuf *err)
+/*
+ * Write an error to `err` and return a nonzero value iff the same
+ * refname appears multiple times in `refnames`. `refnames` must be
+ * sorted on entry to this function.
+ */
+static int ref_update_reject_duplicates(struct string_list *refnames,
+ struct strbuf *err)
{
size_t i, n = refnames->nr;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 58f62ea8a3..ea023a59fc 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3016,12 +3016,6 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (transaction->state != REF_TRANSACTION_PREPARED)
BUG("commit called for transaction that is not prepared");
- string_list_sort(&transaction->refnames);
- if (ref_update_reject_duplicates(&transaction->refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
- }
-
/*
* It's really undefined to call this function in an active
* repository or when there are existing references: we are
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 92db793026..6d3770d0cc 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -142,14 +142,6 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
struct object_id *oid, struct strbuf *referent,
unsigned int *type, int *failure_errno);
-/*
- * Write an error to `err` and return a nonzero value iff the same
- * refname appears multiple times in `refnames`. `refnames` must be
- * sorted on entry to this function.
- */
-int ref_update_reject_duplicates(struct string_list *refnames,
- struct strbuf *err);
-
/*
* Add a ref_update with the specified properties to transaction, and
* return a pointer to the new object. This function does not verify
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v4 4/8] refs/reftable: extract code from the transaction preparation
2025-03-20 11:43 ` [PATCH v4 0/8] refs: introduce support for batched reference updates Karthik Nayak
` (2 preceding siblings ...)
2025-03-20 11:43 ` [PATCH v4 3/8] refs/files: remove duplicate duplicates check Karthik Nayak
@ 2025-03-20 11:43 ` Karthik Nayak
2025-03-20 11:44 ` [PATCH v4 5/8] refs: introduce enum-based transaction error types Karthik Nayak
` (3 subsequent siblings)
7 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-20 11:43 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Extract the core logic for preparing individual reference updates from
`reftable_be_transaction_prepare()` into `prepare_single_update()`. This
dedicated function now handles all validation and preparation steps for
each reference update in the transaction, including object ID
verification, HEAD reference handling, and symref processing.
The refactoring consolidates all reference update validation into a
single logical block, which improves code maintainability and
readability. More importantly, this restructuring lays the groundwork
for implementing batched reference update support in the reftable
backend, which will be introduced in a followup commit.
No functional changes are included in this commit - it is purely a code
reorganization to support future enhancements.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs/reftable-backend.c | 463 +++++++++++++++++++++++++-----------------------
1 file changed, 237 insertions(+), 226 deletions(-)
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index a92c9a2f4f..786df11a03 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1069,6 +1069,239 @@ static int queue_transaction_update(struct reftable_ref_store *refs,
return 0;
}
+static int prepare_single_update(struct reftable_ref_store *refs,
+ struct reftable_transaction_data *tx_data,
+ struct ref_transaction *transaction,
+ struct reftable_backend *be,
+ struct ref_update *u,
+ struct string_list *refnames_to_check,
+ unsigned int head_type,
+ struct strbuf *head_referent,
+ struct strbuf *referent,
+ struct strbuf *err)
+{
+ struct object_id current_oid = {0};
+ const char *rewritten_ref;
+ int ret = 0;
+
+ /*
+ * There is no need to reload the respective backends here as
+ * we have already reloaded them when preparing the transaction
+ * update. And given that the stacks have been locked there
+ * shouldn't have been any concurrent modifications of the
+ * stack.
+ */
+ ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
+ if (ret)
+ return ret;
+
+ /* Verify that the new object ID is valid. */
+ if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
+ !(u->flags & REF_SKIP_OID_VERIFICATION) &&
+ !(u->flags & REF_LOG_ONLY)) {
+ struct object *o = parse_object(refs->base.repo, &u->new_oid);
+ if (!o) {
+ strbuf_addf(err,
+ _("trying to write ref '%s' with nonexistent object %s"),
+ u->refname, oid_to_hex(&u->new_oid));
+ return -1;
+ }
+
+ if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
+ strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
+ oid_to_hex(&u->new_oid), u->refname);
+ return -1;
+ }
+ }
+
+ /*
+ * When we update the reference that HEAD points to we enqueue
+ * a second log-only update for HEAD so that its reflog is
+ * updated accordingly.
+ */
+ if (head_type == REF_ISSYMREF &&
+ !(u->flags & REF_LOG_ONLY) &&
+ !(u->flags & REF_UPDATE_VIA_HEAD) &&
+ !strcmp(rewritten_ref, head_referent->buf)) {
+ /*
+ * First make sure that HEAD is not already in the
+ * transaction. This check is O(lg N) in the transaction
+ * size, but it happens at most once per transaction.
+ */
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
+ /* An entry already existed */
+ strbuf_addf(err,
+ _("multiple updates for 'HEAD' (including one "
+ "via its referent '%s') are not allowed"),
+ u->refname);
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ ref_transaction_add_update(
+ transaction, "HEAD",
+ u->flags | REF_LOG_ONLY | REF_NO_DEREF,
+ &u->new_oid, &u->old_oid, NULL, NULL, NULL,
+ u->msg);
+ }
+
+ ret = reftable_backend_read_ref(be, rewritten_ref,
+ ¤t_oid, referent, &u->type);
+ if (ret < 0)
+ return ret;
+ if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
+ /*
+ * The reference does not exist, and we either have no
+ * old object ID or expect the reference to not exist.
+ * We can thus skip below safety checks as well as the
+ * symref splitting. But we do want to verify that
+ * there is no conflicting reference here so that we
+ * can output a proper error message instead of failing
+ * at a later point.
+ */
+ string_list_append(refnames_to_check, u->refname);
+
+ /*
+ * There is no need to write the reference deletion
+ * when the reference in question doesn't exist.
+ */
+ if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
+ ret = queue_transaction_update(refs, tx_data, u,
+ ¤t_oid, err);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+ }
+ if (ret > 0) {
+ /* The reference does not exist, but we expected it to. */
+ strbuf_addf(err, _("cannot lock ref '%s': "
+
+
+ "unable to resolve reference '%s'"),
+ ref_update_original_update_refname(u), u->refname);
+ return -1;
+ }
+
+ if (u->type & REF_ISSYMREF) {
+ /*
+ * The reftable stack is locked at this point already,
+ * so it is safe to call `refs_resolve_ref_unsafe()`
+ * here without causing races.
+ */
+ const char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,
+ ¤t_oid, NULL);
+
+ if (u->flags & REF_NO_DEREF) {
+ if (u->flags & REF_HAVE_OLD && !resolved) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "error reading reference"), u->refname);
+ return -1;
+ }
+ } else {
+ struct ref_update *new_update;
+ int new_flags;
+
+ new_flags = u->flags;
+ if (!strcmp(rewritten_ref, "HEAD"))
+ new_flags |= REF_UPDATE_VIA_HEAD;
+
+ if (string_list_has_string(&transaction->refnames, referent->buf)) {
+ strbuf_addf(err,
+ _("multiple updates for '%s' (including one "
+ "via symref '%s') are not allowed"),
+ referent->buf, u->refname);
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ /*
+ * If we are updating a symref (eg. HEAD), we should also
+ * update the branch that the symref points to.
+ *
+ * This is generic functionality, and would be better
+ * done in refs.c, but the current implementation is
+ * intertwined with the locking in files-backend.c.
+ */
+ new_update = ref_transaction_add_update(
+ transaction, referent->buf, new_flags,
+ u->new_target ? NULL : &u->new_oid,
+ u->old_target ? NULL : &u->old_oid,
+ u->new_target, u->old_target,
+ u->committer_info, u->msg);
+
+ new_update->parent_update = u;
+
+ /*
+ * Change the symbolic ref update to log only. Also, it
+ * doesn't need to check its old OID value, as that will be
+ * done when new_update is processed.
+ */
+ u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
+ u->flags &= ~REF_HAVE_OLD;
+ }
+ }
+
+ /*
+ * Verify that the old object matches our expectations. Note
+ * that the error messages here do not make a lot of sense in
+ * the context of the reftable backend as we never lock
+ * individual refs. But the error messages match what the files
+ * backend returns, which keeps our tests happy.
+ */
+ if (u->old_target) {
+ if (!(u->type & REF_ISSYMREF)) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "expected symref with target '%s': "
+ "but is a regular ref"),
+ ref_update_original_update_refname(u),
+ u->old_target);
+ return -1;
+ }
+
+ if (ref_update_check_old_target(referent->buf, u, err)) {
+ return -1;
+ }
+ } else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
+ if (is_null_oid(&u->old_oid)) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "reference already exists"),
+ ref_update_original_update_refname(u));
+ return TRANSACTION_CREATE_EXISTS;
+ }
+ else if (is_null_oid(¤t_oid))
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "reference is missing but expected %s"),
+ ref_update_original_update_refname(u),
+ oid_to_hex(&u->old_oid));
+ else
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "is at %s but expected %s"),
+ ref_update_original_update_refname(u),
+ oid_to_hex(¤t_oid),
+ oid_to_hex(&u->old_oid));
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ /*
+ * If all of the following conditions are true:
+ *
+ * - We're not about to write a symref.
+ * - We're not about to write a log-only entry.
+ * - Old and new object ID are different.
+ *
+ * Then we're essentially doing a no-op update that can be
+ * skipped. This is not only for the sake of efficiency, but
+ * also skips writing unneeded reflog entries.
+ */
+ if ((u->type & REF_ISSYMREF) ||
+ (u->flags & REF_LOG_ONLY) ||
+ (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))
+ return queue_transaction_update(refs, tx_data, u,
+ ¤t_oid, err);
+
+ return 0;
+}
+
static int reftable_be_transaction_prepare(struct ref_store *ref_store,
struct ref_transaction *transaction,
struct strbuf *err)
@@ -1133,234 +1366,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
ret = 0;
for (i = 0; i < transaction->nr; i++) {
- struct ref_update *u = transaction->updates[i];
- struct object_id current_oid = {0};
- const char *rewritten_ref;
-
- /*
- * There is no need to reload the respective backends here as
- * we have already reloaded them when preparing the transaction
- * update. And given that the stacks have been locked there
- * shouldn't have been any concurrent modifications of the
- * stack.
- */
- ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
+ ret = prepare_single_update(refs, tx_data, transaction, be,
+ transaction->updates[i],
+ &refnames_to_check, head_type,
+ &head_referent, &referent, err);
if (ret)
goto done;
-
- /* Verify that the new object ID is valid. */
- if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
- !(u->flags & REF_SKIP_OID_VERIFICATION) &&
- !(u->flags & REF_LOG_ONLY)) {
- struct object *o = parse_object(refs->base.repo, &u->new_oid);
- if (!o) {
- strbuf_addf(err,
- _("trying to write ref '%s' with nonexistent object %s"),
- u->refname, oid_to_hex(&u->new_oid));
- ret = -1;
- goto done;
- }
-
- if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
- strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
- oid_to_hex(&u->new_oid), u->refname);
- ret = -1;
- goto done;
- }
- }
-
- /*
- * When we update the reference that HEAD points to we enqueue
- * a second log-only update for HEAD so that its reflog is
- * updated accordingly.
- */
- if (head_type == REF_ISSYMREF &&
- !(u->flags & REF_LOG_ONLY) &&
- !(u->flags & REF_UPDATE_VIA_HEAD) &&
- !strcmp(rewritten_ref, head_referent.buf)) {
- /*
- * First make sure that HEAD is not already in the
- * transaction. This check is O(lg N) in the transaction
- * size, but it happens at most once per transaction.
- */
- if (string_list_has_string(&transaction->refnames, "HEAD")) {
- /* An entry already existed */
- strbuf_addf(err,
- _("multiple updates for 'HEAD' (including one "
- "via its referent '%s') are not allowed"),
- u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
-
- ref_transaction_add_update(
- transaction, "HEAD",
- u->flags | REF_LOG_ONLY | REF_NO_DEREF,
- &u->new_oid, &u->old_oid, NULL, NULL, NULL,
- u->msg);
- }
-
- ret = reftable_backend_read_ref(be, rewritten_ref,
- ¤t_oid, &referent, &u->type);
- if (ret < 0)
- goto done;
- if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
- /*
- * The reference does not exist, and we either have no
- * old object ID or expect the reference to not exist.
- * We can thus skip below safety checks as well as the
- * symref splitting. But we do want to verify that
- * there is no conflicting reference here so that we
- * can output a proper error message instead of failing
- * at a later point.
- */
- string_list_append(&refnames_to_check, u->refname);
-
- /*
- * There is no need to write the reference deletion
- * when the reference in question doesn't exist.
- */
- if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
- ret = queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
- if (ret)
- goto done;
- }
-
- continue;
- }
- if (ret > 0) {
- /* The reference does not exist, but we expected it to. */
- strbuf_addf(err, _("cannot lock ref '%s': "
- "unable to resolve reference '%s'"),
- ref_update_original_update_refname(u), u->refname);
- ret = -1;
- goto done;
- }
-
- if (u->type & REF_ISSYMREF) {
- /*
- * The reftable stack is locked at this point already,
- * so it is safe to call `refs_resolve_ref_unsafe()`
- * here without causing races.
- */
- const char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,
- ¤t_oid, NULL);
-
- if (u->flags & REF_NO_DEREF) {
- if (u->flags & REF_HAVE_OLD && !resolved) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "error reading reference"), u->refname);
- ret = -1;
- goto done;
- }
- } else {
- struct ref_update *new_update;
- int new_flags;
-
- new_flags = u->flags;
- if (!strcmp(rewritten_ref, "HEAD"))
- new_flags |= REF_UPDATE_VIA_HEAD;
-
- if (string_list_has_string(&transaction->refnames, referent.buf)) {
- strbuf_addf(err,
- _("multiple updates for '%s' (including one "
- "via symref '%s') are not allowed"),
- referent.buf, u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
-
- /*
- * If we are updating a symref (eg. HEAD), we should also
- * update the branch that the symref points to.
- *
- * This is generic functionality, and would be better
- * done in refs.c, but the current implementation is
- * intertwined with the locking in files-backend.c.
- */
- new_update = ref_transaction_add_update(
- transaction, referent.buf, new_flags,
- u->new_target ? NULL : &u->new_oid,
- u->old_target ? NULL : &u->old_oid,
- u->new_target, u->old_target,
- u->committer_info, u->msg);
-
- new_update->parent_update = u;
-
- /*
- * Change the symbolic ref update to log only. Also, it
- * doesn't need to check its old OID value, as that will be
- * done when new_update is processed.
- */
- u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
- u->flags &= ~REF_HAVE_OLD;
- }
- }
-
- /*
- * Verify that the old object matches our expectations. Note
- * that the error messages here do not make a lot of sense in
- * the context of the reftable backend as we never lock
- * individual refs. But the error messages match what the files
- * backend returns, which keeps our tests happy.
- */
- if (u->old_target) {
- if (!(u->type & REF_ISSYMREF)) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "expected symref with target '%s': "
- "but is a regular ref"),
- ref_update_original_update_refname(u),
- u->old_target);
- ret = -1;
- goto done;
- }
-
- if (ref_update_check_old_target(referent.buf, u, err)) {
- ret = -1;
- goto done;
- }
- } else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
- ret = TRANSACTION_NAME_CONFLICT;
- if (is_null_oid(&u->old_oid)) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "reference already exists"),
- ref_update_original_update_refname(u));
- ret = TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(¤t_oid))
- strbuf_addf(err, _("cannot lock ref '%s': "
- "reference is missing but expected %s"),
- ref_update_original_update_refname(u),
- oid_to_hex(&u->old_oid));
- else
- strbuf_addf(err, _("cannot lock ref '%s': "
- "is at %s but expected %s"),
- ref_update_original_update_refname(u),
- oid_to_hex(¤t_oid),
- oid_to_hex(&u->old_oid));
- goto done;
- }
-
- /*
- * If all of the following conditions are true:
- *
- * - We're not about to write a symref.
- * - We're not about to write a log-only entry.
- * - Old and new object ID are different.
- *
- * Then we're essentially doing a no-op update that can be
- * skipped. This is not only for the sake of efficiency, but
- * also skips writing unneeded reflog entries.
- */
- if ((u->type & REF_ISSYMREF) ||
- (u->flags & REF_LOG_ONLY) ||
- (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid))) {
- ret = queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
- if (ret)
- goto done;
- }
}
ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v4 5/8] refs: introduce enum-based transaction error types
2025-03-20 11:43 ` [PATCH v4 0/8] refs: introduce support for batched reference updates Karthik Nayak
` (3 preceding siblings ...)
2025-03-20 11:43 ` [PATCH v4 4/8] refs/reftable: extract code from the transaction preparation Karthik Nayak
@ 2025-03-20 11:44 ` Karthik Nayak
2025-03-20 20:26 ` Patrick Steinhardt
2025-03-20 11:44 ` [PATCH v4 6/8] refs: implement batch reference update support Karthik Nayak
` (2 subsequent siblings)
7 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-03-20 11:44 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Replace preprocessor-defined transaction errors with a strongly-typed
enum `ref_transaction_error`. This change:
- Improves type safety and function signature clarity.
- Makes error handling more explicit and discoverable.
- Maintains existing error cases, while adding new error cases for
common scenarios.
This refactoring paves the way for more comprehensive error handling
which we will utilize in the upcoming commits to add batch reference
update support.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
builtin/fetch.c | 2 +-
refs.c | 49 ++++++------
refs.h | 54 ++++++++-----
refs/files-backend.c | 202 ++++++++++++++++++++++++------------------------
refs/packed-backend.c | 23 +++---
refs/refs-internal.h | 5 +-
refs/reftable-backend.c | 64 +++++++--------
7 files changed, 213 insertions(+), 186 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 95fd0018b9..7615c17faf 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -687,7 +687,7 @@ static int s_update_ref(const char *action,
switch (ref_transaction_commit(our_transaction, &err)) {
case 0:
break;
- case TRANSACTION_NAME_CONFLICT:
+ case REF_TRANSACTION_ERROR_NAME_CONFLICT:
ret = STORE_REF_ERROR_DF_CONFLICT;
goto out;
default:
diff --git a/refs.c b/refs.c
index 61bed9672a..3d0b53d56e 100644
--- a/refs.c
+++ b/refs.c
@@ -2271,7 +2271,7 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
REF_NO_DEREF, logmsg, &err))
goto error_return;
prepret = ref_transaction_prepare(transaction, &err);
- if (prepret && prepret != TRANSACTION_CREATE_EXISTS)
+ if (prepret && prepret != REF_TRANSACTION_ERROR_CREATE_EXISTS)
goto error_return;
} else {
if (ref_transaction_update(transaction, ref, NULL, NULL,
@@ -2289,7 +2289,7 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
}
}
- if (prepret == TRANSACTION_CREATE_EXISTS)
+ if (prepret == REF_TRANSACTION_ERROR_CREATE_EXISTS)
goto cleanup;
if (ref_transaction_commit(transaction, &err))
@@ -2425,7 +2425,7 @@ int ref_transaction_prepare(struct ref_transaction *transaction,
string_list_sort(&transaction->refnames);
if (ref_update_reject_duplicates(&transaction->refnames, err))
- return TRANSACTION_GENERIC_ERROR;
+ return REF_TRANSACTION_ERROR_GENERIC;
ret = refs->be->transaction_prepare(refs, transaction, err);
if (ret)
@@ -2497,19 +2497,19 @@ int ref_transaction_commit(struct ref_transaction *transaction,
return ret;
}
-int refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *refnames,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err)
+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err)
{
struct strbuf dirname = STRBUF_INIT;
struct strbuf referent = STRBUF_INIT;
struct string_list_item *item;
struct ref_iterator *iter = NULL;
struct strset dirnames;
- int ret = -1;
+ int ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
/*
* For the sake of comments in this function, suppose that
@@ -2625,12 +2625,13 @@ int refs_verify_refnames_available(struct ref_store *refs,
return ret;
}
-int refs_verify_refname_available(struct ref_store *refs,
- const char *refname,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err)
+enum ref_transaction_error refs_verify_refname_available(
+ struct ref_store *refs,
+ const char *refname,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err)
{
struct string_list_item item = { .string = (char *) refname };
struct string_list refnames = {
@@ -2818,8 +2819,9 @@ int ref_update_has_null_new_value(struct ref_update *update)
return !update->new_target && is_null_oid(&update->new_oid);
}
-int ref_update_check_old_target(const char *referent, struct ref_update *update,
- struct strbuf *err)
+enum ref_transaction_error ref_update_check_old_target(const char *referent,
+ struct ref_update *update,
+ struct strbuf *err)
{
if (!update->old_target)
BUG("called without old_target set");
@@ -2827,17 +2829,18 @@ int ref_update_check_old_target(const char *referent, struct ref_update *update,
if (!strcmp(referent, update->old_target))
return 0;
- if (!strcmp(referent, ""))
+ if (!strcmp(referent, "")) {
strbuf_addf(err, "verifying symref target: '%s': "
"reference is missing but expected %s",
ref_update_original_update_refname(update),
update->old_target);
- else
- strbuf_addf(err, "verifying symref target: '%s': "
- "is at %s but expected %s",
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ }
+
+ strbuf_addf(err, "verifying symref target: '%s': is at %s but expected %s",
ref_update_original_update_refname(update),
referent, update->old_target);
- return -1;
+ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
}
struct migration_data {
diff --git a/refs.h b/refs.h
index 240e2d8537..dcd83e81e2 100644
--- a/refs.h
+++ b/refs.h
@@ -16,6 +16,29 @@ struct worktree;
enum ref_storage_format ref_storage_format_by_name(const char *name);
const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);
+/*
+ * enum ref_transaction_error represents the following return codes:
+ * REF_TRANSACTION_ERROR_GENERIC error_code: default error code.
+ * REF_TRANSACTION_ERROR_NAME_CONFLICT error_code: ref name conflict like A vs A/B.
+ * REF_TRANSACTION_ERROR_CREATE_EXISTS error_code: ref to be created already exists.
+ * REF_TRANSACTION_ERROR_NONEXISTENT_REF error_code: ref expected but doesn't exist.
+ * REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE error_code: provided old_oid or old_target of
+ * reference doesn't match actual.
+ * REF_TRANSACTION_ERROR_INVALID_NEW_VALUE error_code: provided new_oid or new_target is
+ * invalid.
+ * REF_TRANSACTION_ERROR_EXPECTED_SYMREF error_code: expected ref to be symref, but is a
+ * regular ref.
+ */
+enum ref_transaction_error {
+ REF_TRANSACTION_ERROR_GENERIC = -1,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT = -2,
+ REF_TRANSACTION_ERROR_CREATE_EXISTS = -3,
+ REF_TRANSACTION_ERROR_NONEXISTENT_REF = -4,
+ REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE = -5,
+ REF_TRANSACTION_ERROR_INVALID_NEW_VALUE = -6,
+ REF_TRANSACTION_ERROR_EXPECTED_SYMREF = -7,
+};
+
/*
* Resolve a reference, recursively following symbolic references.
*
@@ -117,24 +140,24 @@ int refs_read_symbolic_ref(struct ref_store *ref_store, const char *refname,
*
* extras and skip must be sorted.
*/
-int refs_verify_refname_available(struct ref_store *refs,
- const char *refname,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
+enum ref_transaction_error refs_verify_refname_available(struct ref_store *refs,
+ const char *refname,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err);
/*
* Same as `refs_verify_refname_available()`, but checking for a list of
* refnames instead of only a single item. This is more efficient in the case
* where one needs to check multiple refnames.
*/
-int refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *refnames,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err);
int refs_ref_exists(struct ref_store *refs, const char *refname);
@@ -830,13 +853,6 @@ int ref_transaction_verify(struct ref_transaction *transaction,
unsigned int flags,
struct strbuf *err);
-/* Naming conflict (for example, the ref names A and A/B conflict). */
-#define TRANSACTION_NAME_CONFLICT -1
-/* When only creation was requested, but the ref already exists. */
-#define TRANSACTION_CREATE_EXISTS -2
-/* All other errors. */
-#define TRANSACTION_GENERIC_ERROR -3
-
/*
* Perform the preparatory stages of committing `transaction`. Acquire
* any needed locks, check preconditions, etc.; basically, do as much
diff --git a/refs/files-backend.c b/refs/files-backend.c
index ea023a59fc..4f27f7652c 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -663,7 +663,7 @@ static void unlock_ref(struct ref_lock *lock)
* broken, lock the reference anyway but clear old_oid.
*
* Return 0 on success. On failure, write an error message to err and
- * return TRANSACTION_NAME_CONFLICT or TRANSACTION_GENERIC_ERROR.
+ * return REF_TRANSACTION_ERROR_NAME_CONFLICT or REF_TRANSACTION_ERROR_GENERIC.
*
* Implementation note: This function is basically
*
@@ -676,19 +676,20 @@ static void unlock_ref(struct ref_lock *lock)
* avoided, namely if we were successfully able to read the ref
* - Generate informative error messages in the case of failure
*/
-static int lock_raw_ref(struct files_ref_store *refs,
- const char *refname, int mustexist,
- struct string_list *refnames_to_check,
- const struct string_list *extras,
- struct ref_lock **lock_p,
- struct strbuf *referent,
- unsigned int *type,
- struct strbuf *err)
-{
+static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
+ const char *refname,
+ int mustexist,
+ struct string_list *refnames_to_check,
+ const struct string_list *extras,
+ struct ref_lock **lock_p,
+ struct strbuf *referent,
+ unsigned int *type,
+ struct strbuf *err)
+{
+ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
struct ref_lock *lock;
struct strbuf ref_file = STRBUF_INIT;
int attempts_remaining = 3;
- int ret = TRANSACTION_GENERIC_ERROR;
int failure_errno;
assert(err);
@@ -728,13 +729,14 @@ static int lock_raw_ref(struct files_ref_store *refs,
strbuf_reset(err);
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
} else {
/*
* The error message set by
* refs_verify_refname_available() is
* OK.
*/
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
} else {
/*
@@ -788,6 +790,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
/* Garden variety missing reference. */
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error_return;
} else {
/*
@@ -820,6 +823,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
/* Garden variety missing reference. */
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error_return;
} else if (remove_dir_recursively(&ref_file,
REMOVE_DIR_EMPTY_ONLY)) {
@@ -830,7 +834,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
* The error message set by
* verify_refname_available() is OK.
*/
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto error_return;
} else {
/*
@@ -1517,10 +1521,11 @@ static int rename_tmp_log(struct files_ref_store *refs, const char *newrefname)
return ret;
}
-static int write_ref_to_lockfile(struct files_ref_store *refs,
- struct ref_lock *lock,
- const struct object_id *oid,
- int skip_oid_verification, struct strbuf *err);
+static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs,
+ struct ref_lock *lock,
+ const struct object_id *oid,
+ int skip_oid_verification,
+ struct strbuf *err);
static int commit_ref_update(struct files_ref_store *refs,
struct ref_lock *lock,
const struct object_id *oid, const char *logmsg,
@@ -1926,10 +1931,11 @@ static int files_log_ref_write(struct files_ref_store *refs,
* Write oid into the open lockfile, then close the lockfile. On
* errors, rollback the lockfile, fill in *err and return -1.
*/
-static int write_ref_to_lockfile(struct files_ref_store *refs,
- struct ref_lock *lock,
- const struct object_id *oid,
- int skip_oid_verification, struct strbuf *err)
+static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs,
+ struct ref_lock *lock,
+ const struct object_id *oid,
+ int skip_oid_verification,
+ struct strbuf *err)
{
static char term = '\n';
struct object *o;
@@ -1943,7 +1949,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
"trying to write ref '%s' with nonexistent object %s",
lock->ref_name, oid_to_hex(oid));
unlock_ref(lock);
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
if (o->type != OBJ_COMMIT && is_branch(lock->ref_name)) {
strbuf_addf(
@@ -1951,7 +1957,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
"trying to write non-commit object %s to branch '%s'",
oid_to_hex(oid), lock->ref_name);
unlock_ref(lock);
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
}
fd = get_lock_file_fd(&lock->lk);
@@ -1962,7 +1968,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
strbuf_addf(err,
"couldn't write '%s'", get_lock_file_path(&lock->lk));
unlock_ref(lock);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
return 0;
}
@@ -2376,9 +2382,10 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st
* If update is a direct update of head_ref (the reference pointed to
* by HEAD), then add an extra REF_LOG_ONLY update for HEAD.
*/
-static int split_head_update(struct ref_update *update,
- struct ref_transaction *transaction,
- const char *head_ref, struct strbuf *err)
+static enum ref_transaction_error split_head_update(struct ref_update *update,
+ struct ref_transaction *transaction,
+ const char *head_ref,
+ struct strbuf *err)
{
struct ref_update *new_update;
@@ -2402,7 +2409,7 @@ static int split_head_update(struct ref_update *update,
"multiple updates for 'HEAD' (including one "
"via its referent '%s') are not allowed",
update->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
new_update = ref_transaction_add_update(
@@ -2430,10 +2437,10 @@ static int split_head_update(struct ref_update *update,
* Note that the new update will itself be subject to splitting when
* the iteration gets to it.
*/
-static int split_symref_update(struct ref_update *update,
- const char *referent,
- struct ref_transaction *transaction,
- struct strbuf *err)
+static enum ref_transaction_error split_symref_update(struct ref_update *update,
+ const char *referent,
+ struct ref_transaction *transaction,
+ struct strbuf *err)
{
struct ref_update *new_update;
unsigned int new_flags;
@@ -2450,7 +2457,7 @@ static int split_symref_update(struct ref_update *update,
"multiple updates for '%s' (including one "
"via symref '%s') are not allowed",
referent, update->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
new_flags = update->flags;
@@ -2491,11 +2498,10 @@ static int split_symref_update(struct ref_update *update,
* everything is OK, return 0; otherwise, write an error message to
* err and return -1.
*/
-static int check_old_oid(struct ref_update *update, struct object_id *oid,
- struct strbuf *err)
+static enum ref_transaction_error check_old_oid(struct ref_update *update,
+ struct object_id *oid,
+ struct strbuf *err)
{
- int ret = TRANSACTION_GENERIC_ERROR;
-
if (!(update->flags & REF_HAVE_OLD) ||
oideq(oid, &update->old_oid))
return 0;
@@ -2504,21 +2510,20 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid,
strbuf_addf(err, "cannot lock ref '%s': "
"reference already exists",
ref_update_original_update_refname(update));
- ret = TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(oid))
+ return REF_TRANSACTION_ERROR_CREATE_EXISTS;
+ } else if (is_null_oid(oid)) {
strbuf_addf(err, "cannot lock ref '%s': "
"reference is missing but expected %s",
ref_update_original_update_refname(update),
oid_to_hex(&update->old_oid));
- else
- strbuf_addf(err, "cannot lock ref '%s': "
- "is at %s but expected %s",
- ref_update_original_update_refname(update),
- oid_to_hex(oid),
- oid_to_hex(&update->old_oid));
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ }
- return ret;
+ strbuf_addf(err, "cannot lock ref '%s': is at %s but expected %s",
+ ref_update_original_update_refname(update), oid_to_hex(oid),
+ oid_to_hex(&update->old_oid));
+
+ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
}
struct files_transaction_backend_data {
@@ -2540,17 +2545,17 @@ struct files_transaction_backend_data {
* - If it is an update of head_ref, add a corresponding REF_LOG_ONLY
* update of HEAD.
*/
-static int lock_ref_for_update(struct files_ref_store *refs,
- struct ref_update *update,
- struct ref_transaction *transaction,
- const char *head_ref,
- struct string_list *refnames_to_check,
- struct strbuf *err)
+static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *refs,
+ struct ref_update *update,
+ struct ref_transaction *transaction,
+ const char *head_ref,
+ struct string_list *refnames_to_check,
+ struct strbuf *err)
{
struct strbuf referent = STRBUF_INIT;
int mustexist = ref_update_expects_existing_old_ref(update);
struct files_transaction_backend_data *backend_data;
- int ret = 0;
+ enum ref_transaction_error ret = 0;
struct ref_lock *lock;
files_assert_main_repository(refs, "lock_ref_for_update");
@@ -2602,22 +2607,17 @@ static int lock_ref_for_update(struct files_ref_store *refs,
strbuf_addf(err, "cannot lock ref '%s': "
"error reading reference",
ref_update_original_update_refname(update));
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
}
- if (update->old_target) {
- if (ref_update_check_old_target(referent.buf, update, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto out;
- }
- } else {
+ if (update->old_target)
+ ret = ref_update_check_old_target(referent.buf, update, err);
+ else
ret = check_old_oid(update, &lock->old_oid, err);
- if (ret) {
- goto out;
- }
- }
+ if (ret)
+ goto out;
} else {
/*
* Create a new update for the reference this
@@ -2644,7 +2644,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
"but is a regular ref"),
ref_update_original_update_refname(update),
update->old_target);
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_EXPECTED_SYMREF;
goto out;
} else {
ret = check_old_oid(update, &lock->old_oid, err);
@@ -2668,14 +2668,14 @@ static int lock_ref_for_update(struct files_ref_store *refs,
if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
if (create_symref_lock(lock, update->new_target, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
if (close_ref_gently(lock)) {
strbuf_addf(err, "couldn't close '%s.lock'",
update->refname);
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
@@ -2693,25 +2693,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
* The reference already has the desired
* value, so we don't need to write it.
*/
- } else if (write_ref_to_lockfile(
- refs, lock, &update->new_oid,
- update->flags & REF_SKIP_OID_VERIFICATION,
- err)) {
- char *write_err = strbuf_detach(err, NULL);
-
- /*
- * The lock was freed upon failure of
- * write_ref_to_lockfile():
- */
- update->backend_data = NULL;
- strbuf_addf(err,
- "cannot update ref '%s': %s",
- update->refname, write_err);
- free(write_err);
- ret = TRANSACTION_GENERIC_ERROR;
- goto out;
} else {
- update->flags |= REF_NEEDS_COMMIT;
+ ret = write_ref_to_lockfile(
+ refs, lock, &update->new_oid,
+ update->flags & REF_SKIP_OID_VERIFICATION,
+ err);
+ if (ret) {
+ char *write_err = strbuf_detach(err, NULL);
+
+ /*
+ * The lock was freed upon failure of
+ * write_ref_to_lockfile():
+ */
+ update->backend_data = NULL;
+ strbuf_addf(err,
+ "cannot update ref '%s': %s",
+ update->refname, write_err);
+ free(write_err);
+ goto out;
+ } else {
+ update->flags |= REF_NEEDS_COMMIT;
+ }
}
}
if (!(update->flags & REF_NEEDS_COMMIT)) {
@@ -2723,7 +2725,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
if (close_ref_gently(lock)) {
strbuf_addf(err, "couldn't close '%s.lock'",
update->refname);
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
}
@@ -2865,7 +2867,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
refs->packed_ref_store,
transaction->flags, err);
if (!packed_transaction) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
@@ -2897,13 +2899,13 @@ static int files_transaction_prepare(struct ref_store *ref_store,
*/
if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
&transaction->refnames, NULL, 0, err)) {
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
}
if (packed_transaction) {
if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
backend_data->packed_refs_locked = 1;
@@ -2934,7 +2936,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
*/
backend_data->packed_transaction = NULL;
if (ref_transaction_abort(packed_transaction, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3035,7 +3037,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
packed_transaction = ref_store_transaction_begin(refs->packed_ref_store,
transaction->flags, err);
if (!packed_transaction) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
@@ -3058,7 +3060,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (!loose_transaction) {
loose_transaction = ref_store_transaction_begin(&refs->base, 0, err);
if (!loose_transaction) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3083,19 +3085,19 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
}
if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
if (refs_verify_refnames_available(&refs->base, &refnames_to_check,
&affected_refnames, NULL, 1, err)) {
packed_refs_unlock(refs->packed_ref_store);
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
}
if (ref_transaction_commit(packed_transaction, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
packed_refs_unlock(refs->packed_ref_store);
@@ -3103,7 +3105,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (loose_transaction) {
if (ref_transaction_prepare(loose_transaction, err) ||
ref_transaction_commit(loose_transaction, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3152,7 +3154,7 @@ static int files_transaction_finish(struct ref_store *ref_store,
if (update->flags & REF_NEEDS_COMMIT ||
update->flags & REF_LOG_ONLY) {
if (parse_and_write_reflog(refs, update, lock, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3171,7 +3173,7 @@ static int files_transaction_finish(struct ref_store *ref_store,
strbuf_addf(err, "couldn't set '%s'", lock->ref_name);
unlock_ref(lock);
update->backend_data = NULL;
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3227,7 +3229,7 @@ static int files_transaction_finish(struct ref_store *ref_store,
strbuf_reset(&sb);
files_ref_path(refs, &sb, lock->ref_name);
if (unlink_or_msg(sb.buf, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 19220d2e99..d90bd815a3 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1326,10 +1326,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
* The packfile must be locked before calling this function and will
* remain locked when it is done.
*/
-static int write_with_updates(struct packed_ref_store *refs,
- struct string_list *updates,
- struct strbuf *err)
+static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,
+ struct string_list *updates,
+ struct strbuf *err)
{
+ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
struct ref_iterator *iter = NULL;
size_t i;
int ok;
@@ -1353,7 +1354,7 @@ static int write_with_updates(struct packed_ref_store *refs,
strbuf_addf(err, "unable to create file %s: %s",
sb.buf, strerror(errno));
strbuf_release(&sb);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
strbuf_release(&sb);
@@ -1409,6 +1410,7 @@ static int write_with_updates(struct packed_ref_store *refs,
strbuf_addf(err, "cannot update ref '%s': "
"reference already exists",
update->refname);
+ ret = REF_TRANSACTION_ERROR_CREATE_EXISTS;
goto error;
} else if (!oideq(&update->old_oid, iter->oid)) {
strbuf_addf(err, "cannot update ref '%s': "
@@ -1416,6 +1418,7 @@ static int write_with_updates(struct packed_ref_store *refs,
update->refname,
oid_to_hex(iter->oid),
oid_to_hex(&update->old_oid));
+ ret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
goto error;
}
}
@@ -1452,6 +1455,7 @@ static int write_with_updates(struct packed_ref_store *refs,
"reference is missing but expected %s",
update->refname,
oid_to_hex(&update->old_oid));
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error;
}
}
@@ -1509,7 +1513,7 @@ static int write_with_updates(struct packed_ref_store *refs,
strerror(errno));
strbuf_release(&sb);
delete_tempfile(&refs->tempfile);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
return 0;
@@ -1521,7 +1525,7 @@ static int write_with_updates(struct packed_ref_store *refs,
error:
ref_iterator_free(iter);
delete_tempfile(&refs->tempfile);
- return -1;
+ return ret;
}
int is_packed_transaction_needed(struct ref_store *ref_store,
@@ -1654,7 +1658,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
"ref_transaction_prepare");
struct packed_transaction_backend_data *data;
- int ret = TRANSACTION_GENERIC_ERROR;
+ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
/*
* Note that we *don't* skip transactions with zero updates,
@@ -1675,7 +1679,8 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
data->own_lock = 1;
}
- if (write_with_updates(refs, &transaction->refnames, err))
+ ret = write_with_updates(refs, &transaction->refnames, err);
+ if (ret)
goto failure;
transaction->state = REF_TRANSACTION_PREPARED;
@@ -1707,7 +1712,7 @@ static int packed_transaction_finish(struct ref_store *ref_store,
ref_store,
REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
"ref_transaction_finish");
- int ret = TRANSACTION_GENERIC_ERROR;
+ int ret = REF_TRANSACTION_ERROR_GENERIC;
char *packed_refs_path;
clear_snapshot(refs);
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 6d3770d0cc..3f1d19abd9 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -770,8 +770,9 @@ int ref_update_has_null_new_value(struct ref_update *update);
* If everything is OK, return 0; otherwise, write an error message to
* err and return -1.
*/
-int ref_update_check_old_target(const char *referent, struct ref_update *update,
- struct strbuf *err);
+enum ref_transaction_error ref_update_check_old_target(const char *referent,
+ struct ref_update *update,
+ struct strbuf *err);
/*
* Check if the ref must exist, this means that the old_oid or
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 786df11a03..bd6b042103 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1069,20 +1069,20 @@ static int queue_transaction_update(struct reftable_ref_store *refs,
return 0;
}
-static int prepare_single_update(struct reftable_ref_store *refs,
- struct reftable_transaction_data *tx_data,
- struct ref_transaction *transaction,
- struct reftable_backend *be,
- struct ref_update *u,
- struct string_list *refnames_to_check,
- unsigned int head_type,
- struct strbuf *head_referent,
- struct strbuf *referent,
- struct strbuf *err)
+static enum ref_transaction_error prepare_single_update(struct reftable_ref_store *refs,
+ struct reftable_transaction_data *tx_data,
+ struct ref_transaction *transaction,
+ struct reftable_backend *be,
+ struct ref_update *u,
+ struct string_list *refnames_to_check,
+ unsigned int head_type,
+ struct strbuf *head_referent,
+ struct strbuf *referent,
+ struct strbuf *err)
{
+ enum ref_transaction_error ret = 0;
struct object_id current_oid = {0};
const char *rewritten_ref;
- int ret = 0;
/*
* There is no need to reload the respective backends here as
@@ -1093,7 +1093,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
*/
ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
if (ret)
- return ret;
+ return REF_TRANSACTION_ERROR_GENERIC;
/* Verify that the new object ID is valid. */
if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
@@ -1104,13 +1104,13 @@ static int prepare_single_update(struct reftable_ref_store *refs,
strbuf_addf(err,
_("trying to write ref '%s' with nonexistent object %s"),
u->refname, oid_to_hex(&u->new_oid));
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
oid_to_hex(&u->new_oid), u->refname);
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
}
@@ -1134,7 +1134,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
_("multiple updates for 'HEAD' (including one "
"via its referent '%s') are not allowed"),
u->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
ref_transaction_add_update(
@@ -1147,7 +1147,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
ret = reftable_backend_read_ref(be, rewritten_ref,
¤t_oid, referent, &u->type);
if (ret < 0)
- return ret;
+ return REF_TRANSACTION_ERROR_GENERIC;
if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
/*
* The reference does not exist, and we either have no
@@ -1168,7 +1168,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
ret = queue_transaction_update(refs, tx_data, u,
¤t_oid, err);
if (ret)
- return ret;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
return 0;
@@ -1180,7 +1180,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
"unable to resolve reference '%s'"),
ref_update_original_update_refname(u), u->refname);
- return -1;
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
}
if (u->type & REF_ISSYMREF) {
@@ -1196,7 +1196,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
if (u->flags & REF_HAVE_OLD && !resolved) {
strbuf_addf(err, _("cannot lock ref '%s': "
"error reading reference"), u->refname);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
} else {
struct ref_update *new_update;
@@ -1211,7 +1211,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
_("multiple updates for '%s' (including one "
"via symref '%s') are not allowed"),
referent->buf, u->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
/*
@@ -1255,31 +1255,32 @@ static int prepare_single_update(struct reftable_ref_store *refs,
"but is a regular ref"),
ref_update_original_update_refname(u),
u->old_target);
- return -1;
+ return REF_TRANSACTION_ERROR_EXPECTED_SYMREF;
}
- if (ref_update_check_old_target(referent->buf, u, err)) {
- return -1;
- }
+ ret = ref_update_check_old_target(referent->buf, u, err);
+ if (ret)
+ return ret;
} else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
if (is_null_oid(&u->old_oid)) {
strbuf_addf(err, _("cannot lock ref '%s': "
"reference already exists"),
ref_update_original_update_refname(u));
- return TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(¤t_oid))
+ return REF_TRANSACTION_ERROR_CREATE_EXISTS;
+ } else if (is_null_oid(¤t_oid)) {
strbuf_addf(err, _("cannot lock ref '%s': "
"reference is missing but expected %s"),
ref_update_original_update_refname(u),
oid_to_hex(&u->old_oid));
- else
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ } else {
strbuf_addf(err, _("cannot lock ref '%s': "
"is at %s but expected %s"),
ref_update_original_update_refname(u),
oid_to_hex(¤t_oid),
oid_to_hex(&u->old_oid));
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
+ }
}
/*
@@ -1296,8 +1297,8 @@ static int prepare_single_update(struct reftable_ref_store *refs,
if ((u->type & REF_ISSYMREF) ||
(u->flags & REF_LOG_ONLY) ||
(u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))
- return queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
+ if (queue_transaction_update(refs, tx_data, u, ¤t_oid, err))
+ return REF_TRANSACTION_ERROR_GENERIC;
return 0;
}
@@ -1385,7 +1386,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->state = REF_TRANSACTION_PREPARED;
done:
- assert(ret != REFTABLE_API_ERROR);
if (ret < 0) {
free_transaction_data(tx_data);
transaction->state = REF_TRANSACTION_CLOSED;
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v4 6/8] refs: implement batch reference update support
2025-03-20 11:43 ` [PATCH v4 0/8] refs: introduce support for batched reference updates Karthik Nayak
` (4 preceding siblings ...)
2025-03-20 11:44 ` [PATCH v4 5/8] refs: introduce enum-based transaction error types Karthik Nayak
@ 2025-03-20 11:44 ` Karthik Nayak
2025-03-20 20:26 ` Patrick Steinhardt
2025-03-20 11:44 ` [PATCH v4 7/8] refs: support rejection in batch updates during F/D checks Karthik Nayak
2025-03-20 11:44 ` [PATCH v4 8/8] update-ref: add --batch-updates flag for stdin mode Karthik Nayak
7 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-03-20 11:44 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
Git supports making reference updates with or without transactions.
Updates with transactions are generally better optimized. But
transactions are all or nothing. This means, if a user wants to batch
updates to take advantage of the optimizations without the hard
requirement that all updates must succeed, there is no way currently to
do so. Particularly with the reftable backend where batching multiple
reference updates is more efficient than performing them sequentially.
Introduce batched update support with a new flag,
'REF_TRANSACTION_ALLOW_FAILURE'. Batched updates while different from
transactions, use the transaction infrastructure under the hood. When
enabled, this flag allows individual reference updates that would
typically cause the entire transaction to fail due to non-system-related
errors to be marked as rejected while permitting other updates to
proceed. System errors referred by 'REF_TRANSACTION_ERROR_GENERIC'
continue to result in the entire transaction failing. This approach
enhances flexibility while preserving transactional integrity where
necessary.
The implementation introduces several key components:
- Add 'rejection_err' field to struct `ref_update` to track failed
updates with failure reason.
- Add a new struct `ref_transaction_rejections` and a field within
`ref_transaction` to this struct to allow quick iteration over
rejected updates.
- Modify reference backends (files, packed, reftable) to handle
partial transactions by using `ref_transaction_set_rejected()`
instead of failing the entire transaction when
`REF_TRANSACTION_ALLOW_FAILURE` is set.
- Add `ref_transaction_for_each_rejected_update()` to let callers
examine which updates were rejected and why.
This foundational change enables batched update support throughout the
reference subsystem. A following commit will expose this capability to
users by adding a `--batch-updates` flag to 'git-update-ref(1)',
providing both a user-facing feature and a testable implementation.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++
refs.h | 22 ++++++++++++++++++
refs/files-backend.c | 12 +++++++++-
refs/packed-backend.c | 27 ++++++++++++++++++++--
refs/refs-internal.h | 26 +++++++++++++++++++++
refs/reftable-backend.c | 12 +++++++++-
6 files changed, 156 insertions(+), 4 deletions(-)
diff --git a/refs.c b/refs.c
index 3d0b53d56e..b34ec198f5 100644
--- a/refs.c
+++ b/refs.c
@@ -1176,6 +1176,10 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,
tr->ref_store = refs;
tr->flags = flags;
string_list_init_dup(&tr->refnames);
+
+ if (flags & REF_TRANSACTION_ALLOW_FAILURE)
+ CALLOC_ARRAY(tr->rejections, 1);
+
return tr;
}
@@ -1206,11 +1210,45 @@ void ref_transaction_free(struct ref_transaction *transaction)
free((char *)transaction->updates[i]->old_target);
free(transaction->updates[i]);
}
+
+ if (transaction->rejections)
+ free(transaction->rejections->update_indices);
+ free(transaction->rejections);
+
string_list_clear(&transaction->refnames, 0);
free(transaction->updates);
free(transaction);
}
+int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,
+ size_t update_idx,
+ enum ref_transaction_error err)
+{
+ if (update_idx >= transaction->nr)
+ BUG("trying to set rejection on invalid update index");
+
+ if (!(transaction->flags & REF_TRANSACTION_ALLOW_FAILURE))
+ return 0;
+
+ if (!transaction->rejections)
+ BUG("transaction not inititalized with failure support");
+
+ /*
+ * Don't accept generic errors, since these errors are not user
+ * input related.
+ */
+ if (err == REF_TRANSACTION_ERROR_GENERIC)
+ return 0;
+
+ transaction->updates[update_idx]->rejection_err = err;
+ ALLOC_GROW(transaction->rejections->update_indices,
+ transaction->rejections->nr + 1,
+ transaction->rejections->alloc);
+ transaction->rejections->update_indices[transaction->rejections->nr++] = update_idx;
+
+ return 1;
+}
+
struct ref_update *ref_transaction_add_update(
struct ref_transaction *transaction,
const char *refname, unsigned int flags,
@@ -1236,6 +1274,7 @@ struct ref_update *ref_transaction_add_update(
transaction->updates[transaction->nr++] = update;
update->flags = flags;
+ update->rejection_err = 0;
update->new_target = xstrdup_or_null(new_target);
update->old_target = xstrdup_or_null(old_target);
@@ -2728,6 +2767,28 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
}
}
+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
+ ref_transaction_for_each_rejected_update_fn cb,
+ void *cb_data)
+{
+ if (!transaction->rejections)
+ return;
+
+ for (size_t i = 0; i < transaction->rejections->nr; i++) {
+ size_t update_index = transaction->rejections->update_indices[i];
+ struct ref_update *update = transaction->updates[update_index];
+
+ if (!update->rejection_err)
+ continue;
+
+ cb(update->refname,
+ (update->flags & REF_HAVE_OLD) ? &update->old_oid : NULL,
+ (update->flags & REF_HAVE_NEW) ? &update->new_oid : NULL,
+ update->old_target, update->new_target,
+ update->rejection_err, cb_data);
+ }
+}
+
int refs_delete_refs(struct ref_store *refs, const char *logmsg,
struct string_list *refnames, unsigned int flags)
{
diff --git a/refs.h b/refs.h
index dcd83e81e2..591b703d59 100644
--- a/refs.h
+++ b/refs.h
@@ -673,6 +673,13 @@ enum ref_transaction_flag {
* either be absent or null_oid.
*/
REF_TRANSACTION_FLAG_INITIAL = (1 << 0),
+
+ /*
+ * The transaction mechanism by default fails all updates if any conflict
+ * is detected. This flag allows transactions to partially apply updates
+ * while rejecting updates which do not match the expected state.
+ */
+ REF_TRANSACTION_ALLOW_FAILURE = (1 << 1),
};
/*
@@ -903,6 +910,21 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
ref_transaction_for_each_queued_update_fn cb,
void *cb_data);
+/*
+ * Execute the given callback function for each of the reference updates which
+ * have been rejected in the given transaction.
+ */
+typedef void ref_transaction_for_each_rejected_update_fn(const char *refname,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
+ enum ref_transaction_error err,
+ void *cb_data);
+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
+ ref_transaction_for_each_rejected_update_fn cb,
+ void *cb_data);
+
/*
* Free `*transaction` and all associated data.
*/
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 4f27f7652c..be758ffff5 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2852,8 +2852,15 @@ static int files_transaction_prepare(struct ref_store *ref_store,
ret = lock_ref_for_update(refs, update, transaction,
head_ref, &refnames_to_check,
err);
- if (ret)
+ if (ret) {
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
+ ret = 0;
+
+ continue;
+ }
goto cleanup;
+ }
if (update->flags & REF_DELETING &&
!(update->flags & REF_LOG_ONLY) &&
@@ -3151,6 +3158,9 @@ static int files_transaction_finish(struct ref_store *ref_store,
struct ref_update *update = transaction->updates[i];
struct ref_lock *lock = update->backend_data;
+ if (update->rejection_err)
+ continue;
+
if (update->flags & REF_NEEDS_COMMIT ||
update->flags & REF_LOG_ONLY) {
if (parse_and_write_reflog(refs, update, lock, err)) {
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index d90bd815a3..7bf57ca948 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1327,10 +1327,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
* remain locked when it is done.
*/
static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,
- struct string_list *updates,
+ struct ref_transaction *transaction,
struct strbuf *err)
{
enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
+ struct string_list *updates = &transaction->refnames;
struct ref_iterator *iter = NULL;
size_t i;
int ok;
@@ -1411,6 +1412,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
"reference already exists",
update->refname);
ret = REF_TRANSACTION_ERROR_CREATE_EXISTS;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
+ ret = 0;
+ continue;
+ }
+
goto error;
} else if (!oideq(&update->old_oid, iter->oid)) {
strbuf_addf(err, "cannot update ref '%s': "
@@ -1419,6 +1427,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
oid_to_hex(iter->oid),
oid_to_hex(&update->old_oid));
ret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
+ ret = 0;
+ continue;
+ }
+
goto error;
}
}
@@ -1456,6 +1471,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
update->refname,
oid_to_hex(&update->old_oid));
ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
+ ret = 0;
+ continue;
+ }
+
goto error;
}
}
@@ -1521,6 +1543,7 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
write_error:
strbuf_addf(err, "error writing to %s: %s",
get_tempfile_path(refs->tempfile), strerror(errno));
+ ret = REF_TRANSACTION_ERROR_GENERIC;
error:
ref_iterator_free(iter);
@@ -1679,7 +1702,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
data->own_lock = 1;
}
- ret = write_with_updates(refs, &transaction->refnames, err);
+ ret = write_with_updates(refs, transaction, err);
if (ret)
goto failure;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 3f1d19abd9..73a5379b73 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -123,6 +123,12 @@ struct ref_update {
*/
uint64_t index;
+ /*
+ * Used in batched reference updates to mark if a given update
+ * was rejected.
+ */
+ enum ref_transaction_error rejection_err;
+
/*
* If this ref_update was split off of a symref update via
* split_symref_update(), then this member points at that
@@ -142,6 +148,13 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
struct object_id *oid, struct strbuf *referent,
unsigned int *type, int *failure_errno);
+/*
+ * Mark a given update as rejected with a given reason.
+ */
+int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,
+ size_t update_idx,
+ enum ref_transaction_error err);
+
/*
* Add a ref_update with the specified properties to transaction, and
* return a pointer to the new object. This function does not verify
@@ -183,6 +196,18 @@ enum ref_transaction_state {
REF_TRANSACTION_CLOSED = 2
};
+/*
+ * Data structure to hold indices of updates which were rejected, for batched
+ * reference updates. While the updates themselves hold the rejection error,
+ * this structure allows a transaction to iterate only over the rejected
+ * updates.
+ */
+struct ref_transaction_rejections {
+ size_t *update_indices;
+ size_t alloc;
+ size_t nr;
+};
+
/*
* Data structure for holding a reference transaction, which can
* consist of checks and updates to multiple references, carried out
@@ -195,6 +220,7 @@ struct ref_transaction {
size_t alloc;
size_t nr;
enum ref_transaction_state state;
+ struct ref_transaction_rejections *rejections;
void *backend_data;
unsigned int flags;
uint64_t max_index;
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index bd6b042103..ed65225763 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1371,8 +1371,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->updates[i],
&refnames_to_check, head_type,
&head_referent, &referent, err);
- if (ret)
+ if (ret) {
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_setlen(err, 0);
+ ret = 0;
+
+ continue;
+ }
goto done;
+ }
}
ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
@@ -1454,6 +1461,9 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
struct reftable_transaction_update *tx_update = &arg->updates[i];
struct ref_update *u = tx_update->update;
+ if (u->rejection_err)
+ continue;
+
/*
* Write a reflog entry when updating a ref to point to
* something new in either of the following cases:
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v4 7/8] refs: support rejection in batch updates during F/D checks
2025-03-20 11:43 ` [PATCH v4 0/8] refs: introduce support for batched reference updates Karthik Nayak
` (5 preceding siblings ...)
2025-03-20 11:44 ` [PATCH v4 6/8] refs: implement batch reference update support Karthik Nayak
@ 2025-03-20 11:44 ` Karthik Nayak
2025-03-24 13:08 ` Patrick Steinhardt
2025-03-20 11:44 ` [PATCH v4 8/8] update-ref: add --batch-updates flag for stdin mode Karthik Nayak
7 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-03-20 11:44 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
The `refs_verify_refnames_available()` is used to batch check refnames
for F/D conflicts. While this is the more performant alternative than
its individual version, it does not provide rejection capabilities on a
single update level. For batched updates, this would mean a rejection of
the entire transaction whenever one reference has a F/D conflict.
Modify the function to call `ref_transaction_maybe_set_rejected()` to
check if a single update can be rejected. Since this function is only
internally used within 'refs/' and we want to pass in a `struct
ref_transaction *` as a variable. We also move and mark
`refs_verify_refnames_available()` to 'refs-internal.h' to be an
internal function.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 28 +++++++++++++++++++++++++++-
refs.h | 12 ------------
refs/files-backend.c | 27 ++++++++++++++++++---------
refs/refs-internal.h | 16 ++++++++++++++++
refs/reftable-backend.c | 11 ++++++++---
5 files changed, 69 insertions(+), 25 deletions(-)
diff --git a/refs.c b/refs.c
index b34ec198f5..f719046f47 100644
--- a/refs.c
+++ b/refs.c
@@ -2540,6 +2540,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
const struct string_list *refnames,
const struct string_list *extras,
const struct string_list *skip,
+ struct ref_transaction *transaction,
unsigned int initial_transaction,
struct strbuf *err)
{
@@ -2560,6 +2561,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
strset_init(&dirnames);
for_each_string_list_item(item, refnames) {
+ const size_t *update_idx = (size_t *)item->util;
const char *refname = item->string;
const char *extra_refname;
struct object_id oid;
@@ -2599,12 +2601,26 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
if (!initial_transaction &&
!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
&type, &ignore_errno)) {
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT)) {
+ strset_remove(&dirnames, dirname.buf);
+ continue;
+ }
+
strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
dirname.buf, refname);
goto cleanup;
}
if (extras && string_list_has_string(extras, dirname.buf)) {
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT)) {
+ strset_remove(&dirnames, dirname.buf);
+ continue;
+ }
+
strbuf_addf(err, _("cannot process '%s' and '%s' at the same time"),
refname, dirname.buf);
goto cleanup;
@@ -2637,6 +2653,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
string_list_has_string(skip, iter->refname))
continue;
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT))
+ continue;
+
strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
iter->refname, refname);
goto cleanup;
@@ -2648,6 +2669,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
extra_refname = find_descendant_ref(dirname.buf, extras, skip);
if (extra_refname) {
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT))
+ continue;
+
strbuf_addf(err, _("cannot process '%s' and '%s' at the same time"),
refname, extra_refname);
goto cleanup;
@@ -2679,7 +2705,7 @@ enum ref_transaction_error refs_verify_refname_available(
};
return refs_verify_refnames_available(refs, &refnames, extras, skip,
- initial_transaction, err);
+ NULL, initial_transaction, err);
}
struct do_for_each_reflog_help {
diff --git a/refs.h b/refs.h
index 591b703d59..1e7ab2532f 100644
--- a/refs.h
+++ b/refs.h
@@ -147,18 +147,6 @@ enum ref_transaction_error refs_verify_refname_available(struct ref_store *refs,
unsigned int initial_transaction,
struct strbuf *err);
-/*
- * Same as `refs_verify_refname_available()`, but checking for a list of
- * refnames instead of only a single item. This is more efficient in the case
- * where one needs to check multiple refnames.
- */
-enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *refnames,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
-
int refs_ref_exists(struct ref_store *refs, const char *refname);
int should_autocreate_reflog(enum log_refs_config log_all_ref_updates,
diff --git a/refs/files-backend.c b/refs/files-backend.c
index be758ffff5..1d50d4013c 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -677,16 +677,18 @@ static void unlock_ref(struct ref_lock *lock)
* - Generate informative error messages in the case of failure
*/
static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
- const char *refname,
+ struct ref_update *update,
+ size_t update_idx,
int mustexist,
struct string_list *refnames_to_check,
const struct string_list *extras,
struct ref_lock **lock_p,
struct strbuf *referent,
- unsigned int *type,
struct strbuf *err)
{
enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
+ const char *refname = update->refname;
+ unsigned int *type = &update->type;
struct ref_lock *lock;
struct strbuf ref_file = STRBUF_INIT;
int attempts_remaining = 3;
@@ -785,6 +787,8 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
type, &failure_errno)) {
+ struct string_list_item *item;
+
if (failure_errno == ENOENT) {
if (mustexist) {
/* Garden variety missing reference. */
@@ -864,7 +868,9 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
* make sure there is no existing packed ref that conflicts
* with refname. This check is deferred so that we can batch it.
*/
- string_list_append(refnames_to_check, refname);
+ item = string_list_append(refnames_to_check, refname);
+ item->util = xmalloc(sizeof(update_idx));
+ memcpy(item->util, &update_idx, sizeof(update_idx));
}
ret = 0;
@@ -2547,6 +2553,7 @@ struct files_transaction_backend_data {
*/
static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *refs,
struct ref_update *update,
+ size_t update_idx,
struct ref_transaction *transaction,
const char *head_ref,
struct string_list *refnames_to_check,
@@ -2575,9 +2582,9 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re
if (lock) {
lock->count++;
} else {
- ret = lock_raw_ref(refs, update->refname, mustexist,
+ ret = lock_raw_ref(refs, update, update_idx, mustexist,
refnames_to_check, &transaction->refnames,
- &lock, &referent, &update->type, err);
+ &lock, &referent, err);
if (ret) {
char *reason;
@@ -2849,7 +2856,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
- ret = lock_ref_for_update(refs, update, transaction,
+ ret = lock_ref_for_update(refs, update, i, transaction,
head_ref, &refnames_to_check,
err);
if (ret) {
@@ -2905,7 +2912,8 @@ static int files_transaction_prepare(struct ref_store *ref_store,
* So instead, we accept the race for now.
*/
if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
- &transaction->refnames, NULL, 0, err)) {
+ &transaction->refnames, NULL, transaction,
+ 0, err)) {
ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
}
@@ -2951,7 +2959,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
cleanup:
free(head_ref);
- string_list_clear(&refnames_to_check, 0);
+ string_list_clear(&refnames_to_check, 1);
if (ret)
files_transaction_cleanup(refs, transaction);
@@ -3097,7 +3105,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
}
if (refs_verify_refnames_available(&refs->base, &refnames_to_check,
- &affected_refnames, NULL, 1, err)) {
+ &affected_refnames, NULL, transaction,
+ 1, err)) {
packed_refs_unlock(refs->packed_ref_store);
ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 73a5379b73..f868870851 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -806,4 +806,20 @@ enum ref_transaction_error ref_update_check_old_target(const char *referent,
*/
int ref_update_expects_existing_old_ref(struct ref_update *update);
+/*
+ * Same as `refs_verify_refname_available()`, but checking for a list of
+ * refnames instead of only a single item. This is more efficient in the case
+ * where one needs to check multiple refnames.
+ *
+ * If using batched updates, then individual updates are marked rejected,
+ * reference backends are then in charge of not committing those updates.
+ */
+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ struct ref_transaction *transaction,
+ unsigned int initial_transaction,
+ struct strbuf *err);
+
#endif /* REFS_REFS_INTERNAL_H */
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index ed65225763..f5429c918b 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1074,6 +1074,7 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor
struct ref_transaction *transaction,
struct reftable_backend *be,
struct ref_update *u,
+ size_t update_idx,
struct string_list *refnames_to_check,
unsigned int head_type,
struct strbuf *head_referent,
@@ -1149,6 +1150,7 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor
if (ret < 0)
return REF_TRANSACTION_ERROR_GENERIC;
if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
+ struct string_list_item *item;
/*
* The reference does not exist, and we either have no
* old object ID or expect the reference to not exist.
@@ -1158,7 +1160,9 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor
* can output a proper error message instead of failing
* at a later point.
*/
- string_list_append(refnames_to_check, u->refname);
+ item = string_list_append(refnames_to_check, u->refname);
+ item->util = xmalloc(sizeof(update_idx));
+ memcpy(item->util, &update_idx, sizeof(update_idx));
/*
* There is no need to write the reference deletion
@@ -1368,7 +1372,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
for (i = 0; i < transaction->nr; i++) {
ret = prepare_single_update(refs, tx_data, transaction, be,
- transaction->updates[i],
+ transaction->updates[i], i,
&refnames_to_check, head_type,
&head_referent, &referent, err);
if (ret) {
@@ -1384,6 +1388,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
&transaction->refnames, NULL,
+ transaction,
transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
err);
if (ret < 0)
@@ -1402,7 +1407,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
}
strbuf_release(&referent);
strbuf_release(&head_referent);
- string_list_clear(&refnames_to_check, 0);
+ string_list_clear(&refnames_to_check, 1);
return ret;
}
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v4 8/8] update-ref: add --batch-updates flag for stdin mode
2025-03-20 11:43 ` [PATCH v4 0/8] refs: introduce support for batched reference updates Karthik Nayak
` (6 preceding siblings ...)
2025-03-20 11:44 ` [PATCH v4 7/8] refs: support rejection in batch updates during F/D checks Karthik Nayak
@ 2025-03-20 11:44 ` Karthik Nayak
2025-03-24 13:08 ` Patrick Steinhardt
7 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-03-20 11:44 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak, ps, jltobler, phillip.wood123
When updating multiple references through stdin, Git's update-ref
command normally aborts the entire transaction if any single update
fails. This atomic behavior prevents partial updates. Introduce a new
batch update system, where the updates the performed together similar
but individual updates are allowed to fail.
Add a new `--batch-updates` flag that allows the transaction to continue
even when individual reference updates fail. This flag can only be used
in `--stdin` mode and builds upon the batch update support added to the
refs subsystem in the previous commits. When enabled, failed updates are
reported in the following format:
rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
Update the documentation to reflect this change and also tests to cover
different scenarios where an update could be rejected.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
Documentation/git-update-ref.adoc | 14 ++-
builtin/update-ref.c | 67 ++++++++++-
t/t1400-update-ref.sh | 233 ++++++++++++++++++++++++++++++++++++++
3 files changed, 306 insertions(+), 8 deletions(-)
diff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc
index 9e6935d38d..5be2c16776 100644
--- a/Documentation/git-update-ref.adoc
+++ b/Documentation/git-update-ref.adoc
@@ -7,8 +7,10 @@ git-update-ref - Update the object name stored in a ref safely
SYNOPSIS
--------
-[verse]
-'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z])
+[synopsis]
+git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
+ [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
+ [-m <reason>] [--no-deref] --stdin [-z] [--batch-updates]
DESCRIPTION
-----------
@@ -57,6 +59,14 @@ performs all modifications together. Specify commands of the form:
With `--create-reflog`, update-ref will create a reflog for each ref
even if one would not ordinarily be created.
+With `--batch-updates`, update-ref executes the updates in a batch but allows
+individual updates to fail due to invalid or incorrect user input, applying only
+the successful updates. However, system-related errors—such as I/O failures or
+memory issues—will result in a full failure of all batched updates. Any failed
+updates will be reported in the following format:
+
+ rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
+
Quote fields containing whitespace as if they were strings in C source
code; i.e., surrounded by double-quotes and with backslash escapes.
Use 40 "0" characters or the empty string to specify a zero value. To
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 1d541e13ad..97e14b279e 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -5,6 +5,7 @@
#include "config.h"
#include "gettext.h"
#include "hash.h"
+#include "hex.h"
#include "refs.h"
#include "object-name.h"
#include "parse-options.h"
@@ -13,7 +14,7 @@
static const char * const git_update_ref_usage[] = {
N_("git update-ref [<options>] -d <refname> [<old-oid>]"),
N_("git update-ref [<options>] <refname> <new-oid> [<old-oid>]"),
- N_("git update-ref [<options>] --stdin [-z]"),
+ N_("git update-ref [<options>] --stdin [-z] [--batch-updates]"),
NULL
};
@@ -565,6 +566,49 @@ static void parse_cmd_abort(struct ref_transaction *transaction,
report_ok("abort");
}
+static void print_rejected_refs(const char *refname,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
+ enum ref_transaction_error err,
+ void *cb_data UNUSED)
+{
+ struct strbuf sb = STRBUF_INIT;
+ const char *reason = "";
+
+ switch (err) {
+ case REF_TRANSACTION_ERROR_NAME_CONFLICT:
+ reason = "refname conflict";
+ break;
+ case REF_TRANSACTION_ERROR_CREATE_EXISTS:
+ reason = "reference already exists";
+ break;
+ case REF_TRANSACTION_ERROR_NONEXISTENT_REF:
+ reason = "reference does not exist";
+ break;
+ case REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE:
+ reason = "incorrect old value provided";
+ break;
+ case REF_TRANSACTION_ERROR_INVALID_NEW_VALUE:
+ reason = "invalid new value provided";
+ break;
+ case REF_TRANSACTION_ERROR_EXPECTED_SYMREF:
+ reason = "expected symref but found regular ref";
+ break;
+ default:
+ reason = "unkown failure";
+ }
+
+ strbuf_addf(&sb, "rejected %s %s %s %s\n", refname,
+ new_oid ? oid_to_hex(new_oid) : new_target,
+ old_oid ? oid_to_hex(old_oid) : old_target,
+ reason);
+
+ fwrite(sb.buf, sb.len, 1, stdout);
+ strbuf_release(&sb);
+}
+
static void parse_cmd_commit(struct ref_transaction *transaction,
const char *next, const char *end UNUSED)
{
@@ -573,6 +617,10 @@ static void parse_cmd_commit(struct ref_transaction *transaction,
die("commit: extra input: %s", next);
if (ref_transaction_commit(transaction, &error))
die("commit: %s", error.buf);
+
+ ref_transaction_for_each_rejected_update(transaction,
+ print_rejected_refs, NULL);
+
report_ok("commit");
ref_transaction_free(transaction);
}
@@ -609,7 +657,7 @@ static const struct parse_cmd {
{ "commit", parse_cmd_commit, 0, UPDATE_REFS_CLOSED },
};
-static void update_refs_stdin(void)
+static void update_refs_stdin(unsigned int flags)
{
struct strbuf input = STRBUF_INIT, err = STRBUF_INIT;
enum update_refs_state state = UPDATE_REFS_OPEN;
@@ -617,7 +665,7 @@ static void update_refs_stdin(void)
int i, j;
transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
- 0, &err);
+ flags, &err);
if (!transaction)
die("%s", err.buf);
@@ -685,7 +733,7 @@ static void update_refs_stdin(void)
*/
state = cmd->state;
transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
- 0, &err);
+ flags, &err);
if (!transaction)
die("%s", err.buf);
@@ -701,6 +749,8 @@ static void update_refs_stdin(void)
/* Commit by default if no transaction was requested. */
if (ref_transaction_commit(transaction, &err))
die("%s", err.buf);
+ ref_transaction_for_each_rejected_update(transaction,
+ print_rejected_refs, NULL);
ref_transaction_free(transaction);
break;
case UPDATE_REFS_STARTED:
@@ -727,6 +777,8 @@ int cmd_update_ref(int argc,
struct object_id oid, oldoid;
int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;
int create_reflog = 0;
+ unsigned int flags = 0;
+
struct option options[] = {
OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
OPT_BOOL('d', NULL, &delete, N_("delete the reference")),
@@ -735,6 +787,8 @@ int cmd_update_ref(int argc,
OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
OPT_BOOL( 0 , "create-reflog", &create_reflog, N_("create a reflog")),
+ OPT_BIT('0', "batch-updates", &flags, N_("batch reference updates"),
+ REF_TRANSACTION_ALLOW_FAILURE),
OPT_END(),
};
@@ -756,9 +810,10 @@ int cmd_update_ref(int argc,
usage_with_options(git_update_ref_usage, options);
if (end_null)
line_termination = '\0';
- update_refs_stdin();
+ update_refs_stdin(flags);
return 0;
- }
+ } else if (flags & REF_TRANSACTION_ALLOW_FAILURE)
+ die("--batch-updates can only be used with --stdin");
if (end_null)
usage_with_options(git_update_ref_usage, options);
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 29045aad43..d29d23cb89 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -2066,6 +2066,239 @@ do
grep "$(git rev-parse $a) $(git rev-parse $a)" actual
'
+ test_expect_success "stdin $type batch-updates" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit commit &&
+ head=$(git rev-parse HEAD) &&
+
+ format_command $type "update refs/heads/ref1" "$head" "$Z" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head" "$Z" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with invalid new_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$(test_oid 001)" "$head" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "invalid new value provided" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with non-commit new_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ head_tree=$(git rev-parse HEAD^{tree}) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head_tree" "$head" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "invalid new value provided" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with non-existent ref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ test_must_fail git rev-parse refs/heads/ref2 &&
+ test_grep -q "reference does not exist" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with dangling symref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git symbolic-ref refs/heads/ref2 refs/heads/nonexistent &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
+ git update-ref $type --no-deref --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ test_must_fail git rev-parse refs/heads/ref2 &&
+ test_grep -q "reference does not exist" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with regular ref as symref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "symref-update refs/heads/ref2" "$old_head" "ref" "refs/heads/nonexistent" >>stdin &&
+ git update-ref $type --no-deref --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "expected symref but found regular ref" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with invalid old_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$Z" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "reference already exists" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with incorrect old oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head" "$old_head" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "incorrect old value provided" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates refname conflict" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref/foo $head &&
+
+ format_command $type "update refs/heads/ref/foo" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref/foo >actual &&
+ test_cmp expect actual &&
+ test_grep -q "refname conflict" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates refname conflict new ref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref/foo $head &&
+
+ format_command $type "update refs/heads/foo" "$old_head" "" >stdin &&
+ format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/foo >actual &&
+ test_cmp expect actual &&
+ test_grep -q "refname conflict" stdout
+ )
+ '
done
test_expect_success 'update-ref should also create reflog for HEAD' '
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH v4 5/8] refs: introduce enum-based transaction error types
2025-03-20 11:44 ` [PATCH v4 5/8] refs: introduce enum-based transaction error types Karthik Nayak
@ 2025-03-20 20:26 ` Patrick Steinhardt
2025-03-24 14:50 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Patrick Steinhardt @ 2025-03-20 20:26 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler, phillip.wood123
On Thu, Mar 20, 2025 at 12:44:00PM +0100, Karthik Nayak wrote:
> diff --git a/refs.h b/refs.h
> index 240e2d8537..dcd83e81e2 100644
> --- a/refs.h
> +++ b/refs.h
> @@ -16,6 +16,29 @@ struct worktree;
> enum ref_storage_format ref_storage_format_by_name(const char *name);
> const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);
>
> +/*
> + * enum ref_transaction_error represents the following return codes:
> + * REF_TRANSACTION_ERROR_GENERIC error_code: default error code.
> + * REF_TRANSACTION_ERROR_NAME_CONFLICT error_code: ref name conflict like A vs A/B.
> + * REF_TRANSACTION_ERROR_CREATE_EXISTS error_code: ref to be created already exists.
> + * REF_TRANSACTION_ERROR_NONEXISTENT_REF error_code: ref expected but doesn't exist.
> + * REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE error_code: provided old_oid or old_target of
> + * reference doesn't match actual.
> + * REF_TRANSACTION_ERROR_INVALID_NEW_VALUE error_code: provided new_oid or new_target is
> + * invalid.
> + * REF_TRANSACTION_ERROR_EXPECTED_SYMREF error_code: expected ref to be symref, but is a
> + * regular ref.
> + */
> +enum ref_transaction_error {
> + REF_TRANSACTION_ERROR_GENERIC = -1,
> + REF_TRANSACTION_ERROR_NAME_CONFLICT = -2,
> + REF_TRANSACTION_ERROR_CREATE_EXISTS = -3,
> + REF_TRANSACTION_ERROR_NONEXISTENT_REF = -4,
> + REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE = -5,
> + REF_TRANSACTION_ERROR_INVALID_NEW_VALUE = -6,
> + REF_TRANSACTION_ERROR_EXPECTED_SYMREF = -7,
> +};
> +
Tiny nit: I think it's generally preferable to document each specific
right next to its definition.
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v4 6/8] refs: implement batch reference update support
2025-03-20 11:44 ` [PATCH v4 6/8] refs: implement batch reference update support Karthik Nayak
@ 2025-03-20 20:26 ` Patrick Steinhardt
2025-03-24 14:54 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Patrick Steinhardt @ 2025-03-20 20:26 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler, phillip.wood123
On Thu, Mar 20, 2025 at 12:44:01PM +0100, Karthik Nayak wrote:
> diff --git a/refs.c b/refs.c
> index 3d0b53d56e..b34ec198f5 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1206,11 +1210,45 @@ void ref_transaction_free(struct ref_transaction *transaction)
> free((char *)transaction->updates[i]->old_target);
> free(transaction->updates[i]);
> }
> +
> + if (transaction->rejections)
> + free(transaction->rejections->update_indices);
> + free(transaction->rejections);
> +
> string_list_clear(&transaction->refnames, 0);
> free(transaction->updates);
> free(transaction);
> }
>
> +int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,
> + size_t update_idx,
> + enum ref_transaction_error err)
> +{
> + if (update_idx >= transaction->nr)
> + BUG("trying to set rejection on invalid update index");
> +
> + if (!(transaction->flags & REF_TRANSACTION_ALLOW_FAILURE))
> + return 0;
> +
> + if (!transaction->rejections)
> + BUG("transaction not inititalized with failure support");
> +
> + /*
> + * Don't accept generic errors, since these errors are not user
> + * input related.
> + */
> + if (err == REF_TRANSACTION_ERROR_GENERIC)
> + return 0;
> +
> + transaction->updates[update_idx]->rejection_err = err;
> + ALLOC_GROW(transaction->rejections->update_indices,
> + transaction->rejections->nr + 1,
> + transaction->rejections->alloc);
> + transaction->rejections->update_indices[transaction->rejections->nr++] = update_idx;
> +
> + return 1;
> +}
If we had a `struct ref_update_rejection` we could store the update
index and rejection errors in the same location, which might be a bit
easier to reason about.
> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
> index d90bd815a3..7bf57ca948 100644
> --- a/refs/packed-backend.c
> +++ b/refs/packed-backend.c
> @@ -1327,10 +1327,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
> * remain locked when it is done.
> */
> static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,
> - struct string_list *updates,
> + struct ref_transaction *transaction,
> struct strbuf *err)
> {
> enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
> + struct string_list *updates = &transaction->refnames;
> struct ref_iterator *iter = NULL;
> size_t i;
> int ok;
> @@ -1411,6 +1412,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
> "reference already exists",
> update->refname);
> ret = REF_TRANSACTION_ERROR_CREATE_EXISTS;
> +
> + if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
> + strbuf_setlen(err, 0);
Nit: you can use `strbuf_reset()` for this and other instances.
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v4 7/8] refs: support rejection in batch updates during F/D checks
2025-03-20 11:44 ` [PATCH v4 7/8] refs: support rejection in batch updates during F/D checks Karthik Nayak
@ 2025-03-24 13:08 ` Patrick Steinhardt
2025-03-24 17:48 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Patrick Steinhardt @ 2025-03-24 13:08 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler, phillip.wood123
On Thu, Mar 20, 2025 at 12:44:02PM +0100, Karthik Nayak wrote:
> diff --git a/refs.c b/refs.c
> index b34ec198f5..f719046f47 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -2540,6 +2540,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
> const struct string_list *refnames,
> const struct string_list *extras,
> const struct string_list *skip,
> + struct ref_transaction *transaction,
> unsigned int initial_transaction,
> struct strbuf *err)
> {
> @@ -2599,12 +2601,26 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
> if (!initial_transaction &&
> !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
> &type, &ignore_errno)) {
> + if (transaction && ref_transaction_maybe_set_rejected(
> + transaction, *update_idx,
> + REF_TRANSACTION_ERROR_NAME_CONFLICT)) {
> + strset_remove(&dirnames, dirname.buf);
> + continue;
> + }
> +
Okay. We have to remove the dirname from `dirnames` again so that the
next reference that creates a reference in the same directory would also
be marked as conflicting. It does have the consequence that we now have
to read the dirname N times again, where N is the number of refs that
are created below that directory.
We could probably improve this by using another map that contains the
conflicting names, right?
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index be758ffff5..1d50d4013c 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -864,7 +868,9 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
> * make sure there is no existing packed ref that conflicts
> * with refname. This check is deferred so that we can batch it.
> */
> - string_list_append(refnames_to_check, refname);
> + item = string_list_append(refnames_to_check, refname);
> + item->util = xmalloc(sizeof(update_idx));
> + memcpy(item->util, &update_idx, sizeof(update_idx));
> }
>
> ret = 0;
Hm, so we have to allocate the `util` field now to store the update
index, which is a bit unfortunate because all of this is part of the hot
loop. We cannot store a direct pointer though because the array of
updates may be reallocated, which would invalidate any pointers pointing
into the array.
I was wondering whether we could abuse an `uintptr_t` and use it to
store the update index as a pointer. It does feel somewhat dirty though.
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v4 8/8] update-ref: add --batch-updates flag for stdin mode
2025-03-20 11:44 ` [PATCH v4 8/8] update-ref: add --batch-updates flag for stdin mode Karthik Nayak
@ 2025-03-24 13:08 ` Patrick Steinhardt
2025-03-24 17:51 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Patrick Steinhardt @ 2025-03-24 13:08 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler, phillip.wood123
On Thu, Mar 20, 2025 at 12:44:03PM +0100, Karthik Nayak wrote:
> diff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc
> index 9e6935d38d..5be2c16776 100644
> --- a/Documentation/git-update-ref.adoc
> +++ b/Documentation/git-update-ref.adoc
> @@ -57,6 +59,14 @@ performs all modifications together. Specify commands of the form:
> With `--create-reflog`, update-ref will create a reflog for each ref
> even if one would not ordinarily be created.
>
> +With `--batch-updates`, update-ref executes the updates in a batch but allows
> +individual updates to fail due to invalid or incorrect user input, applying only
> +the successful updates. However, system-related errors—such as I/O failures or
> +memory issues—will result in a full failure of all batched updates. Any failed
> +updates will be reported in the following format:
> +
> + rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
> +
Does this support NUL-terminated mode? It probably should, and if it
does we should also document the format.
> diff --git a/builtin/update-ref.c b/builtin/update-ref.c
> index 1d541e13ad..97e14b279e 100644
> --- a/builtin/update-ref.c
> +++ b/builtin/update-ref.c
> @@ -735,6 +787,8 @@ int cmd_update_ref(int argc,
> OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
> OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
> OPT_BOOL( 0 , "create-reflog", &create_reflog, N_("create a reflog")),
> + OPT_BIT('0', "batch-updates", &flags, N_("batch reference updates"),
> + REF_TRANSACTION_ALLOW_FAILURE),
> OPT_END(),
> };
>
> @@ -756,9 +810,10 @@ int cmd_update_ref(int argc,
> usage_with_options(git_update_ref_usage, options);
> if (end_null)
> line_termination = '\0';
> - update_refs_stdin();
> + update_refs_stdin(flags);
> return 0;
> - }
> + } else if (flags & REF_TRANSACTION_ALLOW_FAILURE)
> + die("--batch-updates can only be used with --stdin");
Nit: formatting, the `else if` branch should have curly braces.
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v4 5/8] refs: introduce enum-based transaction error types
2025-03-20 20:26 ` Patrick Steinhardt
@ 2025-03-24 14:50 ` Karthik Nayak
2025-03-25 12:31 ` Patrick Steinhardt
0 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-03-24 14:50 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 1905 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> On Thu, Mar 20, 2025 at 12:44:00PM +0100, Karthik Nayak wrote:
>> diff --git a/refs.h b/refs.h
>> index 240e2d8537..dcd83e81e2 100644
>> --- a/refs.h
>> +++ b/refs.h
>> @@ -16,6 +16,29 @@ struct worktree;
>> enum ref_storage_format ref_storage_format_by_name(const char *name);
>> const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);
>>
>> +/*
>> + * enum ref_transaction_error represents the following return codes:
>> + * REF_TRANSACTION_ERROR_GENERIC error_code: default error code.
>> + * REF_TRANSACTION_ERROR_NAME_CONFLICT error_code: ref name conflict like A vs A/B.
>> + * REF_TRANSACTION_ERROR_CREATE_EXISTS error_code: ref to be created already exists.
>> + * REF_TRANSACTION_ERROR_NONEXISTENT_REF error_code: ref expected but doesn't exist.
>> + * REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE error_code: provided old_oid or old_target of
>> + * reference doesn't match actual.
>> + * REF_TRANSACTION_ERROR_INVALID_NEW_VALUE error_code: provided new_oid or new_target is
>> + * invalid.
>> + * REF_TRANSACTION_ERROR_EXPECTED_SYMREF error_code: expected ref to be symref, but is a
>> + * regular ref.
>> + */
>> +enum ref_transaction_error {
>> + REF_TRANSACTION_ERROR_GENERIC = -1,
>> + REF_TRANSACTION_ERROR_NAME_CONFLICT = -2,
>> + REF_TRANSACTION_ERROR_CREATE_EXISTS = -3,
>> + REF_TRANSACTION_ERROR_NONEXISTENT_REF = -4,
>> + REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE = -5,
>> + REF_TRANSACTION_ERROR_INVALID_NEW_VALUE = -6,
>> + REF_TRANSACTION_ERROR_EXPECTED_SYMREF = -7,
>> +};
>> +
>
> Tiny nit: I think it's generally preferable to document each specific
> right next to its definition.
>
Idk about this, I based it off on `enum bisect_error` which is similar,
but I also see the same in `enum scld_error`.
I'm okay to change this though, perhaps makes sense to document the
preferred style however.
> Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v4 6/8] refs: implement batch reference update support
2025-03-20 20:26 ` Patrick Steinhardt
@ 2025-03-24 14:54 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-24 14:54 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 3364 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> On Thu, Mar 20, 2025 at 12:44:01PM +0100, Karthik Nayak wrote:
>> diff --git a/refs.c b/refs.c
>> index 3d0b53d56e..b34ec198f5 100644
>> --- a/refs.c
>> +++ b/refs.c
>> @@ -1206,11 +1210,45 @@ void ref_transaction_free(struct ref_transaction *transaction)
>> free((char *)transaction->updates[i]->old_target);
>> free(transaction->updates[i]);
>> }
>> +
>> + if (transaction->rejections)
>> + free(transaction->rejections->update_indices);
>> + free(transaction->rejections);
>> +
>> string_list_clear(&transaction->refnames, 0);
>> free(transaction->updates);
>> free(transaction);
>> }
>>
>> +int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,
>> + size_t update_idx,
>> + enum ref_transaction_error err)
>> +{
>> + if (update_idx >= transaction->nr)
>> + BUG("trying to set rejection on invalid update index");
>> +
>> + if (!(transaction->flags & REF_TRANSACTION_ALLOW_FAILURE))
>> + return 0;
>> +
>> + if (!transaction->rejections)
>> + BUG("transaction not inititalized with failure support");
>> +
>> + /*
>> + * Don't accept generic errors, since these errors are not user
>> + * input related.
>> + */
>> + if (err == REF_TRANSACTION_ERROR_GENERIC)
>> + return 0;
>> +
>> + transaction->updates[update_idx]->rejection_err = err;
>> + ALLOC_GROW(transaction->rejections->update_indices,
>> + transaction->rejections->nr + 1,
>> + transaction->rejections->alloc);
>> + transaction->rejections->update_indices[transaction->rejections->nr++] = update_idx;
>> +
>> + return 1;
>> +}
>
> If we had a `struct ref_update_rejection` we could store the update
> index and rejection errors in the same location, which might be a bit
> easier to reason about.
>
I struggled with this a bit, I was thinking the same at the start. But
then I was also thinking that a rejection is specific to an update so it
should lie in `ref_update`, however tracking failed updates in a
transaction is specific to a transaction and should lie in
`ref_transaction`. It probably would be easier code-wise to have them
both in the same place, but it didn't feel like the correct place.
>> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
>> index d90bd815a3..7bf57ca948 100644
>> --- a/refs/packed-backend.c
>> +++ b/refs/packed-backend.c
>> @@ -1327,10 +1327,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
>> * remain locked when it is done.
>> */
>> static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,
>> - struct string_list *updates,
>> + struct ref_transaction *transaction,
>> struct strbuf *err)
>> {
>> enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
>> + struct string_list *updates = &transaction->refnames;
>> struct ref_iterator *iter = NULL;
>> size_t i;
>> int ok;
>> @@ -1411,6 +1412,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
>> "reference already exists",
>> update->refname);
>> ret = REF_TRANSACTION_ERROR_CREATE_EXISTS;
>> +
>> + if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
>> + strbuf_setlen(err, 0);
>
> Nit: you can use `strbuf_reset()` for this and other instances.
>
Yes, indeed, will change!
> Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v4 7/8] refs: support rejection in batch updates during F/D checks
2025-03-24 13:08 ` Patrick Steinhardt
@ 2025-03-24 17:48 ` Karthik Nayak
2025-03-25 12:31 ` Patrick Steinhardt
0 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-03-24 17:48 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 3921 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> On Thu, Mar 20, 2025 at 12:44:02PM +0100, Karthik Nayak wrote:
>> diff --git a/refs.c b/refs.c
>> index b34ec198f5..f719046f47 100644
>> --- a/refs.c
>> +++ b/refs.c
>> @@ -2540,6 +2540,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
>> const struct string_list *refnames,
>> const struct string_list *extras,
>> const struct string_list *skip,
>> + struct ref_transaction *transaction,
>> unsigned int initial_transaction,
>> struct strbuf *err)
>> {
>> @@ -2599,12 +2601,26 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
>> if (!initial_transaction &&
>> !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
>> &type, &ignore_errno)) {
>> + if (transaction && ref_transaction_maybe_set_rejected(
>> + transaction, *update_idx,
>> + REF_TRANSACTION_ERROR_NAME_CONFLICT)) {
>> + strset_remove(&dirnames, dirname.buf);
>> + continue;
>> + }
>> +
>
> Okay. We have to remove the dirname from `dirnames` again so that the
> next reference that creates a reference in the same directory would also
> be marked as conflicting. It does have the consequence that we now have
> to read the dirname N times again, where N is the number of refs that
> are created below that directory.
>
> We could probably improve this by using another map that contains the
> conflicting names, right?
>
Yes that's definitely possible, I will go ahead and add it!
>> diff --git a/refs/files-backend.c b/refs/files-backend.c
>> index be758ffff5..1d50d4013c 100644
>> --- a/refs/files-backend.c
>> +++ b/refs/files-backend.c
>> @@ -864,7 +868,9 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
>> * make sure there is no existing packed ref that conflicts
>> * with refname. This check is deferred so that we can batch it.
>> */
>> - string_list_append(refnames_to_check, refname);
>> + item = string_list_append(refnames_to_check, refname);
>> + item->util = xmalloc(sizeof(update_idx));
>> + memcpy(item->util, &update_idx, sizeof(update_idx));
>> }
>>
>> ret = 0;
>
> Hm, so we have to allocate the `util` field now to store the update
> index, which is a bit unfortunate because all of this is part of the hot
> loop. We cannot store a direct pointer though because the array of
> updates may be reallocated, which would invalidate any pointers pointing
> into the array.
>
Yes, your inference is on point.
> I was wondering whether we could abuse an `uintptr_t` and use it to
> store the update index as a pointer. It does feel somewhat dirty though.
>
We can do something like this, it would be _clever_ but does feel very
hacky.
I did so some benchmarking here
Benchmark 1: update-ref: create many refs (refformat = files, refcount
= 100000, revision = master)
Time (mean ± σ): 7.396 s ± 0.175 s [User: 0.962 s, System: 6.312 s]
Range (min … max): 7.145 s … 7.688 s 10 runs
Benchmark 2: update-ref: create many refs (refformat = files, refcount
= 100000, revision = b4/245-partially-atomic-ref-updates)
Time (mean ± σ): 7.514 s ± 0.144 s [User: 0.919 s, System: 6.438 s]
Range (min … max): 7.297 s … 7.750 s 10 runs
Summary
update-ref: create many refs (refformat = files, refcount = 100000,
revision = master) ran
1.02 ± 0.03 times faster than update-ref: create many refs
(refformat = files, refcount = 100000, revision =
b4/245-partially-atomic-ref-updates)
Overall the perf degradation is very minimal. I also looked at the
flamegraph to see if there is something. But most of the time seems to
be spent in IO (reading refs and creating locks).
So I think we should be ok here, wdyt?
> Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v4 8/8] update-ref: add --batch-updates flag for stdin mode
2025-03-24 13:08 ` Patrick Steinhardt
@ 2025-03-24 17:51 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-24 17:51 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git, jltobler, phillip.wood123
[-- Attachment #1: Type: text/plain, Size: 2606 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> On Thu, Mar 20, 2025 at 12:44:03PM +0100, Karthik Nayak wrote:
>> diff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc
>> index 9e6935d38d..5be2c16776 100644
>> --- a/Documentation/git-update-ref.adoc
>> +++ b/Documentation/git-update-ref.adoc
>> @@ -57,6 +59,14 @@ performs all modifications together. Specify commands of the form:
>> With `--create-reflog`, update-ref will create a reflog for each ref
>> even if one would not ordinarily be created.
>>
>> +With `--batch-updates`, update-ref executes the updates in a batch but allows
>> +individual updates to fail due to invalid or incorrect user input, applying only
>> +the successful updates. However, system-related errors—such as I/O failures or
>> +memory issues—will result in a full failure of all batched updates. Any failed
>> +updates will be reported in the following format:
>> +
>> + rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
>> +
>
> Does this support NUL-terminated mode? It probably should, and if it
> does we should also document the format.
>
It only does for inputs. So there is nothing to be done for outputs.
I actually added support for '-z' mode here, but there was an assumption
on my part that this is for both input/output.
Phillip corrected my assumption in the first version of this series [1].
[1]: https://lore.kernel.org/all/ceda422e-8c8e-4a1d-aaab-9a7a2fc009dd@gmail.com/
>> diff --git a/builtin/update-ref.c b/builtin/update-ref.c
>> index 1d541e13ad..97e14b279e 100644
>> --- a/builtin/update-ref.c
>> +++ b/builtin/update-ref.c
>> @@ -735,6 +787,8 @@ int cmd_update_ref(int argc,
>> OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
>> OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
>> OPT_BOOL( 0 , "create-reflog", &create_reflog, N_("create a reflog")),
>> + OPT_BIT('0', "batch-updates", &flags, N_("batch reference updates"),
>> + REF_TRANSACTION_ALLOW_FAILURE),
>> OPT_END(),
>> };
>>
>> @@ -756,9 +810,10 @@ int cmd_update_ref(int argc,
>> usage_with_options(git_update_ref_usage, options);
>> if (end_null)
>> line_termination = '\0';
>> - update_refs_stdin();
>> + update_refs_stdin(flags);
>> return 0;
>> - }
>> + } else if (flags & REF_TRANSACTION_ALLOW_FAILURE)
>> + die("--batch-updates can only be used with --stdin");
>
> Nit: formatting, the `else if` branch should have curly braces.
Ah! Thanks, will change!
> Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v4 5/8] refs: introduce enum-based transaction error types
2025-03-24 14:50 ` Karthik Nayak
@ 2025-03-25 12:31 ` Patrick Steinhardt
0 siblings, 0 replies; 143+ messages in thread
From: Patrick Steinhardt @ 2025-03-25 12:31 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler, phillip.wood123
On Mon, Mar 24, 2025 at 02:50:56PM +0000, Karthik Nayak wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > On Thu, Mar 20, 2025 at 12:44:00PM +0100, Karthik Nayak wrote:
> >> diff --git a/refs.h b/refs.h
> >> index 240e2d8537..dcd83e81e2 100644
> >> --- a/refs.h
> >> +++ b/refs.h
> >> @@ -16,6 +16,29 @@ struct worktree;
> >> enum ref_storage_format ref_storage_format_by_name(const char *name);
> >> const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);
> >>
> >> +/*
> >> + * enum ref_transaction_error represents the following return codes:
> >> + * REF_TRANSACTION_ERROR_GENERIC error_code: default error code.
> >> + * REF_TRANSACTION_ERROR_NAME_CONFLICT error_code: ref name conflict like A vs A/B.
> >> + * REF_TRANSACTION_ERROR_CREATE_EXISTS error_code: ref to be created already exists.
> >> + * REF_TRANSACTION_ERROR_NONEXISTENT_REF error_code: ref expected but doesn't exist.
> >> + * REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE error_code: provided old_oid or old_target of
> >> + * reference doesn't match actual.
> >> + * REF_TRANSACTION_ERROR_INVALID_NEW_VALUE error_code: provided new_oid or new_target is
> >> + * invalid.
> >> + * REF_TRANSACTION_ERROR_EXPECTED_SYMREF error_code: expected ref to be symref, but is a
> >> + * regular ref.
> >> + */
> >> +enum ref_transaction_error {
> >> + REF_TRANSACTION_ERROR_GENERIC = -1,
> >> + REF_TRANSACTION_ERROR_NAME_CONFLICT = -2,
> >> + REF_TRANSACTION_ERROR_CREATE_EXISTS = -3,
> >> + REF_TRANSACTION_ERROR_NONEXISTENT_REF = -4,
> >> + REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE = -5,
> >> + REF_TRANSACTION_ERROR_INVALID_NEW_VALUE = -6,
> >> + REF_TRANSACTION_ERROR_EXPECTED_SYMREF = -7,
> >> +};
> >> +
> >
> > Tiny nit: I think it's generally preferable to document each specific
> > right next to its definition.
> >
>
> Idk about this, I based it off on `enum bisect_error` which is similar,
> but I also see the same in `enum scld_error`.
>
> I'm okay to change this though, perhaps makes sense to document the
> preferred style however.
Probably would make sense, yes. The argument for why it's preferable to
do this inline is that it's harder for the list to grow stale. It's
trivial to see when a comment needs to be removed/updated.
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v4 7/8] refs: support rejection in batch updates during F/D checks
2025-03-24 17:48 ` Karthik Nayak
@ 2025-03-25 12:31 ` Patrick Steinhardt
0 siblings, 0 replies; 143+ messages in thread
From: Patrick Steinhardt @ 2025-03-25 12:31 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler, phillip.wood123
On Mon, Mar 24, 2025 at 05:48:32PM +0000, Karthik Nayak wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> > On Thu, Mar 20, 2025 at 12:44:02PM +0100, Karthik Nayak wrote:
> >> diff --git a/refs/files-backend.c b/refs/files-backend.c
> >> index be758ffff5..1d50d4013c 100644
> >> --- a/refs/files-backend.c
> >> +++ b/refs/files-backend.c
> >> @@ -864,7 +868,9 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
> >> * make sure there is no existing packed ref that conflicts
> >> * with refname. This check is deferred so that we can batch it.
> >> */
> >> - string_list_append(refnames_to_check, refname);
> >> + item = string_list_append(refnames_to_check, refname);
> >> + item->util = xmalloc(sizeof(update_idx));
> >> + memcpy(item->util, &update_idx, sizeof(update_idx));
> >> }
> >>
> >> ret = 0;
> >
> > Hm, so we have to allocate the `util` field now to store the update
> > index, which is a bit unfortunate because all of this is part of the hot
> > loop. We cannot store a direct pointer though because the array of
> > updates may be reallocated, which would invalidate any pointers pointing
> > into the array.
> >
>
> Yes, your inference is on point.
>
> > I was wondering whether we could abuse an `uintptr_t` and use it to
> > store the update index as a pointer. It does feel somewhat dirty though.
> >
>
> We can do something like this, it would be _clever_ but does feel very
> hacky.
>
> I did so some benchmarking here
>
> Benchmark 1: update-ref: create many refs (refformat = files, refcount
> = 100000, revision = master)
> Time (mean ± σ): 7.396 s ± 0.175 s [User: 0.962 s, System: 6.312 s]
> Range (min … max): 7.145 s … 7.688 s 10 runs
>
> Benchmark 2: update-ref: create many refs (refformat = files, refcount
> = 100000, revision = b4/245-partially-atomic-ref-updates)
> Time (mean ± σ): 7.514 s ± 0.144 s [User: 0.919 s, System: 6.438 s]
> Range (min … max): 7.297 s … 7.750 s 10 runs
>
> Summary
> update-ref: create many refs (refformat = files, refcount = 100000,
> revision = master) ran
> 1.02 ± 0.03 times faster than update-ref: create many refs
> (refformat = files, refcount = 100000, revision =
> b4/245-partially-atomic-ref-updates)
>
> Overall the perf degradation is very minimal. I also looked at the
> flamegraph to see if there is something. But most of the time seems to
> be spent in IO (reading refs and creating locks).
>
> So I think we should be ok here, wdyt?
Okay. I'm not a huge fan of using the `->util` pointer like this, but I
don't have a better idea either, and the performance regression seems
acceptable. So let's roll with it until somebody has a better idea.
Thanks for doing the benchmark!
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH v5 0/8] refs: introduce support for batched reference updates
2025-02-07 7:34 [PATCH 0/6] refs: introduce support for partial reference transactions Karthik Nayak
` (9 preceding siblings ...)
2025-03-20 11:43 ` [PATCH v4 0/8] refs: introduce support for batched reference updates Karthik Nayak
@ 2025-03-27 11:13 ` Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
` (8 more replies)
2025-04-08 8:51 ` [PATCH v6 " Karthik Nayak
11 siblings, 9 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-27 11:13 UTC (permalink / raw)
To: git; +Cc: jltobler, phillip.wood123, gitster, ps, Karthik Nayak
Git supports making reference updates with or without transactions.
Updates with transactions are generally better optimized. But
transactions are all or nothing. This means, if a user wants to batch
updates to take advantage of the optimizations without the hard
requirement that all updates must succeed, there is no way currently to
do so. Particularly with the reftable backend where batching multiple
reference updates is more efficient than performing them sequentially.
This series introduces support for batched reference updates without
transactions allowing individual reference updates to fail while letting
others proceed. This capability is exposed through git-update-ref's
`--allow-partial` flag, which can be used in `--stdin` mode to batch
updates and handle failures gracefully. Under the hood, these batched
updates still use the transactions infrastructure, while modifying
sections to allow partial failures.
The changes are structured to carefully build up this functionality:
First, we clean up and consolidate the reference update checking logic.
This includes removing duplicate checks in the files backend and moving
refname tracking to the generic layer, which simplifies the codebase and
prepares it for the new feature.
We then restructure the reftable backend's transaction preparation code,
extracting the update validation logic into a dedicated function. This
not only improves code organization but sets the stage for implementing
partial transaction support.
To ensure we only skip errors which are user-oriented, we introduce
typed errors for transactions with 'enum ref_transaction_error'. We
extend the existing errors to include other scenarios and use this new
errors throughout the refs code.
With this groundwork in place, we implement the core batch update
support in the refs subsystem. This adds the necessary infrastructure to
track and report rejected updates while allowing transactions to
proceed. All reference backends are modified to support this behavior
when enabled.
Finally, we expose this functionality to users through
git-update-ref(1)'s `--allow-partial` flag, complete with test coverage
and documentation. The flag is specifically limited to `--stdin` mode
where batching multiple updates is most relevant.
This enhancement improves Git's flexibility in handling reference
updates while maintaining the safety of atomic transactions by default.
It's particularly valuable for tools and workflows that need to handle
reference update failures gracefully without abandoning the entire batch
of updates.
This series is based on top of 683c54c999 (Git 2.49, 2025-03-14) with
Patrick's series 'refs: batch refname availability checks' [1] merged
in.
[1]: https://lore.kernel.org/all/20250217-pks-update-ref-optimization-v1-0-a2b6d87a24af@pks.im/
---
Changes in v5:
- Inline the comments around the 'ref_transaction_error'.
- Use 'strbuf_reset()' wherever possible instead of 'strbuf_setlen(err, 0)'.
- Use an extra 'conflicting_dirnames' strset in 'refs_verify_refnames_available()' to track
dirnames which were found to be conflicting, this is to avoid re-reading those dirnames.
- Add curly braces style mismatch in if..else block.
- Link to v4: https://lore.kernel.org/r/20250320-245-partially-atomic-ref-updates-v4-0-3dcc1b311dc9@gmail.com
Changes in v4:
- Rebased on top of 2.49 since there was a long time between the
previous iteration and we have a new release.
- Changed the naming to say 'batched' updates instead of 'partial
transactions'. While we still use the transaction infrastructure
underneath, the new naming causes less ambiguity.
- Clean up some of the commit messages.
- Raise BUG for invalid update index while setting rejections.
- Fix an incorrect early return.
- Link to v3: https://lore.kernel.org/r/20250305-245-partially-atomic-ref-updates-v3-0-0c64e3052354@gmail.com
Changes in v3:
- Changed 'transaction_error' to 'ref_transaction_error' along with the
error names. Removed 'TRANSACTION_OK' since it can potentially be
missed instead of simply 'return 0'.
- Rename 'ref_transaction_set_rejected' to
'ref_transaction_maybe_set_rejected' and move logic around error
checks to within this function.
- Add a new struct 'ref_transaction_rejections' to track the rejections
within a transaction. This allows us to only iterate over rejected
updates.
- Add a new commit to also support partial transactions within the
batched F/D checks.
- Remove NUL delimited outputs in 'git-update-ref(1)'.
- Remove translations for plumbing outputs.
- Other small cleanups in the commit message and code.
Changes in v2:
- Introduce and use structured errors. This consolidates the errors
and their handling between the ref backends.
- In the previous version, we skipped over all failures. This include
system failures such as low memory or IO problems. Let's instead, only
skip user-oriented failures, such as invalid old OID and so on.
- Change the rejection function name to `ref_transaction_set_rejected()`.
- Modify the commit messages and documentation to be a little more
verbose.
- Link to v1: https://lore.kernel.org/r/20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com
---
Documentation/git-update-ref.adoc | 14 +-
builtin/fetch.c | 2 +-
builtin/update-ref.c | 66 ++++-
refs.c | 171 +++++++++++--
refs.h | 70 ++++--
refs/files-backend.c | 314 +++++++++++-------------
refs/packed-backend.c | 69 +++---
refs/refs-internal.h | 51 +++-
refs/reftable-backend.c | 502 +++++++++++++++++++-------------------
t/t1400-update-ref.sh | 233 ++++++++++++++++++
10 files changed, 969 insertions(+), 523 deletions(-)
Karthik Nayak (8):
refs/files: remove redundant check in split_symref_update()
refs: move duplicate refname update check to generic layer
refs/files: remove duplicate duplicates check
refs/reftable: extract code from the transaction preparation
refs: introduce enum-based transaction error types
refs: implement batch reference update support
refs: support rejection in batch updates during F/D checks
update-ref: add --batch-updates flag for stdin mode
---
Range-diff versus v4:
1: c682fce9d0 = 1: cae24142a1 refs/files: remove redundant check in split_symref_update()
2: 7483120888 = 2: 239aecdb0f refs: move duplicate refname update check to generic layer
3: e54c9042b5 = 3: 06404dd350 refs/files: remove duplicate duplicates check
4: 4f905880af = 4: a3e645aa37 refs/reftable: extract code from the transaction preparation
5: 7846fc43f5 ! 5: 2615bfe78e refs: introduce enum-based transaction error types
@@ refs.h: struct worktree;
enum ref_storage_format ref_storage_format_by_name(const char *name);
const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);
-+/*
-+ * enum ref_transaction_error represents the following return codes:
-+ * REF_TRANSACTION_ERROR_GENERIC error_code: default error code.
-+ * REF_TRANSACTION_ERROR_NAME_CONFLICT error_code: ref name conflict like A vs A/B.
-+ * REF_TRANSACTION_ERROR_CREATE_EXISTS error_code: ref to be created already exists.
-+ * REF_TRANSACTION_ERROR_NONEXISTENT_REF error_code: ref expected but doesn't exist.
-+ * REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE error_code: provided old_oid or old_target of
-+ * reference doesn't match actual.
-+ * REF_TRANSACTION_ERROR_INVALID_NEW_VALUE error_code: provided new_oid or new_target is
-+ * invalid.
-+ * REF_TRANSACTION_ERROR_EXPECTED_SYMREF error_code: expected ref to be symref, but is a
-+ * regular ref.
-+ */
+enum ref_transaction_error {
++ /* Default error code */
+ REF_TRANSACTION_ERROR_GENERIC = -1,
++ /* Ref name conflict like A vs A/B */
+ REF_TRANSACTION_ERROR_NAME_CONFLICT = -2,
++ /* Ref to be created already exists */
+ REF_TRANSACTION_ERROR_CREATE_EXISTS = -3,
++ /* ref expected but doesn't exist */
+ REF_TRANSACTION_ERROR_NONEXISTENT_REF = -4,
++ /* Provided old_oid or old_target of reference doesn't match actual */
+ REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE = -5,
++ /* Provided new_oid or new_target is invalid */
+ REF_TRANSACTION_ERROR_INVALID_NEW_VALUE = -6,
++ /* Expected ref to be symref, but is a regular ref */
+ REF_TRANSACTION_ERROR_EXPECTED_SYMREF = -7,
+};
+
6: 398b93689a ! 6: d5c1c77b0d refs: implement batch reference update support
@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref
- if (ret)
+ if (ret) {
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
-+ strbuf_setlen(err, 0);
++ strbuf_reset(err);
+ ret = 0;
+
+ continue;
@@ refs/packed-backend.c: static enum ref_transaction_error write_with_updates(stru
ret = REF_TRANSACTION_ERROR_CREATE_EXISTS;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
-+ strbuf_setlen(err, 0);
++ strbuf_reset(err);
+ ret = 0;
+ continue;
+ }
@@ refs/packed-backend.c: static enum ref_transaction_error write_with_updates(stru
ret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
-+ strbuf_setlen(err, 0);
++ strbuf_reset(err);
+ ret = 0;
+ continue;
+ }
@@ refs/packed-backend.c: static enum ref_transaction_error write_with_updates(stru
ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
-+ strbuf_setlen(err, 0);
++ strbuf_reset(err);
+ ret = 0;
+ continue;
+ }
@@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
- if (ret)
+ if (ret) {
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
-+ strbuf_setlen(err, 0);
++ strbuf_reset(err);
+ ret = 0;
+
+ continue;
7: 965cd76097 ! 7: 4bb4902631 refs: support rejection in batch updates during F/D checks
@@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_sto
struct strbuf *err)
{
@@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
+ struct strbuf referent = STRBUF_INIT;
+ struct string_list_item *item;
+ struct ref_iterator *iter = NULL;
++ struct strset conflicting_dirnames;
+ struct strset dirnames;
+ int ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
+
+@@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
+
+ assert(err);
+
++ strset_init(&conflicting_dirnames);
strset_init(&dirnames);
for_each_string_list_item(item, refnames) {
@@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_sto
const char *extra_refname;
struct object_id oid;
@@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
+ continue;
+
if (!initial_transaction &&
- !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
- &type, &ignore_errno)) {
+- !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
+- &type, &ignore_errno)) {
++ (strset_contains(&conflicting_dirnames, dirname.buf) ||
++ !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
++ &type, &ignore_errno))) {
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT)) {
+ strset_remove(&dirnames, dirname.buf);
++ strset_add(&conflicting_dirnames, dirname.buf);
+ continue;
+ }
+
@@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_sto
strbuf_addf(err, _("cannot process '%s' and '%s' at the same time"),
refname, extra_refname);
goto cleanup;
+@@ refs.c: enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
+ cleanup:
+ strbuf_release(&referent);
+ strbuf_release(&dirname);
++ strset_clear(&conflicting_dirnames);
+ strset_clear(&dirnames);
+ ref_iterator_free(iter);
+ return ret;
@@ refs.c: enum ref_transaction_error refs_verify_refname_available(
};
8: ed58c67cd7 ! 8: 674630f77c update-ref: add --batch-updates flag for stdin mode
@@ builtin/update-ref.c: int cmd_update_ref(int argc,
- update_refs_stdin();
+ update_refs_stdin(flags);
return 0;
-- }
-+ } else if (flags & REF_TRANSACTION_ALLOW_FAILURE)
++ } else if (flags & REF_TRANSACTION_ALLOW_FAILURE) {
+ die("--batch-updates can only be used with --stdin");
+ }
if (end_null)
- usage_with_options(git_update_ref_usage, options);
## t/t1400-update-ref.sh ##
@@ t/t1400-update-ref.sh: do
---
base-commit: 679c868f5fffadd1f7e8e49d4d87d745ee36ffb7
change-id: 20241206-245-partially-atomic-ref-updates-9fe8b080345c
Thanks
- Karthik
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH v5 1/8] refs/files: remove redundant check in split_symref_update()
2025-03-27 11:13 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Karthik Nayak
@ 2025-03-27 11:13 ` Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 2/8] refs: move duplicate refname update check to generic layer Karthik Nayak
` (7 subsequent siblings)
8 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-27 11:13 UTC (permalink / raw)
To: git; +Cc: jltobler, phillip.wood123, gitster, ps, Karthik Nayak
In `split_symref_update()`, there were two checks for duplicate
refnames:
- At the start, `string_list_has_string()` ensures the refname is not
already in `affected_refnames`, preventing duplicates from being
added.
- After adding the refname, another check verifies whether the newly
inserted item has a `util` value.
The second check is unnecessary because the first one guarantees that
`string_list_insert()` will never encounter a preexisting entry.
The `item->util` field is assigned to validate that a rename doesn't
already exist in the list. The validation is done after the first check.
As this check is removed, clean up the validation and the assignment of
this field in `split_head_update()` and `files_transaction_prepare()`.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs/files-backend.c | 20 +++-----------------
1 file changed, 3 insertions(+), 17 deletions(-)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index ff54a4bb7e..15559a09c5 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2382,7 +2382,6 @@ static int split_head_update(struct ref_update *update,
struct string_list *affected_refnames,
struct strbuf *err)
{
- struct string_list_item *item;
struct ref_update *new_update;
if ((update->flags & REF_LOG_ONLY) ||
@@ -2421,8 +2420,7 @@ static int split_head_update(struct ref_update *update,
*/
if (strcmp(new_update->refname, "HEAD"))
BUG("%s unexpectedly not 'HEAD'", new_update->refname);
- item = string_list_insert(affected_refnames, new_update->refname);
- item->util = new_update;
+ string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2441,7 +2439,6 @@ static int split_symref_update(struct ref_update *update,
struct string_list *affected_refnames,
struct strbuf *err)
{
- struct string_list_item *item;
struct ref_update *new_update;
unsigned int new_flags;
@@ -2496,11 +2493,7 @@ static int split_symref_update(struct ref_update *update,
* be valid as long as affected_refnames is in use, and NOT
* referent, which might soon be freed by our caller.
*/
- item = string_list_insert(affected_refnames, new_update->refname);
- if (item->util)
- BUG("%s unexpectedly found in affected_refnames",
- new_update->refname);
- item->util = new_update;
+ string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2834,7 +2827,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
*/
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
- struct string_list_item *item;
if ((update->flags & REF_IS_PRUNING) &&
!(update->flags & REF_NO_DEREF))
@@ -2843,13 +2835,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
if (update->flags & REF_LOG_ONLY)
continue;
- item = string_list_append(&affected_refnames, update->refname);
- /*
- * We store a pointer to update in item->util, but at
- * the moment we never use the value of this field
- * except to check whether it is non-NULL.
- */
- item->util = update;
+ string_list_append(&affected_refnames, update->refname);
}
string_list_sort(&affected_refnames);
if (ref_update_reject_duplicates(&affected_refnames, err)) {
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v5 2/8] refs: move duplicate refname update check to generic layer
2025-03-27 11:13 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
@ 2025-03-27 11:13 ` Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 3/8] refs/files: remove duplicate duplicates check Karthik Nayak
` (6 subsequent siblings)
8 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-27 11:13 UTC (permalink / raw)
To: git; +Cc: jltobler, phillip.wood123, gitster, ps, Karthik Nayak
Move the tracking of refnames in `affected_refnames` from individual
backends into the generic layer in 'refs.c'. This centralizes the
duplicate refname detection that was previously handled separately by
each backend.
Make some changes to accommodate this move:
- Add a `string_list` field `refnames` to `ref_transaction` to contain
all the references in a transaction. This field is updated whenever
a new update is added via `ref_transaction_add_update`, so manual
additions in reference backends are dropped.
- Modify the backends to use this field internally as needed. The
backends need to check if an update for refname already exists when
splitting symrefs or adding an update for 'HEAD'.
- In the reftable backend, within `reftable_be_transaction_prepare()`,
move the `string_list_has_string()` check above
`ref_transaction_add_update()`. Since `ref_transaction_add_update()`
automatically adds the refname to `transaction->refnames`,
performing the check after will always return true, so we perform
the check before adding the update.
This helps reduce duplication of functionality between the backends and
makes it easier to make changes in a more centralized manner.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 17 +++++++++++++
refs/files-backend.c | 67 +++++++++++--------------------------------------
refs/packed-backend.c | 25 +-----------------
refs/refs-internal.h | 2 ++
refs/reftable-backend.c | 54 +++++++++++++--------------------------
5 files changed, 51 insertions(+), 114 deletions(-)
diff --git a/refs.c b/refs.c
index 2ac9d8ebd0..504bf2063e 100644
--- a/refs.c
+++ b/refs.c
@@ -1175,6 +1175,7 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,
CALLOC_ARRAY(tr, 1);
tr->ref_store = refs;
tr->flags = flags;
+ string_list_init_dup(&tr->refnames);
return tr;
}
@@ -1205,6 +1206,7 @@ void ref_transaction_free(struct ref_transaction *transaction)
free((char *)transaction->updates[i]->old_target);
free(transaction->updates[i]);
}
+ string_list_clear(&transaction->refnames, 0);
free(transaction->updates);
free(transaction);
}
@@ -1218,6 +1220,7 @@ struct ref_update *ref_transaction_add_update(
const char *committer_info,
const char *msg)
{
+ struct string_list_item *item;
struct ref_update *update;
if (transaction->state != REF_TRANSACTION_OPEN)
@@ -1245,6 +1248,16 @@ struct ref_update *ref_transaction_add_update(
update->msg = normalize_reflog_message(msg);
}
+ /*
+ * This list is generally used by the backends to avoid duplicates.
+ * But we do support multiple log updates for a given refname within
+ * a single transaction.
+ */
+ if (!(update->flags & REF_LOG_ONLY)) {
+ item = string_list_append(&transaction->refnames, refname);
+ item->util = update;
+ }
+
return update;
}
@@ -2405,6 +2418,10 @@ int ref_transaction_prepare(struct ref_transaction *transaction,
return -1;
}
+ string_list_sort(&transaction->refnames);
+ if (ref_update_reject_duplicates(&transaction->refnames, err))
+ return TRANSACTION_GENERIC_ERROR;
+
ret = refs->be->transaction_prepare(refs, transaction, err);
if (ret)
return ret;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 15559a09c5..58f62ea8a3 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2378,9 +2378,7 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st
*/
static int split_head_update(struct ref_update *update,
struct ref_transaction *transaction,
- const char *head_ref,
- struct string_list *affected_refnames,
- struct strbuf *err)
+ const char *head_ref, struct strbuf *err)
{
struct ref_update *new_update;
@@ -2398,7 +2396,7 @@ static int split_head_update(struct ref_update *update,
* transaction. This check is O(lg N) in the transaction
* size, but it happens at most once per transaction.
*/
- if (string_list_has_string(affected_refnames, "HEAD")) {
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
/* An entry already existed */
strbuf_addf(err,
"multiple updates for 'HEAD' (including one "
@@ -2420,7 +2418,6 @@ static int split_head_update(struct ref_update *update,
*/
if (strcmp(new_update->refname, "HEAD"))
BUG("%s unexpectedly not 'HEAD'", new_update->refname);
- string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2436,7 +2433,6 @@ static int split_head_update(struct ref_update *update,
static int split_symref_update(struct ref_update *update,
const char *referent,
struct ref_transaction *transaction,
- struct string_list *affected_refnames,
struct strbuf *err)
{
struct ref_update *new_update;
@@ -2448,7 +2444,7 @@ static int split_symref_update(struct ref_update *update,
* size, but it happens at most once per symref in a
* transaction.
*/
- if (string_list_has_string(affected_refnames, referent)) {
+ if (string_list_has_string(&transaction->refnames, referent)) {
/* An entry already exists */
strbuf_addf(err,
"multiple updates for '%s' (including one "
@@ -2486,15 +2482,6 @@ static int split_symref_update(struct ref_update *update,
update->flags |= REF_LOG_ONLY | REF_NO_DEREF;
update->flags &= ~REF_HAVE_OLD;
- /*
- * Add the referent. This insertion is O(N) in the transaction
- * size, but it happens at most once per symref in a
- * transaction. Make sure to add new_update->refname, which will
- * be valid as long as affected_refnames is in use, and NOT
- * referent, which might soon be freed by our caller.
- */
- string_list_insert(affected_refnames, new_update->refname);
-
return 0;
}
@@ -2558,7 +2545,6 @@ static int lock_ref_for_update(struct files_ref_store *refs,
struct ref_transaction *transaction,
const char *head_ref,
struct string_list *refnames_to_check,
- struct string_list *affected_refnames,
struct strbuf *err)
{
struct strbuf referent = STRBUF_INIT;
@@ -2575,8 +2561,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
update->flags |= REF_DELETING;
if (head_ref) {
- ret = split_head_update(update, transaction, head_ref,
- affected_refnames, err);
+ ret = split_head_update(update, transaction, head_ref, err);
if (ret)
goto out;
}
@@ -2586,9 +2571,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,
lock->count++;
} else {
ret = lock_raw_ref(refs, update->refname, mustexist,
- refnames_to_check, affected_refnames,
- &lock, &referent,
- &update->type, err);
+ refnames_to_check, &transaction->refnames,
+ &lock, &referent, &update->type, err);
if (ret) {
char *reason;
@@ -2642,9 +2626,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,
* of processing the split-off update, so we
* don't have to do it here.
*/
- ret = split_symref_update(update,
- referent.buf, transaction,
- affected_refnames, err);
+ ret = split_symref_update(update, referent.buf,
+ transaction, err);
if (ret)
goto out;
}
@@ -2799,7 +2782,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
"ref_transaction_prepare");
size_t i;
int ret = 0;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
struct string_list refnames_to_check = STRING_LIST_INIT_NODUP;
char *head_ref = NULL;
int head_type;
@@ -2818,12 +2800,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
transaction->backend_data = backend_data;
/*
- * Fail if a refname appears more than once in the
- * transaction. (If we end up splitting up any updates using
- * split_symref_update() or split_head_update(), those
- * functions will check that the new updates don't have the
- * same refname as any existing ones.) Also fail if any of the
- * updates use REF_IS_PRUNING without REF_NO_DEREF.
+ * Fail if any of the updates use REF_IS_PRUNING without REF_NO_DEREF.
*/
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
@@ -2831,16 +2808,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
if ((update->flags & REF_IS_PRUNING) &&
!(update->flags & REF_NO_DEREF))
BUG("REF_IS_PRUNING set without REF_NO_DEREF");
-
- if (update->flags & REF_LOG_ONLY)
- continue;
-
- string_list_append(&affected_refnames, update->refname);
- }
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
}
/*
@@ -2882,7 +2849,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
ret = lock_ref_for_update(refs, update, transaction,
head_ref, &refnames_to_check,
- &affected_refnames, err);
+ err);
if (ret)
goto cleanup;
@@ -2929,7 +2896,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
* So instead, we accept the race for now.
*/
if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
- &affected_refnames, NULL, 0, err)) {
+ &transaction->refnames, NULL, 0, err)) {
ret = TRANSACTION_NAME_CONFLICT;
goto cleanup;
}
@@ -2975,7 +2942,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
cleanup:
free(head_ref);
- string_list_clear(&affected_refnames, 0);
string_list_clear(&refnames_to_check, 0);
if (ret)
@@ -3050,13 +3016,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (transaction->state != REF_TRANSACTION_PREPARED)
BUG("commit called for transaction that is not prepared");
- /* Fail if a refname appears more than once in the transaction: */
- for (i = 0; i < transaction->nr; i++)
- if (!(transaction->updates[i]->flags & REF_LOG_ONLY))
- string_list_append(&affected_refnames,
- transaction->updates[i]->refname);
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
+ string_list_sort(&transaction->refnames);
+ if (ref_update_reject_duplicates(&transaction->refnames, err)) {
ret = TRANSACTION_GENERIC_ERROR;
goto cleanup;
}
@@ -3074,7 +3035,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
* that we are creating already exists.
*/
if (refs_for_each_rawref(&refs->base, ref_present,
- &affected_refnames))
+ &transaction->refnames))
BUG("initial ref transaction called with existing refs");
packed_transaction = ref_store_transaction_begin(refs->packed_ref_store,
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index f4c82ba2c7..19220d2e99 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1622,8 +1622,6 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
struct packed_transaction_backend_data {
/* True iff the transaction owns the packed-refs lock. */
int own_lock;
-
- struct string_list updates;
};
static void packed_transaction_cleanup(struct packed_ref_store *refs,
@@ -1632,8 +1630,6 @@ static void packed_transaction_cleanup(struct packed_ref_store *refs,
struct packed_transaction_backend_data *data = transaction->backend_data;
if (data) {
- string_list_clear(&data->updates, 0);
-
if (is_tempfile_active(refs->tempfile))
delete_tempfile(&refs->tempfile);
@@ -1658,7 +1654,6 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
"ref_transaction_prepare");
struct packed_transaction_backend_data *data;
- size_t i;
int ret = TRANSACTION_GENERIC_ERROR;
/*
@@ -1671,34 +1666,16 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
*/
CALLOC_ARRAY(data, 1);
- string_list_init_nodup(&data->updates);
transaction->backend_data = data;
- /*
- * Stick the updates in a string list by refname so that we
- * can sort them:
- */
- for (i = 0; i < transaction->nr; i++) {
- struct ref_update *update = transaction->updates[i];
- struct string_list_item *item =
- string_list_append(&data->updates, update->refname);
-
- /* Store a pointer to update in item->util: */
- item->util = update;
- }
- string_list_sort(&data->updates);
-
- if (ref_update_reject_duplicates(&data->updates, err))
- goto failure;
-
if (!is_lock_file_locked(&refs->lock)) {
if (packed_refs_lock(ref_store, 0, err))
goto failure;
data->own_lock = 1;
}
- if (write_with_updates(refs, &data->updates, err))
+ if (write_with_updates(refs, &transaction->refnames, err))
goto failure;
transaction->state = REF_TRANSACTION_PREPARED;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index e5862757a7..92db793026 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -3,6 +3,7 @@
#include "refs.h"
#include "iterator.h"
+#include "string-list.h"
struct fsck_options;
struct ref_transaction;
@@ -198,6 +199,7 @@ enum ref_transaction_state {
struct ref_transaction {
struct ref_store *ref_store;
struct ref_update **updates;
+ struct string_list refnames;
size_t alloc;
size_t nr;
enum ref_transaction_state state;
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index ae434cd248..a92c9a2f4f 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1076,7 +1076,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
struct reftable_ref_store *refs =
reftable_be_downcast(ref_store, REF_STORE_WRITE|REF_STORE_MAIN, "ref_transaction_prepare");
struct strbuf referent = STRBUF_INIT, head_referent = STRBUF_INIT;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
struct string_list refnames_to_check = STRING_LIST_INIT_NODUP;
struct reftable_transaction_data *tx_data = NULL;
struct reftable_backend *be;
@@ -1101,10 +1100,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->updates[i], err);
if (ret)
goto done;
-
- if (!(transaction->updates[i]->flags & REF_LOG_ONLY))
- string_list_append(&affected_refnames,
- transaction->updates[i]->refname);
}
/*
@@ -1116,17 +1111,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
tx_data->args[i].updates_alloc = tx_data->args[i].updates_expected;
}
- /*
- * Fail if a refname appears more than once in the transaction.
- * This code is taken from the files backend and is a good candidate to
- * be moved into the generic layer.
- */
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto done;
- }
-
/*
* TODO: it's dubious whether we should reload the stack that "HEAD"
* belongs to or not. In theory, it may happen that we only modify
@@ -1194,14 +1178,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
!(u->flags & REF_LOG_ONLY) &&
!(u->flags & REF_UPDATE_VIA_HEAD) &&
!strcmp(rewritten_ref, head_referent.buf)) {
- struct ref_update *new_update;
-
/*
* First make sure that HEAD is not already in the
* transaction. This check is O(lg N) in the transaction
* size, but it happens at most once per transaction.
*/
- if (string_list_has_string(&affected_refnames, "HEAD")) {
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
/* An entry already existed */
strbuf_addf(err,
_("multiple updates for 'HEAD' (including one "
@@ -1211,12 +1193,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
goto done;
}
- new_update = ref_transaction_add_update(
- transaction, "HEAD",
- u->flags | REF_LOG_ONLY | REF_NO_DEREF,
- &u->new_oid, &u->old_oid, NULL, NULL, NULL,
- u->msg);
- string_list_insert(&affected_refnames, new_update->refname);
+ ref_transaction_add_update(
+ transaction, "HEAD",
+ u->flags | REF_LOG_ONLY | REF_NO_DEREF,
+ &u->new_oid, &u->old_oid, NULL, NULL, NULL,
+ u->msg);
}
ret = reftable_backend_read_ref(be, rewritten_ref,
@@ -1281,6 +1262,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
if (!strcmp(rewritten_ref, "HEAD"))
new_flags |= REF_UPDATE_VIA_HEAD;
+ if (string_list_has_string(&transaction->refnames, referent.buf)) {
+ strbuf_addf(err,
+ _("multiple updates for '%s' (including one "
+ "via symref '%s') are not allowed"),
+ referent.buf, u->refname);
+ ret = TRANSACTION_NAME_CONFLICT;
+ goto done;
+ }
+
/*
* If we are updating a symref (eg. HEAD), we should also
* update the branch that the symref points to.
@@ -1305,16 +1295,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
*/
u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
u->flags &= ~REF_HAVE_OLD;
-
- if (string_list_has_string(&affected_refnames, new_update->refname)) {
- strbuf_addf(err,
- _("multiple updates for '%s' (including one "
- "via symref '%s') are not allowed"),
- referent.buf, u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
- string_list_insert(&affected_refnames, new_update->refname);
}
}
@@ -1383,7 +1363,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
}
}
- ret = refs_verify_refnames_available(ref_store, &refnames_to_check, &affected_refnames, NULL,
+ ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
+ &transaction->refnames, NULL,
transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
err);
if (ret < 0)
@@ -1401,7 +1382,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
strbuf_addf(err, _("reftable: transaction prepare: %s"),
reftable_error_str(ret));
}
- string_list_clear(&affected_refnames, 0);
strbuf_release(&referent);
strbuf_release(&head_referent);
string_list_clear(&refnames_to_check, 0);
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v5 3/8] refs/files: remove duplicate duplicates check
2025-03-27 11:13 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 2/8] refs: move duplicate refname update check to generic layer Karthik Nayak
@ 2025-03-27 11:13 ` Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 4/8] refs/reftable: extract code from the transaction preparation Karthik Nayak
` (5 subsequent siblings)
8 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-27 11:13 UTC (permalink / raw)
To: git; +Cc: jltobler, phillip.wood123, gitster, ps, Karthik Nayak
Within the files reference backend's transaction's 'finish' phase, a
verification step is currently performed wherein the refnames list is
sorted and examined for multiple updates targeting the same refname.
It has been observed that this verification is redundant, as an
identical check is already executed during the transaction's 'prepare'
stage. Since the refnames list remains unmodified following the
'prepare' stage, this secondary verification can be safely eliminated.
The duplicate check has been removed accordingly, and the
`ref_update_reject_duplicates()` function has been marked as static, as
its usage is now confined to 'refs.c'.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 9 +++++++--
refs/files-backend.c | 6 ------
refs/refs-internal.h | 8 --------
3 files changed, 7 insertions(+), 16 deletions(-)
diff --git a/refs.c b/refs.c
index 504bf2063e..61bed9672a 100644
--- a/refs.c
+++ b/refs.c
@@ -2303,8 +2303,13 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
return ret;
}
-int ref_update_reject_duplicates(struct string_list *refnames,
- struct strbuf *err)
+/*
+ * Write an error to `err` and return a nonzero value iff the same
+ * refname appears multiple times in `refnames`. `refnames` must be
+ * sorted on entry to this function.
+ */
+static int ref_update_reject_duplicates(struct string_list *refnames,
+ struct strbuf *err)
{
size_t i, n = refnames->nr;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 58f62ea8a3..ea023a59fc 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3016,12 +3016,6 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (transaction->state != REF_TRANSACTION_PREPARED)
BUG("commit called for transaction that is not prepared");
- string_list_sort(&transaction->refnames);
- if (ref_update_reject_duplicates(&transaction->refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
- }
-
/*
* It's really undefined to call this function in an active
* repository or when there are existing references: we are
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 92db793026..6d3770d0cc 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -142,14 +142,6 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
struct object_id *oid, struct strbuf *referent,
unsigned int *type, int *failure_errno);
-/*
- * Write an error to `err` and return a nonzero value iff the same
- * refname appears multiple times in `refnames`. `refnames` must be
- * sorted on entry to this function.
- */
-int ref_update_reject_duplicates(struct string_list *refnames,
- struct strbuf *err);
-
/*
* Add a ref_update with the specified properties to transaction, and
* return a pointer to the new object. This function does not verify
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v5 4/8] refs/reftable: extract code from the transaction preparation
2025-03-27 11:13 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Karthik Nayak
` (2 preceding siblings ...)
2025-03-27 11:13 ` [PATCH v5 3/8] refs/files: remove duplicate duplicates check Karthik Nayak
@ 2025-03-27 11:13 ` Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 5/8] refs: introduce enum-based transaction error types Karthik Nayak
` (4 subsequent siblings)
8 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-27 11:13 UTC (permalink / raw)
To: git; +Cc: jltobler, phillip.wood123, gitster, ps, Karthik Nayak
Extract the core logic for preparing individual reference updates from
`reftable_be_transaction_prepare()` into `prepare_single_update()`. This
dedicated function now handles all validation and preparation steps for
each reference update in the transaction, including object ID
verification, HEAD reference handling, and symref processing.
The refactoring consolidates all reference update validation into a
single logical block, which improves code maintainability and
readability. More importantly, this restructuring lays the groundwork
for implementing batched reference update support in the reftable
backend, which will be introduced in a followup commit.
No functional changes are included in this commit - it is purely a code
reorganization to support future enhancements.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs/reftable-backend.c | 463 +++++++++++++++++++++++++-----------------------
1 file changed, 237 insertions(+), 226 deletions(-)
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index a92c9a2f4f..786df11a03 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1069,6 +1069,239 @@ static int queue_transaction_update(struct reftable_ref_store *refs,
return 0;
}
+static int prepare_single_update(struct reftable_ref_store *refs,
+ struct reftable_transaction_data *tx_data,
+ struct ref_transaction *transaction,
+ struct reftable_backend *be,
+ struct ref_update *u,
+ struct string_list *refnames_to_check,
+ unsigned int head_type,
+ struct strbuf *head_referent,
+ struct strbuf *referent,
+ struct strbuf *err)
+{
+ struct object_id current_oid = {0};
+ const char *rewritten_ref;
+ int ret = 0;
+
+ /*
+ * There is no need to reload the respective backends here as
+ * we have already reloaded them when preparing the transaction
+ * update. And given that the stacks have been locked there
+ * shouldn't have been any concurrent modifications of the
+ * stack.
+ */
+ ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
+ if (ret)
+ return ret;
+
+ /* Verify that the new object ID is valid. */
+ if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
+ !(u->flags & REF_SKIP_OID_VERIFICATION) &&
+ !(u->flags & REF_LOG_ONLY)) {
+ struct object *o = parse_object(refs->base.repo, &u->new_oid);
+ if (!o) {
+ strbuf_addf(err,
+ _("trying to write ref '%s' with nonexistent object %s"),
+ u->refname, oid_to_hex(&u->new_oid));
+ return -1;
+ }
+
+ if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
+ strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
+ oid_to_hex(&u->new_oid), u->refname);
+ return -1;
+ }
+ }
+
+ /*
+ * When we update the reference that HEAD points to we enqueue
+ * a second log-only update for HEAD so that its reflog is
+ * updated accordingly.
+ */
+ if (head_type == REF_ISSYMREF &&
+ !(u->flags & REF_LOG_ONLY) &&
+ !(u->flags & REF_UPDATE_VIA_HEAD) &&
+ !strcmp(rewritten_ref, head_referent->buf)) {
+ /*
+ * First make sure that HEAD is not already in the
+ * transaction. This check is O(lg N) in the transaction
+ * size, but it happens at most once per transaction.
+ */
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
+ /* An entry already existed */
+ strbuf_addf(err,
+ _("multiple updates for 'HEAD' (including one "
+ "via its referent '%s') are not allowed"),
+ u->refname);
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ ref_transaction_add_update(
+ transaction, "HEAD",
+ u->flags | REF_LOG_ONLY | REF_NO_DEREF,
+ &u->new_oid, &u->old_oid, NULL, NULL, NULL,
+ u->msg);
+ }
+
+ ret = reftable_backend_read_ref(be, rewritten_ref,
+ ¤t_oid, referent, &u->type);
+ if (ret < 0)
+ return ret;
+ if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
+ /*
+ * The reference does not exist, and we either have no
+ * old object ID or expect the reference to not exist.
+ * We can thus skip below safety checks as well as the
+ * symref splitting. But we do want to verify that
+ * there is no conflicting reference here so that we
+ * can output a proper error message instead of failing
+ * at a later point.
+ */
+ string_list_append(refnames_to_check, u->refname);
+
+ /*
+ * There is no need to write the reference deletion
+ * when the reference in question doesn't exist.
+ */
+ if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
+ ret = queue_transaction_update(refs, tx_data, u,
+ ¤t_oid, err);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+ }
+ if (ret > 0) {
+ /* The reference does not exist, but we expected it to. */
+ strbuf_addf(err, _("cannot lock ref '%s': "
+
+
+ "unable to resolve reference '%s'"),
+ ref_update_original_update_refname(u), u->refname);
+ return -1;
+ }
+
+ if (u->type & REF_ISSYMREF) {
+ /*
+ * The reftable stack is locked at this point already,
+ * so it is safe to call `refs_resolve_ref_unsafe()`
+ * here without causing races.
+ */
+ const char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,
+ ¤t_oid, NULL);
+
+ if (u->flags & REF_NO_DEREF) {
+ if (u->flags & REF_HAVE_OLD && !resolved) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "error reading reference"), u->refname);
+ return -1;
+ }
+ } else {
+ struct ref_update *new_update;
+ int new_flags;
+
+ new_flags = u->flags;
+ if (!strcmp(rewritten_ref, "HEAD"))
+ new_flags |= REF_UPDATE_VIA_HEAD;
+
+ if (string_list_has_string(&transaction->refnames, referent->buf)) {
+ strbuf_addf(err,
+ _("multiple updates for '%s' (including one "
+ "via symref '%s') are not allowed"),
+ referent->buf, u->refname);
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ /*
+ * If we are updating a symref (eg. HEAD), we should also
+ * update the branch that the symref points to.
+ *
+ * This is generic functionality, and would be better
+ * done in refs.c, but the current implementation is
+ * intertwined with the locking in files-backend.c.
+ */
+ new_update = ref_transaction_add_update(
+ transaction, referent->buf, new_flags,
+ u->new_target ? NULL : &u->new_oid,
+ u->old_target ? NULL : &u->old_oid,
+ u->new_target, u->old_target,
+ u->committer_info, u->msg);
+
+ new_update->parent_update = u;
+
+ /*
+ * Change the symbolic ref update to log only. Also, it
+ * doesn't need to check its old OID value, as that will be
+ * done when new_update is processed.
+ */
+ u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
+ u->flags &= ~REF_HAVE_OLD;
+ }
+ }
+
+ /*
+ * Verify that the old object matches our expectations. Note
+ * that the error messages here do not make a lot of sense in
+ * the context of the reftable backend as we never lock
+ * individual refs. But the error messages match what the files
+ * backend returns, which keeps our tests happy.
+ */
+ if (u->old_target) {
+ if (!(u->type & REF_ISSYMREF)) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "expected symref with target '%s': "
+ "but is a regular ref"),
+ ref_update_original_update_refname(u),
+ u->old_target);
+ return -1;
+ }
+
+ if (ref_update_check_old_target(referent->buf, u, err)) {
+ return -1;
+ }
+ } else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
+ if (is_null_oid(&u->old_oid)) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "reference already exists"),
+ ref_update_original_update_refname(u));
+ return TRANSACTION_CREATE_EXISTS;
+ }
+ else if (is_null_oid(¤t_oid))
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "reference is missing but expected %s"),
+ ref_update_original_update_refname(u),
+ oid_to_hex(&u->old_oid));
+ else
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "is at %s but expected %s"),
+ ref_update_original_update_refname(u),
+ oid_to_hex(¤t_oid),
+ oid_to_hex(&u->old_oid));
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ /*
+ * If all of the following conditions are true:
+ *
+ * - We're not about to write a symref.
+ * - We're not about to write a log-only entry.
+ * - Old and new object ID are different.
+ *
+ * Then we're essentially doing a no-op update that can be
+ * skipped. This is not only for the sake of efficiency, but
+ * also skips writing unneeded reflog entries.
+ */
+ if ((u->type & REF_ISSYMREF) ||
+ (u->flags & REF_LOG_ONLY) ||
+ (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))
+ return queue_transaction_update(refs, tx_data, u,
+ ¤t_oid, err);
+
+ return 0;
+}
+
static int reftable_be_transaction_prepare(struct ref_store *ref_store,
struct ref_transaction *transaction,
struct strbuf *err)
@@ -1133,234 +1366,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
ret = 0;
for (i = 0; i < transaction->nr; i++) {
- struct ref_update *u = transaction->updates[i];
- struct object_id current_oid = {0};
- const char *rewritten_ref;
-
- /*
- * There is no need to reload the respective backends here as
- * we have already reloaded them when preparing the transaction
- * update. And given that the stacks have been locked there
- * shouldn't have been any concurrent modifications of the
- * stack.
- */
- ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
+ ret = prepare_single_update(refs, tx_data, transaction, be,
+ transaction->updates[i],
+ &refnames_to_check, head_type,
+ &head_referent, &referent, err);
if (ret)
goto done;
-
- /* Verify that the new object ID is valid. */
- if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
- !(u->flags & REF_SKIP_OID_VERIFICATION) &&
- !(u->flags & REF_LOG_ONLY)) {
- struct object *o = parse_object(refs->base.repo, &u->new_oid);
- if (!o) {
- strbuf_addf(err,
- _("trying to write ref '%s' with nonexistent object %s"),
- u->refname, oid_to_hex(&u->new_oid));
- ret = -1;
- goto done;
- }
-
- if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
- strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
- oid_to_hex(&u->new_oid), u->refname);
- ret = -1;
- goto done;
- }
- }
-
- /*
- * When we update the reference that HEAD points to we enqueue
- * a second log-only update for HEAD so that its reflog is
- * updated accordingly.
- */
- if (head_type == REF_ISSYMREF &&
- !(u->flags & REF_LOG_ONLY) &&
- !(u->flags & REF_UPDATE_VIA_HEAD) &&
- !strcmp(rewritten_ref, head_referent.buf)) {
- /*
- * First make sure that HEAD is not already in the
- * transaction. This check is O(lg N) in the transaction
- * size, but it happens at most once per transaction.
- */
- if (string_list_has_string(&transaction->refnames, "HEAD")) {
- /* An entry already existed */
- strbuf_addf(err,
- _("multiple updates for 'HEAD' (including one "
- "via its referent '%s') are not allowed"),
- u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
-
- ref_transaction_add_update(
- transaction, "HEAD",
- u->flags | REF_LOG_ONLY | REF_NO_DEREF,
- &u->new_oid, &u->old_oid, NULL, NULL, NULL,
- u->msg);
- }
-
- ret = reftable_backend_read_ref(be, rewritten_ref,
- ¤t_oid, &referent, &u->type);
- if (ret < 0)
- goto done;
- if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
- /*
- * The reference does not exist, and we either have no
- * old object ID or expect the reference to not exist.
- * We can thus skip below safety checks as well as the
- * symref splitting. But we do want to verify that
- * there is no conflicting reference here so that we
- * can output a proper error message instead of failing
- * at a later point.
- */
- string_list_append(&refnames_to_check, u->refname);
-
- /*
- * There is no need to write the reference deletion
- * when the reference in question doesn't exist.
- */
- if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
- ret = queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
- if (ret)
- goto done;
- }
-
- continue;
- }
- if (ret > 0) {
- /* The reference does not exist, but we expected it to. */
- strbuf_addf(err, _("cannot lock ref '%s': "
- "unable to resolve reference '%s'"),
- ref_update_original_update_refname(u), u->refname);
- ret = -1;
- goto done;
- }
-
- if (u->type & REF_ISSYMREF) {
- /*
- * The reftable stack is locked at this point already,
- * so it is safe to call `refs_resolve_ref_unsafe()`
- * here without causing races.
- */
- const char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,
- ¤t_oid, NULL);
-
- if (u->flags & REF_NO_DEREF) {
- if (u->flags & REF_HAVE_OLD && !resolved) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "error reading reference"), u->refname);
- ret = -1;
- goto done;
- }
- } else {
- struct ref_update *new_update;
- int new_flags;
-
- new_flags = u->flags;
- if (!strcmp(rewritten_ref, "HEAD"))
- new_flags |= REF_UPDATE_VIA_HEAD;
-
- if (string_list_has_string(&transaction->refnames, referent.buf)) {
- strbuf_addf(err,
- _("multiple updates for '%s' (including one "
- "via symref '%s') are not allowed"),
- referent.buf, u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
-
- /*
- * If we are updating a symref (eg. HEAD), we should also
- * update the branch that the symref points to.
- *
- * This is generic functionality, and would be better
- * done in refs.c, but the current implementation is
- * intertwined with the locking in files-backend.c.
- */
- new_update = ref_transaction_add_update(
- transaction, referent.buf, new_flags,
- u->new_target ? NULL : &u->new_oid,
- u->old_target ? NULL : &u->old_oid,
- u->new_target, u->old_target,
- u->committer_info, u->msg);
-
- new_update->parent_update = u;
-
- /*
- * Change the symbolic ref update to log only. Also, it
- * doesn't need to check its old OID value, as that will be
- * done when new_update is processed.
- */
- u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
- u->flags &= ~REF_HAVE_OLD;
- }
- }
-
- /*
- * Verify that the old object matches our expectations. Note
- * that the error messages here do not make a lot of sense in
- * the context of the reftable backend as we never lock
- * individual refs. But the error messages match what the files
- * backend returns, which keeps our tests happy.
- */
- if (u->old_target) {
- if (!(u->type & REF_ISSYMREF)) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "expected symref with target '%s': "
- "but is a regular ref"),
- ref_update_original_update_refname(u),
- u->old_target);
- ret = -1;
- goto done;
- }
-
- if (ref_update_check_old_target(referent.buf, u, err)) {
- ret = -1;
- goto done;
- }
- } else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
- ret = TRANSACTION_NAME_CONFLICT;
- if (is_null_oid(&u->old_oid)) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "reference already exists"),
- ref_update_original_update_refname(u));
- ret = TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(¤t_oid))
- strbuf_addf(err, _("cannot lock ref '%s': "
- "reference is missing but expected %s"),
- ref_update_original_update_refname(u),
- oid_to_hex(&u->old_oid));
- else
- strbuf_addf(err, _("cannot lock ref '%s': "
- "is at %s but expected %s"),
- ref_update_original_update_refname(u),
- oid_to_hex(¤t_oid),
- oid_to_hex(&u->old_oid));
- goto done;
- }
-
- /*
- * If all of the following conditions are true:
- *
- * - We're not about to write a symref.
- * - We're not about to write a log-only entry.
- * - Old and new object ID are different.
- *
- * Then we're essentially doing a no-op update that can be
- * skipped. This is not only for the sake of efficiency, but
- * also skips writing unneeded reflog entries.
- */
- if ((u->type & REF_ISSYMREF) ||
- (u->flags & REF_LOG_ONLY) ||
- (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid))) {
- ret = queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
- if (ret)
- goto done;
- }
}
ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v5 5/8] refs: introduce enum-based transaction error types
2025-03-27 11:13 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Karthik Nayak
` (3 preceding siblings ...)
2025-03-27 11:13 ` [PATCH v5 4/8] refs/reftable: extract code from the transaction preparation Karthik Nayak
@ 2025-03-27 11:13 ` Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 6/8] refs: implement batch reference update support Karthik Nayak
` (3 subsequent siblings)
8 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-27 11:13 UTC (permalink / raw)
To: git; +Cc: jltobler, phillip.wood123, gitster, ps, Karthik Nayak
Replace preprocessor-defined transaction errors with a strongly-typed
enum `ref_transaction_error`. This change:
- Improves type safety and function signature clarity.
- Makes error handling more explicit and discoverable.
- Maintains existing error cases, while adding new error cases for
common scenarios.
This refactoring paves the way for more comprehensive error handling
which we will utilize in the upcoming commits to add batch reference
update support.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
builtin/fetch.c | 2 +-
refs.c | 49 ++++++------
refs.h | 48 +++++++-----
refs/files-backend.c | 202 ++++++++++++++++++++++++------------------------
refs/packed-backend.c | 23 +++---
refs/refs-internal.h | 5 +-
refs/reftable-backend.c | 64 +++++++--------
7 files changed, 207 insertions(+), 186 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 95fd0018b9..7615c17faf 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -687,7 +687,7 @@ static int s_update_ref(const char *action,
switch (ref_transaction_commit(our_transaction, &err)) {
case 0:
break;
- case TRANSACTION_NAME_CONFLICT:
+ case REF_TRANSACTION_ERROR_NAME_CONFLICT:
ret = STORE_REF_ERROR_DF_CONFLICT;
goto out;
default:
diff --git a/refs.c b/refs.c
index 61bed9672a..3d0b53d56e 100644
--- a/refs.c
+++ b/refs.c
@@ -2271,7 +2271,7 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
REF_NO_DEREF, logmsg, &err))
goto error_return;
prepret = ref_transaction_prepare(transaction, &err);
- if (prepret && prepret != TRANSACTION_CREATE_EXISTS)
+ if (prepret && prepret != REF_TRANSACTION_ERROR_CREATE_EXISTS)
goto error_return;
} else {
if (ref_transaction_update(transaction, ref, NULL, NULL,
@@ -2289,7 +2289,7 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
}
}
- if (prepret == TRANSACTION_CREATE_EXISTS)
+ if (prepret == REF_TRANSACTION_ERROR_CREATE_EXISTS)
goto cleanup;
if (ref_transaction_commit(transaction, &err))
@@ -2425,7 +2425,7 @@ int ref_transaction_prepare(struct ref_transaction *transaction,
string_list_sort(&transaction->refnames);
if (ref_update_reject_duplicates(&transaction->refnames, err))
- return TRANSACTION_GENERIC_ERROR;
+ return REF_TRANSACTION_ERROR_GENERIC;
ret = refs->be->transaction_prepare(refs, transaction, err);
if (ret)
@@ -2497,19 +2497,19 @@ int ref_transaction_commit(struct ref_transaction *transaction,
return ret;
}
-int refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *refnames,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err)
+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err)
{
struct strbuf dirname = STRBUF_INIT;
struct strbuf referent = STRBUF_INIT;
struct string_list_item *item;
struct ref_iterator *iter = NULL;
struct strset dirnames;
- int ret = -1;
+ int ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
/*
* For the sake of comments in this function, suppose that
@@ -2625,12 +2625,13 @@ int refs_verify_refnames_available(struct ref_store *refs,
return ret;
}
-int refs_verify_refname_available(struct ref_store *refs,
- const char *refname,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err)
+enum ref_transaction_error refs_verify_refname_available(
+ struct ref_store *refs,
+ const char *refname,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err)
{
struct string_list_item item = { .string = (char *) refname };
struct string_list refnames = {
@@ -2818,8 +2819,9 @@ int ref_update_has_null_new_value(struct ref_update *update)
return !update->new_target && is_null_oid(&update->new_oid);
}
-int ref_update_check_old_target(const char *referent, struct ref_update *update,
- struct strbuf *err)
+enum ref_transaction_error ref_update_check_old_target(const char *referent,
+ struct ref_update *update,
+ struct strbuf *err)
{
if (!update->old_target)
BUG("called without old_target set");
@@ -2827,17 +2829,18 @@ int ref_update_check_old_target(const char *referent, struct ref_update *update,
if (!strcmp(referent, update->old_target))
return 0;
- if (!strcmp(referent, ""))
+ if (!strcmp(referent, "")) {
strbuf_addf(err, "verifying symref target: '%s': "
"reference is missing but expected %s",
ref_update_original_update_refname(update),
update->old_target);
- else
- strbuf_addf(err, "verifying symref target: '%s': "
- "is at %s but expected %s",
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ }
+
+ strbuf_addf(err, "verifying symref target: '%s': is at %s but expected %s",
ref_update_original_update_refname(update),
referent, update->old_target);
- return -1;
+ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
}
struct migration_data {
diff --git a/refs.h b/refs.h
index 240e2d8537..f009cdae7d 100644
--- a/refs.h
+++ b/refs.h
@@ -16,6 +16,23 @@ struct worktree;
enum ref_storage_format ref_storage_format_by_name(const char *name);
const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);
+enum ref_transaction_error {
+ /* Default error code */
+ REF_TRANSACTION_ERROR_GENERIC = -1,
+ /* Ref name conflict like A vs A/B */
+ REF_TRANSACTION_ERROR_NAME_CONFLICT = -2,
+ /* Ref to be created already exists */
+ REF_TRANSACTION_ERROR_CREATE_EXISTS = -3,
+ /* ref expected but doesn't exist */
+ REF_TRANSACTION_ERROR_NONEXISTENT_REF = -4,
+ /* Provided old_oid or old_target of reference doesn't match actual */
+ REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE = -5,
+ /* Provided new_oid or new_target is invalid */
+ REF_TRANSACTION_ERROR_INVALID_NEW_VALUE = -6,
+ /* Expected ref to be symref, but is a regular ref */
+ REF_TRANSACTION_ERROR_EXPECTED_SYMREF = -7,
+};
+
/*
* Resolve a reference, recursively following symbolic references.
*
@@ -117,24 +134,24 @@ int refs_read_symbolic_ref(struct ref_store *ref_store, const char *refname,
*
* extras and skip must be sorted.
*/
-int refs_verify_refname_available(struct ref_store *refs,
- const char *refname,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
+enum ref_transaction_error refs_verify_refname_available(struct ref_store *refs,
+ const char *refname,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err);
/*
* Same as `refs_verify_refname_available()`, but checking for a list of
* refnames instead of only a single item. This is more efficient in the case
* where one needs to check multiple refnames.
*/
-int refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *refnames,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err);
int refs_ref_exists(struct ref_store *refs, const char *refname);
@@ -830,13 +847,6 @@ int ref_transaction_verify(struct ref_transaction *transaction,
unsigned int flags,
struct strbuf *err);
-/* Naming conflict (for example, the ref names A and A/B conflict). */
-#define TRANSACTION_NAME_CONFLICT -1
-/* When only creation was requested, but the ref already exists. */
-#define TRANSACTION_CREATE_EXISTS -2
-/* All other errors. */
-#define TRANSACTION_GENERIC_ERROR -3
-
/*
* Perform the preparatory stages of committing `transaction`. Acquire
* any needed locks, check preconditions, etc.; basically, do as much
diff --git a/refs/files-backend.c b/refs/files-backend.c
index ea023a59fc..4f27f7652c 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -663,7 +663,7 @@ static void unlock_ref(struct ref_lock *lock)
* broken, lock the reference anyway but clear old_oid.
*
* Return 0 on success. On failure, write an error message to err and
- * return TRANSACTION_NAME_CONFLICT or TRANSACTION_GENERIC_ERROR.
+ * return REF_TRANSACTION_ERROR_NAME_CONFLICT or REF_TRANSACTION_ERROR_GENERIC.
*
* Implementation note: This function is basically
*
@@ -676,19 +676,20 @@ static void unlock_ref(struct ref_lock *lock)
* avoided, namely if we were successfully able to read the ref
* - Generate informative error messages in the case of failure
*/
-static int lock_raw_ref(struct files_ref_store *refs,
- const char *refname, int mustexist,
- struct string_list *refnames_to_check,
- const struct string_list *extras,
- struct ref_lock **lock_p,
- struct strbuf *referent,
- unsigned int *type,
- struct strbuf *err)
-{
+static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
+ const char *refname,
+ int mustexist,
+ struct string_list *refnames_to_check,
+ const struct string_list *extras,
+ struct ref_lock **lock_p,
+ struct strbuf *referent,
+ unsigned int *type,
+ struct strbuf *err)
+{
+ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
struct ref_lock *lock;
struct strbuf ref_file = STRBUF_INIT;
int attempts_remaining = 3;
- int ret = TRANSACTION_GENERIC_ERROR;
int failure_errno;
assert(err);
@@ -728,13 +729,14 @@ static int lock_raw_ref(struct files_ref_store *refs,
strbuf_reset(err);
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
} else {
/*
* The error message set by
* refs_verify_refname_available() is
* OK.
*/
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
} else {
/*
@@ -788,6 +790,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
/* Garden variety missing reference. */
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error_return;
} else {
/*
@@ -820,6 +823,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
/* Garden variety missing reference. */
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error_return;
} else if (remove_dir_recursively(&ref_file,
REMOVE_DIR_EMPTY_ONLY)) {
@@ -830,7 +834,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
* The error message set by
* verify_refname_available() is OK.
*/
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto error_return;
} else {
/*
@@ -1517,10 +1521,11 @@ static int rename_tmp_log(struct files_ref_store *refs, const char *newrefname)
return ret;
}
-static int write_ref_to_lockfile(struct files_ref_store *refs,
- struct ref_lock *lock,
- const struct object_id *oid,
- int skip_oid_verification, struct strbuf *err);
+static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs,
+ struct ref_lock *lock,
+ const struct object_id *oid,
+ int skip_oid_verification,
+ struct strbuf *err);
static int commit_ref_update(struct files_ref_store *refs,
struct ref_lock *lock,
const struct object_id *oid, const char *logmsg,
@@ -1926,10 +1931,11 @@ static int files_log_ref_write(struct files_ref_store *refs,
* Write oid into the open lockfile, then close the lockfile. On
* errors, rollback the lockfile, fill in *err and return -1.
*/
-static int write_ref_to_lockfile(struct files_ref_store *refs,
- struct ref_lock *lock,
- const struct object_id *oid,
- int skip_oid_verification, struct strbuf *err)
+static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs,
+ struct ref_lock *lock,
+ const struct object_id *oid,
+ int skip_oid_verification,
+ struct strbuf *err)
{
static char term = '\n';
struct object *o;
@@ -1943,7 +1949,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
"trying to write ref '%s' with nonexistent object %s",
lock->ref_name, oid_to_hex(oid));
unlock_ref(lock);
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
if (o->type != OBJ_COMMIT && is_branch(lock->ref_name)) {
strbuf_addf(
@@ -1951,7 +1957,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
"trying to write non-commit object %s to branch '%s'",
oid_to_hex(oid), lock->ref_name);
unlock_ref(lock);
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
}
fd = get_lock_file_fd(&lock->lk);
@@ -1962,7 +1968,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
strbuf_addf(err,
"couldn't write '%s'", get_lock_file_path(&lock->lk));
unlock_ref(lock);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
return 0;
}
@@ -2376,9 +2382,10 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st
* If update is a direct update of head_ref (the reference pointed to
* by HEAD), then add an extra REF_LOG_ONLY update for HEAD.
*/
-static int split_head_update(struct ref_update *update,
- struct ref_transaction *transaction,
- const char *head_ref, struct strbuf *err)
+static enum ref_transaction_error split_head_update(struct ref_update *update,
+ struct ref_transaction *transaction,
+ const char *head_ref,
+ struct strbuf *err)
{
struct ref_update *new_update;
@@ -2402,7 +2409,7 @@ static int split_head_update(struct ref_update *update,
"multiple updates for 'HEAD' (including one "
"via its referent '%s') are not allowed",
update->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
new_update = ref_transaction_add_update(
@@ -2430,10 +2437,10 @@ static int split_head_update(struct ref_update *update,
* Note that the new update will itself be subject to splitting when
* the iteration gets to it.
*/
-static int split_symref_update(struct ref_update *update,
- const char *referent,
- struct ref_transaction *transaction,
- struct strbuf *err)
+static enum ref_transaction_error split_symref_update(struct ref_update *update,
+ const char *referent,
+ struct ref_transaction *transaction,
+ struct strbuf *err)
{
struct ref_update *new_update;
unsigned int new_flags;
@@ -2450,7 +2457,7 @@ static int split_symref_update(struct ref_update *update,
"multiple updates for '%s' (including one "
"via symref '%s') are not allowed",
referent, update->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
new_flags = update->flags;
@@ -2491,11 +2498,10 @@ static int split_symref_update(struct ref_update *update,
* everything is OK, return 0; otherwise, write an error message to
* err and return -1.
*/
-static int check_old_oid(struct ref_update *update, struct object_id *oid,
- struct strbuf *err)
+static enum ref_transaction_error check_old_oid(struct ref_update *update,
+ struct object_id *oid,
+ struct strbuf *err)
{
- int ret = TRANSACTION_GENERIC_ERROR;
-
if (!(update->flags & REF_HAVE_OLD) ||
oideq(oid, &update->old_oid))
return 0;
@@ -2504,21 +2510,20 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid,
strbuf_addf(err, "cannot lock ref '%s': "
"reference already exists",
ref_update_original_update_refname(update));
- ret = TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(oid))
+ return REF_TRANSACTION_ERROR_CREATE_EXISTS;
+ } else if (is_null_oid(oid)) {
strbuf_addf(err, "cannot lock ref '%s': "
"reference is missing but expected %s",
ref_update_original_update_refname(update),
oid_to_hex(&update->old_oid));
- else
- strbuf_addf(err, "cannot lock ref '%s': "
- "is at %s but expected %s",
- ref_update_original_update_refname(update),
- oid_to_hex(oid),
- oid_to_hex(&update->old_oid));
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ }
- return ret;
+ strbuf_addf(err, "cannot lock ref '%s': is at %s but expected %s",
+ ref_update_original_update_refname(update), oid_to_hex(oid),
+ oid_to_hex(&update->old_oid));
+
+ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
}
struct files_transaction_backend_data {
@@ -2540,17 +2545,17 @@ struct files_transaction_backend_data {
* - If it is an update of head_ref, add a corresponding REF_LOG_ONLY
* update of HEAD.
*/
-static int lock_ref_for_update(struct files_ref_store *refs,
- struct ref_update *update,
- struct ref_transaction *transaction,
- const char *head_ref,
- struct string_list *refnames_to_check,
- struct strbuf *err)
+static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *refs,
+ struct ref_update *update,
+ struct ref_transaction *transaction,
+ const char *head_ref,
+ struct string_list *refnames_to_check,
+ struct strbuf *err)
{
struct strbuf referent = STRBUF_INIT;
int mustexist = ref_update_expects_existing_old_ref(update);
struct files_transaction_backend_data *backend_data;
- int ret = 0;
+ enum ref_transaction_error ret = 0;
struct ref_lock *lock;
files_assert_main_repository(refs, "lock_ref_for_update");
@@ -2602,22 +2607,17 @@ static int lock_ref_for_update(struct files_ref_store *refs,
strbuf_addf(err, "cannot lock ref '%s': "
"error reading reference",
ref_update_original_update_refname(update));
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
}
- if (update->old_target) {
- if (ref_update_check_old_target(referent.buf, update, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto out;
- }
- } else {
+ if (update->old_target)
+ ret = ref_update_check_old_target(referent.buf, update, err);
+ else
ret = check_old_oid(update, &lock->old_oid, err);
- if (ret) {
- goto out;
- }
- }
+ if (ret)
+ goto out;
} else {
/*
* Create a new update for the reference this
@@ -2644,7 +2644,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
"but is a regular ref"),
ref_update_original_update_refname(update),
update->old_target);
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_EXPECTED_SYMREF;
goto out;
} else {
ret = check_old_oid(update, &lock->old_oid, err);
@@ -2668,14 +2668,14 @@ static int lock_ref_for_update(struct files_ref_store *refs,
if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
if (create_symref_lock(lock, update->new_target, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
if (close_ref_gently(lock)) {
strbuf_addf(err, "couldn't close '%s.lock'",
update->refname);
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
@@ -2693,25 +2693,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
* The reference already has the desired
* value, so we don't need to write it.
*/
- } else if (write_ref_to_lockfile(
- refs, lock, &update->new_oid,
- update->flags & REF_SKIP_OID_VERIFICATION,
- err)) {
- char *write_err = strbuf_detach(err, NULL);
-
- /*
- * The lock was freed upon failure of
- * write_ref_to_lockfile():
- */
- update->backend_data = NULL;
- strbuf_addf(err,
- "cannot update ref '%s': %s",
- update->refname, write_err);
- free(write_err);
- ret = TRANSACTION_GENERIC_ERROR;
- goto out;
} else {
- update->flags |= REF_NEEDS_COMMIT;
+ ret = write_ref_to_lockfile(
+ refs, lock, &update->new_oid,
+ update->flags & REF_SKIP_OID_VERIFICATION,
+ err);
+ if (ret) {
+ char *write_err = strbuf_detach(err, NULL);
+
+ /*
+ * The lock was freed upon failure of
+ * write_ref_to_lockfile():
+ */
+ update->backend_data = NULL;
+ strbuf_addf(err,
+ "cannot update ref '%s': %s",
+ update->refname, write_err);
+ free(write_err);
+ goto out;
+ } else {
+ update->flags |= REF_NEEDS_COMMIT;
+ }
}
}
if (!(update->flags & REF_NEEDS_COMMIT)) {
@@ -2723,7 +2725,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
if (close_ref_gently(lock)) {
strbuf_addf(err, "couldn't close '%s.lock'",
update->refname);
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
}
@@ -2865,7 +2867,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
refs->packed_ref_store,
transaction->flags, err);
if (!packed_transaction) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
@@ -2897,13 +2899,13 @@ static int files_transaction_prepare(struct ref_store *ref_store,
*/
if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
&transaction->refnames, NULL, 0, err)) {
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
}
if (packed_transaction) {
if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
backend_data->packed_refs_locked = 1;
@@ -2934,7 +2936,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
*/
backend_data->packed_transaction = NULL;
if (ref_transaction_abort(packed_transaction, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3035,7 +3037,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
packed_transaction = ref_store_transaction_begin(refs->packed_ref_store,
transaction->flags, err);
if (!packed_transaction) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
@@ -3058,7 +3060,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (!loose_transaction) {
loose_transaction = ref_store_transaction_begin(&refs->base, 0, err);
if (!loose_transaction) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3083,19 +3085,19 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
}
if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
if (refs_verify_refnames_available(&refs->base, &refnames_to_check,
&affected_refnames, NULL, 1, err)) {
packed_refs_unlock(refs->packed_ref_store);
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
}
if (ref_transaction_commit(packed_transaction, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
packed_refs_unlock(refs->packed_ref_store);
@@ -3103,7 +3105,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (loose_transaction) {
if (ref_transaction_prepare(loose_transaction, err) ||
ref_transaction_commit(loose_transaction, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3152,7 +3154,7 @@ static int files_transaction_finish(struct ref_store *ref_store,
if (update->flags & REF_NEEDS_COMMIT ||
update->flags & REF_LOG_ONLY) {
if (parse_and_write_reflog(refs, update, lock, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3171,7 +3173,7 @@ static int files_transaction_finish(struct ref_store *ref_store,
strbuf_addf(err, "couldn't set '%s'", lock->ref_name);
unlock_ref(lock);
update->backend_data = NULL;
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3227,7 +3229,7 @@ static int files_transaction_finish(struct ref_store *ref_store,
strbuf_reset(&sb);
files_ref_path(refs, &sb, lock->ref_name);
if (unlink_or_msg(sb.buf, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 19220d2e99..d90bd815a3 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1326,10 +1326,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
* The packfile must be locked before calling this function and will
* remain locked when it is done.
*/
-static int write_with_updates(struct packed_ref_store *refs,
- struct string_list *updates,
- struct strbuf *err)
+static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,
+ struct string_list *updates,
+ struct strbuf *err)
{
+ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
struct ref_iterator *iter = NULL;
size_t i;
int ok;
@@ -1353,7 +1354,7 @@ static int write_with_updates(struct packed_ref_store *refs,
strbuf_addf(err, "unable to create file %s: %s",
sb.buf, strerror(errno));
strbuf_release(&sb);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
strbuf_release(&sb);
@@ -1409,6 +1410,7 @@ static int write_with_updates(struct packed_ref_store *refs,
strbuf_addf(err, "cannot update ref '%s': "
"reference already exists",
update->refname);
+ ret = REF_TRANSACTION_ERROR_CREATE_EXISTS;
goto error;
} else if (!oideq(&update->old_oid, iter->oid)) {
strbuf_addf(err, "cannot update ref '%s': "
@@ -1416,6 +1418,7 @@ static int write_with_updates(struct packed_ref_store *refs,
update->refname,
oid_to_hex(iter->oid),
oid_to_hex(&update->old_oid));
+ ret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
goto error;
}
}
@@ -1452,6 +1455,7 @@ static int write_with_updates(struct packed_ref_store *refs,
"reference is missing but expected %s",
update->refname,
oid_to_hex(&update->old_oid));
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error;
}
}
@@ -1509,7 +1513,7 @@ static int write_with_updates(struct packed_ref_store *refs,
strerror(errno));
strbuf_release(&sb);
delete_tempfile(&refs->tempfile);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
return 0;
@@ -1521,7 +1525,7 @@ static int write_with_updates(struct packed_ref_store *refs,
error:
ref_iterator_free(iter);
delete_tempfile(&refs->tempfile);
- return -1;
+ return ret;
}
int is_packed_transaction_needed(struct ref_store *ref_store,
@@ -1654,7 +1658,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
"ref_transaction_prepare");
struct packed_transaction_backend_data *data;
- int ret = TRANSACTION_GENERIC_ERROR;
+ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
/*
* Note that we *don't* skip transactions with zero updates,
@@ -1675,7 +1679,8 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
data->own_lock = 1;
}
- if (write_with_updates(refs, &transaction->refnames, err))
+ ret = write_with_updates(refs, &transaction->refnames, err);
+ if (ret)
goto failure;
transaction->state = REF_TRANSACTION_PREPARED;
@@ -1707,7 +1712,7 @@ static int packed_transaction_finish(struct ref_store *ref_store,
ref_store,
REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
"ref_transaction_finish");
- int ret = TRANSACTION_GENERIC_ERROR;
+ int ret = REF_TRANSACTION_ERROR_GENERIC;
char *packed_refs_path;
clear_snapshot(refs);
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 6d3770d0cc..3f1d19abd9 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -770,8 +770,9 @@ int ref_update_has_null_new_value(struct ref_update *update);
* If everything is OK, return 0; otherwise, write an error message to
* err and return -1.
*/
-int ref_update_check_old_target(const char *referent, struct ref_update *update,
- struct strbuf *err);
+enum ref_transaction_error ref_update_check_old_target(const char *referent,
+ struct ref_update *update,
+ struct strbuf *err);
/*
* Check if the ref must exist, this means that the old_oid or
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 786df11a03..bd6b042103 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1069,20 +1069,20 @@ static int queue_transaction_update(struct reftable_ref_store *refs,
return 0;
}
-static int prepare_single_update(struct reftable_ref_store *refs,
- struct reftable_transaction_data *tx_data,
- struct ref_transaction *transaction,
- struct reftable_backend *be,
- struct ref_update *u,
- struct string_list *refnames_to_check,
- unsigned int head_type,
- struct strbuf *head_referent,
- struct strbuf *referent,
- struct strbuf *err)
+static enum ref_transaction_error prepare_single_update(struct reftable_ref_store *refs,
+ struct reftable_transaction_data *tx_data,
+ struct ref_transaction *transaction,
+ struct reftable_backend *be,
+ struct ref_update *u,
+ struct string_list *refnames_to_check,
+ unsigned int head_type,
+ struct strbuf *head_referent,
+ struct strbuf *referent,
+ struct strbuf *err)
{
+ enum ref_transaction_error ret = 0;
struct object_id current_oid = {0};
const char *rewritten_ref;
- int ret = 0;
/*
* There is no need to reload the respective backends here as
@@ -1093,7 +1093,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
*/
ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
if (ret)
- return ret;
+ return REF_TRANSACTION_ERROR_GENERIC;
/* Verify that the new object ID is valid. */
if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
@@ -1104,13 +1104,13 @@ static int prepare_single_update(struct reftable_ref_store *refs,
strbuf_addf(err,
_("trying to write ref '%s' with nonexistent object %s"),
u->refname, oid_to_hex(&u->new_oid));
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
oid_to_hex(&u->new_oid), u->refname);
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
}
@@ -1134,7 +1134,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
_("multiple updates for 'HEAD' (including one "
"via its referent '%s') are not allowed"),
u->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
ref_transaction_add_update(
@@ -1147,7 +1147,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
ret = reftable_backend_read_ref(be, rewritten_ref,
¤t_oid, referent, &u->type);
if (ret < 0)
- return ret;
+ return REF_TRANSACTION_ERROR_GENERIC;
if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
/*
* The reference does not exist, and we either have no
@@ -1168,7 +1168,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
ret = queue_transaction_update(refs, tx_data, u,
¤t_oid, err);
if (ret)
- return ret;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
return 0;
@@ -1180,7 +1180,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
"unable to resolve reference '%s'"),
ref_update_original_update_refname(u), u->refname);
- return -1;
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
}
if (u->type & REF_ISSYMREF) {
@@ -1196,7 +1196,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
if (u->flags & REF_HAVE_OLD && !resolved) {
strbuf_addf(err, _("cannot lock ref '%s': "
"error reading reference"), u->refname);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
} else {
struct ref_update *new_update;
@@ -1211,7 +1211,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
_("multiple updates for '%s' (including one "
"via symref '%s') are not allowed"),
referent->buf, u->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
/*
@@ -1255,31 +1255,32 @@ static int prepare_single_update(struct reftable_ref_store *refs,
"but is a regular ref"),
ref_update_original_update_refname(u),
u->old_target);
- return -1;
+ return REF_TRANSACTION_ERROR_EXPECTED_SYMREF;
}
- if (ref_update_check_old_target(referent->buf, u, err)) {
- return -1;
- }
+ ret = ref_update_check_old_target(referent->buf, u, err);
+ if (ret)
+ return ret;
} else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
if (is_null_oid(&u->old_oid)) {
strbuf_addf(err, _("cannot lock ref '%s': "
"reference already exists"),
ref_update_original_update_refname(u));
- return TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(¤t_oid))
+ return REF_TRANSACTION_ERROR_CREATE_EXISTS;
+ } else if (is_null_oid(¤t_oid)) {
strbuf_addf(err, _("cannot lock ref '%s': "
"reference is missing but expected %s"),
ref_update_original_update_refname(u),
oid_to_hex(&u->old_oid));
- else
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ } else {
strbuf_addf(err, _("cannot lock ref '%s': "
"is at %s but expected %s"),
ref_update_original_update_refname(u),
oid_to_hex(¤t_oid),
oid_to_hex(&u->old_oid));
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
+ }
}
/*
@@ -1296,8 +1297,8 @@ static int prepare_single_update(struct reftable_ref_store *refs,
if ((u->type & REF_ISSYMREF) ||
(u->flags & REF_LOG_ONLY) ||
(u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))
- return queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
+ if (queue_transaction_update(refs, tx_data, u, ¤t_oid, err))
+ return REF_TRANSACTION_ERROR_GENERIC;
return 0;
}
@@ -1385,7 +1386,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->state = REF_TRANSACTION_PREPARED;
done:
- assert(ret != REFTABLE_API_ERROR);
if (ret < 0) {
free_transaction_data(tx_data);
transaction->state = REF_TRANSACTION_CLOSED;
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v5 6/8] refs: implement batch reference update support
2025-03-27 11:13 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Karthik Nayak
` (4 preceding siblings ...)
2025-03-27 11:13 ` [PATCH v5 5/8] refs: introduce enum-based transaction error types Karthik Nayak
@ 2025-03-27 11:13 ` Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 7/8] refs: support rejection in batch updates during F/D checks Karthik Nayak
` (2 subsequent siblings)
8 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-27 11:13 UTC (permalink / raw)
To: git; +Cc: jltobler, phillip.wood123, gitster, ps, Karthik Nayak
Git supports making reference updates with or without transactions.
Updates with transactions are generally better optimized. But
transactions are all or nothing. This means, if a user wants to batch
updates to take advantage of the optimizations without the hard
requirement that all updates must succeed, there is no way currently to
do so. Particularly with the reftable backend where batching multiple
reference updates is more efficient than performing them sequentially.
Introduce batched update support with a new flag,
'REF_TRANSACTION_ALLOW_FAILURE'. Batched updates while different from
transactions, use the transaction infrastructure under the hood. When
enabled, this flag allows individual reference updates that would
typically cause the entire transaction to fail due to non-system-related
errors to be marked as rejected while permitting other updates to
proceed. System errors referred by 'REF_TRANSACTION_ERROR_GENERIC'
continue to result in the entire transaction failing. This approach
enhances flexibility while preserving transactional integrity where
necessary.
The implementation introduces several key components:
- Add 'rejection_err' field to struct `ref_update` to track failed
updates with failure reason.
- Add a new struct `ref_transaction_rejections` and a field within
`ref_transaction` to this struct to allow quick iteration over
rejected updates.
- Modify reference backends (files, packed, reftable) to handle
partial transactions by using `ref_transaction_set_rejected()`
instead of failing the entire transaction when
`REF_TRANSACTION_ALLOW_FAILURE` is set.
- Add `ref_transaction_for_each_rejected_update()` to let callers
examine which updates were rejected and why.
This foundational change enables batched update support throughout the
reference subsystem. A following commit will expose this capability to
users by adding a `--batch-updates` flag to 'git-update-ref(1)',
providing both a user-facing feature and a testable implementation.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++
refs.h | 22 ++++++++++++++++++
refs/files-backend.c | 12 +++++++++-
refs/packed-backend.c | 27 ++++++++++++++++++++--
refs/refs-internal.h | 26 +++++++++++++++++++++
refs/reftable-backend.c | 12 +++++++++-
6 files changed, 156 insertions(+), 4 deletions(-)
diff --git a/refs.c b/refs.c
index 3d0b53d56e..b34ec198f5 100644
--- a/refs.c
+++ b/refs.c
@@ -1176,6 +1176,10 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,
tr->ref_store = refs;
tr->flags = flags;
string_list_init_dup(&tr->refnames);
+
+ if (flags & REF_TRANSACTION_ALLOW_FAILURE)
+ CALLOC_ARRAY(tr->rejections, 1);
+
return tr;
}
@@ -1206,11 +1210,45 @@ void ref_transaction_free(struct ref_transaction *transaction)
free((char *)transaction->updates[i]->old_target);
free(transaction->updates[i]);
}
+
+ if (transaction->rejections)
+ free(transaction->rejections->update_indices);
+ free(transaction->rejections);
+
string_list_clear(&transaction->refnames, 0);
free(transaction->updates);
free(transaction);
}
+int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,
+ size_t update_idx,
+ enum ref_transaction_error err)
+{
+ if (update_idx >= transaction->nr)
+ BUG("trying to set rejection on invalid update index");
+
+ if (!(transaction->flags & REF_TRANSACTION_ALLOW_FAILURE))
+ return 0;
+
+ if (!transaction->rejections)
+ BUG("transaction not inititalized with failure support");
+
+ /*
+ * Don't accept generic errors, since these errors are not user
+ * input related.
+ */
+ if (err == REF_TRANSACTION_ERROR_GENERIC)
+ return 0;
+
+ transaction->updates[update_idx]->rejection_err = err;
+ ALLOC_GROW(transaction->rejections->update_indices,
+ transaction->rejections->nr + 1,
+ transaction->rejections->alloc);
+ transaction->rejections->update_indices[transaction->rejections->nr++] = update_idx;
+
+ return 1;
+}
+
struct ref_update *ref_transaction_add_update(
struct ref_transaction *transaction,
const char *refname, unsigned int flags,
@@ -1236,6 +1274,7 @@ struct ref_update *ref_transaction_add_update(
transaction->updates[transaction->nr++] = update;
update->flags = flags;
+ update->rejection_err = 0;
update->new_target = xstrdup_or_null(new_target);
update->old_target = xstrdup_or_null(old_target);
@@ -2728,6 +2767,28 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
}
}
+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
+ ref_transaction_for_each_rejected_update_fn cb,
+ void *cb_data)
+{
+ if (!transaction->rejections)
+ return;
+
+ for (size_t i = 0; i < transaction->rejections->nr; i++) {
+ size_t update_index = transaction->rejections->update_indices[i];
+ struct ref_update *update = transaction->updates[update_index];
+
+ if (!update->rejection_err)
+ continue;
+
+ cb(update->refname,
+ (update->flags & REF_HAVE_OLD) ? &update->old_oid : NULL,
+ (update->flags & REF_HAVE_NEW) ? &update->new_oid : NULL,
+ update->old_target, update->new_target,
+ update->rejection_err, cb_data);
+ }
+}
+
int refs_delete_refs(struct ref_store *refs, const char *logmsg,
struct string_list *refnames, unsigned int flags)
{
diff --git a/refs.h b/refs.h
index f009cdae7d..c48c800478 100644
--- a/refs.h
+++ b/refs.h
@@ -667,6 +667,13 @@ enum ref_transaction_flag {
* either be absent or null_oid.
*/
REF_TRANSACTION_FLAG_INITIAL = (1 << 0),
+
+ /*
+ * The transaction mechanism by default fails all updates if any conflict
+ * is detected. This flag allows transactions to partially apply updates
+ * while rejecting updates which do not match the expected state.
+ */
+ REF_TRANSACTION_ALLOW_FAILURE = (1 << 1),
};
/*
@@ -897,6 +904,21 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
ref_transaction_for_each_queued_update_fn cb,
void *cb_data);
+/*
+ * Execute the given callback function for each of the reference updates which
+ * have been rejected in the given transaction.
+ */
+typedef void ref_transaction_for_each_rejected_update_fn(const char *refname,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
+ enum ref_transaction_error err,
+ void *cb_data);
+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
+ ref_transaction_for_each_rejected_update_fn cb,
+ void *cb_data);
+
/*
* Free `*transaction` and all associated data.
*/
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 4f27f7652c..256c69b942 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2852,8 +2852,15 @@ static int files_transaction_prepare(struct ref_store *ref_store,
ret = lock_ref_for_update(refs, update, transaction,
head_ref, &refnames_to_check,
err);
- if (ret)
+ if (ret) {
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_reset(err);
+ ret = 0;
+
+ continue;
+ }
goto cleanup;
+ }
if (update->flags & REF_DELETING &&
!(update->flags & REF_LOG_ONLY) &&
@@ -3151,6 +3158,9 @@ static int files_transaction_finish(struct ref_store *ref_store,
struct ref_update *update = transaction->updates[i];
struct ref_lock *lock = update->backend_data;
+ if (update->rejection_err)
+ continue;
+
if (update->flags & REF_NEEDS_COMMIT ||
update->flags & REF_LOG_ONLY) {
if (parse_and_write_reflog(refs, update, lock, err)) {
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index d90bd815a3..debca86a2b 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1327,10 +1327,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
* remain locked when it is done.
*/
static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,
- struct string_list *updates,
+ struct ref_transaction *transaction,
struct strbuf *err)
{
enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
+ struct string_list *updates = &transaction->refnames;
struct ref_iterator *iter = NULL;
size_t i;
int ok;
@@ -1411,6 +1412,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
"reference already exists",
update->refname);
ret = REF_TRANSACTION_ERROR_CREATE_EXISTS;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_reset(err);
+ ret = 0;
+ continue;
+ }
+
goto error;
} else if (!oideq(&update->old_oid, iter->oid)) {
strbuf_addf(err, "cannot update ref '%s': "
@@ -1419,6 +1427,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
oid_to_hex(iter->oid),
oid_to_hex(&update->old_oid));
ret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_reset(err);
+ ret = 0;
+ continue;
+ }
+
goto error;
}
}
@@ -1456,6 +1471,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
update->refname,
oid_to_hex(&update->old_oid));
ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_reset(err);
+ ret = 0;
+ continue;
+ }
+
goto error;
}
}
@@ -1521,6 +1543,7 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
write_error:
strbuf_addf(err, "error writing to %s: %s",
get_tempfile_path(refs->tempfile), strerror(errno));
+ ret = REF_TRANSACTION_ERROR_GENERIC;
error:
ref_iterator_free(iter);
@@ -1679,7 +1702,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
data->own_lock = 1;
}
- ret = write_with_updates(refs, &transaction->refnames, err);
+ ret = write_with_updates(refs, transaction, err);
if (ret)
goto failure;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 3f1d19abd9..73a5379b73 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -123,6 +123,12 @@ struct ref_update {
*/
uint64_t index;
+ /*
+ * Used in batched reference updates to mark if a given update
+ * was rejected.
+ */
+ enum ref_transaction_error rejection_err;
+
/*
* If this ref_update was split off of a symref update via
* split_symref_update(), then this member points at that
@@ -142,6 +148,13 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
struct object_id *oid, struct strbuf *referent,
unsigned int *type, int *failure_errno);
+/*
+ * Mark a given update as rejected with a given reason.
+ */
+int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,
+ size_t update_idx,
+ enum ref_transaction_error err);
+
/*
* Add a ref_update with the specified properties to transaction, and
* return a pointer to the new object. This function does not verify
@@ -183,6 +196,18 @@ enum ref_transaction_state {
REF_TRANSACTION_CLOSED = 2
};
+/*
+ * Data structure to hold indices of updates which were rejected, for batched
+ * reference updates. While the updates themselves hold the rejection error,
+ * this structure allows a transaction to iterate only over the rejected
+ * updates.
+ */
+struct ref_transaction_rejections {
+ size_t *update_indices;
+ size_t alloc;
+ size_t nr;
+};
+
/*
* Data structure for holding a reference transaction, which can
* consist of checks and updates to multiple references, carried out
@@ -195,6 +220,7 @@ struct ref_transaction {
size_t alloc;
size_t nr;
enum ref_transaction_state state;
+ struct ref_transaction_rejections *rejections;
void *backend_data;
unsigned int flags;
uint64_t max_index;
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index bd6b042103..5db4a108b9 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1371,8 +1371,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->updates[i],
&refnames_to_check, head_type,
&head_referent, &referent, err);
- if (ret)
+ if (ret) {
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_reset(err);
+ ret = 0;
+
+ continue;
+ }
goto done;
+ }
}
ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
@@ -1454,6 +1461,9 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
struct reftable_transaction_update *tx_update = &arg->updates[i];
struct ref_update *u = tx_update->update;
+ if (u->rejection_err)
+ continue;
+
/*
* Write a reflog entry when updating a ref to point to
* something new in either of the following cases:
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v5 7/8] refs: support rejection in batch updates during F/D checks
2025-03-27 11:13 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Karthik Nayak
` (5 preceding siblings ...)
2025-03-27 11:13 ` [PATCH v5 6/8] refs: implement batch reference update support Karthik Nayak
@ 2025-03-27 11:13 ` Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 8/8] update-ref: add --batch-updates flag for stdin mode Karthik Nayak
2025-03-28 9:24 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Patrick Steinhardt
8 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-27 11:13 UTC (permalink / raw)
To: git; +Cc: jltobler, phillip.wood123, gitster, ps, Karthik Nayak
The `refs_verify_refnames_available()` is used to batch check refnames
for F/D conflicts. While this is the more performant alternative than
its individual version, it does not provide rejection capabilities on a
single update level. For batched updates, this would mean a rejection of
the entire transaction whenever one reference has a F/D conflict.
Modify the function to call `ref_transaction_maybe_set_rejected()` to
check if a single update can be rejected. Since this function is only
internally used within 'refs/' and we want to pass in a `struct
ref_transaction *` as a variable. We also move and mark
`refs_verify_refnames_available()` to 'refs-internal.h' to be an
internal function.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 37 ++++++++++++++++++++++++++++++++++---
refs.h | 12 ------------
refs/files-backend.c | 27 ++++++++++++++++++---------
refs/refs-internal.h | 16 ++++++++++++++++
refs/reftable-backend.c | 11 ++++++++---
5 files changed, 76 insertions(+), 27 deletions(-)
diff --git a/refs.c b/refs.c
index b34ec198f5..41d6247e70 100644
--- a/refs.c
+++ b/refs.c
@@ -2540,6 +2540,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
const struct string_list *refnames,
const struct string_list *extras,
const struct string_list *skip,
+ struct ref_transaction *transaction,
unsigned int initial_transaction,
struct strbuf *err)
{
@@ -2547,6 +2548,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
struct strbuf referent = STRBUF_INIT;
struct string_list_item *item;
struct ref_iterator *iter = NULL;
+ struct strset conflicting_dirnames;
struct strset dirnames;
int ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
@@ -2557,9 +2559,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
assert(err);
+ strset_init(&conflicting_dirnames);
strset_init(&dirnames);
for_each_string_list_item(item, refnames) {
+ const size_t *update_idx = (size_t *)item->util;
const char *refname = item->string;
const char *extra_refname;
struct object_id oid;
@@ -2597,14 +2601,30 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
continue;
if (!initial_transaction &&
- !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
- &type, &ignore_errno)) {
+ (strset_contains(&conflicting_dirnames, dirname.buf) ||
+ !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
+ &type, &ignore_errno))) {
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT)) {
+ strset_remove(&dirnames, dirname.buf);
+ strset_add(&conflicting_dirnames, dirname.buf);
+ continue;
+ }
+
strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
dirname.buf, refname);
goto cleanup;
}
if (extras && string_list_has_string(extras, dirname.buf)) {
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT)) {
+ strset_remove(&dirnames, dirname.buf);
+ continue;
+ }
+
strbuf_addf(err, _("cannot process '%s' and '%s' at the same time"),
refname, dirname.buf);
goto cleanup;
@@ -2637,6 +2657,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
string_list_has_string(skip, iter->refname))
continue;
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT))
+ continue;
+
strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
iter->refname, refname);
goto cleanup;
@@ -2648,6 +2673,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
extra_refname = find_descendant_ref(dirname.buf, extras, skip);
if (extra_refname) {
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT))
+ continue;
+
strbuf_addf(err, _("cannot process '%s' and '%s' at the same time"),
refname, extra_refname);
goto cleanup;
@@ -2659,6 +2689,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
cleanup:
strbuf_release(&referent);
strbuf_release(&dirname);
+ strset_clear(&conflicting_dirnames);
strset_clear(&dirnames);
ref_iterator_free(iter);
return ret;
@@ -2679,7 +2710,7 @@ enum ref_transaction_error refs_verify_refname_available(
};
return refs_verify_refnames_available(refs, &refnames, extras, skip,
- initial_transaction, err);
+ NULL, initial_transaction, err);
}
struct do_for_each_reflog_help {
diff --git a/refs.h b/refs.h
index c48c800478..46a6008e07 100644
--- a/refs.h
+++ b/refs.h
@@ -141,18 +141,6 @@ enum ref_transaction_error refs_verify_refname_available(struct ref_store *refs,
unsigned int initial_transaction,
struct strbuf *err);
-/*
- * Same as `refs_verify_refname_available()`, but checking for a list of
- * refnames instead of only a single item. This is more efficient in the case
- * where one needs to check multiple refnames.
- */
-enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *refnames,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
-
int refs_ref_exists(struct ref_store *refs, const char *refname);
int should_autocreate_reflog(enum log_refs_config log_all_ref_updates,
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 256c69b942..b96a511977 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -677,16 +677,18 @@ static void unlock_ref(struct ref_lock *lock)
* - Generate informative error messages in the case of failure
*/
static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
- const char *refname,
+ struct ref_update *update,
+ size_t update_idx,
int mustexist,
struct string_list *refnames_to_check,
const struct string_list *extras,
struct ref_lock **lock_p,
struct strbuf *referent,
- unsigned int *type,
struct strbuf *err)
{
enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
+ const char *refname = update->refname;
+ unsigned int *type = &update->type;
struct ref_lock *lock;
struct strbuf ref_file = STRBUF_INIT;
int attempts_remaining = 3;
@@ -785,6 +787,8 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
type, &failure_errno)) {
+ struct string_list_item *item;
+
if (failure_errno == ENOENT) {
if (mustexist) {
/* Garden variety missing reference. */
@@ -864,7 +868,9 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
* make sure there is no existing packed ref that conflicts
* with refname. This check is deferred so that we can batch it.
*/
- string_list_append(refnames_to_check, refname);
+ item = string_list_append(refnames_to_check, refname);
+ item->util = xmalloc(sizeof(update_idx));
+ memcpy(item->util, &update_idx, sizeof(update_idx));
}
ret = 0;
@@ -2547,6 +2553,7 @@ struct files_transaction_backend_data {
*/
static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *refs,
struct ref_update *update,
+ size_t update_idx,
struct ref_transaction *transaction,
const char *head_ref,
struct string_list *refnames_to_check,
@@ -2575,9 +2582,9 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re
if (lock) {
lock->count++;
} else {
- ret = lock_raw_ref(refs, update->refname, mustexist,
+ ret = lock_raw_ref(refs, update, update_idx, mustexist,
refnames_to_check, &transaction->refnames,
- &lock, &referent, &update->type, err);
+ &lock, &referent, err);
if (ret) {
char *reason;
@@ -2849,7 +2856,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
- ret = lock_ref_for_update(refs, update, transaction,
+ ret = lock_ref_for_update(refs, update, i, transaction,
head_ref, &refnames_to_check,
err);
if (ret) {
@@ -2905,7 +2912,8 @@ static int files_transaction_prepare(struct ref_store *ref_store,
* So instead, we accept the race for now.
*/
if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
- &transaction->refnames, NULL, 0, err)) {
+ &transaction->refnames, NULL, transaction,
+ 0, err)) {
ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
}
@@ -2951,7 +2959,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
cleanup:
free(head_ref);
- string_list_clear(&refnames_to_check, 0);
+ string_list_clear(&refnames_to_check, 1);
if (ret)
files_transaction_cleanup(refs, transaction);
@@ -3097,7 +3105,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
}
if (refs_verify_refnames_available(&refs->base, &refnames_to_check,
- &affected_refnames, NULL, 1, err)) {
+ &affected_refnames, NULL, transaction,
+ 1, err)) {
packed_refs_unlock(refs->packed_ref_store);
ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 73a5379b73..f868870851 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -806,4 +806,20 @@ enum ref_transaction_error ref_update_check_old_target(const char *referent,
*/
int ref_update_expects_existing_old_ref(struct ref_update *update);
+/*
+ * Same as `refs_verify_refname_available()`, but checking for a list of
+ * refnames instead of only a single item. This is more efficient in the case
+ * where one needs to check multiple refnames.
+ *
+ * If using batched updates, then individual updates are marked rejected,
+ * reference backends are then in charge of not committing those updates.
+ */
+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ struct ref_transaction *transaction,
+ unsigned int initial_transaction,
+ struct strbuf *err);
+
#endif /* REFS_REFS_INTERNAL_H */
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 5db4a108b9..4c3817f4ec 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1074,6 +1074,7 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor
struct ref_transaction *transaction,
struct reftable_backend *be,
struct ref_update *u,
+ size_t update_idx,
struct string_list *refnames_to_check,
unsigned int head_type,
struct strbuf *head_referent,
@@ -1149,6 +1150,7 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor
if (ret < 0)
return REF_TRANSACTION_ERROR_GENERIC;
if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
+ struct string_list_item *item;
/*
* The reference does not exist, and we either have no
* old object ID or expect the reference to not exist.
@@ -1158,7 +1160,9 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor
* can output a proper error message instead of failing
* at a later point.
*/
- string_list_append(refnames_to_check, u->refname);
+ item = string_list_append(refnames_to_check, u->refname);
+ item->util = xmalloc(sizeof(update_idx));
+ memcpy(item->util, &update_idx, sizeof(update_idx));
/*
* There is no need to write the reference deletion
@@ -1368,7 +1372,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
for (i = 0; i < transaction->nr; i++) {
ret = prepare_single_update(refs, tx_data, transaction, be,
- transaction->updates[i],
+ transaction->updates[i], i,
&refnames_to_check, head_type,
&head_referent, &referent, err);
if (ret) {
@@ -1384,6 +1388,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
&transaction->refnames, NULL,
+ transaction,
transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
err);
if (ret < 0)
@@ -1402,7 +1407,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
}
strbuf_release(&referent);
strbuf_release(&head_referent);
- string_list_clear(&refnames_to_check, 0);
+ string_list_clear(&refnames_to_check, 1);
return ret;
}
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v5 8/8] update-ref: add --batch-updates flag for stdin mode
2025-03-27 11:13 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Karthik Nayak
` (6 preceding siblings ...)
2025-03-27 11:13 ` [PATCH v5 7/8] refs: support rejection in batch updates during F/D checks Karthik Nayak
@ 2025-03-27 11:13 ` Karthik Nayak
2025-03-28 13:00 ` Jean-Noël AVILA
2025-03-28 9:24 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Patrick Steinhardt
8 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-03-27 11:13 UTC (permalink / raw)
To: git; +Cc: jltobler, phillip.wood123, gitster, ps, Karthik Nayak
When updating multiple references through stdin, Git's update-ref
command normally aborts the entire transaction if any single update
fails. This atomic behavior prevents partial updates. Introduce a new
batch update system, where the updates the performed together similar
but individual updates are allowed to fail.
Add a new `--batch-updates` flag that allows the transaction to continue
even when individual reference updates fail. This flag can only be used
in `--stdin` mode and builds upon the batch update support added to the
refs subsystem in the previous commits. When enabled, failed updates are
reported in the following format:
rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
Update the documentation to reflect this change and also tests to cover
different scenarios where an update could be rejected.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
Documentation/git-update-ref.adoc | 14 ++-
builtin/update-ref.c | 66 ++++++++++-
t/t1400-update-ref.sh | 233 ++++++++++++++++++++++++++++++++++++++
3 files changed, 306 insertions(+), 7 deletions(-)
diff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc
index 9e6935d38d..5be2c16776 100644
--- a/Documentation/git-update-ref.adoc
+++ b/Documentation/git-update-ref.adoc
@@ -7,8 +7,10 @@ git-update-ref - Update the object name stored in a ref safely
SYNOPSIS
--------
-[verse]
-'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z])
+[synopsis]
+git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
+ [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
+ [-m <reason>] [--no-deref] --stdin [-z] [--batch-updates]
DESCRIPTION
-----------
@@ -57,6 +59,14 @@ performs all modifications together. Specify commands of the form:
With `--create-reflog`, update-ref will create a reflog for each ref
even if one would not ordinarily be created.
+With `--batch-updates`, update-ref executes the updates in a batch but allows
+individual updates to fail due to invalid or incorrect user input, applying only
+the successful updates. However, system-related errors—such as I/O failures or
+memory issues—will result in a full failure of all batched updates. Any failed
+updates will be reported in the following format:
+
+ rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
+
Quote fields containing whitespace as if they were strings in C source
code; i.e., surrounded by double-quotes and with backslash escapes.
Use 40 "0" characters or the empty string to specify a zero value. To
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 1d541e13ad..111d6473ad 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -5,6 +5,7 @@
#include "config.h"
#include "gettext.h"
#include "hash.h"
+#include "hex.h"
#include "refs.h"
#include "object-name.h"
#include "parse-options.h"
@@ -13,7 +14,7 @@
static const char * const git_update_ref_usage[] = {
N_("git update-ref [<options>] -d <refname> [<old-oid>]"),
N_("git update-ref [<options>] <refname> <new-oid> [<old-oid>]"),
- N_("git update-ref [<options>] --stdin [-z]"),
+ N_("git update-ref [<options>] --stdin [-z] [--batch-updates]"),
NULL
};
@@ -565,6 +566,49 @@ static void parse_cmd_abort(struct ref_transaction *transaction,
report_ok("abort");
}
+static void print_rejected_refs(const char *refname,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
+ enum ref_transaction_error err,
+ void *cb_data UNUSED)
+{
+ struct strbuf sb = STRBUF_INIT;
+ const char *reason = "";
+
+ switch (err) {
+ case REF_TRANSACTION_ERROR_NAME_CONFLICT:
+ reason = "refname conflict";
+ break;
+ case REF_TRANSACTION_ERROR_CREATE_EXISTS:
+ reason = "reference already exists";
+ break;
+ case REF_TRANSACTION_ERROR_NONEXISTENT_REF:
+ reason = "reference does not exist";
+ break;
+ case REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE:
+ reason = "incorrect old value provided";
+ break;
+ case REF_TRANSACTION_ERROR_INVALID_NEW_VALUE:
+ reason = "invalid new value provided";
+ break;
+ case REF_TRANSACTION_ERROR_EXPECTED_SYMREF:
+ reason = "expected symref but found regular ref";
+ break;
+ default:
+ reason = "unkown failure";
+ }
+
+ strbuf_addf(&sb, "rejected %s %s %s %s\n", refname,
+ new_oid ? oid_to_hex(new_oid) : new_target,
+ old_oid ? oid_to_hex(old_oid) : old_target,
+ reason);
+
+ fwrite(sb.buf, sb.len, 1, stdout);
+ strbuf_release(&sb);
+}
+
static void parse_cmd_commit(struct ref_transaction *transaction,
const char *next, const char *end UNUSED)
{
@@ -573,6 +617,10 @@ static void parse_cmd_commit(struct ref_transaction *transaction,
die("commit: extra input: %s", next);
if (ref_transaction_commit(transaction, &error))
die("commit: %s", error.buf);
+
+ ref_transaction_for_each_rejected_update(transaction,
+ print_rejected_refs, NULL);
+
report_ok("commit");
ref_transaction_free(transaction);
}
@@ -609,7 +657,7 @@ static const struct parse_cmd {
{ "commit", parse_cmd_commit, 0, UPDATE_REFS_CLOSED },
};
-static void update_refs_stdin(void)
+static void update_refs_stdin(unsigned int flags)
{
struct strbuf input = STRBUF_INIT, err = STRBUF_INIT;
enum update_refs_state state = UPDATE_REFS_OPEN;
@@ -617,7 +665,7 @@ static void update_refs_stdin(void)
int i, j;
transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
- 0, &err);
+ flags, &err);
if (!transaction)
die("%s", err.buf);
@@ -685,7 +733,7 @@ static void update_refs_stdin(void)
*/
state = cmd->state;
transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
- 0, &err);
+ flags, &err);
if (!transaction)
die("%s", err.buf);
@@ -701,6 +749,8 @@ static void update_refs_stdin(void)
/* Commit by default if no transaction was requested. */
if (ref_transaction_commit(transaction, &err))
die("%s", err.buf);
+ ref_transaction_for_each_rejected_update(transaction,
+ print_rejected_refs, NULL);
ref_transaction_free(transaction);
break;
case UPDATE_REFS_STARTED:
@@ -727,6 +777,8 @@ int cmd_update_ref(int argc,
struct object_id oid, oldoid;
int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;
int create_reflog = 0;
+ unsigned int flags = 0;
+
struct option options[] = {
OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
OPT_BOOL('d', NULL, &delete, N_("delete the reference")),
@@ -735,6 +787,8 @@ int cmd_update_ref(int argc,
OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
OPT_BOOL( 0 , "create-reflog", &create_reflog, N_("create a reflog")),
+ OPT_BIT('0', "batch-updates", &flags, N_("batch reference updates"),
+ REF_TRANSACTION_ALLOW_FAILURE),
OPT_END(),
};
@@ -756,8 +810,10 @@ int cmd_update_ref(int argc,
usage_with_options(git_update_ref_usage, options);
if (end_null)
line_termination = '\0';
- update_refs_stdin();
+ update_refs_stdin(flags);
return 0;
+ } else if (flags & REF_TRANSACTION_ALLOW_FAILURE) {
+ die("--batch-updates can only be used with --stdin");
}
if (end_null)
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 29045aad43..d29d23cb89 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -2066,6 +2066,239 @@ do
grep "$(git rev-parse $a) $(git rev-parse $a)" actual
'
+ test_expect_success "stdin $type batch-updates" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit commit &&
+ head=$(git rev-parse HEAD) &&
+
+ format_command $type "update refs/heads/ref1" "$head" "$Z" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head" "$Z" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with invalid new_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$(test_oid 001)" "$head" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "invalid new value provided" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with non-commit new_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ head_tree=$(git rev-parse HEAD^{tree}) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head_tree" "$head" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "invalid new value provided" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with non-existent ref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ test_must_fail git rev-parse refs/heads/ref2 &&
+ test_grep -q "reference does not exist" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with dangling symref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git symbolic-ref refs/heads/ref2 refs/heads/nonexistent &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
+ git update-ref $type --no-deref --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ test_must_fail git rev-parse refs/heads/ref2 &&
+ test_grep -q "reference does not exist" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with regular ref as symref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "symref-update refs/heads/ref2" "$old_head" "ref" "refs/heads/nonexistent" >>stdin &&
+ git update-ref $type --no-deref --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "expected symref but found regular ref" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with invalid old_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$Z" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "reference already exists" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with incorrect old oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head" "$old_head" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "incorrect old value provided" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates refname conflict" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref/foo $head &&
+
+ format_command $type "update refs/heads/ref/foo" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref/foo >actual &&
+ test_cmp expect actual &&
+ test_grep -q "refname conflict" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates refname conflict new ref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref/foo $head &&
+
+ format_command $type "update refs/heads/foo" "$old_head" "" >stdin &&
+ format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/foo >actual &&
+ test_cmp expect actual &&
+ test_grep -q "refname conflict" stdout
+ )
+ '
done
test_expect_success 'update-ref should also create reflog for HEAD' '
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH v5 0/8] refs: introduce support for batched reference updates
2025-03-27 11:13 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Karthik Nayak
` (7 preceding siblings ...)
2025-03-27 11:13 ` [PATCH v5 8/8] update-ref: add --batch-updates flag for stdin mode Karthik Nayak
@ 2025-03-28 9:24 ` Patrick Steinhardt
8 siblings, 0 replies; 143+ messages in thread
From: Patrick Steinhardt @ 2025-03-28 9:24 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler, phillip.wood123, gitster
On Thu, Mar 27, 2025 at 12:13:24PM +0100, Karthik Nayak wrote:
> Changes in v5:
> - Inline the comments around the 'ref_transaction_error'.
> - Use 'strbuf_reset()' wherever possible instead of 'strbuf_setlen(err, 0)'.
> - Use an extra 'conflicting_dirnames' strset in 'refs_verify_refnames_available()' to track
> dirnames which were found to be conflicting, this is to avoid re-reading those dirnames.
> - Add curly braces style mismatch in if..else block.
> - Link to v4: https://lore.kernel.org/r/20250320-245-partially-atomic-ref-updates-v4-0-3dcc1b311dc9@gmail.com
Thanks, the series looks good to me judging by the range diff.
Patrick
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v5 8/8] update-ref: add --batch-updates flag for stdin mode
2025-03-27 11:13 ` [PATCH v5 8/8] update-ref: add --batch-updates flag for stdin mode Karthik Nayak
@ 2025-03-28 13:00 ` Jean-Noël AVILA
2025-03-29 16:36 ` Junio C Hamano
0 siblings, 1 reply; 143+ messages in thread
From: Jean-Noël AVILA @ 2025-03-28 13:00 UTC (permalink / raw)
To: git, Karthik Nayak; +Cc: jltobler, phillip.wood123, gitster, ps, Karthik Nayak
On Thursday, 27 March 2025 12:13:32 CET Karthik Nayak wrote:
> When updating multiple references through stdin, Git's update-ref
> command normally aborts the entire transaction if any single update
> fails. This atomic behavior prevents partial updates. Introduce a new
> batch update system, where the updates the performed together similar
> but individual updates are allowed to fail.
>
> Add a new `--batch-updates` flag that allows the transaction to continue
> even when individual reference updates fail. This flag can only be used
> in `--stdin` mode and builds upon the batch update support added to the
> refs subsystem in the previous commits. When enabled, failed updates are
> reported in the following format:
>
> rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP
> <rejection-reason> LF
>
> Update the documentation to reflect this change and also tests to cover
> different scenarios where an update could be rejected.
>
> Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
> ---
> Documentation/git-update-ref.adoc | 14 ++-
> builtin/update-ref.c | 66 ++++++++++-
> t/t1400-update-ref.sh | 233
> ++++++++++++++++++++++++++++++++++++++ 3 files changed, 306 insertions(+), 7
> deletions(-)
>
> diff --git a/Documentation/git-update-ref.adoc
> b/Documentation/git-update-ref.adoc index 9e6935d38d..5be2c16776 100644
> --- a/Documentation/git-update-ref.adoc
> +++ b/Documentation/git-update-ref.adoc
> @@ -7,8 +7,10 @@ git-update-ref - Update the object name stored in a ref
> safely
>
> SYNOPSIS
> --------
> -[verse]
> -'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] |
> [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z]) +[synopsis]
> +git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
> + [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid>
> [<old-oid>] + [-m <reason>] [--no-deref] --stdin [-z]
> [--batch-updates]
In the case of expressing alternative command line invocations, you need to
repeat the "git update-ref" command on each line. Otherwise, it means that
this is the continuation of possible options of one command
>
> DESCRIPTION
> -----------
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v5 8/8] update-ref: add --batch-updates flag for stdin mode
2025-03-28 13:00 ` Jean-Noël AVILA
@ 2025-03-29 16:36 ` Junio C Hamano
2025-03-29 18:18 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Junio C Hamano @ 2025-03-29 16:36 UTC (permalink / raw)
To: Jean-Noël AVILA; +Cc: git, Karthik Nayak, jltobler, phillip.wood123, ps
Jean-Noël AVILA <jn.avila@free.fr> writes:
>> SYNOPSIS
>> --------
>> -[verse]
>> -'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] |
>> [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z]) +[synopsis]
>> +git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
>> + [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid>
>> [<old-oid>] + [-m <reason>] [--no-deref] --stdin [-z]
>> [--batch-updates]
>
> In the case of expressing alternative command line invocations, you need to
> repeat the "git update-ref" command on each line. Otherwise, it means that
> this is the continuation of possible options of one command
Like this?
diff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc
index 5be2c16776..9310ce9768 100644
--- a/Documentation/git-update-ref.adoc
+++ b/Documentation/git-update-ref.adoc
@@ -9,8 +9,8 @@ SYNOPSIS
--------
[synopsis]
git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
- [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
- [-m <reason>] [--no-deref] --stdin [-z] [--batch-updates]
+git update-ref [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
+git update-ref [-m <reason>] [--no-deref] --stdin [-z] [--batch-updates]
DESCRIPTION
-----------
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH v5 8/8] update-ref: add --batch-updates flag for stdin mode
2025-03-29 16:36 ` Junio C Hamano
@ 2025-03-29 18:18 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-03-29 18:18 UTC (permalink / raw)
To: Junio C Hamano, Jean-Noël AVILA; +Cc: git, jltobler, phillip.wood123, ps
[-- Attachment #1: Type: text/plain, Size: 1613 bytes --]
Junio C Hamano <gitster@pobox.com> writes:
> Jean-Noël AVILA <jn.avila@free.fr> writes:
>
>>> SYNOPSIS
>>> --------
>>> -[verse]
>>> -'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] |
>>> [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z]) +[synopsis]
>>> +git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
>>> + [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid>
>>> [<old-oid>] + [-m <reason>] [--no-deref] --stdin [-z]
>>> [--batch-updates]
>>
>> In the case of expressing alternative command line invocations, you need to
>> repeat the "git update-ref" command on each line. Otherwise, it means that
>> this is the continuation of possible options of one command
>
>
> Like this?
>
> diff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc
> index 5be2c16776..9310ce9768 100644
> --- a/Documentation/git-update-ref.adoc
> +++ b/Documentation/git-update-ref.adoc
> @@ -9,8 +9,8 @@ SYNOPSIS
> --------
> [synopsis]
> git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
> - [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
> - [-m <reason>] [--no-deref] --stdin [-z] [--batch-updates]
> +git update-ref [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
> +git update-ref [-m <reason>] [--no-deref] --stdin [-z] [--batch-updates]
>
> DESCRIPTION
> -----------
I think you also caught a whitespace issue here! I'll add this locally
nevertheless, but will hold on re-rolling! Thanks both
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH v6 0/8] refs: introduce support for batched reference updates
2025-02-07 7:34 [PATCH 0/6] refs: introduce support for partial reference transactions Karthik Nayak
` (10 preceding siblings ...)
2025-03-27 11:13 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Karthik Nayak
@ 2025-04-08 8:51 ` Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
` (7 more replies)
11 siblings, 8 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-04-08 8:51 UTC (permalink / raw)
To: karthik.188; +Cc: git, jltobler, ps, jn.avila, gitster
Git supports making reference updates with or without transactions.
Updates with transactions are generally better optimized. But
transactions are all or nothing. This means, if a user wants to batch
updates to take advantage of the optimizations without the hard
requirement that all updates must succeed, there is no way currently to
do so. Particularly with the reftable backend where batching multiple
reference updates is more efficient than performing them sequentially.
This series introduces support for batched reference updates without
transactions allowing individual reference updates to fail while letting
others proceed. This capability is exposed through git-update-ref's
`--allow-partial` flag, which can be used in `--stdin` mode to batch
updates and handle failures gracefully. Under the hood, these batched
updates still use the transactions infrastructure, while modifying
sections to allow partial failures.
The changes are structured to carefully build up this functionality:
First, we clean up and consolidate the reference update checking logic.
This includes removing duplicate checks in the files backend and moving
refname tracking to the generic layer, which simplifies the codebase and
prepares it for the new feature.
We then restructure the reftable backend's transaction preparation code,
extracting the update validation logic into a dedicated function. This
not only improves code organization but sets the stage for implementing
partial transaction support.
To ensure we only skip errors which are user-oriented, we introduce
typed errors for transactions with 'enum ref_transaction_error'. We
extend the existing errors to include other scenarios and use this new
errors throughout the refs code.
With this groundwork in place, we implement the core batch update
support in the refs subsystem. This adds the necessary infrastructure to
track and report rejected updates while allowing transactions to
proceed. All reference backends are modified to support this behavior
when enabled.
Finally, we expose this functionality to users through
git-update-ref(1)'s `--allow-partial` flag, complete with test coverage
and documentation. The flag is specifically limited to `--stdin` mode
where batching multiple updates is most relevant.
This enhancement improves Git's flexibility in handling reference
updates while maintaining the safety of atomic transactions by default.
It's particularly valuable for tools and workflows that need to handle
reference update failures gracefully without abandoning the entire batch
of updates.
This series is based on top of 683c54c999 (Git 2.49, 2025-03-14) with
Patrick's series 'refs: batch refname availability checks' [1] merged
in.
[1]: https://lore.kernel.org/all/20250217-pks-update-ref-optimization-v1-0-a2b6d87a24af@pks.im/
---
Changes in v6:
- The documentation for 'git update-ref' didn't repeat the command, giving the intention
that newlines added were continuation of options rather than alternative invocations.
- Link to v5: https://lore.kernel.org/all/20250327-245-partially-atomic-ref-updates-v5-0-4db2a3e34404@gmail.com
Changes in v5:
- Inline the comments around the 'ref_transaction_error'.
- Use 'strbuf_reset()' wherever possible instead of 'strbuf_setlen(err, 0)'.
- Use an extra 'conflicting_dirnames' strset in 'refs_verify_refnames_available()' to track
dirnames which were found to be conflicting, this is to avoid re-reading those dirnames.
- Add curly braces style mismatch in if..else block.
- Link to v4: https://lore.kernel.org/r/20250320-245-partially-atomic-ref-updates-v4-0-3dcc1b311dc9@gmail.com
Changes in v4:
- Rebased on top of 2.49 since there was a long time between the
previous iteration and we have a new release.
- Changed the naming to say 'batched' updates instead of 'partial
transactions'. While we still use the transaction infrastructure
underneath, the new naming causes less ambiguity.
- Clean up some of the commit messages.
- Raise BUG for invalid update index while setting rejections.
- Fix an incorrect early return.
- Link to v3: https://lore.kernel.org/r/20250305-245-partially-atomic-ref-updates-v3-0-0c64e3052354@gmail.com
Changes in v3:
- Changed 'transaction_error' to 'ref_transaction_error' along with the
error names. Removed 'TRANSACTION_OK' since it can potentially be
missed instead of simply 'return 0'.
- Rename 'ref_transaction_set_rejected' to
'ref_transaction_maybe_set_rejected' and move logic around error
checks to within this function.
- Add a new struct 'ref_transaction_rejections' to track the rejections
within a transaction. This allows us to only iterate over rejected
updates.
- Add a new commit to also support partial transactions within the
batched F/D checks.
- Remove NUL delimited outputs in 'git-update-ref(1)'.
- Remove translations for plumbing outputs.
- Other small cleanups in the commit message and code.
Changes in v2:
- Introduce and use structured errors. This consolidates the errors
and their handling between the ref backends.
- In the previous version, we skipped over all failures. This include
system failures such as low memory or IO problems. Let's instead, only
skip user-oriented failures, such as invalid old OID and so on.
- Change the rejection function name to `ref_transaction_set_rejected()`.
- Modify the commit messages and documentation to be a little more
verbose.
- Link to v1: https://lore.kernel.org/r/20250207-245-partially-atomic-ref-updates-v1-0-e6a3690ff23a@gmail.com
---
Karthik Nayak (8):
refs/files: remove redundant check in split_symref_update()
refs: move duplicate refname update check to generic layer
refs/files: remove duplicate duplicates check
refs/reftable: extract code from the transaction preparation
refs: introduce enum-based transaction error types
refs: implement batch reference update support
refs: support rejection in batch updates during F/D checks
update-ref: add --batch-updates flag for stdin mode
Documentation/git-update-ref.adoc | 14 +-
builtin/fetch.c | 2 +-
builtin/update-ref.c | 66 +++-
refs.c | 171 ++++++++--
refs.h | 70 +++--
refs/files-backend.c | 314 ++++++++-----------
refs/packed-backend.c | 69 ++--
refs/refs-internal.h | 51 ++-
refs/reftable-backend.c | 502 +++++++++++++++---------------
t/t1400-update-ref.sh | 233 ++++++++++++++
10 files changed, 969 insertions(+), 523 deletions(-)
---
Range-diff versus v5:
1: cae24142a1 = 1: cae24142a1 refs/files: remove redundant check in split_symref_update()
2: 239aecdb0f = 2: 239aecdb0f refs: move duplicate refname update check to generic layer
3: 06404dd350 = 3: 06404dd350 refs/files: remove duplicate duplicates check
4: a3e645aa37 = 4: a3e645aa37 refs/reftable: extract code from the transaction preparation
5: 2615bfe78e = 5: 2615bfe78e refs: introduce enum-based transaction error types
6: d5c1c77b0d = 6: d5c1c77b0d refs: implement batch reference update support
7: 4bb4902631 = 7: 4bb4902631 refs: support rejection in batch updates during F/D checks
8: 674630f77c ! 8: ed92beaf18 update-ref: add --batch-updates flag for stdin mode
@@ Documentation/git-update-ref.adoc: git-update-ref - Update the object name store
-'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z])
+[synopsis]
+git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
-+ [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
-+ [-m <reason>] [--no-deref] --stdin [-z] [--batch-updates]
++git update-ref [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
++git update-ref [-m <reason>] [--no-deref] --stdin [-z] [--batch-updates]
DESCRIPTION
-----------
--
2.48.1
^ permalink raw reply [flat|nested] 143+ messages in thread
* [PATCH v6 1/8] refs/files: remove redundant check in split_symref_update()
2025-04-08 8:51 ` [PATCH v6 " Karthik Nayak
@ 2025-04-08 8:51 ` Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 2/8] refs: move duplicate refname update check to generic layer Karthik Nayak
` (6 subsequent siblings)
7 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-04-08 8:51 UTC (permalink / raw)
To: karthik.188; +Cc: git, jltobler, ps, jn.avila, gitster
In `split_symref_update()`, there were two checks for duplicate
refnames:
- At the start, `string_list_has_string()` ensures the refname is not
already in `affected_refnames`, preventing duplicates from being
added.
- After adding the refname, another check verifies whether the newly
inserted item has a `util` value.
The second check is unnecessary because the first one guarantees that
`string_list_insert()` will never encounter a preexisting entry.
The `item->util` field is assigned to validate that a rename doesn't
already exist in the list. The validation is done after the first check.
As this check is removed, clean up the validation and the assignment of
this field in `split_head_update()` and `files_transaction_prepare()`.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs/files-backend.c | 20 +++-----------------
1 file changed, 3 insertions(+), 17 deletions(-)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index ff54a4bb7e..15559a09c5 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2382,7 +2382,6 @@ static int split_head_update(struct ref_update *update,
struct string_list *affected_refnames,
struct strbuf *err)
{
- struct string_list_item *item;
struct ref_update *new_update;
if ((update->flags & REF_LOG_ONLY) ||
@@ -2421,8 +2420,7 @@ static int split_head_update(struct ref_update *update,
*/
if (strcmp(new_update->refname, "HEAD"))
BUG("%s unexpectedly not 'HEAD'", new_update->refname);
- item = string_list_insert(affected_refnames, new_update->refname);
- item->util = new_update;
+ string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2441,7 +2439,6 @@ static int split_symref_update(struct ref_update *update,
struct string_list *affected_refnames,
struct strbuf *err)
{
- struct string_list_item *item;
struct ref_update *new_update;
unsigned int new_flags;
@@ -2496,11 +2493,7 @@ static int split_symref_update(struct ref_update *update,
* be valid as long as affected_refnames is in use, and NOT
* referent, which might soon be freed by our caller.
*/
- item = string_list_insert(affected_refnames, new_update->refname);
- if (item->util)
- BUG("%s unexpectedly found in affected_refnames",
- new_update->refname);
- item->util = new_update;
+ string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2834,7 +2827,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
*/
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
- struct string_list_item *item;
if ((update->flags & REF_IS_PRUNING) &&
!(update->flags & REF_NO_DEREF))
@@ -2843,13 +2835,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
if (update->flags & REF_LOG_ONLY)
continue;
- item = string_list_append(&affected_refnames, update->refname);
- /*
- * We store a pointer to update in item->util, but at
- * the moment we never use the value of this field
- * except to check whether it is non-NULL.
- */
- item->util = update;
+ string_list_append(&affected_refnames, update->refname);
}
string_list_sort(&affected_refnames);
if (ref_update_reject_duplicates(&affected_refnames, err)) {
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v6 2/8] refs: move duplicate refname update check to generic layer
2025-04-08 8:51 ` [PATCH v6 " Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
@ 2025-04-08 8:51 ` Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 3/8] refs/files: remove duplicate duplicates check Karthik Nayak
` (5 subsequent siblings)
7 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-04-08 8:51 UTC (permalink / raw)
To: karthik.188; +Cc: git, jltobler, ps, jn.avila, gitster
Move the tracking of refnames in `affected_refnames` from individual
backends into the generic layer in 'refs.c'. This centralizes the
duplicate refname detection that was previously handled separately by
each backend.
Make some changes to accommodate this move:
- Add a `string_list` field `refnames` to `ref_transaction` to contain
all the references in a transaction. This field is updated whenever
a new update is added via `ref_transaction_add_update`, so manual
additions in reference backends are dropped.
- Modify the backends to use this field internally as needed. The
backends need to check if an update for refname already exists when
splitting symrefs or adding an update for 'HEAD'.
- In the reftable backend, within `reftable_be_transaction_prepare()`,
move the `string_list_has_string()` check above
`ref_transaction_add_update()`. Since `ref_transaction_add_update()`
automatically adds the refname to `transaction->refnames`,
performing the check after will always return true, so we perform
the check before adding the update.
This helps reduce duplication of functionality between the backends and
makes it easier to make changes in a more centralized manner.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 17 +++++++++++
refs/files-backend.c | 67 +++++++++--------------------------------
refs/packed-backend.c | 25 +--------------
refs/refs-internal.h | 2 ++
refs/reftable-backend.c | 54 +++++++++++----------------------
5 files changed, 51 insertions(+), 114 deletions(-)
diff --git a/refs.c b/refs.c
index 2ac9d8ebd0..504bf2063e 100644
--- a/refs.c
+++ b/refs.c
@@ -1175,6 +1175,7 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,
CALLOC_ARRAY(tr, 1);
tr->ref_store = refs;
tr->flags = flags;
+ string_list_init_dup(&tr->refnames);
return tr;
}
@@ -1205,6 +1206,7 @@ void ref_transaction_free(struct ref_transaction *transaction)
free((char *)transaction->updates[i]->old_target);
free(transaction->updates[i]);
}
+ string_list_clear(&transaction->refnames, 0);
free(transaction->updates);
free(transaction);
}
@@ -1218,6 +1220,7 @@ struct ref_update *ref_transaction_add_update(
const char *committer_info,
const char *msg)
{
+ struct string_list_item *item;
struct ref_update *update;
if (transaction->state != REF_TRANSACTION_OPEN)
@@ -1245,6 +1248,16 @@ struct ref_update *ref_transaction_add_update(
update->msg = normalize_reflog_message(msg);
}
+ /*
+ * This list is generally used by the backends to avoid duplicates.
+ * But we do support multiple log updates for a given refname within
+ * a single transaction.
+ */
+ if (!(update->flags & REF_LOG_ONLY)) {
+ item = string_list_append(&transaction->refnames, refname);
+ item->util = update;
+ }
+
return update;
}
@@ -2405,6 +2418,10 @@ int ref_transaction_prepare(struct ref_transaction *transaction,
return -1;
}
+ string_list_sort(&transaction->refnames);
+ if (ref_update_reject_duplicates(&transaction->refnames, err))
+ return TRANSACTION_GENERIC_ERROR;
+
ret = refs->be->transaction_prepare(refs, transaction, err);
if (ret)
return ret;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 15559a09c5..58f62ea8a3 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2378,9 +2378,7 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st
*/
static int split_head_update(struct ref_update *update,
struct ref_transaction *transaction,
- const char *head_ref,
- struct string_list *affected_refnames,
- struct strbuf *err)
+ const char *head_ref, struct strbuf *err)
{
struct ref_update *new_update;
@@ -2398,7 +2396,7 @@ static int split_head_update(struct ref_update *update,
* transaction. This check is O(lg N) in the transaction
* size, but it happens at most once per transaction.
*/
- if (string_list_has_string(affected_refnames, "HEAD")) {
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
/* An entry already existed */
strbuf_addf(err,
"multiple updates for 'HEAD' (including one "
@@ -2420,7 +2418,6 @@ static int split_head_update(struct ref_update *update,
*/
if (strcmp(new_update->refname, "HEAD"))
BUG("%s unexpectedly not 'HEAD'", new_update->refname);
- string_list_insert(affected_refnames, new_update->refname);
return 0;
}
@@ -2436,7 +2433,6 @@ static int split_head_update(struct ref_update *update,
static int split_symref_update(struct ref_update *update,
const char *referent,
struct ref_transaction *transaction,
- struct string_list *affected_refnames,
struct strbuf *err)
{
struct ref_update *new_update;
@@ -2448,7 +2444,7 @@ static int split_symref_update(struct ref_update *update,
* size, but it happens at most once per symref in a
* transaction.
*/
- if (string_list_has_string(affected_refnames, referent)) {
+ if (string_list_has_string(&transaction->refnames, referent)) {
/* An entry already exists */
strbuf_addf(err,
"multiple updates for '%s' (including one "
@@ -2486,15 +2482,6 @@ static int split_symref_update(struct ref_update *update,
update->flags |= REF_LOG_ONLY | REF_NO_DEREF;
update->flags &= ~REF_HAVE_OLD;
- /*
- * Add the referent. This insertion is O(N) in the transaction
- * size, but it happens at most once per symref in a
- * transaction. Make sure to add new_update->refname, which will
- * be valid as long as affected_refnames is in use, and NOT
- * referent, which might soon be freed by our caller.
- */
- string_list_insert(affected_refnames, new_update->refname);
-
return 0;
}
@@ -2558,7 +2545,6 @@ static int lock_ref_for_update(struct files_ref_store *refs,
struct ref_transaction *transaction,
const char *head_ref,
struct string_list *refnames_to_check,
- struct string_list *affected_refnames,
struct strbuf *err)
{
struct strbuf referent = STRBUF_INIT;
@@ -2575,8 +2561,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
update->flags |= REF_DELETING;
if (head_ref) {
- ret = split_head_update(update, transaction, head_ref,
- affected_refnames, err);
+ ret = split_head_update(update, transaction, head_ref, err);
if (ret)
goto out;
}
@@ -2586,9 +2571,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,
lock->count++;
} else {
ret = lock_raw_ref(refs, update->refname, mustexist,
- refnames_to_check, affected_refnames,
- &lock, &referent,
- &update->type, err);
+ refnames_to_check, &transaction->refnames,
+ &lock, &referent, &update->type, err);
if (ret) {
char *reason;
@@ -2642,9 +2626,8 @@ static int lock_ref_for_update(struct files_ref_store *refs,
* of processing the split-off update, so we
* don't have to do it here.
*/
- ret = split_symref_update(update,
- referent.buf, transaction,
- affected_refnames, err);
+ ret = split_symref_update(update, referent.buf,
+ transaction, err);
if (ret)
goto out;
}
@@ -2799,7 +2782,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
"ref_transaction_prepare");
size_t i;
int ret = 0;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
struct string_list refnames_to_check = STRING_LIST_INIT_NODUP;
char *head_ref = NULL;
int head_type;
@@ -2818,12 +2800,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
transaction->backend_data = backend_data;
/*
- * Fail if a refname appears more than once in the
- * transaction. (If we end up splitting up any updates using
- * split_symref_update() or split_head_update(), those
- * functions will check that the new updates don't have the
- * same refname as any existing ones.) Also fail if any of the
- * updates use REF_IS_PRUNING without REF_NO_DEREF.
+ * Fail if any of the updates use REF_IS_PRUNING without REF_NO_DEREF.
*/
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
@@ -2831,16 +2808,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
if ((update->flags & REF_IS_PRUNING) &&
!(update->flags & REF_NO_DEREF))
BUG("REF_IS_PRUNING set without REF_NO_DEREF");
-
- if (update->flags & REF_LOG_ONLY)
- continue;
-
- string_list_append(&affected_refnames, update->refname);
- }
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
}
/*
@@ -2882,7 +2849,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
ret = lock_ref_for_update(refs, update, transaction,
head_ref, &refnames_to_check,
- &affected_refnames, err);
+ err);
if (ret)
goto cleanup;
@@ -2929,7 +2896,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
* So instead, we accept the race for now.
*/
if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
- &affected_refnames, NULL, 0, err)) {
+ &transaction->refnames, NULL, 0, err)) {
ret = TRANSACTION_NAME_CONFLICT;
goto cleanup;
}
@@ -2975,7 +2942,6 @@ static int files_transaction_prepare(struct ref_store *ref_store,
cleanup:
free(head_ref);
- string_list_clear(&affected_refnames, 0);
string_list_clear(&refnames_to_check, 0);
if (ret)
@@ -3050,13 +3016,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (transaction->state != REF_TRANSACTION_PREPARED)
BUG("commit called for transaction that is not prepared");
- /* Fail if a refname appears more than once in the transaction: */
- for (i = 0; i < transaction->nr; i++)
- if (!(transaction->updates[i]->flags & REF_LOG_ONLY))
- string_list_append(&affected_refnames,
- transaction->updates[i]->refname);
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
+ string_list_sort(&transaction->refnames);
+ if (ref_update_reject_duplicates(&transaction->refnames, err)) {
ret = TRANSACTION_GENERIC_ERROR;
goto cleanup;
}
@@ -3074,7 +3035,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
* that we are creating already exists.
*/
if (refs_for_each_rawref(&refs->base, ref_present,
- &affected_refnames))
+ &transaction->refnames))
BUG("initial ref transaction called with existing refs");
packed_transaction = ref_store_transaction_begin(refs->packed_ref_store,
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index f4c82ba2c7..19220d2e99 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1622,8 +1622,6 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
struct packed_transaction_backend_data {
/* True iff the transaction owns the packed-refs lock. */
int own_lock;
-
- struct string_list updates;
};
static void packed_transaction_cleanup(struct packed_ref_store *refs,
@@ -1632,8 +1630,6 @@ static void packed_transaction_cleanup(struct packed_ref_store *refs,
struct packed_transaction_backend_data *data = transaction->backend_data;
if (data) {
- string_list_clear(&data->updates, 0);
-
if (is_tempfile_active(refs->tempfile))
delete_tempfile(&refs->tempfile);
@@ -1658,7 +1654,6 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
"ref_transaction_prepare");
struct packed_transaction_backend_data *data;
- size_t i;
int ret = TRANSACTION_GENERIC_ERROR;
/*
@@ -1671,34 +1666,16 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
*/
CALLOC_ARRAY(data, 1);
- string_list_init_nodup(&data->updates);
transaction->backend_data = data;
- /*
- * Stick the updates in a string list by refname so that we
- * can sort them:
- */
- for (i = 0; i < transaction->nr; i++) {
- struct ref_update *update = transaction->updates[i];
- struct string_list_item *item =
- string_list_append(&data->updates, update->refname);
-
- /* Store a pointer to update in item->util: */
- item->util = update;
- }
- string_list_sort(&data->updates);
-
- if (ref_update_reject_duplicates(&data->updates, err))
- goto failure;
-
if (!is_lock_file_locked(&refs->lock)) {
if (packed_refs_lock(ref_store, 0, err))
goto failure;
data->own_lock = 1;
}
- if (write_with_updates(refs, &data->updates, err))
+ if (write_with_updates(refs, &transaction->refnames, err))
goto failure;
transaction->state = REF_TRANSACTION_PREPARED;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index e5862757a7..92db793026 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -3,6 +3,7 @@
#include "refs.h"
#include "iterator.h"
+#include "string-list.h"
struct fsck_options;
struct ref_transaction;
@@ -198,6 +199,7 @@ enum ref_transaction_state {
struct ref_transaction {
struct ref_store *ref_store;
struct ref_update **updates;
+ struct string_list refnames;
size_t alloc;
size_t nr;
enum ref_transaction_state state;
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index ae434cd248..a92c9a2f4f 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1076,7 +1076,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
struct reftable_ref_store *refs =
reftable_be_downcast(ref_store, REF_STORE_WRITE|REF_STORE_MAIN, "ref_transaction_prepare");
struct strbuf referent = STRBUF_INIT, head_referent = STRBUF_INIT;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
struct string_list refnames_to_check = STRING_LIST_INIT_NODUP;
struct reftable_transaction_data *tx_data = NULL;
struct reftable_backend *be;
@@ -1101,10 +1100,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->updates[i], err);
if (ret)
goto done;
-
- if (!(transaction->updates[i]->flags & REF_LOG_ONLY))
- string_list_append(&affected_refnames,
- transaction->updates[i]->refname);
}
/*
@@ -1116,17 +1111,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
tx_data->args[i].updates_alloc = tx_data->args[i].updates_expected;
}
- /*
- * Fail if a refname appears more than once in the transaction.
- * This code is taken from the files backend and is a good candidate to
- * be moved into the generic layer.
- */
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto done;
- }
-
/*
* TODO: it's dubious whether we should reload the stack that "HEAD"
* belongs to or not. In theory, it may happen that we only modify
@@ -1194,14 +1178,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
!(u->flags & REF_LOG_ONLY) &&
!(u->flags & REF_UPDATE_VIA_HEAD) &&
!strcmp(rewritten_ref, head_referent.buf)) {
- struct ref_update *new_update;
-
/*
* First make sure that HEAD is not already in the
* transaction. This check is O(lg N) in the transaction
* size, but it happens at most once per transaction.
*/
- if (string_list_has_string(&affected_refnames, "HEAD")) {
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
/* An entry already existed */
strbuf_addf(err,
_("multiple updates for 'HEAD' (including one "
@@ -1211,12 +1193,11 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
goto done;
}
- new_update = ref_transaction_add_update(
- transaction, "HEAD",
- u->flags | REF_LOG_ONLY | REF_NO_DEREF,
- &u->new_oid, &u->old_oid, NULL, NULL, NULL,
- u->msg);
- string_list_insert(&affected_refnames, new_update->refname);
+ ref_transaction_add_update(
+ transaction, "HEAD",
+ u->flags | REF_LOG_ONLY | REF_NO_DEREF,
+ &u->new_oid, &u->old_oid, NULL, NULL, NULL,
+ u->msg);
}
ret = reftable_backend_read_ref(be, rewritten_ref,
@@ -1281,6 +1262,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
if (!strcmp(rewritten_ref, "HEAD"))
new_flags |= REF_UPDATE_VIA_HEAD;
+ if (string_list_has_string(&transaction->refnames, referent.buf)) {
+ strbuf_addf(err,
+ _("multiple updates for '%s' (including one "
+ "via symref '%s') are not allowed"),
+ referent.buf, u->refname);
+ ret = TRANSACTION_NAME_CONFLICT;
+ goto done;
+ }
+
/*
* If we are updating a symref (eg. HEAD), we should also
* update the branch that the symref points to.
@@ -1305,16 +1295,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
*/
u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
u->flags &= ~REF_HAVE_OLD;
-
- if (string_list_has_string(&affected_refnames, new_update->refname)) {
- strbuf_addf(err,
- _("multiple updates for '%s' (including one "
- "via symref '%s') are not allowed"),
- referent.buf, u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
- string_list_insert(&affected_refnames, new_update->refname);
}
}
@@ -1383,7 +1363,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
}
}
- ret = refs_verify_refnames_available(ref_store, &refnames_to_check, &affected_refnames, NULL,
+ ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
+ &transaction->refnames, NULL,
transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
err);
if (ret < 0)
@@ -1401,7 +1382,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
strbuf_addf(err, _("reftable: transaction prepare: %s"),
reftable_error_str(ret));
}
- string_list_clear(&affected_refnames, 0);
strbuf_release(&referent);
strbuf_release(&head_referent);
string_list_clear(&refnames_to_check, 0);
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v6 3/8] refs/files: remove duplicate duplicates check
2025-04-08 8:51 ` [PATCH v6 " Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 2/8] refs: move duplicate refname update check to generic layer Karthik Nayak
@ 2025-04-08 8:51 ` Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 4/8] refs/reftable: extract code from the transaction preparation Karthik Nayak
` (4 subsequent siblings)
7 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-04-08 8:51 UTC (permalink / raw)
To: karthik.188; +Cc: git, jltobler, ps, jn.avila, gitster
Within the files reference backend's transaction's 'finish' phase, a
verification step is currently performed wherein the refnames list is
sorted and examined for multiple updates targeting the same refname.
It has been observed that this verification is redundant, as an
identical check is already executed during the transaction's 'prepare'
stage. Since the refnames list remains unmodified following the
'prepare' stage, this secondary verification can be safely eliminated.
The duplicate check has been removed accordingly, and the
`ref_update_reject_duplicates()` function has been marked as static, as
its usage is now confined to 'refs.c'.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 9 +++++++--
refs/files-backend.c | 6 ------
refs/refs-internal.h | 8 --------
3 files changed, 7 insertions(+), 16 deletions(-)
diff --git a/refs.c b/refs.c
index 504bf2063e..61bed9672a 100644
--- a/refs.c
+++ b/refs.c
@@ -2303,8 +2303,13 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
return ret;
}
-int ref_update_reject_duplicates(struct string_list *refnames,
- struct strbuf *err)
+/*
+ * Write an error to `err` and return a nonzero value iff the same
+ * refname appears multiple times in `refnames`. `refnames` must be
+ * sorted on entry to this function.
+ */
+static int ref_update_reject_duplicates(struct string_list *refnames,
+ struct strbuf *err)
{
size_t i, n = refnames->nr;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 58f62ea8a3..ea023a59fc 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3016,12 +3016,6 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (transaction->state != REF_TRANSACTION_PREPARED)
BUG("commit called for transaction that is not prepared");
- string_list_sort(&transaction->refnames);
- if (ref_update_reject_duplicates(&transaction->refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
- }
-
/*
* It's really undefined to call this function in an active
* repository or when there are existing references: we are
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 92db793026..6d3770d0cc 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -142,14 +142,6 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
struct object_id *oid, struct strbuf *referent,
unsigned int *type, int *failure_errno);
-/*
- * Write an error to `err` and return a nonzero value iff the same
- * refname appears multiple times in `refnames`. `refnames` must be
- * sorted on entry to this function.
- */
-int ref_update_reject_duplicates(struct string_list *refnames,
- struct strbuf *err);
-
/*
* Add a ref_update with the specified properties to transaction, and
* return a pointer to the new object. This function does not verify
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v6 4/8] refs/reftable: extract code from the transaction preparation
2025-04-08 8:51 ` [PATCH v6 " Karthik Nayak
` (2 preceding siblings ...)
2025-04-08 8:51 ` [PATCH v6 3/8] refs/files: remove duplicate duplicates check Karthik Nayak
@ 2025-04-08 8:51 ` Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 5/8] refs: introduce enum-based transaction error types Karthik Nayak
` (3 subsequent siblings)
7 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-04-08 8:51 UTC (permalink / raw)
To: karthik.188; +Cc: git, jltobler, ps, jn.avila, gitster
Extract the core logic for preparing individual reference updates from
`reftable_be_transaction_prepare()` into `prepare_single_update()`. This
dedicated function now handles all validation and preparation steps for
each reference update in the transaction, including object ID
verification, HEAD reference handling, and symref processing.
The refactoring consolidates all reference update validation into a
single logical block, which improves code maintainability and
readability. More importantly, this restructuring lays the groundwork
for implementing batched reference update support in the reftable
backend, which will be introduced in a followup commit.
No functional changes are included in this commit - it is purely a code
reorganization to support future enhancements.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs/reftable-backend.c | 463 ++++++++++++++++++++--------------------
1 file changed, 237 insertions(+), 226 deletions(-)
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index a92c9a2f4f..786df11a03 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1069,6 +1069,239 @@ static int queue_transaction_update(struct reftable_ref_store *refs,
return 0;
}
+static int prepare_single_update(struct reftable_ref_store *refs,
+ struct reftable_transaction_data *tx_data,
+ struct ref_transaction *transaction,
+ struct reftable_backend *be,
+ struct ref_update *u,
+ struct string_list *refnames_to_check,
+ unsigned int head_type,
+ struct strbuf *head_referent,
+ struct strbuf *referent,
+ struct strbuf *err)
+{
+ struct object_id current_oid = {0};
+ const char *rewritten_ref;
+ int ret = 0;
+
+ /*
+ * There is no need to reload the respective backends here as
+ * we have already reloaded them when preparing the transaction
+ * update. And given that the stacks have been locked there
+ * shouldn't have been any concurrent modifications of the
+ * stack.
+ */
+ ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
+ if (ret)
+ return ret;
+
+ /* Verify that the new object ID is valid. */
+ if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
+ !(u->flags & REF_SKIP_OID_VERIFICATION) &&
+ !(u->flags & REF_LOG_ONLY)) {
+ struct object *o = parse_object(refs->base.repo, &u->new_oid);
+ if (!o) {
+ strbuf_addf(err,
+ _("trying to write ref '%s' with nonexistent object %s"),
+ u->refname, oid_to_hex(&u->new_oid));
+ return -1;
+ }
+
+ if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
+ strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
+ oid_to_hex(&u->new_oid), u->refname);
+ return -1;
+ }
+ }
+
+ /*
+ * When we update the reference that HEAD points to we enqueue
+ * a second log-only update for HEAD so that its reflog is
+ * updated accordingly.
+ */
+ if (head_type == REF_ISSYMREF &&
+ !(u->flags & REF_LOG_ONLY) &&
+ !(u->flags & REF_UPDATE_VIA_HEAD) &&
+ !strcmp(rewritten_ref, head_referent->buf)) {
+ /*
+ * First make sure that HEAD is not already in the
+ * transaction. This check is O(lg N) in the transaction
+ * size, but it happens at most once per transaction.
+ */
+ if (string_list_has_string(&transaction->refnames, "HEAD")) {
+ /* An entry already existed */
+ strbuf_addf(err,
+ _("multiple updates for 'HEAD' (including one "
+ "via its referent '%s') are not allowed"),
+ u->refname);
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ ref_transaction_add_update(
+ transaction, "HEAD",
+ u->flags | REF_LOG_ONLY | REF_NO_DEREF,
+ &u->new_oid, &u->old_oid, NULL, NULL, NULL,
+ u->msg);
+ }
+
+ ret = reftable_backend_read_ref(be, rewritten_ref,
+ ¤t_oid, referent, &u->type);
+ if (ret < 0)
+ return ret;
+ if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
+ /*
+ * The reference does not exist, and we either have no
+ * old object ID or expect the reference to not exist.
+ * We can thus skip below safety checks as well as the
+ * symref splitting. But we do want to verify that
+ * there is no conflicting reference here so that we
+ * can output a proper error message instead of failing
+ * at a later point.
+ */
+ string_list_append(refnames_to_check, u->refname);
+
+ /*
+ * There is no need to write the reference deletion
+ * when the reference in question doesn't exist.
+ */
+ if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
+ ret = queue_transaction_update(refs, tx_data, u,
+ ¤t_oid, err);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+ }
+ if (ret > 0) {
+ /* The reference does not exist, but we expected it to. */
+ strbuf_addf(err, _("cannot lock ref '%s': "
+
+
+ "unable to resolve reference '%s'"),
+ ref_update_original_update_refname(u), u->refname);
+ return -1;
+ }
+
+ if (u->type & REF_ISSYMREF) {
+ /*
+ * The reftable stack is locked at this point already,
+ * so it is safe to call `refs_resolve_ref_unsafe()`
+ * here without causing races.
+ */
+ const char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,
+ ¤t_oid, NULL);
+
+ if (u->flags & REF_NO_DEREF) {
+ if (u->flags & REF_HAVE_OLD && !resolved) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "error reading reference"), u->refname);
+ return -1;
+ }
+ } else {
+ struct ref_update *new_update;
+ int new_flags;
+
+ new_flags = u->flags;
+ if (!strcmp(rewritten_ref, "HEAD"))
+ new_flags |= REF_UPDATE_VIA_HEAD;
+
+ if (string_list_has_string(&transaction->refnames, referent->buf)) {
+ strbuf_addf(err,
+ _("multiple updates for '%s' (including one "
+ "via symref '%s') are not allowed"),
+ referent->buf, u->refname);
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ /*
+ * If we are updating a symref (eg. HEAD), we should also
+ * update the branch that the symref points to.
+ *
+ * This is generic functionality, and would be better
+ * done in refs.c, but the current implementation is
+ * intertwined with the locking in files-backend.c.
+ */
+ new_update = ref_transaction_add_update(
+ transaction, referent->buf, new_flags,
+ u->new_target ? NULL : &u->new_oid,
+ u->old_target ? NULL : &u->old_oid,
+ u->new_target, u->old_target,
+ u->committer_info, u->msg);
+
+ new_update->parent_update = u;
+
+ /*
+ * Change the symbolic ref update to log only. Also, it
+ * doesn't need to check its old OID value, as that will be
+ * done when new_update is processed.
+ */
+ u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
+ u->flags &= ~REF_HAVE_OLD;
+ }
+ }
+
+ /*
+ * Verify that the old object matches our expectations. Note
+ * that the error messages here do not make a lot of sense in
+ * the context of the reftable backend as we never lock
+ * individual refs. But the error messages match what the files
+ * backend returns, which keeps our tests happy.
+ */
+ if (u->old_target) {
+ if (!(u->type & REF_ISSYMREF)) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "expected symref with target '%s': "
+ "but is a regular ref"),
+ ref_update_original_update_refname(u),
+ u->old_target);
+ return -1;
+ }
+
+ if (ref_update_check_old_target(referent->buf, u, err)) {
+ return -1;
+ }
+ } else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
+ if (is_null_oid(&u->old_oid)) {
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "reference already exists"),
+ ref_update_original_update_refname(u));
+ return TRANSACTION_CREATE_EXISTS;
+ }
+ else if (is_null_oid(¤t_oid))
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "reference is missing but expected %s"),
+ ref_update_original_update_refname(u),
+ oid_to_hex(&u->old_oid));
+ else
+ strbuf_addf(err, _("cannot lock ref '%s': "
+ "is at %s but expected %s"),
+ ref_update_original_update_refname(u),
+ oid_to_hex(¤t_oid),
+ oid_to_hex(&u->old_oid));
+ return TRANSACTION_NAME_CONFLICT;
+ }
+
+ /*
+ * If all of the following conditions are true:
+ *
+ * - We're not about to write a symref.
+ * - We're not about to write a log-only entry.
+ * - Old and new object ID are different.
+ *
+ * Then we're essentially doing a no-op update that can be
+ * skipped. This is not only for the sake of efficiency, but
+ * also skips writing unneeded reflog entries.
+ */
+ if ((u->type & REF_ISSYMREF) ||
+ (u->flags & REF_LOG_ONLY) ||
+ (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))
+ return queue_transaction_update(refs, tx_data, u,
+ ¤t_oid, err);
+
+ return 0;
+}
+
static int reftable_be_transaction_prepare(struct ref_store *ref_store,
struct ref_transaction *transaction,
struct strbuf *err)
@@ -1133,234 +1366,12 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
ret = 0;
for (i = 0; i < transaction->nr; i++) {
- struct ref_update *u = transaction->updates[i];
- struct object_id current_oid = {0};
- const char *rewritten_ref;
-
- /*
- * There is no need to reload the respective backends here as
- * we have already reloaded them when preparing the transaction
- * update. And given that the stacks have been locked there
- * shouldn't have been any concurrent modifications of the
- * stack.
- */
- ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
+ ret = prepare_single_update(refs, tx_data, transaction, be,
+ transaction->updates[i],
+ &refnames_to_check, head_type,
+ &head_referent, &referent, err);
if (ret)
goto done;
-
- /* Verify that the new object ID is valid. */
- if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
- !(u->flags & REF_SKIP_OID_VERIFICATION) &&
- !(u->flags & REF_LOG_ONLY)) {
- struct object *o = parse_object(refs->base.repo, &u->new_oid);
- if (!o) {
- strbuf_addf(err,
- _("trying to write ref '%s' with nonexistent object %s"),
- u->refname, oid_to_hex(&u->new_oid));
- ret = -1;
- goto done;
- }
-
- if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
- strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
- oid_to_hex(&u->new_oid), u->refname);
- ret = -1;
- goto done;
- }
- }
-
- /*
- * When we update the reference that HEAD points to we enqueue
- * a second log-only update for HEAD so that its reflog is
- * updated accordingly.
- */
- if (head_type == REF_ISSYMREF &&
- !(u->flags & REF_LOG_ONLY) &&
- !(u->flags & REF_UPDATE_VIA_HEAD) &&
- !strcmp(rewritten_ref, head_referent.buf)) {
- /*
- * First make sure that HEAD is not already in the
- * transaction. This check is O(lg N) in the transaction
- * size, but it happens at most once per transaction.
- */
- if (string_list_has_string(&transaction->refnames, "HEAD")) {
- /* An entry already existed */
- strbuf_addf(err,
- _("multiple updates for 'HEAD' (including one "
- "via its referent '%s') are not allowed"),
- u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
-
- ref_transaction_add_update(
- transaction, "HEAD",
- u->flags | REF_LOG_ONLY | REF_NO_DEREF,
- &u->new_oid, &u->old_oid, NULL, NULL, NULL,
- u->msg);
- }
-
- ret = reftable_backend_read_ref(be, rewritten_ref,
- ¤t_oid, &referent, &u->type);
- if (ret < 0)
- goto done;
- if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
- /*
- * The reference does not exist, and we either have no
- * old object ID or expect the reference to not exist.
- * We can thus skip below safety checks as well as the
- * symref splitting. But we do want to verify that
- * there is no conflicting reference here so that we
- * can output a proper error message instead of failing
- * at a later point.
- */
- string_list_append(&refnames_to_check, u->refname);
-
- /*
- * There is no need to write the reference deletion
- * when the reference in question doesn't exist.
- */
- if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
- ret = queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
- if (ret)
- goto done;
- }
-
- continue;
- }
- if (ret > 0) {
- /* The reference does not exist, but we expected it to. */
- strbuf_addf(err, _("cannot lock ref '%s': "
- "unable to resolve reference '%s'"),
- ref_update_original_update_refname(u), u->refname);
- ret = -1;
- goto done;
- }
-
- if (u->type & REF_ISSYMREF) {
- /*
- * The reftable stack is locked at this point already,
- * so it is safe to call `refs_resolve_ref_unsafe()`
- * here without causing races.
- */
- const char *resolved = refs_resolve_ref_unsafe(&refs->base, u->refname, 0,
- ¤t_oid, NULL);
-
- if (u->flags & REF_NO_DEREF) {
- if (u->flags & REF_HAVE_OLD && !resolved) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "error reading reference"), u->refname);
- ret = -1;
- goto done;
- }
- } else {
- struct ref_update *new_update;
- int new_flags;
-
- new_flags = u->flags;
- if (!strcmp(rewritten_ref, "HEAD"))
- new_flags |= REF_UPDATE_VIA_HEAD;
-
- if (string_list_has_string(&transaction->refnames, referent.buf)) {
- strbuf_addf(err,
- _("multiple updates for '%s' (including one "
- "via symref '%s') are not allowed"),
- referent.buf, u->refname);
- ret = TRANSACTION_NAME_CONFLICT;
- goto done;
- }
-
- /*
- * If we are updating a symref (eg. HEAD), we should also
- * update the branch that the symref points to.
- *
- * This is generic functionality, and would be better
- * done in refs.c, but the current implementation is
- * intertwined with the locking in files-backend.c.
- */
- new_update = ref_transaction_add_update(
- transaction, referent.buf, new_flags,
- u->new_target ? NULL : &u->new_oid,
- u->old_target ? NULL : &u->old_oid,
- u->new_target, u->old_target,
- u->committer_info, u->msg);
-
- new_update->parent_update = u;
-
- /*
- * Change the symbolic ref update to log only. Also, it
- * doesn't need to check its old OID value, as that will be
- * done when new_update is processed.
- */
- u->flags |= REF_LOG_ONLY | REF_NO_DEREF;
- u->flags &= ~REF_HAVE_OLD;
- }
- }
-
- /*
- * Verify that the old object matches our expectations. Note
- * that the error messages here do not make a lot of sense in
- * the context of the reftable backend as we never lock
- * individual refs. But the error messages match what the files
- * backend returns, which keeps our tests happy.
- */
- if (u->old_target) {
- if (!(u->type & REF_ISSYMREF)) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "expected symref with target '%s': "
- "but is a regular ref"),
- ref_update_original_update_refname(u),
- u->old_target);
- ret = -1;
- goto done;
- }
-
- if (ref_update_check_old_target(referent.buf, u, err)) {
- ret = -1;
- goto done;
- }
- } else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
- ret = TRANSACTION_NAME_CONFLICT;
- if (is_null_oid(&u->old_oid)) {
- strbuf_addf(err, _("cannot lock ref '%s': "
- "reference already exists"),
- ref_update_original_update_refname(u));
- ret = TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(¤t_oid))
- strbuf_addf(err, _("cannot lock ref '%s': "
- "reference is missing but expected %s"),
- ref_update_original_update_refname(u),
- oid_to_hex(&u->old_oid));
- else
- strbuf_addf(err, _("cannot lock ref '%s': "
- "is at %s but expected %s"),
- ref_update_original_update_refname(u),
- oid_to_hex(¤t_oid),
- oid_to_hex(&u->old_oid));
- goto done;
- }
-
- /*
- * If all of the following conditions are true:
- *
- * - We're not about to write a symref.
- * - We're not about to write a log-only entry.
- * - Old and new object ID are different.
- *
- * Then we're essentially doing a no-op update that can be
- * skipped. This is not only for the sake of efficiency, but
- * also skips writing unneeded reflog entries.
- */
- if ((u->type & REF_ISSYMREF) ||
- (u->flags & REF_LOG_ONLY) ||
- (u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid))) {
- ret = queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
- if (ret)
- goto done;
- }
}
ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v6 5/8] refs: introduce enum-based transaction error types
2025-04-08 8:51 ` [PATCH v6 " Karthik Nayak
` (3 preceding siblings ...)
2025-04-08 8:51 ` [PATCH v6 4/8] refs/reftable: extract code from the transaction preparation Karthik Nayak
@ 2025-04-08 8:51 ` Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 6/8] refs: implement batch reference update support Karthik Nayak
` (2 subsequent siblings)
7 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-04-08 8:51 UTC (permalink / raw)
To: karthik.188; +Cc: git, jltobler, ps, jn.avila, gitster
Replace preprocessor-defined transaction errors with a strongly-typed
enum `ref_transaction_error`. This change:
- Improves type safety and function signature clarity.
- Makes error handling more explicit and discoverable.
- Maintains existing error cases, while adding new error cases for
common scenarios.
This refactoring paves the way for more comprehensive error handling
which we will utilize in the upcoming commits to add batch reference
update support.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
builtin/fetch.c | 2 +-
refs.c | 49 +++++-----
refs.h | 48 ++++++----
refs/files-backend.c | 202 ++++++++++++++++++++--------------------
refs/packed-backend.c | 23 +++--
refs/refs-internal.h | 5 +-
refs/reftable-backend.c | 64 ++++++-------
7 files changed, 207 insertions(+), 186 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 95fd0018b9..7615c17faf 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -687,7 +687,7 @@ static int s_update_ref(const char *action,
switch (ref_transaction_commit(our_transaction, &err)) {
case 0:
break;
- case TRANSACTION_NAME_CONFLICT:
+ case REF_TRANSACTION_ERROR_NAME_CONFLICT:
ret = STORE_REF_ERROR_DF_CONFLICT;
goto out;
default:
diff --git a/refs.c b/refs.c
index 61bed9672a..3d0b53d56e 100644
--- a/refs.c
+++ b/refs.c
@@ -2271,7 +2271,7 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
REF_NO_DEREF, logmsg, &err))
goto error_return;
prepret = ref_transaction_prepare(transaction, &err);
- if (prepret && prepret != TRANSACTION_CREATE_EXISTS)
+ if (prepret && prepret != REF_TRANSACTION_ERROR_CREATE_EXISTS)
goto error_return;
} else {
if (ref_transaction_update(transaction, ref, NULL, NULL,
@@ -2289,7 +2289,7 @@ int refs_update_symref_extended(struct ref_store *refs, const char *ref,
}
}
- if (prepret == TRANSACTION_CREATE_EXISTS)
+ if (prepret == REF_TRANSACTION_ERROR_CREATE_EXISTS)
goto cleanup;
if (ref_transaction_commit(transaction, &err))
@@ -2425,7 +2425,7 @@ int ref_transaction_prepare(struct ref_transaction *transaction,
string_list_sort(&transaction->refnames);
if (ref_update_reject_duplicates(&transaction->refnames, err))
- return TRANSACTION_GENERIC_ERROR;
+ return REF_TRANSACTION_ERROR_GENERIC;
ret = refs->be->transaction_prepare(refs, transaction, err);
if (ret)
@@ -2497,19 +2497,19 @@ int ref_transaction_commit(struct ref_transaction *transaction,
return ret;
}
-int refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *refnames,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err)
+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err)
{
struct strbuf dirname = STRBUF_INIT;
struct strbuf referent = STRBUF_INIT;
struct string_list_item *item;
struct ref_iterator *iter = NULL;
struct strset dirnames;
- int ret = -1;
+ int ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
/*
* For the sake of comments in this function, suppose that
@@ -2625,12 +2625,13 @@ int refs_verify_refnames_available(struct ref_store *refs,
return ret;
}
-int refs_verify_refname_available(struct ref_store *refs,
- const char *refname,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err)
+enum ref_transaction_error refs_verify_refname_available(
+ struct ref_store *refs,
+ const char *refname,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err)
{
struct string_list_item item = { .string = (char *) refname };
struct string_list refnames = {
@@ -2818,8 +2819,9 @@ int ref_update_has_null_new_value(struct ref_update *update)
return !update->new_target && is_null_oid(&update->new_oid);
}
-int ref_update_check_old_target(const char *referent, struct ref_update *update,
- struct strbuf *err)
+enum ref_transaction_error ref_update_check_old_target(const char *referent,
+ struct ref_update *update,
+ struct strbuf *err)
{
if (!update->old_target)
BUG("called without old_target set");
@@ -2827,17 +2829,18 @@ int ref_update_check_old_target(const char *referent, struct ref_update *update,
if (!strcmp(referent, update->old_target))
return 0;
- if (!strcmp(referent, ""))
+ if (!strcmp(referent, "")) {
strbuf_addf(err, "verifying symref target: '%s': "
"reference is missing but expected %s",
ref_update_original_update_refname(update),
update->old_target);
- else
- strbuf_addf(err, "verifying symref target: '%s': "
- "is at %s but expected %s",
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ }
+
+ strbuf_addf(err, "verifying symref target: '%s': is at %s but expected %s",
ref_update_original_update_refname(update),
referent, update->old_target);
- return -1;
+ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
}
struct migration_data {
diff --git a/refs.h b/refs.h
index 240e2d8537..f009cdae7d 100644
--- a/refs.h
+++ b/refs.h
@@ -16,6 +16,23 @@ struct worktree;
enum ref_storage_format ref_storage_format_by_name(const char *name);
const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format);
+enum ref_transaction_error {
+ /* Default error code */
+ REF_TRANSACTION_ERROR_GENERIC = -1,
+ /* Ref name conflict like A vs A/B */
+ REF_TRANSACTION_ERROR_NAME_CONFLICT = -2,
+ /* Ref to be created already exists */
+ REF_TRANSACTION_ERROR_CREATE_EXISTS = -3,
+ /* ref expected but doesn't exist */
+ REF_TRANSACTION_ERROR_NONEXISTENT_REF = -4,
+ /* Provided old_oid or old_target of reference doesn't match actual */
+ REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE = -5,
+ /* Provided new_oid or new_target is invalid */
+ REF_TRANSACTION_ERROR_INVALID_NEW_VALUE = -6,
+ /* Expected ref to be symref, but is a regular ref */
+ REF_TRANSACTION_ERROR_EXPECTED_SYMREF = -7,
+};
+
/*
* Resolve a reference, recursively following symbolic references.
*
@@ -117,24 +134,24 @@ int refs_read_symbolic_ref(struct ref_store *ref_store, const char *refname,
*
* extras and skip must be sorted.
*/
-int refs_verify_refname_available(struct ref_store *refs,
- const char *refname,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
+enum ref_transaction_error refs_verify_refname_available(struct ref_store *refs,
+ const char *refname,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err);
/*
* Same as `refs_verify_refname_available()`, but checking for a list of
* refnames instead of only a single item. This is more efficient in the case
* where one needs to check multiple refnames.
*/
-int refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *refnames,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ unsigned int initial_transaction,
+ struct strbuf *err);
int refs_ref_exists(struct ref_store *refs, const char *refname);
@@ -830,13 +847,6 @@ int ref_transaction_verify(struct ref_transaction *transaction,
unsigned int flags,
struct strbuf *err);
-/* Naming conflict (for example, the ref names A and A/B conflict). */
-#define TRANSACTION_NAME_CONFLICT -1
-/* When only creation was requested, but the ref already exists. */
-#define TRANSACTION_CREATE_EXISTS -2
-/* All other errors. */
-#define TRANSACTION_GENERIC_ERROR -3
-
/*
* Perform the preparatory stages of committing `transaction`. Acquire
* any needed locks, check preconditions, etc.; basically, do as much
diff --git a/refs/files-backend.c b/refs/files-backend.c
index ea023a59fc..4f27f7652c 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -663,7 +663,7 @@ static void unlock_ref(struct ref_lock *lock)
* broken, lock the reference anyway but clear old_oid.
*
* Return 0 on success. On failure, write an error message to err and
- * return TRANSACTION_NAME_CONFLICT or TRANSACTION_GENERIC_ERROR.
+ * return REF_TRANSACTION_ERROR_NAME_CONFLICT or REF_TRANSACTION_ERROR_GENERIC.
*
* Implementation note: This function is basically
*
@@ -676,19 +676,20 @@ static void unlock_ref(struct ref_lock *lock)
* avoided, namely if we were successfully able to read the ref
* - Generate informative error messages in the case of failure
*/
-static int lock_raw_ref(struct files_ref_store *refs,
- const char *refname, int mustexist,
- struct string_list *refnames_to_check,
- const struct string_list *extras,
- struct ref_lock **lock_p,
- struct strbuf *referent,
- unsigned int *type,
- struct strbuf *err)
-{
+static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
+ const char *refname,
+ int mustexist,
+ struct string_list *refnames_to_check,
+ const struct string_list *extras,
+ struct ref_lock **lock_p,
+ struct strbuf *referent,
+ unsigned int *type,
+ struct strbuf *err)
+{
+ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
struct ref_lock *lock;
struct strbuf ref_file = STRBUF_INIT;
int attempts_remaining = 3;
- int ret = TRANSACTION_GENERIC_ERROR;
int failure_errno;
assert(err);
@@ -728,13 +729,14 @@ static int lock_raw_ref(struct files_ref_store *refs,
strbuf_reset(err);
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
} else {
/*
* The error message set by
* refs_verify_refname_available() is
* OK.
*/
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
} else {
/*
@@ -788,6 +790,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
/* Garden variety missing reference. */
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error_return;
} else {
/*
@@ -820,6 +823,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
/* Garden variety missing reference. */
strbuf_addf(err, "unable to resolve reference '%s'",
refname);
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error_return;
} else if (remove_dir_recursively(&ref_file,
REMOVE_DIR_EMPTY_ONLY)) {
@@ -830,7 +834,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
* The error message set by
* verify_refname_available() is OK.
*/
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto error_return;
} else {
/*
@@ -1517,10 +1521,11 @@ static int rename_tmp_log(struct files_ref_store *refs, const char *newrefname)
return ret;
}
-static int write_ref_to_lockfile(struct files_ref_store *refs,
- struct ref_lock *lock,
- const struct object_id *oid,
- int skip_oid_verification, struct strbuf *err);
+static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs,
+ struct ref_lock *lock,
+ const struct object_id *oid,
+ int skip_oid_verification,
+ struct strbuf *err);
static int commit_ref_update(struct files_ref_store *refs,
struct ref_lock *lock,
const struct object_id *oid, const char *logmsg,
@@ -1926,10 +1931,11 @@ static int files_log_ref_write(struct files_ref_store *refs,
* Write oid into the open lockfile, then close the lockfile. On
* errors, rollback the lockfile, fill in *err and return -1.
*/
-static int write_ref_to_lockfile(struct files_ref_store *refs,
- struct ref_lock *lock,
- const struct object_id *oid,
- int skip_oid_verification, struct strbuf *err)
+static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs,
+ struct ref_lock *lock,
+ const struct object_id *oid,
+ int skip_oid_verification,
+ struct strbuf *err)
{
static char term = '\n';
struct object *o;
@@ -1943,7 +1949,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
"trying to write ref '%s' with nonexistent object %s",
lock->ref_name, oid_to_hex(oid));
unlock_ref(lock);
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
if (o->type != OBJ_COMMIT && is_branch(lock->ref_name)) {
strbuf_addf(
@@ -1951,7 +1957,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
"trying to write non-commit object %s to branch '%s'",
oid_to_hex(oid), lock->ref_name);
unlock_ref(lock);
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
}
fd = get_lock_file_fd(&lock->lk);
@@ -1962,7 +1968,7 @@ static int write_ref_to_lockfile(struct files_ref_store *refs,
strbuf_addf(err,
"couldn't write '%s'", get_lock_file_path(&lock->lk));
unlock_ref(lock);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
return 0;
}
@@ -2376,9 +2382,10 @@ static struct ref_iterator *files_reflog_iterator_begin(struct ref_store *ref_st
* If update is a direct update of head_ref (the reference pointed to
* by HEAD), then add an extra REF_LOG_ONLY update for HEAD.
*/
-static int split_head_update(struct ref_update *update,
- struct ref_transaction *transaction,
- const char *head_ref, struct strbuf *err)
+static enum ref_transaction_error split_head_update(struct ref_update *update,
+ struct ref_transaction *transaction,
+ const char *head_ref,
+ struct strbuf *err)
{
struct ref_update *new_update;
@@ -2402,7 +2409,7 @@ static int split_head_update(struct ref_update *update,
"multiple updates for 'HEAD' (including one "
"via its referent '%s') are not allowed",
update->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
new_update = ref_transaction_add_update(
@@ -2430,10 +2437,10 @@ static int split_head_update(struct ref_update *update,
* Note that the new update will itself be subject to splitting when
* the iteration gets to it.
*/
-static int split_symref_update(struct ref_update *update,
- const char *referent,
- struct ref_transaction *transaction,
- struct strbuf *err)
+static enum ref_transaction_error split_symref_update(struct ref_update *update,
+ const char *referent,
+ struct ref_transaction *transaction,
+ struct strbuf *err)
{
struct ref_update *new_update;
unsigned int new_flags;
@@ -2450,7 +2457,7 @@ static int split_symref_update(struct ref_update *update,
"multiple updates for '%s' (including one "
"via symref '%s') are not allowed",
referent, update->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
new_flags = update->flags;
@@ -2491,11 +2498,10 @@ static int split_symref_update(struct ref_update *update,
* everything is OK, return 0; otherwise, write an error message to
* err and return -1.
*/
-static int check_old_oid(struct ref_update *update, struct object_id *oid,
- struct strbuf *err)
+static enum ref_transaction_error check_old_oid(struct ref_update *update,
+ struct object_id *oid,
+ struct strbuf *err)
{
- int ret = TRANSACTION_GENERIC_ERROR;
-
if (!(update->flags & REF_HAVE_OLD) ||
oideq(oid, &update->old_oid))
return 0;
@@ -2504,21 +2510,20 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid,
strbuf_addf(err, "cannot lock ref '%s': "
"reference already exists",
ref_update_original_update_refname(update));
- ret = TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(oid))
+ return REF_TRANSACTION_ERROR_CREATE_EXISTS;
+ } else if (is_null_oid(oid)) {
strbuf_addf(err, "cannot lock ref '%s': "
"reference is missing but expected %s",
ref_update_original_update_refname(update),
oid_to_hex(&update->old_oid));
- else
- strbuf_addf(err, "cannot lock ref '%s': "
- "is at %s but expected %s",
- ref_update_original_update_refname(update),
- oid_to_hex(oid),
- oid_to_hex(&update->old_oid));
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ }
- return ret;
+ strbuf_addf(err, "cannot lock ref '%s': is at %s but expected %s",
+ ref_update_original_update_refname(update), oid_to_hex(oid),
+ oid_to_hex(&update->old_oid));
+
+ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
}
struct files_transaction_backend_data {
@@ -2540,17 +2545,17 @@ struct files_transaction_backend_data {
* - If it is an update of head_ref, add a corresponding REF_LOG_ONLY
* update of HEAD.
*/
-static int lock_ref_for_update(struct files_ref_store *refs,
- struct ref_update *update,
- struct ref_transaction *transaction,
- const char *head_ref,
- struct string_list *refnames_to_check,
- struct strbuf *err)
+static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *refs,
+ struct ref_update *update,
+ struct ref_transaction *transaction,
+ const char *head_ref,
+ struct string_list *refnames_to_check,
+ struct strbuf *err)
{
struct strbuf referent = STRBUF_INIT;
int mustexist = ref_update_expects_existing_old_ref(update);
struct files_transaction_backend_data *backend_data;
- int ret = 0;
+ enum ref_transaction_error ret = 0;
struct ref_lock *lock;
files_assert_main_repository(refs, "lock_ref_for_update");
@@ -2602,22 +2607,17 @@ static int lock_ref_for_update(struct files_ref_store *refs,
strbuf_addf(err, "cannot lock ref '%s': "
"error reading reference",
ref_update_original_update_refname(update));
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
}
- if (update->old_target) {
- if (ref_update_check_old_target(referent.buf, update, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto out;
- }
- } else {
+ if (update->old_target)
+ ret = ref_update_check_old_target(referent.buf, update, err);
+ else
ret = check_old_oid(update, &lock->old_oid, err);
- if (ret) {
- goto out;
- }
- }
+ if (ret)
+ goto out;
} else {
/*
* Create a new update for the reference this
@@ -2644,7 +2644,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
"but is a regular ref"),
ref_update_original_update_refname(update),
update->old_target);
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_EXPECTED_SYMREF;
goto out;
} else {
ret = check_old_oid(update, &lock->old_oid, err);
@@ -2668,14 +2668,14 @@ static int lock_ref_for_update(struct files_ref_store *refs,
if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
if (create_symref_lock(lock, update->new_target, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
if (close_ref_gently(lock)) {
strbuf_addf(err, "couldn't close '%s.lock'",
update->refname);
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
@@ -2693,25 +2693,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
* The reference already has the desired
* value, so we don't need to write it.
*/
- } else if (write_ref_to_lockfile(
- refs, lock, &update->new_oid,
- update->flags & REF_SKIP_OID_VERIFICATION,
- err)) {
- char *write_err = strbuf_detach(err, NULL);
-
- /*
- * The lock was freed upon failure of
- * write_ref_to_lockfile():
- */
- update->backend_data = NULL;
- strbuf_addf(err,
- "cannot update ref '%s': %s",
- update->refname, write_err);
- free(write_err);
- ret = TRANSACTION_GENERIC_ERROR;
- goto out;
} else {
- update->flags |= REF_NEEDS_COMMIT;
+ ret = write_ref_to_lockfile(
+ refs, lock, &update->new_oid,
+ update->flags & REF_SKIP_OID_VERIFICATION,
+ err);
+ if (ret) {
+ char *write_err = strbuf_detach(err, NULL);
+
+ /*
+ * The lock was freed upon failure of
+ * write_ref_to_lockfile():
+ */
+ update->backend_data = NULL;
+ strbuf_addf(err,
+ "cannot update ref '%s': %s",
+ update->refname, write_err);
+ free(write_err);
+ goto out;
+ } else {
+ update->flags |= REF_NEEDS_COMMIT;
+ }
}
}
if (!(update->flags & REF_NEEDS_COMMIT)) {
@@ -2723,7 +2725,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
if (close_ref_gently(lock)) {
strbuf_addf(err, "couldn't close '%s.lock'",
update->refname);
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto out;
}
}
@@ -2865,7 +2867,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
refs->packed_ref_store,
transaction->flags, err);
if (!packed_transaction) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
@@ -2897,13 +2899,13 @@ static int files_transaction_prepare(struct ref_store *ref_store,
*/
if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
&transaction->refnames, NULL, 0, err)) {
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
}
if (packed_transaction) {
if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
backend_data->packed_refs_locked = 1;
@@ -2934,7 +2936,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
*/
backend_data->packed_transaction = NULL;
if (ref_transaction_abort(packed_transaction, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3035,7 +3037,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
packed_transaction = ref_store_transaction_begin(refs->packed_ref_store,
transaction->flags, err);
if (!packed_transaction) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
@@ -3058,7 +3060,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (!loose_transaction) {
loose_transaction = ref_store_transaction_begin(&refs->base, 0, err);
if (!loose_transaction) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3083,19 +3085,19 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
}
if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
if (refs_verify_refnames_available(&refs->base, &refnames_to_check,
&affected_refnames, NULL, 1, err)) {
packed_refs_unlock(refs->packed_ref_store);
- ret = TRANSACTION_NAME_CONFLICT;
+ ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
}
if (ref_transaction_commit(packed_transaction, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
packed_refs_unlock(refs->packed_ref_store);
@@ -3103,7 +3105,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if (loose_transaction) {
if (ref_transaction_prepare(loose_transaction, err) ||
ref_transaction_commit(loose_transaction, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3152,7 +3154,7 @@ static int files_transaction_finish(struct ref_store *ref_store,
if (update->flags & REF_NEEDS_COMMIT ||
update->flags & REF_LOG_ONLY) {
if (parse_and_write_reflog(refs, update, lock, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3171,7 +3173,7 @@ static int files_transaction_finish(struct ref_store *ref_store,
strbuf_addf(err, "couldn't set '%s'", lock->ref_name);
unlock_ref(lock);
update->backend_data = NULL;
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
@@ -3227,7 +3229,7 @@ static int files_transaction_finish(struct ref_store *ref_store,
strbuf_reset(&sb);
files_ref_path(refs, &sb, lock->ref_name);
if (unlink_or_msg(sb.buf, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ ret = REF_TRANSACTION_ERROR_GENERIC;
goto cleanup;
}
}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 19220d2e99..d90bd815a3 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1326,10 +1326,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
* The packfile must be locked before calling this function and will
* remain locked when it is done.
*/
-static int write_with_updates(struct packed_ref_store *refs,
- struct string_list *updates,
- struct strbuf *err)
+static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,
+ struct string_list *updates,
+ struct strbuf *err)
{
+ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
struct ref_iterator *iter = NULL;
size_t i;
int ok;
@@ -1353,7 +1354,7 @@ static int write_with_updates(struct packed_ref_store *refs,
strbuf_addf(err, "unable to create file %s: %s",
sb.buf, strerror(errno));
strbuf_release(&sb);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
strbuf_release(&sb);
@@ -1409,6 +1410,7 @@ static int write_with_updates(struct packed_ref_store *refs,
strbuf_addf(err, "cannot update ref '%s': "
"reference already exists",
update->refname);
+ ret = REF_TRANSACTION_ERROR_CREATE_EXISTS;
goto error;
} else if (!oideq(&update->old_oid, iter->oid)) {
strbuf_addf(err, "cannot update ref '%s': "
@@ -1416,6 +1418,7 @@ static int write_with_updates(struct packed_ref_store *refs,
update->refname,
oid_to_hex(iter->oid),
oid_to_hex(&update->old_oid));
+ ret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
goto error;
}
}
@@ -1452,6 +1455,7 @@ static int write_with_updates(struct packed_ref_store *refs,
"reference is missing but expected %s",
update->refname,
oid_to_hex(&update->old_oid));
+ ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
goto error;
}
}
@@ -1509,7 +1513,7 @@ static int write_with_updates(struct packed_ref_store *refs,
strerror(errno));
strbuf_release(&sb);
delete_tempfile(&refs->tempfile);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
return 0;
@@ -1521,7 +1525,7 @@ static int write_with_updates(struct packed_ref_store *refs,
error:
ref_iterator_free(iter);
delete_tempfile(&refs->tempfile);
- return -1;
+ return ret;
}
int is_packed_transaction_needed(struct ref_store *ref_store,
@@ -1654,7 +1658,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
"ref_transaction_prepare");
struct packed_transaction_backend_data *data;
- int ret = TRANSACTION_GENERIC_ERROR;
+ enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
/*
* Note that we *don't* skip transactions with zero updates,
@@ -1675,7 +1679,8 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
data->own_lock = 1;
}
- if (write_with_updates(refs, &transaction->refnames, err))
+ ret = write_with_updates(refs, &transaction->refnames, err);
+ if (ret)
goto failure;
transaction->state = REF_TRANSACTION_PREPARED;
@@ -1707,7 +1712,7 @@ static int packed_transaction_finish(struct ref_store *ref_store,
ref_store,
REF_STORE_READ | REF_STORE_WRITE | REF_STORE_ODB,
"ref_transaction_finish");
- int ret = TRANSACTION_GENERIC_ERROR;
+ int ret = REF_TRANSACTION_ERROR_GENERIC;
char *packed_refs_path;
clear_snapshot(refs);
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 6d3770d0cc..3f1d19abd9 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -770,8 +770,9 @@ int ref_update_has_null_new_value(struct ref_update *update);
* If everything is OK, return 0; otherwise, write an error message to
* err and return -1.
*/
-int ref_update_check_old_target(const char *referent, struct ref_update *update,
- struct strbuf *err);
+enum ref_transaction_error ref_update_check_old_target(const char *referent,
+ struct ref_update *update,
+ struct strbuf *err);
/*
* Check if the ref must exist, this means that the old_oid or
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 786df11a03..bd6b042103 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1069,20 +1069,20 @@ static int queue_transaction_update(struct reftable_ref_store *refs,
return 0;
}
-static int prepare_single_update(struct reftable_ref_store *refs,
- struct reftable_transaction_data *tx_data,
- struct ref_transaction *transaction,
- struct reftable_backend *be,
- struct ref_update *u,
- struct string_list *refnames_to_check,
- unsigned int head_type,
- struct strbuf *head_referent,
- struct strbuf *referent,
- struct strbuf *err)
+static enum ref_transaction_error prepare_single_update(struct reftable_ref_store *refs,
+ struct reftable_transaction_data *tx_data,
+ struct ref_transaction *transaction,
+ struct reftable_backend *be,
+ struct ref_update *u,
+ struct string_list *refnames_to_check,
+ unsigned int head_type,
+ struct strbuf *head_referent,
+ struct strbuf *referent,
+ struct strbuf *err)
{
+ enum ref_transaction_error ret = 0;
struct object_id current_oid = {0};
const char *rewritten_ref;
- int ret = 0;
/*
* There is no need to reload the respective backends here as
@@ -1093,7 +1093,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
*/
ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0);
if (ret)
- return ret;
+ return REF_TRANSACTION_ERROR_GENERIC;
/* Verify that the new object ID is valid. */
if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) &&
@@ -1104,13 +1104,13 @@ static int prepare_single_update(struct reftable_ref_store *refs,
strbuf_addf(err,
_("trying to write ref '%s' with nonexistent object %s"),
u->refname, oid_to_hex(&u->new_oid));
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
if (o->type != OBJ_COMMIT && is_branch(u->refname)) {
strbuf_addf(err, _("trying to write non-commit object %s to branch '%s'"),
oid_to_hex(&u->new_oid), u->refname);
- return -1;
+ return REF_TRANSACTION_ERROR_INVALID_NEW_VALUE;
}
}
@@ -1134,7 +1134,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
_("multiple updates for 'HEAD' (including one "
"via its referent '%s') are not allowed"),
u->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
ref_transaction_add_update(
@@ -1147,7 +1147,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
ret = reftable_backend_read_ref(be, rewritten_ref,
¤t_oid, referent, &u->type);
if (ret < 0)
- return ret;
+ return REF_TRANSACTION_ERROR_GENERIC;
if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
/*
* The reference does not exist, and we either have no
@@ -1168,7 +1168,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
ret = queue_transaction_update(refs, tx_data, u,
¤t_oid, err);
if (ret)
- return ret;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
return 0;
@@ -1180,7 +1180,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
"unable to resolve reference '%s'"),
ref_update_original_update_refname(u), u->refname);
- return -1;
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
}
if (u->type & REF_ISSYMREF) {
@@ -1196,7 +1196,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
if (u->flags & REF_HAVE_OLD && !resolved) {
strbuf_addf(err, _("cannot lock ref '%s': "
"error reading reference"), u->refname);
- return -1;
+ return REF_TRANSACTION_ERROR_GENERIC;
}
} else {
struct ref_update *new_update;
@@ -1211,7 +1211,7 @@ static int prepare_single_update(struct reftable_ref_store *refs,
_("multiple updates for '%s' (including one "
"via symref '%s') are not allowed"),
referent->buf, u->refname);
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_NAME_CONFLICT;
}
/*
@@ -1255,31 +1255,32 @@ static int prepare_single_update(struct reftable_ref_store *refs,
"but is a regular ref"),
ref_update_original_update_refname(u),
u->old_target);
- return -1;
+ return REF_TRANSACTION_ERROR_EXPECTED_SYMREF;
}
- if (ref_update_check_old_target(referent->buf, u, err)) {
- return -1;
- }
+ ret = ref_update_check_old_target(referent->buf, u, err);
+ if (ret)
+ return ret;
} else if ((u->flags & REF_HAVE_OLD) && !oideq(¤t_oid, &u->old_oid)) {
if (is_null_oid(&u->old_oid)) {
strbuf_addf(err, _("cannot lock ref '%s': "
"reference already exists"),
ref_update_original_update_refname(u));
- return TRANSACTION_CREATE_EXISTS;
- }
- else if (is_null_oid(¤t_oid))
+ return REF_TRANSACTION_ERROR_CREATE_EXISTS;
+ } else if (is_null_oid(¤t_oid)) {
strbuf_addf(err, _("cannot lock ref '%s': "
"reference is missing but expected %s"),
ref_update_original_update_refname(u),
oid_to_hex(&u->old_oid));
- else
+ return REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+ } else {
strbuf_addf(err, _("cannot lock ref '%s': "
"is at %s but expected %s"),
ref_update_original_update_refname(u),
oid_to_hex(¤t_oid),
oid_to_hex(&u->old_oid));
- return TRANSACTION_NAME_CONFLICT;
+ return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
+ }
}
/*
@@ -1296,8 +1297,8 @@ static int prepare_single_update(struct reftable_ref_store *refs,
if ((u->type & REF_ISSYMREF) ||
(u->flags & REF_LOG_ONLY) ||
(u->flags & REF_HAVE_NEW && !oideq(¤t_oid, &u->new_oid)))
- return queue_transaction_update(refs, tx_data, u,
- ¤t_oid, err);
+ if (queue_transaction_update(refs, tx_data, u, ¤t_oid, err))
+ return REF_TRANSACTION_ERROR_GENERIC;
return 0;
}
@@ -1385,7 +1386,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->state = REF_TRANSACTION_PREPARED;
done:
- assert(ret != REFTABLE_API_ERROR);
if (ret < 0) {
free_transaction_data(tx_data);
transaction->state = REF_TRANSACTION_CLOSED;
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v6 6/8] refs: implement batch reference update support
2025-04-08 8:51 ` [PATCH v6 " Karthik Nayak
` (4 preceding siblings ...)
2025-04-08 8:51 ` [PATCH v6 5/8] refs: introduce enum-based transaction error types Karthik Nayak
@ 2025-04-08 8:51 ` Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 7/8] refs: support rejection in batch updates during F/D checks Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 8/8] update-ref: add --batch-updates flag for stdin mode Karthik Nayak
7 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-04-08 8:51 UTC (permalink / raw)
To: karthik.188; +Cc: git, jltobler, ps, jn.avila, gitster
Git supports making reference updates with or without transactions.
Updates with transactions are generally better optimized. But
transactions are all or nothing. This means, if a user wants to batch
updates to take advantage of the optimizations without the hard
requirement that all updates must succeed, there is no way currently to
do so. Particularly with the reftable backend where batching multiple
reference updates is more efficient than performing them sequentially.
Introduce batched update support with a new flag,
'REF_TRANSACTION_ALLOW_FAILURE'. Batched updates while different from
transactions, use the transaction infrastructure under the hood. When
enabled, this flag allows individual reference updates that would
typically cause the entire transaction to fail due to non-system-related
errors to be marked as rejected while permitting other updates to
proceed. System errors referred by 'REF_TRANSACTION_ERROR_GENERIC'
continue to result in the entire transaction failing. This approach
enhances flexibility while preserving transactional integrity where
necessary.
The implementation introduces several key components:
- Add 'rejection_err' field to struct `ref_update` to track failed
updates with failure reason.
- Add a new struct `ref_transaction_rejections` and a field within
`ref_transaction` to this struct to allow quick iteration over
rejected updates.
- Modify reference backends (files, packed, reftable) to handle
partial transactions by using `ref_transaction_set_rejected()`
instead of failing the entire transaction when
`REF_TRANSACTION_ALLOW_FAILURE` is set.
- Add `ref_transaction_for_each_rejected_update()` to let callers
examine which updates were rejected and why.
This foundational change enables batched update support throughout the
reference subsystem. A following commit will expose this capability to
users by adding a `--batch-updates` flag to 'git-update-ref(1)',
providing both a user-facing feature and a testable implementation.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 61 +++++++++++++++++++++++++++++++++++++++++
refs.h | 22 +++++++++++++++
refs/files-backend.c | 12 +++++++-
refs/packed-backend.c | 27 ++++++++++++++++--
refs/refs-internal.h | 26 ++++++++++++++++++
refs/reftable-backend.c | 12 +++++++-
6 files changed, 156 insertions(+), 4 deletions(-)
diff --git a/refs.c b/refs.c
index 3d0b53d56e..b34ec198f5 100644
--- a/refs.c
+++ b/refs.c
@@ -1176,6 +1176,10 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs,
tr->ref_store = refs;
tr->flags = flags;
string_list_init_dup(&tr->refnames);
+
+ if (flags & REF_TRANSACTION_ALLOW_FAILURE)
+ CALLOC_ARRAY(tr->rejections, 1);
+
return tr;
}
@@ -1206,11 +1210,45 @@ void ref_transaction_free(struct ref_transaction *transaction)
free((char *)transaction->updates[i]->old_target);
free(transaction->updates[i]);
}
+
+ if (transaction->rejections)
+ free(transaction->rejections->update_indices);
+ free(transaction->rejections);
+
string_list_clear(&transaction->refnames, 0);
free(transaction->updates);
free(transaction);
}
+int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,
+ size_t update_idx,
+ enum ref_transaction_error err)
+{
+ if (update_idx >= transaction->nr)
+ BUG("trying to set rejection on invalid update index");
+
+ if (!(transaction->flags & REF_TRANSACTION_ALLOW_FAILURE))
+ return 0;
+
+ if (!transaction->rejections)
+ BUG("transaction not inititalized with failure support");
+
+ /*
+ * Don't accept generic errors, since these errors are not user
+ * input related.
+ */
+ if (err == REF_TRANSACTION_ERROR_GENERIC)
+ return 0;
+
+ transaction->updates[update_idx]->rejection_err = err;
+ ALLOC_GROW(transaction->rejections->update_indices,
+ transaction->rejections->nr + 1,
+ transaction->rejections->alloc);
+ transaction->rejections->update_indices[transaction->rejections->nr++] = update_idx;
+
+ return 1;
+}
+
struct ref_update *ref_transaction_add_update(
struct ref_transaction *transaction,
const char *refname, unsigned int flags,
@@ -1236,6 +1274,7 @@ struct ref_update *ref_transaction_add_update(
transaction->updates[transaction->nr++] = update;
update->flags = flags;
+ update->rejection_err = 0;
update->new_target = xstrdup_or_null(new_target);
update->old_target = xstrdup_or_null(old_target);
@@ -2728,6 +2767,28 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
}
}
+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
+ ref_transaction_for_each_rejected_update_fn cb,
+ void *cb_data)
+{
+ if (!transaction->rejections)
+ return;
+
+ for (size_t i = 0; i < transaction->rejections->nr; i++) {
+ size_t update_index = transaction->rejections->update_indices[i];
+ struct ref_update *update = transaction->updates[update_index];
+
+ if (!update->rejection_err)
+ continue;
+
+ cb(update->refname,
+ (update->flags & REF_HAVE_OLD) ? &update->old_oid : NULL,
+ (update->flags & REF_HAVE_NEW) ? &update->new_oid : NULL,
+ update->old_target, update->new_target,
+ update->rejection_err, cb_data);
+ }
+}
+
int refs_delete_refs(struct ref_store *refs, const char *logmsg,
struct string_list *refnames, unsigned int flags)
{
diff --git a/refs.h b/refs.h
index f009cdae7d..c48c800478 100644
--- a/refs.h
+++ b/refs.h
@@ -667,6 +667,13 @@ enum ref_transaction_flag {
* either be absent or null_oid.
*/
REF_TRANSACTION_FLAG_INITIAL = (1 << 0),
+
+ /*
+ * The transaction mechanism by default fails all updates if any conflict
+ * is detected. This flag allows transactions to partially apply updates
+ * while rejecting updates which do not match the expected state.
+ */
+ REF_TRANSACTION_ALLOW_FAILURE = (1 << 1),
};
/*
@@ -897,6 +904,21 @@ void ref_transaction_for_each_queued_update(struct ref_transaction *transaction,
ref_transaction_for_each_queued_update_fn cb,
void *cb_data);
+/*
+ * Execute the given callback function for each of the reference updates which
+ * have been rejected in the given transaction.
+ */
+typedef void ref_transaction_for_each_rejected_update_fn(const char *refname,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
+ enum ref_transaction_error err,
+ void *cb_data);
+void ref_transaction_for_each_rejected_update(struct ref_transaction *transaction,
+ ref_transaction_for_each_rejected_update_fn cb,
+ void *cb_data);
+
/*
* Free `*transaction` and all associated data.
*/
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 4f27f7652c..256c69b942 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2852,8 +2852,15 @@ static int files_transaction_prepare(struct ref_store *ref_store,
ret = lock_ref_for_update(refs, update, transaction,
head_ref, &refnames_to_check,
err);
- if (ret)
+ if (ret) {
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_reset(err);
+ ret = 0;
+
+ continue;
+ }
goto cleanup;
+ }
if (update->flags & REF_DELETING &&
!(update->flags & REF_LOG_ONLY) &&
@@ -3151,6 +3158,9 @@ static int files_transaction_finish(struct ref_store *ref_store,
struct ref_update *update = transaction->updates[i];
struct ref_lock *lock = update->backend_data;
+ if (update->rejection_err)
+ continue;
+
if (update->flags & REF_NEEDS_COMMIT ||
update->flags & REF_LOG_ONLY) {
if (parse_and_write_reflog(refs, update, lock, err)) {
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index d90bd815a3..debca86a2b 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1327,10 +1327,11 @@ static int packed_ref_store_remove_on_disk(struct ref_store *ref_store,
* remain locked when it is done.
*/
static enum ref_transaction_error write_with_updates(struct packed_ref_store *refs,
- struct string_list *updates,
+ struct ref_transaction *transaction,
struct strbuf *err)
{
enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
+ struct string_list *updates = &transaction->refnames;
struct ref_iterator *iter = NULL;
size_t i;
int ok;
@@ -1411,6 +1412,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
"reference already exists",
update->refname);
ret = REF_TRANSACTION_ERROR_CREATE_EXISTS;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_reset(err);
+ ret = 0;
+ continue;
+ }
+
goto error;
} else if (!oideq(&update->old_oid, iter->oid)) {
strbuf_addf(err, "cannot update ref '%s': "
@@ -1419,6 +1427,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
oid_to_hex(iter->oid),
oid_to_hex(&update->old_oid));
ret = REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_reset(err);
+ ret = 0;
+ continue;
+ }
+
goto error;
}
}
@@ -1456,6 +1471,13 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
update->refname,
oid_to_hex(&update->old_oid));
ret = REF_TRANSACTION_ERROR_NONEXISTENT_REF;
+
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_reset(err);
+ ret = 0;
+ continue;
+ }
+
goto error;
}
}
@@ -1521,6 +1543,7 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
write_error:
strbuf_addf(err, "error writing to %s: %s",
get_tempfile_path(refs->tempfile), strerror(errno));
+ ret = REF_TRANSACTION_ERROR_GENERIC;
error:
ref_iterator_free(iter);
@@ -1679,7 +1702,7 @@ static int packed_transaction_prepare(struct ref_store *ref_store,
data->own_lock = 1;
}
- ret = write_with_updates(refs, &transaction->refnames, err);
+ ret = write_with_updates(refs, transaction, err);
if (ret)
goto failure;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 3f1d19abd9..73a5379b73 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -123,6 +123,12 @@ struct ref_update {
*/
uint64_t index;
+ /*
+ * Used in batched reference updates to mark if a given update
+ * was rejected.
+ */
+ enum ref_transaction_error rejection_err;
+
/*
* If this ref_update was split off of a symref update via
* split_symref_update(), then this member points at that
@@ -142,6 +148,13 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
struct object_id *oid, struct strbuf *referent,
unsigned int *type, int *failure_errno);
+/*
+ * Mark a given update as rejected with a given reason.
+ */
+int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction,
+ size_t update_idx,
+ enum ref_transaction_error err);
+
/*
* Add a ref_update with the specified properties to transaction, and
* return a pointer to the new object. This function does not verify
@@ -183,6 +196,18 @@ enum ref_transaction_state {
REF_TRANSACTION_CLOSED = 2
};
+/*
+ * Data structure to hold indices of updates which were rejected, for batched
+ * reference updates. While the updates themselves hold the rejection error,
+ * this structure allows a transaction to iterate only over the rejected
+ * updates.
+ */
+struct ref_transaction_rejections {
+ size_t *update_indices;
+ size_t alloc;
+ size_t nr;
+};
+
/*
* Data structure for holding a reference transaction, which can
* consist of checks and updates to multiple references, carried out
@@ -195,6 +220,7 @@ struct ref_transaction {
size_t alloc;
size_t nr;
enum ref_transaction_state state;
+ struct ref_transaction_rejections *rejections;
void *backend_data;
unsigned int flags;
uint64_t max_index;
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index bd6b042103..5db4a108b9 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1371,8 +1371,15 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
transaction->updates[i],
&refnames_to_check, head_type,
&head_referent, &referent, err);
- if (ret)
+ if (ret) {
+ if (ref_transaction_maybe_set_rejected(transaction, i, ret)) {
+ strbuf_reset(err);
+ ret = 0;
+
+ continue;
+ }
goto done;
+ }
}
ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
@@ -1454,6 +1461,9 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
struct reftable_transaction_update *tx_update = &arg->updates[i];
struct ref_update *u = tx_update->update;
+ if (u->rejection_err)
+ continue;
+
/*
* Write a reflog entry when updating a ref to point to
* something new in either of the following cases:
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v6 7/8] refs: support rejection in batch updates during F/D checks
2025-04-08 8:51 ` [PATCH v6 " Karthik Nayak
` (5 preceding siblings ...)
2025-04-08 8:51 ` [PATCH v6 6/8] refs: implement batch reference update support Karthik Nayak
@ 2025-04-08 8:51 ` Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 8/8] update-ref: add --batch-updates flag for stdin mode Karthik Nayak
7 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-04-08 8:51 UTC (permalink / raw)
To: karthik.188; +Cc: git, jltobler, ps, jn.avila, gitster
The `refs_verify_refnames_available()` is used to batch check refnames
for F/D conflicts. While this is the more performant alternative than
its individual version, it does not provide rejection capabilities on a
single update level. For batched updates, this would mean a rejection of
the entire transaction whenever one reference has a F/D conflict.
Modify the function to call `ref_transaction_maybe_set_rejected()` to
check if a single update can be rejected. Since this function is only
internally used within 'refs/' and we want to pass in a `struct
ref_transaction *` as a variable. We also move and mark
`refs_verify_refnames_available()` to 'refs-internal.h' to be an
internal function.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
refs.c | 37 ++++++++++++++++++++++++++++++++++---
refs.h | 12 ------------
refs/files-backend.c | 27 ++++++++++++++++++---------
refs/refs-internal.h | 16 ++++++++++++++++
refs/reftable-backend.c | 11 ++++++++---
5 files changed, 76 insertions(+), 27 deletions(-)
diff --git a/refs.c b/refs.c
index b34ec198f5..41d6247e70 100644
--- a/refs.c
+++ b/refs.c
@@ -2540,6 +2540,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
const struct string_list *refnames,
const struct string_list *extras,
const struct string_list *skip,
+ struct ref_transaction *transaction,
unsigned int initial_transaction,
struct strbuf *err)
{
@@ -2547,6 +2548,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
struct strbuf referent = STRBUF_INIT;
struct string_list_item *item;
struct ref_iterator *iter = NULL;
+ struct strset conflicting_dirnames;
struct strset dirnames;
int ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
@@ -2557,9 +2559,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
assert(err);
+ strset_init(&conflicting_dirnames);
strset_init(&dirnames);
for_each_string_list_item(item, refnames) {
+ const size_t *update_idx = (size_t *)item->util;
const char *refname = item->string;
const char *extra_refname;
struct object_id oid;
@@ -2597,14 +2601,30 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
continue;
if (!initial_transaction &&
- !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
- &type, &ignore_errno)) {
+ (strset_contains(&conflicting_dirnames, dirname.buf) ||
+ !refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
+ &type, &ignore_errno))) {
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT)) {
+ strset_remove(&dirnames, dirname.buf);
+ strset_add(&conflicting_dirnames, dirname.buf);
+ continue;
+ }
+
strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
dirname.buf, refname);
goto cleanup;
}
if (extras && string_list_has_string(extras, dirname.buf)) {
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT)) {
+ strset_remove(&dirnames, dirname.buf);
+ continue;
+ }
+
strbuf_addf(err, _("cannot process '%s' and '%s' at the same time"),
refname, dirname.buf);
goto cleanup;
@@ -2637,6 +2657,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
string_list_has_string(skip, iter->refname))
continue;
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT))
+ continue;
+
strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
iter->refname, refname);
goto cleanup;
@@ -2648,6 +2673,11 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
extra_refname = find_descendant_ref(dirname.buf, extras, skip);
if (extra_refname) {
+ if (transaction && ref_transaction_maybe_set_rejected(
+ transaction, *update_idx,
+ REF_TRANSACTION_ERROR_NAME_CONFLICT))
+ continue;
+
strbuf_addf(err, _("cannot process '%s' and '%s' at the same time"),
refname, extra_refname);
goto cleanup;
@@ -2659,6 +2689,7 @@ enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs
cleanup:
strbuf_release(&referent);
strbuf_release(&dirname);
+ strset_clear(&conflicting_dirnames);
strset_clear(&dirnames);
ref_iterator_free(iter);
return ret;
@@ -2679,7 +2710,7 @@ enum ref_transaction_error refs_verify_refname_available(
};
return refs_verify_refnames_available(refs, &refnames, extras, skip,
- initial_transaction, err);
+ NULL, initial_transaction, err);
}
struct do_for_each_reflog_help {
diff --git a/refs.h b/refs.h
index c48c800478..46a6008e07 100644
--- a/refs.h
+++ b/refs.h
@@ -141,18 +141,6 @@ enum ref_transaction_error refs_verify_refname_available(struct ref_store *refs,
unsigned int initial_transaction,
struct strbuf *err);
-/*
- * Same as `refs_verify_refname_available()`, but checking for a list of
- * refnames instead of only a single item. This is more efficient in the case
- * where one needs to check multiple refnames.
- */
-enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
- const struct string_list *refnames,
- const struct string_list *extras,
- const struct string_list *skip,
- unsigned int initial_transaction,
- struct strbuf *err);
-
int refs_ref_exists(struct ref_store *refs, const char *refname);
int should_autocreate_reflog(enum log_refs_config log_all_ref_updates,
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 256c69b942..b96a511977 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -677,16 +677,18 @@ static void unlock_ref(struct ref_lock *lock)
* - Generate informative error messages in the case of failure
*/
static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
- const char *refname,
+ struct ref_update *update,
+ size_t update_idx,
int mustexist,
struct string_list *refnames_to_check,
const struct string_list *extras,
struct ref_lock **lock_p,
struct strbuf *referent,
- unsigned int *type,
struct strbuf *err)
{
enum ref_transaction_error ret = REF_TRANSACTION_ERROR_GENERIC;
+ const char *refname = update->refname;
+ unsigned int *type = &update->type;
struct ref_lock *lock;
struct strbuf ref_file = STRBUF_INIT;
int attempts_remaining = 3;
@@ -785,6 +787,8 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
type, &failure_errno)) {
+ struct string_list_item *item;
+
if (failure_errno == ENOENT) {
if (mustexist) {
/* Garden variety missing reference. */
@@ -864,7 +868,9 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
* make sure there is no existing packed ref that conflicts
* with refname. This check is deferred so that we can batch it.
*/
- string_list_append(refnames_to_check, refname);
+ item = string_list_append(refnames_to_check, refname);
+ item->util = xmalloc(sizeof(update_idx));
+ memcpy(item->util, &update_idx, sizeof(update_idx));
}
ret = 0;
@@ -2547,6 +2553,7 @@ struct files_transaction_backend_data {
*/
static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *refs,
struct ref_update *update,
+ size_t update_idx,
struct ref_transaction *transaction,
const char *head_ref,
struct string_list *refnames_to_check,
@@ -2575,9 +2582,9 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re
if (lock) {
lock->count++;
} else {
- ret = lock_raw_ref(refs, update->refname, mustexist,
+ ret = lock_raw_ref(refs, update, update_idx, mustexist,
refnames_to_check, &transaction->refnames,
- &lock, &referent, &update->type, err);
+ &lock, &referent, err);
if (ret) {
char *reason;
@@ -2849,7 +2856,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
- ret = lock_ref_for_update(refs, update, transaction,
+ ret = lock_ref_for_update(refs, update, i, transaction,
head_ref, &refnames_to_check,
err);
if (ret) {
@@ -2905,7 +2912,8 @@ static int files_transaction_prepare(struct ref_store *ref_store,
* So instead, we accept the race for now.
*/
if (refs_verify_refnames_available(refs->packed_ref_store, &refnames_to_check,
- &transaction->refnames, NULL, 0, err)) {
+ &transaction->refnames, NULL, transaction,
+ 0, err)) {
ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
}
@@ -2951,7 +2959,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
cleanup:
free(head_ref);
- string_list_clear(&refnames_to_check, 0);
+ string_list_clear(&refnames_to_check, 1);
if (ret)
files_transaction_cleanup(refs, transaction);
@@ -3097,7 +3105,8 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
}
if (refs_verify_refnames_available(&refs->base, &refnames_to_check,
- &affected_refnames, NULL, 1, err)) {
+ &affected_refnames, NULL, transaction,
+ 1, err)) {
packed_refs_unlock(refs->packed_ref_store);
ret = REF_TRANSACTION_ERROR_NAME_CONFLICT;
goto cleanup;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 73a5379b73..f868870851 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -806,4 +806,20 @@ enum ref_transaction_error ref_update_check_old_target(const char *referent,
*/
int ref_update_expects_existing_old_ref(struct ref_update *update);
+/*
+ * Same as `refs_verify_refname_available()`, but checking for a list of
+ * refnames instead of only a single item. This is more efficient in the case
+ * where one needs to check multiple refnames.
+ *
+ * If using batched updates, then individual updates are marked rejected,
+ * reference backends are then in charge of not committing those updates.
+ */
+enum ref_transaction_error refs_verify_refnames_available(struct ref_store *refs,
+ const struct string_list *refnames,
+ const struct string_list *extras,
+ const struct string_list *skip,
+ struct ref_transaction *transaction,
+ unsigned int initial_transaction,
+ struct strbuf *err);
+
#endif /* REFS_REFS_INTERNAL_H */
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 5db4a108b9..4c3817f4ec 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1074,6 +1074,7 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor
struct ref_transaction *transaction,
struct reftable_backend *be,
struct ref_update *u,
+ size_t update_idx,
struct string_list *refnames_to_check,
unsigned int head_type,
struct strbuf *head_referent,
@@ -1149,6 +1150,7 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor
if (ret < 0)
return REF_TRANSACTION_ERROR_GENERIC;
if (ret > 0 && !ref_update_expects_existing_old_ref(u)) {
+ struct string_list_item *item;
/*
* The reference does not exist, and we either have no
* old object ID or expect the reference to not exist.
@@ -1158,7 +1160,9 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor
* can output a proper error message instead of failing
* at a later point.
*/
- string_list_append(refnames_to_check, u->refname);
+ item = string_list_append(refnames_to_check, u->refname);
+ item->util = xmalloc(sizeof(update_idx));
+ memcpy(item->util, &update_idx, sizeof(update_idx));
/*
* There is no need to write the reference deletion
@@ -1368,7 +1372,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
for (i = 0; i < transaction->nr; i++) {
ret = prepare_single_update(refs, tx_data, transaction, be,
- transaction->updates[i],
+ transaction->updates[i], i,
&refnames_to_check, head_type,
&head_referent, &referent, err);
if (ret) {
@@ -1384,6 +1388,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
ret = refs_verify_refnames_available(ref_store, &refnames_to_check,
&transaction->refnames, NULL,
+ transaction,
transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
err);
if (ret < 0)
@@ -1402,7 +1407,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
}
strbuf_release(&referent);
strbuf_release(&head_referent);
- string_list_clear(&refnames_to_check, 0);
+ string_list_clear(&refnames_to_check, 1);
return ret;
}
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* [PATCH v6 8/8] update-ref: add --batch-updates flag for stdin mode
2025-04-08 8:51 ` [PATCH v6 " Karthik Nayak
` (6 preceding siblings ...)
2025-04-08 8:51 ` [PATCH v6 7/8] refs: support rejection in batch updates during F/D checks Karthik Nayak
@ 2025-04-08 8:51 ` Karthik Nayak
2025-04-08 15:02 ` Junio C Hamano
7 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-04-08 8:51 UTC (permalink / raw)
To: karthik.188; +Cc: git, jltobler, ps, jn.avila, gitster
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=y, Size: 15880 bytes --]
When updating multiple references through stdin, Git's update-ref
command normally aborts the entire transaction if any single update
fails. This atomic behavior prevents partial updates. Introduce a new
batch update system, where the updates the performed together similar
but individual updates are allowed to fail.
Add a new `--batch-updates` flag that allows the transaction to continue
even when individual reference updates fail. This flag can only be used
in `--stdin` mode and builds upon the batch update support added to the
refs subsystem in the previous commits. When enabled, failed updates are
reported in the following format:
rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
Update the documentation to reflect this change and also tests to cover
different scenarios where an update could be rejected.
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
Documentation/git-update-ref.adoc | 14 +-
builtin/update-ref.c | 66 ++++++++-
t/t1400-update-ref.sh | 233 ++++++++++++++++++++++++++++++
3 files changed, 306 insertions(+), 7 deletions(-)
diff --git a/Documentation/git-update-ref.adoc b/Documentation/git-update-ref.adoc
index 9e6935d38d..9310ce9768 100644
--- a/Documentation/git-update-ref.adoc
+++ b/Documentation/git-update-ref.adoc
@@ -7,8 +7,10 @@ git-update-ref - Update the object name stored in a ref safely
SYNOPSIS
--------
-[verse]
-'git update-ref' [-m <reason>] [--no-deref] (-d <ref> [<old-oid>] | [--create-reflog] <ref> <new-oid> [<old-oid>] | --stdin [-z])
+[synopsis]
+git update-ref [-m <reason>] [--no-deref] -d <ref> [<old-oid>]
+git update-ref [-m <reason>] [--no-deref] [--create-reflog] <ref> <new-oid> [<old-oid>]
+git update-ref [-m <reason>] [--no-deref] --stdin [-z] [--batch-updates]
DESCRIPTION
-----------
@@ -57,6 +59,14 @@ performs all modifications together. Specify commands of the form:
With `--create-reflog`, update-ref will create a reflog for each ref
even if one would not ordinarily be created.
+With `--batch-updates`, update-ref executes the updates in a batch but allows
+individual updates to fail due to invalid or incorrect user input, applying only
+the successful updates. However, system-related errors—such as I/O failures or
+memory issues—will result in a full failure of all batched updates. Any failed
+updates will be reported in the following format:
+
+ rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
+
Quote fields containing whitespace as if they were strings in C source
code; i.e., surrounded by double-quotes and with backslash escapes.
Use 40 "0" characters or the empty string to specify a zero value. To
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 1d541e13ad..111d6473ad 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -5,6 +5,7 @@
#include "config.h"
#include "gettext.h"
#include "hash.h"
+#include "hex.h"
#include "refs.h"
#include "object-name.h"
#include "parse-options.h"
@@ -13,7 +14,7 @@
static const char * const git_update_ref_usage[] = {
N_("git update-ref [<options>] -d <refname> [<old-oid>]"),
N_("git update-ref [<options>] <refname> <new-oid> [<old-oid>]"),
- N_("git update-ref [<options>] --stdin [-z]"),
+ N_("git update-ref [<options>] --stdin [-z] [--batch-updates]"),
NULL
};
@@ -565,6 +566,49 @@ static void parse_cmd_abort(struct ref_transaction *transaction,
report_ok("abort");
}
+static void print_rejected_refs(const char *refname,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
+ const char *old_target,
+ const char *new_target,
+ enum ref_transaction_error err,
+ void *cb_data UNUSED)
+{
+ struct strbuf sb = STRBUF_INIT;
+ const char *reason = "";
+
+ switch (err) {
+ case REF_TRANSACTION_ERROR_NAME_CONFLICT:
+ reason = "refname conflict";
+ break;
+ case REF_TRANSACTION_ERROR_CREATE_EXISTS:
+ reason = "reference already exists";
+ break;
+ case REF_TRANSACTION_ERROR_NONEXISTENT_REF:
+ reason = "reference does not exist";
+ break;
+ case REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE:
+ reason = "incorrect old value provided";
+ break;
+ case REF_TRANSACTION_ERROR_INVALID_NEW_VALUE:
+ reason = "invalid new value provided";
+ break;
+ case REF_TRANSACTION_ERROR_EXPECTED_SYMREF:
+ reason = "expected symref but found regular ref";
+ break;
+ default:
+ reason = "unkown failure";
+ }
+
+ strbuf_addf(&sb, "rejected %s %s %s %s\n", refname,
+ new_oid ? oid_to_hex(new_oid) : new_target,
+ old_oid ? oid_to_hex(old_oid) : old_target,
+ reason);
+
+ fwrite(sb.buf, sb.len, 1, stdout);
+ strbuf_release(&sb);
+}
+
static void parse_cmd_commit(struct ref_transaction *transaction,
const char *next, const char *end UNUSED)
{
@@ -573,6 +617,10 @@ static void parse_cmd_commit(struct ref_transaction *transaction,
die("commit: extra input: %s", next);
if (ref_transaction_commit(transaction, &error))
die("commit: %s", error.buf);
+
+ ref_transaction_for_each_rejected_update(transaction,
+ print_rejected_refs, NULL);
+
report_ok("commit");
ref_transaction_free(transaction);
}
@@ -609,7 +657,7 @@ static const struct parse_cmd {
{ "commit", parse_cmd_commit, 0, UPDATE_REFS_CLOSED },
};
-static void update_refs_stdin(void)
+static void update_refs_stdin(unsigned int flags)
{
struct strbuf input = STRBUF_INIT, err = STRBUF_INIT;
enum update_refs_state state = UPDATE_REFS_OPEN;
@@ -617,7 +665,7 @@ static void update_refs_stdin(void)
int i, j;
transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
- 0, &err);
+ flags, &err);
if (!transaction)
die("%s", err.buf);
@@ -685,7 +733,7 @@ static void update_refs_stdin(void)
*/
state = cmd->state;
transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
- 0, &err);
+ flags, &err);
if (!transaction)
die("%s", err.buf);
@@ -701,6 +749,8 @@ static void update_refs_stdin(void)
/* Commit by default if no transaction was requested. */
if (ref_transaction_commit(transaction, &err))
die("%s", err.buf);
+ ref_transaction_for_each_rejected_update(transaction,
+ print_rejected_refs, NULL);
ref_transaction_free(transaction);
break;
case UPDATE_REFS_STARTED:
@@ -727,6 +777,8 @@ int cmd_update_ref(int argc,
struct object_id oid, oldoid;
int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;
int create_reflog = 0;
+ unsigned int flags = 0;
+
struct option options[] = {
OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
OPT_BOOL('d', NULL, &delete, N_("delete the reference")),
@@ -735,6 +787,8 @@ int cmd_update_ref(int argc,
OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
OPT_BOOL( 0 , "create-reflog", &create_reflog, N_("create a reflog")),
+ OPT_BIT('0', "batch-updates", &flags, N_("batch reference updates"),
+ REF_TRANSACTION_ALLOW_FAILURE),
OPT_END(),
};
@@ -756,8 +810,10 @@ int cmd_update_ref(int argc,
usage_with_options(git_update_ref_usage, options);
if (end_null)
line_termination = '\0';
- update_refs_stdin();
+ update_refs_stdin(flags);
return 0;
+ } else if (flags & REF_TRANSACTION_ALLOW_FAILURE) {
+ die("--batch-updates can only be used with --stdin");
}
if (end_null)
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 29045aad43..d29d23cb89 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -2066,6 +2066,239 @@ do
grep "$(git rev-parse $a) $(git rev-parse $a)" actual
'
+ test_expect_success "stdin $type batch-updates" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit commit &&
+ head=$(git rev-parse HEAD) &&
+
+ format_command $type "update refs/heads/ref1" "$head" "$Z" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head" "$Z" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with invalid new_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$(test_oid 001)" "$head" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "invalid new value provided" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with non-commit new_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ head_tree=$(git rev-parse HEAD^{tree}) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head_tree" "$head" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "invalid new value provided" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with non-existent ref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ test_must_fail git rev-parse refs/heads/ref2 &&
+ test_grep -q "reference does not exist" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with dangling symref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git symbolic-ref refs/heads/ref2 refs/heads/nonexistent &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
+ git update-ref $type --no-deref --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ test_must_fail git rev-parse refs/heads/ref2 &&
+ test_grep -q "reference does not exist" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with regular ref as symref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "symref-update refs/heads/ref2" "$old_head" "ref" "refs/heads/nonexistent" >>stdin &&
+ git update-ref $type --no-deref --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "expected symref but found regular ref" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with invalid old_oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$old_head" "$Z" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "reference already exists" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates with incorrect old oid" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref1 $head &&
+ git update-ref refs/heads/ref2 $head &&
+
+ format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref2" "$head" "$old_head" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref1 >actual &&
+ test_cmp expect actual &&
+ echo $head >expect &&
+ git rev-parse refs/heads/ref2 >actual &&
+ test_cmp expect actual &&
+ test_grep -q "incorrect old value provided" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates refname conflict" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref/foo $head &&
+
+ format_command $type "update refs/heads/ref/foo" "$old_head" "$head" >stdin &&
+ format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/ref/foo >actual &&
+ test_cmp expect actual &&
+ test_grep -q "refname conflict" stdout
+ )
+ '
+
+ test_expect_success "stdin $type batch-updates refname conflict new ref" '
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+ test_commit one &&
+ old_head=$(git rev-parse HEAD) &&
+ test_commit two &&
+ head=$(git rev-parse HEAD) &&
+ git update-ref refs/heads/ref/foo $head &&
+
+ format_command $type "update refs/heads/foo" "$old_head" "" >stdin &&
+ format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
+ git update-ref $type --stdin --batch-updates <stdin >stdout &&
+ echo $old_head >expect &&
+ git rev-parse refs/heads/foo >actual &&
+ test_cmp expect actual &&
+ test_grep -q "refname conflict" stdout
+ )
+ '
done
test_expect_success 'update-ref should also create reflog for HEAD' '
--
2.48.1
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH v6 8/8] update-ref: add --batch-updates flag for stdin mode
2025-04-08 8:51 ` [PATCH v6 8/8] update-ref: add --batch-updates flag for stdin mode Karthik Nayak
@ 2025-04-08 15:02 ` Junio C Hamano
2025-04-08 15:26 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Junio C Hamano @ 2025-04-08 15:02 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler, ps, jn.avila
Karthik Nayak <karthik.188@gmail.com> writes:
> Content-Type: text/plain; charset=y
Please don't ;-).
More practically, is there something we can do to avoid this
happening in send-email? It may be a not-so-uncommon end user
mistake that we would rather help our users avoid.
> When updating multiple references through stdin, Git's update-ref
> ...
Will replace and queue. Let me mark the topic for 'next'.
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v6 8/8] update-ref: add --batch-updates flag for stdin mode
2025-04-08 15:02 ` Junio C Hamano
@ 2025-04-08 15:26 ` Karthik Nayak
2025-04-08 17:37 ` Junio C Hamano
0 siblings, 1 reply; 143+ messages in thread
From: Karthik Nayak @ 2025-04-08 15:26 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, jltobler, ps, jn.avila
[-- Attachment #1: Type: text/plain, Size: 818 bytes --]
Junio C Hamano <gitster@pobox.com> writes:
> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> Content-Type: text/plain; charset=y
>
> Please don't ;-).
>
> More practically, is there something we can do to avoid this
> happening in send-email? It may be a not-so-uncommon end user
> mistake that we would rather help our users avoid.
>
This seems like this was in response to the following question:
Which 8bit encoding should I declare [UTF-8]?
Which I should have just clicked 'Enter' on, but typed 'y' as 'yes
please pick UTF-8'. Which again confirms the encoding, which I
presumably didn't read. So I guess the problem Exists Between Keyboard
and Chair.
>> When updating multiple references through stdin, Git's update-ref
>> ...
>
> Will replace and queue. Let me mark the topic for 'next'.
Thanks!
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 143+ messages in thread
* Re: [PATCH v6 8/8] update-ref: add --batch-updates flag for stdin mode
2025-04-08 15:26 ` Karthik Nayak
@ 2025-04-08 17:37 ` Junio C Hamano
2025-04-10 11:23 ` Karthik Nayak
0 siblings, 1 reply; 143+ messages in thread
From: Junio C Hamano @ 2025-04-08 17:37 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git, jltobler, ps, jn.avila
Karthik Nayak <karthik.188@gmail.com> writes:
> Junio C Hamano <gitster@pobox.com> writes:
>
>> Karthik Nayak <karthik.188@gmail.com> writes:
>>
>>> Content-Type: text/plain; charset=y
>>
>> Please don't ;-).
>>
>> More practically, is there something we can do to avoid this
>> happening in send-email? It may be a not-so-uncommon end user
>> mistake that we would rather help our users avoid.
>>
>
> This seems like this was in response to the following question:
> Which 8bit encoding should I declare [UTF-8]?
>
> Which I should have just clicked 'Enter' on, but typed 'y' as 'yes
> please pick UTF-8'. Which again confirms the encoding, which I
> presumably didn't read. So I guess the problem Exists Between Keyboard
> and Chair.
OK, we have seen enough people got burned by 'y', and made 852a15d7
(send-email: ask confirmation if given encoding name is very short,
2015-02-13) as a response exactly for that problem, but it is not
effective as we wished X-<.
If there were a better validation method than "4 bytes or longer" we
currently use for valid values for "charset=$auto_8bit_encoding", we
could lose confirm_only from the call to ask() that asks the
question, but I do not know if that is feasible.
Another more obvious alternative is to do something ugly like this
patch, I suppose? Just like <ENTER> is taken as "I take the default
value presented", this makes yes<ENTER> mean the same thing.
There is one question that asks yes/no question with default set to
'n', which would be broken by the patch below, so it needs a bit
more thought, though.
git-send-email.perl | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git c/git-send-email.perl w/git-send-email.perl
index 798d59b84f..8b942e5bcf 100755
--- c/git-send-email.perl
+++ w/git-send-email.perl
@@ -986,7 +986,8 @@ sub ask {
print "\n";
return defined $default ? $default : undef;
}
- if ($resp eq '' and defined $default) {
+ if (defined $default &&
+ ($resp eq '' || $resp =~ /^y(?:es)$/i)) {
return $default;
}
if (!defined $valid_re or $resp =~ /$valid_re/) {
^ permalink raw reply related [flat|nested] 143+ messages in thread
* Re: [PATCH v6 8/8] update-ref: add --batch-updates flag for stdin mode
2025-04-08 17:37 ` Junio C Hamano
@ 2025-04-10 11:23 ` Karthik Nayak
0 siblings, 0 replies; 143+ messages in thread
From: Karthik Nayak @ 2025-04-10 11:23 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, jltobler, ps, jn.avila
[-- Attachment #1: Type: text/plain, Size: 3334 bytes --]
Junio C Hamano <gitster@pobox.com> writes:
> Karthik Nayak <karthik.188@gmail.com> writes:
>
>> Junio C Hamano <gitster@pobox.com> writes:
>>
>>> Karthik Nayak <karthik.188@gmail.com> writes:
>>>
>>>> Content-Type: text/plain; charset=y
>>>
>>> Please don't ;-).
>>>
>>> More practically, is there something we can do to avoid this
>>> happening in send-email? It may be a not-so-uncommon end user
>>> mistake that we would rather help our users avoid.
>>>
>>
>> This seems like this was in response to the following question:
>> Which 8bit encoding should I declare [UTF-8]?
>>
>> Which I should have just clicked 'Enter' on, but typed 'y' as 'yes
>> please pick UTF-8'. Which again confirms the encoding, which I
>> presumably didn't read. So I guess the problem Exists Between Keyboard
>> and Chair.
>
> OK, we have seen enough people got burned by 'y', and made 852a15d7
> (send-email: ask confirmation if given encoding name is very short,
> 2015-02-13) as a response exactly for that problem, but it is not
> effective as we wished X-<.
>
I see.
> If there were a better validation method than "4 bytes or longer" we
> currently use for valid values for "charset=$auto_8bit_encoding", we
> could lose confirm_only from the call to ask() that asks the
> question, but I do not know if that is feasible.
>
> Another more obvious alternative is to do something ugly like this
> patch, I suppose? Just like <ENTER> is taken as "I take the default
> value presented", this makes yes<ENTER> mean the same thing.
>
> There is one question that asks yes/no question with default set to
> 'n', which would be broken by the patch below, so it needs a bit
> more thought, though.
>
Yes, this would be an issue. I think what would be nice is perhaps an
option like $yes_default.
> git-send-email.perl | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git c/git-send-email.perl w/git-send-email.perl
> index 798d59b84f..8b942e5bcf 100755
> --- c/git-send-email.perl
> +++ w/git-send-email.perl
> @@ -986,7 +986,8 @@ sub ask {
> print "\n";
> return defined $default ? $default : undef;
> }
> - if ($resp eq '' and defined $default) {
> + if (defined $default &&
> + ($resp eq '' || $resp =~ /^y(?:es)$/i)) {
> return $default;
> }
> if (!defined $valid_re or $resp =~ /$valid_re/) {
Going on top of your patch, something like:
-->8--
diff --git a/git-send-email.perl b/git-send-email.perl
index 798d59b84f..318699d26c 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -974,6 +974,7 @@ sub ask {
my $valid_re = $arg{valid_re};
my $default = $arg{default};
my $confirm_only = $arg{confirm_only};
+ my $yes_is_default = $arg{yes_is_default};
my $resp;
my $i = 0;
my $term = term();
@@ -989,6 +990,10 @@ sub ask {
if ($resp eq '' and defined $default) {
return $default;
}
+ if (defined $default and defined $yes_is_default
+ and $resp =~ /^y(?:es)$/i) {
+ return $default;
+ }
if (!defined $valid_re or $resp =~ /$valid_re/) {
return $resp;
}
@@ -1031,7 +1036,7 @@ sub file_declares_8bit_cte {
}
$auto_8bit_encoding = ask(__("Which 8bit encoding should I declare
[UTF-8]? "),
valid_re => qr/.{4}/, confirm_only => 1,
- default => "UTF-8");
+ default => "UTF-8", yes_is_default => true);
}
if (!$force) {
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply related [flat|nested] 143+ messages in thread
end of thread, other threads:[~2025-04-10 11:23 UTC | newest]
Thread overview: 143+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-02-07 7:34 [PATCH 0/6] refs: introduce support for partial reference transactions Karthik Nayak
2025-02-07 7:34 ` [PATCH 1/6] refs/files: remove duplicate check in `split_symref_update()` Karthik Nayak
2025-02-07 16:12 ` Patrick Steinhardt
2025-02-11 6:35 ` Karthik Nayak
2025-02-07 7:34 ` [PATCH 2/6] refs: move duplicate refname update check to generic layer Karthik Nayak
2025-02-07 16:12 ` Patrick Steinhardt
2025-02-11 10:33 ` Karthik Nayak
2025-02-07 7:34 ` [PATCH 3/6] refs/files: remove duplicate duplicates check Karthik Nayak
2025-02-07 16:12 ` Patrick Steinhardt
2025-02-07 7:34 ` [PATCH 4/6] refs/reftable: extract code from the transaction preparation Karthik Nayak
2025-02-07 7:34 ` [PATCH 5/6] refs: implement partial reference transaction support Karthik Nayak
2025-02-07 16:12 ` Patrick Steinhardt
2025-02-21 10:33 ` Karthik Nayak
2025-02-07 7:34 ` [PATCH 6/6] update-ref: add --allow-partial flag for stdin mode Karthik Nayak
2025-02-07 16:12 ` Patrick Steinhardt
2025-02-21 11:45 ` Karthik Nayak
2025-02-11 17:03 ` [PATCH 0/6] refs: introduce support for partial reference transactions Phillip Wood
2025-02-11 17:40 ` Phillip Wood
2025-02-12 12:36 ` Karthik Nayak
2025-02-12 12:34 ` Karthik Nayak
2025-02-19 14:34 ` Phillip Wood
2025-02-19 15:10 ` Patrick Steinhardt
2025-02-21 11:50 ` Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 0/7] " Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 1/7] refs/files: remove redundant check in split_symref_update() Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 2/7] refs: move duplicate refname update check to generic layer Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 3/7] refs/files: remove duplicate duplicates check Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 4/7] refs/reftable: extract code from the transaction preparation Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 5/7] refs: introduce enum-based transaction error types Karthik Nayak
2025-02-25 11:08 ` Patrick Steinhardt
2025-03-03 20:12 ` Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 6/7] refs: implement partial reference transaction support Karthik Nayak
2025-02-25 11:07 ` Patrick Steinhardt
2025-03-03 20:17 ` Karthik Nayak
2025-02-25 14:57 ` Phillip Wood
2025-03-03 20:21 ` Karthik Nayak
2025-03-04 10:31 ` Phillip Wood
2025-03-05 14:20 ` Karthik Nayak
2025-02-25 9:29 ` [PATCH v2 7/7] update-ref: add --allow-partial flag for stdin mode Karthik Nayak
2025-02-25 11:08 ` Patrick Steinhardt
2025-03-03 20:22 ` Karthik Nayak
2025-02-25 14:59 ` Phillip Wood
2025-03-03 20:34 ` Karthik Nayak
2025-03-05 17:38 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Karthik Nayak
2025-03-05 17:38 ` [PATCH v3 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
2025-03-05 21:20 ` Junio C Hamano
2025-03-06 9:13 ` Karthik Nayak
2025-03-05 17:38 ` [PATCH v3 2/8] refs: move duplicate refname update check to generic layer Karthik Nayak
2025-03-05 21:56 ` Junio C Hamano
2025-03-06 9:46 ` Karthik Nayak
2025-03-05 17:38 ` [PATCH v3 3/8] refs/files: remove duplicate duplicates check Karthik Nayak
2025-03-05 17:38 ` [PATCH v3 4/8] refs/reftable: extract code from the transaction preparation Karthik Nayak
2025-03-05 17:39 ` [PATCH v3 5/8] refs: introduce enum-based transaction error types Karthik Nayak
2025-03-05 17:39 ` [PATCH v3 6/8] refs: implement partial reference transaction support Karthik Nayak
2025-03-07 19:50 ` Jeff King
2025-03-07 20:46 ` Junio C Hamano
2025-03-07 20:48 ` Junio C Hamano
2025-03-07 21:05 ` Karthik Nayak
2025-03-07 22:54 ` [PATCH] config.mak.dev: enable -Wunreachable-code Jeff King
2025-03-07 23:28 ` Junio C Hamano
2025-03-08 3:23 ` Jeff King
2025-03-10 15:40 ` Junio C Hamano
2025-03-10 16:04 ` Jeff King
2025-03-10 18:50 ` Junio C Hamano
2025-03-14 16:10 ` Jeff King
2025-03-14 16:13 ` Jeff King
2025-03-14 17:27 ` Junio C Hamano
2025-03-14 17:40 ` Junio C Hamano
2025-03-14 17:43 ` Patrick Steinhardt
2025-03-14 18:53 ` Jeff King
2025-03-14 19:50 ` Junio C Hamano
2025-03-14 17:15 ` Junio C Hamano
2025-03-14 21:09 ` [PATCH v2 0/3] -Wunreachable-code Junio C Hamano
2025-03-14 21:09 ` [PATCH v2 1/3] config.mak.dev: enable -Wunreachable-code Junio C Hamano
2025-03-14 21:09 ` [PATCH v2 2/3] run-command: use errno to check for sigfillset() error Junio C Hamano
2025-03-17 21:30 ` Taylor Blau
2025-03-17 23:12 ` Junio C Hamano
2025-03-18 0:36 ` Junio C Hamano
2025-03-14 21:09 ` [PATCH v2 3/3] git-compat-util: add NOT_A_CONST macro and use it in atfork_prepare() Junio C Hamano
2025-03-14 22:29 ` Junio C Hamano
2025-03-17 18:00 ` Jeff King
2025-03-17 23:53 ` [PATCH v3 0/3] -Wunreachable-code Junio C Hamano
2025-03-17 23:53 ` [PATCH v3 1/3] run-command: use errno to check for sigfillset() error Junio C Hamano
2025-03-17 23:53 ` [PATCH v3 2/3] git-compat-util: add NOT_CONSTANT macro and use it in atfork_prepare() Junio C Hamano
2025-03-18 0:20 ` Jeff King
2025-03-18 0:28 ` Junio C Hamano
2025-03-18 22:04 ` Calvin Wan
2025-03-18 22:26 ` Calvin Wan
2025-03-18 23:55 ` Junio C Hamano
2025-03-17 23:53 ` [PATCH v3 3/3] config.mak.dev: enable -Wunreachable-code Junio C Hamano
2025-03-18 0:18 ` [PATCH v3 0/3] -Wunreachable-code Jeff King
2025-03-07 21:02 ` [PATCH v3 6/8] refs: implement partial reference transaction support Karthik Nayak
2025-03-07 19:57 ` Jeff King
2025-03-07 21:07 ` Karthik Nayak
2025-03-05 17:39 ` [PATCH v3 7/8] refs: support partial update rejections during F/D checks Karthik Nayak
2025-03-05 17:39 ` [PATCH v3 8/8] update-ref: add --allow-partial flag for stdin mode Karthik Nayak
2025-03-05 19:28 ` [PATCH v3 0/8] refs: introduce support for partial reference transactions Junio C Hamano
2025-03-06 9:06 ` Karthik Nayak
2025-03-20 11:43 ` [PATCH v4 0/8] refs: introduce support for batched reference updates Karthik Nayak
2025-03-20 11:43 ` [PATCH v4 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
2025-03-20 11:43 ` [PATCH v4 2/8] refs: move duplicate refname update check to generic layer Karthik Nayak
2025-03-20 11:43 ` [PATCH v4 3/8] refs/files: remove duplicate duplicates check Karthik Nayak
2025-03-20 11:43 ` [PATCH v4 4/8] refs/reftable: extract code from the transaction preparation Karthik Nayak
2025-03-20 11:44 ` [PATCH v4 5/8] refs: introduce enum-based transaction error types Karthik Nayak
2025-03-20 20:26 ` Patrick Steinhardt
2025-03-24 14:50 ` Karthik Nayak
2025-03-25 12:31 ` Patrick Steinhardt
2025-03-20 11:44 ` [PATCH v4 6/8] refs: implement batch reference update support Karthik Nayak
2025-03-20 20:26 ` Patrick Steinhardt
2025-03-24 14:54 ` Karthik Nayak
2025-03-20 11:44 ` [PATCH v4 7/8] refs: support rejection in batch updates during F/D checks Karthik Nayak
2025-03-24 13:08 ` Patrick Steinhardt
2025-03-24 17:48 ` Karthik Nayak
2025-03-25 12:31 ` Patrick Steinhardt
2025-03-20 11:44 ` [PATCH v4 8/8] update-ref: add --batch-updates flag for stdin mode Karthik Nayak
2025-03-24 13:08 ` Patrick Steinhardt
2025-03-24 17:51 ` Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 2/8] refs: move duplicate refname update check to generic layer Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 3/8] refs/files: remove duplicate duplicates check Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 4/8] refs/reftable: extract code from the transaction preparation Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 5/8] refs: introduce enum-based transaction error types Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 6/8] refs: implement batch reference update support Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 7/8] refs: support rejection in batch updates during F/D checks Karthik Nayak
2025-03-27 11:13 ` [PATCH v5 8/8] update-ref: add --batch-updates flag for stdin mode Karthik Nayak
2025-03-28 13:00 ` Jean-Noël AVILA
2025-03-29 16:36 ` Junio C Hamano
2025-03-29 18:18 ` Karthik Nayak
2025-03-28 9:24 ` [PATCH v5 0/8] refs: introduce support for batched reference updates Patrick Steinhardt
2025-04-08 8:51 ` [PATCH v6 " Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 1/8] refs/files: remove redundant check in split_symref_update() Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 2/8] refs: move duplicate refname update check to generic layer Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 3/8] refs/files: remove duplicate duplicates check Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 4/8] refs/reftable: extract code from the transaction preparation Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 5/8] refs: introduce enum-based transaction error types Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 6/8] refs: implement batch reference update support Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 7/8] refs: support rejection in batch updates during F/D checks Karthik Nayak
2025-04-08 8:51 ` [PATCH v6 8/8] update-ref: add --batch-updates flag for stdin mode Karthik Nayak
2025-04-08 15:02 ` Junio C Hamano
2025-04-08 15:26 ` Karthik Nayak
2025-04-08 17:37 ` Junio C Hamano
2025-04-10 11:23 ` Karthik Nayak
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).