* Lost files after git stash && git stash pop
@ 2023-07-21 17:31 Till Friebe
2023-07-22 21:44 ` Torsten Bögershausen
2023-08-08 17:26 ` [PATCH v1 1/1] git stash needing mkdir deletes untracked file tboegi
0 siblings, 2 replies; 15+ messages in thread
From: Till Friebe @ 2023-07-21 17:31 UTC (permalink / raw)
To: git
Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.
What did you do before the bug happened? (Steps to reproduce your issue)
```
git init
mkdir README
touch README/README
git add .
git commit -m "Init project"
echo "Test" > README/README
mv README/README README2
rmdir README
mv README2 README
git stash
git stash pop
```
What did you expect to happen? (Expected behavior)
I expected that after the `git stash pop` the README file would be back.
What happened instead? (Actual behavior)
This README with "Test" file was deleted and I lost 5 hours of work.
What's different between what you expected and what actually happened?
The file doesn't exist anymore and I can't recover it.
Anything else you want to add:
This is just a reproducible example.
Please review the rest of the bug report below.
You can delete any lines you don't wish to share.
[System Info]
git version:
git version 2.39.2 (Apple Git-143)
cpu: arm64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
uname: Darwin 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:22:20 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T6000 arm64
compiler info: clang: 14.0.3 (clang-1403.0.22.14.1)
libc info: no libc information available
$SHELL (typically, interactive shell): /bin/zsh
[Enabled Hooks]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Lost files after git stash && git stash pop
2023-07-21 17:31 Lost files after git stash && git stash pop Till Friebe
@ 2023-07-22 21:44 ` Torsten Bögershausen
2023-07-23 10:01 ` Phillip Wood
2023-08-08 17:26 ` [PATCH v1 1/1] git stash needing mkdir deletes untracked file tboegi
1 sibling, 1 reply; 15+ messages in thread
From: Torsten Bögershausen @ 2023-07-22 21:44 UTC (permalink / raw)
To: Till Friebe; +Cc: git
On Fri, Jul 21, 2023 at 07:31:53PM +0200, Till Friebe wrote:
> Thank you for filling out a Git bug report!
> Please answer the following questions to help us understand your issue.
>
> What did you do before the bug happened? (Steps to reproduce your issue)
> ```
> git init
> mkdir README
> touch README/README
> git add .
> git commit -m "Init project"
> echo "Test" > README/README
> mv README/README README2
> rmdir README
> mv README2 README
> git stash
> git stash pop
> ```
>
> What did you expect to happen? (Expected behavior)
> I expected that after the `git stash pop` the README file would be back.
>
> What happened instead? (Actual behavior)
> This README with "Test" file was deleted and I lost 5 hours of work.
That is always sad to hear, when work is lost.
However, I personally wonder if this is a bug or not.
First, Git is told to track a file called README/README
Then the file is removed, without telling Git.
And a new, unkown file appers on disk (which collides with the name
of the directory)
Using this sequence could have told Git, what is going on:
git mv README/README README2
rmdir README
git mv README2 README
(a temporary branch may be checked out, with the option
to merge-squash the final result)
An other alternative could be to tell `git stash` to care
about untracked file(s):
git stash -u
git stash pop
Which will refuse to apply the stash.
A third alternative could be to keep the file inside an
editor, to have the content still available.
However, it would/could be nice, if files are not simply deleted,
but saved into a "lost+found" folder, or a wastebasket kind of thing.
But which files ?
Those that are untracked ?
They may be important (local config files, passwords, help scripts, ...)
or not (.o files from a C compiler).
In some older discussions they had been named "precious" files.
But, as far as I remember, there was no easy solution.
In that sense I don't have a better answer.
Others may have.
Thanks for reporting, it make me read [1] and come to the conclusion
that it is sometimes safer to checkout out a temporary branch, commit
everything and clean up later, rather than relying too much on
`git stash`
<https://stackoverflow.com/questions/835501/how-do-you-stash-an-untracked-file>
>
> What's different between what you expected and what actually happened?
> The file doesn't exist anymore and I can't recover it.
>
> Anything else you want to add:
> This is just a reproducible example.
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Lost files after git stash && git stash pop
2023-07-22 21:44 ` Torsten Bögershausen
@ 2023-07-23 10:01 ` Phillip Wood
2023-07-23 20:52 ` Torsten Bögershausen
0 siblings, 1 reply; 15+ messages in thread
From: Phillip Wood @ 2023-07-23 10:01 UTC (permalink / raw)
To: Torsten Bögershausen, Till Friebe; +Cc: git
On 22/07/2023 22:44, Torsten Bögershausen wrote:
> On Fri, Jul 21, 2023 at 07:31:53PM +0200, Till Friebe wrote:
>> Thank you for filling out a Git bug report!
>> Please answer the following questions to help us understand your issue.
>>
>> What did you do before the bug happened? (Steps to reproduce your issue)
>> ```
>> git init
>> mkdir README
>> touch README/README
>> git add .
>> git commit -m "Init project"
>> echo "Test" > README/README
>> mv README/README README2
>> rmdir README
>> mv README2 README
>> git stash
>> git stash pop
>> ```
>>
>> What did you expect to happen? (Expected behavior)
>> I expected that after the `git stash pop` the README file would be back.
>>
>> What happened instead? (Actual behavior)
>> This README with "Test" file was deleted and I lost 5 hours of work.
>
> That is always sad to hear, when work is lost.
Indeed it is. Thanks Till for providing an easy reproducer.
> However, I personally wonder if this is a bug or not.
I think whenever git overwrites an untracked file without the user
passing some option indicating that they want to do so it is a bug. For
example "git checkout" refuses to overwrite untracked files by default.
Sadly this seems to be a known bug in do_push_stash() where we are using
"git reset --hard" to remove the stashed changes from the working copy.
This was documented in 94b7f1563a (Comment important codepaths regarding
nuking untracked files/dirs, 2021-09-27). The stash implementation does
a lot of necessary forking of subprocesses, in this case I think it
would be better to call unpack_trees() directly with
UNPACK_RESET_PROTECT_UNTRACKED.
Best Wishes
Phillip
> First, Git is told to track a file called README/README
> Then the file is removed, without telling Git.
> And a new, unkown file appers on disk (which collides with the name
> of the directory)
>
> Using this sequence could have told Git, what is going on:
> git mv README/README README2
> rmdir README
> git mv README2 README
>
> (a temporary branch may be checked out, with the option
> to merge-squash the final result)
>
>
> An other alternative could be to tell `git stash` to care
> about untracked file(s):
>
> git stash -u
> git stash pop
>
> Which will refuse to apply the stash.
>
> A third alternative could be to keep the file inside an
> editor, to have the content still available.
>
> However, it would/could be nice, if files are not simply deleted,
> but saved into a "lost+found" folder, or a wastebasket kind of thing.
>
> But which files ?
> Those that are untracked ?
> They may be important (local config files, passwords, help scripts, ...)
> or not (.o files from a C compiler).
>
> In some older discussions they had been named "precious" files.
> But, as far as I remember, there was no easy solution.
> In that sense I don't have a better answer.
> Others may have.
>
> Thanks for reporting, it make me read [1] and come to the conclusion
> that it is sometimes safer to checkout out a temporary branch, commit
> everything and clean up later, rather than relying too much on
> `git stash`
>
>
> <https://stackoverflow.com/questions/835501/how-do-you-stash-an-untracked-file>
>>
>> What's different between what you expected and what actually happened?
>> The file doesn't exist anymore and I can't recover it.
>>
>> Anything else you want to add:
>> This is just a reproducible example.
>>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Lost files after git stash && git stash pop
2023-07-23 10:01 ` Phillip Wood
@ 2023-07-23 20:52 ` Torsten Bögershausen
2023-07-24 9:59 ` Phillip Wood
0 siblings, 1 reply; 15+ messages in thread
From: Torsten Bögershausen @ 2023-07-23 20:52 UTC (permalink / raw)
To: phillip.wood; +Cc: Till Friebe, git
On Sun, Jul 23, 2023 at 11:01:29AM +0100, Phillip Wood wrote:
> On 22/07/2023 22:44, Torsten Bögershausen wrote:
> > On Fri, Jul 21, 2023 at 07:31:53PM +0200, Till Friebe wrote:
> > > Thank you for filling out a Git bug report!
> > > Please answer the following questions to help us understand your issue.
> > >
> > > What did you do before the bug happened? (Steps to reproduce your issue)
> > > ```
> > > git init
> > > mkdir README
> > > touch README/README
> > > git add .
> > > git commit -m "Init project"
> > > echo "Test" > README/README
> > > mv README/README README2
> > > rmdir README
> > > mv README2 README
> > > git stash
> > > git stash pop
> > > ```
> > >
> > > What did you expect to happen? (Expected behavior)
> > > I expected that after the `git stash pop` the README file would be back.
> > >
> > > What happened instead? (Actual behavior)
> > > This README with "Test" file was deleted and I lost 5 hours of work.
> >
> > That is always sad to hear, when work is lost.
>
> Indeed it is. Thanks Till for providing an easy reproducer.
>
> > However, I personally wonder if this is a bug or not.
>
> I think whenever git overwrites an untracked file without the user passing
> some option indicating that they want to do so it is a bug.
OK, agreed after reading the next sentence.
> For example "git
> checkout" refuses to overwrite untracked files by default. Sadly this seems
> to be a known bug in do_push_stash() where we are using "git reset --hard"
> to remove the stashed changes from the working copy. This was documented in
> 94b7f1563a (Comment important codepaths regarding nuking untracked
> files/dirs, 2021-09-27). The stash implementation does a lot of necessary
> forking of subprocesses, in this case I think it would be better to call
> unpack_trees() directly with UNPACK_RESET_PROTECT_UNTRACKED.
Thanks for the fast response.
This is not an area of Git, where I have much understanding of the code.
But is seems as if pop_stash() in builtin/stash.c
(and the called functions) seems to be the problem here ?
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Lost files after git stash && git stash pop
2023-07-23 20:52 ` Torsten Bögershausen
@ 2023-07-24 9:59 ` Phillip Wood
0 siblings, 0 replies; 15+ messages in thread
From: Phillip Wood @ 2023-07-24 9:59 UTC (permalink / raw)
To: Torsten Bögershausen, phillip.wood; +Cc: Till Friebe, git
Hi Torsten
On 23/07/2023 21:52, Torsten Bögershausen wrote:
>> I think whenever git overwrites an untracked file without the user passing
>> some option indicating that they want to do so it is a bug.
>
> OK, agreed after reading the next sentence.
>
>> For example "git
>> checkout" refuses to overwrite untracked files by default. Sadly this seems
>> to be a known bug in do_push_stash() where we are using "git reset --hard"
>> to remove the stashed changes from the working copy. This was documented in
>> 94b7f1563a (Comment important codepaths regarding nuking untracked
>> files/dirs, 2021-09-27). The stash implementation does a lot of necessary
>> forking of subprocesses, in this case I think it would be better to call
>> unpack_trees() directly with UNPACK_RESET_PROTECT_UNTRACKED.
>
> Thanks for the fast response.
>
> This is not an area of Git, where I have much understanding of the code.
> But is seems as if pop_stash() in builtin/stash.c
> (and the called functions) seems to be the problem here ?
Confusingly it is creating the stash that deletes the untracked file because
it recreates README/README. do_push_stash() in builtin/stash.c is the culprit
I think. I had hoped the diff below would fix the problem but it does not
seem to and breaks half a dozen test cases that seem to rely on removing
untracked files. Unfortunately I don't really have time to dig any
further at the moment.
Best Wishes
Phillip
diff --git a/builtin/stash.c b/builtin/stash.c
index fe64cde9ce3..c8bbfe56d26 100644
--- a/builtin/stash.c
+++ b/builtin/stash.c
@@ -29,6 +29,8 @@
#include "exec-cmd.h"
#include "reflog.h"
#include "add-interactive.h"
+#include "reset.h"
+#include "submodule.h"
#define INCLUDE_ALL_FILES 2
@@ -336,7 +338,7 @@ static int apply_cached(struct strbuf *out)
return pipe_command(&cp, out->buf, out->len, NULL, 0, NULL, 0);
}
-static int reset_head(void)
+static int stash_reset_head(void)
{
struct child_process cp = CHILD_PROCESS_INIT;
@@ -569,7 +571,7 @@ static int do_apply_stash(const char *prefix, struct stash_info *info,
get_index_file(), 0, NULL))
return error(_("could not save index tree"));
- reset_head();
+ stash_reset_head();
discard_index(&the_index);
repo_read_index(the_repository);
}
@@ -1649,12 +1651,14 @@ static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int q
goto done;
}
} else {
- struct child_process cp = CHILD_PROCESS_INIT;
- cp.git_cmd = 1;
- /* BUG: this nukes untracked files in the way */
- strvec_pushl(&cp.args, "reset", "--hard", "-q",
- "--no-recurse-submodules", NULL);
- if (run_command(&cp)) {
+ struct reset_head_opts opts = {
+ .flags = RESET_HEAD_HARD,
+ };
+
+ if (should_update_submodules())
+ BUG("stash should not update submodules");
+
+ if (reset_head(the_repository, &opts)) {
ret = -1;
goto done;
}
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v1 1/1] git stash needing mkdir deletes untracked file
2023-07-21 17:31 Lost files after git stash && git stash pop Till Friebe
2023-07-22 21:44 ` Torsten Bögershausen
@ 2023-08-08 17:26 ` tboegi
2023-08-08 18:03 ` Torsten Bögershausen
` (2 more replies)
1 sibling, 3 replies; 15+ messages in thread
From: tboegi @ 2023-08-08 17:26 UTC (permalink / raw)
To: tboegi, git, friebetill, phillip.wood123
From: Torsten Bögershausen <tboegi@web.de>
The following sequence leads to loss of work:
git init
mkdir README
touch README/README
git add .
git commit -m "Init project"
echo "Test" > README/README
mv README/README README2
rmdir README
mv README2 README
git stash
git stash pop
The problem is, that `git stash` needs to create the directory README/
and to be able to do this, the file README needs to be removed.
And this is, where the work was lost.
There are different possibilities preventing this loss of work:
a)
`git stash` does refuse the removel of the untracked file,
when a directory with the same name needs to be created
There is a small problem here:
In the ideal world, the stash would do nothing at all,
and not do anything but complain.
The current code makes this hard to achieve
An other solution could be to do as much stash work as possible,
but stop when the file/directory conflict is detected.
This would create some inconsistent state.
b) Create the directory as needed, but rename the file before doing that.
This would let the `git stash` proceed as usual and create a "new" file,
which may be surprising for some worlflows.
This change goes for b), as it seems the most intuitive solution for
Git users.
Introdue a new function rename_to_untracked_or_warn() and use it
in create_directories() in entry.c
Reported-by: Till Friebe <friebetill@gmail.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
---
entry.c | 25 ++++++++++++++++++++++++-
t/t3903-stash.sh | 23 +++++++++++++++++++++++
2 files changed, 47 insertions(+), 1 deletion(-)
diff --git a/entry.c b/entry.c
index 43767f9043..76d8a0762d 100644
--- a/entry.c
+++ b/entry.c
@@ -15,6 +15,28 @@
#include "entry.h"
#include "parallel-checkout.h"
+static int rename_to_untracked_or_warn(const char *file)
+{
+ const size_t file_name_len = strlen(file);
+ const static char *dot_untracked = ".untracked";
+ const size_t dot_un_len = strlen(dot_untracked);
+ struct strbuf sb;
+ int ret;
+
+ strbuf_init(&sb, file_name_len + dot_un_len);
+ strbuf_add(&sb, file, file_name_len);
+ strbuf_add(&sb, dot_untracked, dot_un_len);
+ ret = rename(file, sb.buf);
+
+ if (ret) {
+ int saved_errno = errno;
+ warning_errno(_("unable rename '%s' into '%s'"), file, sb.buf);
+ errno = saved_errno;
+ }
+ strbuf_release(&sb);
+ return ret;
+}
+
static void create_directories(const char *path, int path_len,
const struct checkout *state)
{
@@ -48,7 +70,8 @@ static void create_directories(const char *path, int path_len,
*/
if (mkdir(buf, 0777)) {
if (errno == EEXIST && state->force &&
- !unlink_or_warn(buf) && !mkdir(buf, 0777))
+ !rename_to_untracked_or_warn(buf) &&
+ !mkdir(buf, 0777))
continue;
die_errno("cannot create directory at '%s'", buf);
}
diff --git a/t/t3903-stash.sh b/t/t3903-stash.sh
index 0b3dfeaea2..1a210f8a5a 100755
--- a/t/t3903-stash.sh
+++ b/t/t3903-stash.sh
@@ -1512,4 +1512,27 @@ test_expect_success 'restore untracked files even when we hit conflicts' '
)
'
+test_expect_success 'stash mkdir README needed - README.untracked created' '
+ git init mkdir_needed_file_untracked &&
+ (
+ cd mkdir_needed_file_untracked &&
+ mkdir README &&
+ touch README/README &&
+ git add . &&
+ git commit -m "Add README/README" &&
+ echo Version2 > README/README &&
+ mv README/README README2 &&
+ rmdir README &&
+ mv README2 README &&
+ git stash &&
+ test_path_is_file README.untracked &&
+ echo Version2 >expect &&
+ test_cmp expect README.untracked &&
+ rm expect &&
+ git stash pop &&
+ test_path_is_file README.untracked &&
+ echo Version2 >expect &&
+ test_cmp expect README.untracked
+ )
+'
test_done
--
2.41.0.394.ge43f4fd0bd
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v1 1/1] git stash needing mkdir deletes untracked file
2023-08-08 17:26 ` [PATCH v1 1/1] git stash needing mkdir deletes untracked file tboegi
@ 2023-08-08 18:03 ` Torsten Bögershausen
2023-08-08 19:28 ` Eric Sunshine
2023-08-09 13:15 ` Phillip Wood
2 siblings, 0 replies; 15+ messages in thread
From: Torsten Bögershausen @ 2023-08-08 18:03 UTC (permalink / raw)
To: git, friebetill, phillip.wood123
On Tue, Aug 08, 2023 at 07:26:24PM +0200, tboegi@web.de wrote:
> From: Torsten Bögershausen <tboegi@web.de>
> .
... The following sequence leads to loss of work:
I just realized that this breaks other tests:
t1092-sparse-checkout-compatibility.sh not ok 17 - diff with renames and conflicts
t1092-sparse-checkout-compatibility.sh not ok 18 - diff with directory/file conflicts
However, comments are welcome:
Is it a good idea to create file called "filename.untracked" ?
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v1 1/1] git stash needing mkdir deletes untracked file
2023-08-08 17:26 ` [PATCH v1 1/1] git stash needing mkdir deletes untracked file tboegi
2023-08-08 18:03 ` Torsten Bögershausen
@ 2023-08-08 19:28 ` Eric Sunshine
2023-08-09 13:15 ` Phillip Wood
2 siblings, 0 replies; 15+ messages in thread
From: Eric Sunshine @ 2023-08-08 19:28 UTC (permalink / raw)
To: tboegi; +Cc: git, friebetill, phillip.wood123
On Tue, Aug 8, 2023 at 3:15 PM <tboegi@web.de> wrote:
> The following sequence leads to loss of work:
> git init
> mkdir README
> touch README/README
> git add .
> git commit -m "Init project"
> echo "Test" > README/README
> mv README/README README2
> rmdir README
> mv README2 README
> git stash
> git stash pop
>
> The problem is, that `git stash` needs to create the directory README/
> and to be able to do this, the file README needs to be removed.
> And this is, where the work was lost.
> There are different possibilities preventing this loss of work:
> a)
> `git stash` does refuse the removel of the untracked file,
s/removel/removal/
> when a directory with the same name needs to be created
s/$/./
> There is a small problem here:
> In the ideal world, the stash would do nothing at all,
> and not do anything but complain.
> The current code makes this hard to achieve
s/$/./
> An other solution could be to do as much stash work as possible,
s/An other/Another/
> but stop when the file/directory conflict is detected.
> This would create some inconsistent state.
>
> b) Create the directory as needed, but rename the file before doing that.
> This would let the `git stash` proceed as usual and create a "new" file,
> which may be surprising for some worlflows.
s/worlflows/workflows/
> This change goes for b), as it seems the most intuitive solution for
> Git users.
>
> Introdue a new function rename_to_untracked_or_warn() and use it
s/Introdue/Introduce/
> in create_directories() in entry.c
>
> Reported-by: Till Friebe <friebetill@gmail.com>
> Signed-off-by: Torsten Bögershausen <tboegi@web.de>
> ---
> diff --git a/entry.c b/entry.c
> @@ -15,6 +15,28 @@
> +static int rename_to_untracked_or_warn(const char *file)
> +{
> + const size_t file_name_len = strlen(file);
> + const static char *dot_untracked = ".untracked";
> + const size_t dot_un_len = strlen(dot_untracked);
> + struct strbuf sb;
> + int ret;
> +
> + strbuf_init(&sb, file_name_len + dot_un_len);
> + strbuf_add(&sb, file, file_name_len);
> + strbuf_add(&sb, dot_untracked, dot_un_len);
> + ret = rename(file, sb.buf);
This could probably all be simplified to:
char *to = xstrfmt("%s.untracked", file);
ret = rename(...);
...
free(to);
If there is already a file named "foo.untracked", then this will
overwrite it, thus potentially losing work, right? I wonder if it
makes sense to be a bit more careful.
> + if (ret) {
> + int saved_errno = errno;
> + warning_errno(_("unable rename '%s' into '%s'"), file, sb.buf);
> + errno = saved_errno;
> + }
> + strbuf_release(&sb);
> + return ret;
> +}
Do we want to give the user some warning/notification that their file,
as a safety precaution, got renamed to "foo.untracked"?
> diff --git a/t/t3903-stash.sh b/t/t3903-stash.sh
> @@ -1512,4 +1512,27 @@ test_expect_success 'restore untracked files even when we hit conflicts' '
> +test_expect_success 'stash mkdir README needed - README.untracked created' '
> + git init mkdir_needed_file_untracked &&
> + (
> + cd mkdir_needed_file_untracked &&
> + mkdir README &&
> + touch README/README &&
s/touch/>/
> + git add . &&
> + git commit -m "Add README/README" &&
> + echo Version2 > README/README &&
s/> R/>R/
> + mv README/README README2 &&
> + rmdir README &&
> + mv README2 README &&
> + git stash &&
> + test_path_is_file README.untracked &&
> + echo Version2 >expect &&
> + test_cmp expect README.untracked &&
> + rm expect &&
> + git stash pop &&
> + test_path_is_file README.untracked &&
> + echo Version2 >expect &&
> + test_cmp expect README.untracked
> + )
> +'
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v1 1/1] git stash needing mkdir deletes untracked file
2023-08-08 17:26 ` [PATCH v1 1/1] git stash needing mkdir deletes untracked file tboegi
2023-08-08 18:03 ` Torsten Bögershausen
2023-08-08 19:28 ` Eric Sunshine
@ 2023-08-09 13:15 ` Phillip Wood
2023-08-09 18:47 ` Torsten Bögershausen
2023-08-09 20:57 ` Junio C Hamano
2 siblings, 2 replies; 15+ messages in thread
From: Phillip Wood @ 2023-08-09 13:15 UTC (permalink / raw)
To: tboegi, git, friebetill; +Cc: Junio C Hamano
Hi Torsten
Thanks for working on this. I've cc'd Junio for his unpack_trees()
knowledge.
On 08/08/2023 18:26, tboegi@web.de wrote:
> From: Torsten Bögershausen <tboegi@web.de>
>
> The following sequence leads to loss of work:
> git init
> mkdir README
> touch README/README
> git add .
> git commit -m "Init project"
> echo "Test" > README/README
> mv README/README README2
> rmdir README
> mv README2 README
> git stash
> git stash pop
>
> The problem is, that `git stash` needs to create the directory README/
> and to be able to do this, the file README needs to be removed.
> And this is, where the work was lost.
> There are different possibilities preventing this loss of work:
> a)
> `git stash` does refuse the removel of the untracked file,
> when a directory with the same name needs to be created
> There is a small problem here:
> In the ideal world, the stash would do nothing at all,
> and not do anything but complain.
> The current code makes this hard to achieve
> An other solution could be to do as much stash work as possible,
> but stop when the file/directory conflict is detected.
> This would create some inconsistent state.
>
> b) Create the directory as needed, but rename the file before doing that.
> This would let the `git stash` proceed as usual and create a "new" file,
> which may be surprising for some worlflows.
>
> This change goes for b), as it seems the most intuitive solution for
> Git users.
>
> Introdue a new function rename_to_untracked_or_warn() and use it
> in create_directories() in entry.c
Although this change is framed in terms of changes to "git stash push" I
think the underlying issue and this patch actually affects all users of
unpack_trees(). For example if "README" is untracked then
git checkout <rev> README
will currently fail if <rev>:README is a blob but will succeed and
remove the untracked file if <rev>:README is a tree.
I'm far from an expert in this area but I think we might want to
understand why unpack_trees() sets state->force when it calls
checkout_entry() before making any changes.
Best Wishes
Phillip
> Reported-by: Till Friebe <friebetill@gmail.com>
> Signed-off-by: Torsten Bögershausen <tboegi@web.de>
> ---
> entry.c | 25 ++++++++++++++++++++++++-
> t/t3903-stash.sh | 23 +++++++++++++++++++++++
> 2 files changed, 47 insertions(+), 1 deletion(-)
>
> diff --git a/entry.c b/entry.c
> index 43767f9043..76d8a0762d 100644
> --- a/entry.c
> +++ b/entry.c
> @@ -15,6 +15,28 @@
> #include "entry.h"
> #include "parallel-checkout.h"
>
> +static int rename_to_untracked_or_warn(const char *file)
> +{
> + const size_t file_name_len = strlen(file);
> + const static char *dot_untracked = ".untracked";
> + const size_t dot_un_len = strlen(dot_untracked);
> + struct strbuf sb;
> + int ret;
> +
> + strbuf_init(&sb, file_name_len + dot_un_len);
> + strbuf_add(&sb, file, file_name_len);
> + strbuf_add(&sb, dot_untracked, dot_un_len);
> + ret = rename(file, sb.buf);
> +
> + if (ret) {
> + int saved_errno = errno;
> + warning_errno(_("unable rename '%s' into '%s'"), file, sb.buf);
> + errno = saved_errno;
> + }
> + strbuf_release(&sb);
> + return ret;
> +}
> +
> static void create_directories(const char *path, int path_len,
> const struct checkout *state)
> {
> @@ -48,7 +70,8 @@ static void create_directories(const char *path, int path_len,
> */
> if (mkdir(buf, 0777)) {
> if (errno == EEXIST && state->force &&
> - !unlink_or_warn(buf) && !mkdir(buf, 0777))
> + !rename_to_untracked_or_warn(buf) &&
> + !mkdir(buf, 0777))
> continue;
> die_errno("cannot create directory at '%s'", buf);
> }
> diff --git a/t/t3903-stash.sh b/t/t3903-stash.sh
> index 0b3dfeaea2..1a210f8a5a 100755
> --- a/t/t3903-stash.sh
> +++ b/t/t3903-stash.sh
> @@ -1512,4 +1512,27 @@ test_expect_success 'restore untracked files even when we hit conflicts' '
> )
> '
>
> +test_expect_success 'stash mkdir README needed - README.untracked created' '
> + git init mkdir_needed_file_untracked &&
> + (
> + cd mkdir_needed_file_untracked &&
> + mkdir README &&
> + touch README/README &&
> + git add . &&
> + git commit -m "Add README/README" &&
> + echo Version2 > README/README &&
> + mv README/README README2 &&
> + rmdir README &&
> + mv README2 README &&
> + git stash &&
> + test_path_is_file README.untracked &&
> + echo Version2 >expect &&
> + test_cmp expect README.untracked &&
> + rm expect &&
> + git stash pop &&
> + test_path_is_file README.untracked &&
> + echo Version2 >expect &&
> + test_cmp expect README.untracked
> + )
> +'
> test_done
> --
> 2.41.0.394.ge43f4fd0bd
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v1 1/1] git stash needing mkdir deletes untracked file
2023-08-09 13:15 ` Phillip Wood
@ 2023-08-09 18:47 ` Torsten Bögershausen
2023-08-15 9:15 ` Phillip Wood
2023-08-09 20:57 ` Junio C Hamano
1 sibling, 1 reply; 15+ messages in thread
From: Torsten Bögershausen @ 2023-08-09 18:47 UTC (permalink / raw)
To: phillip.wood; +Cc: git, friebetill, Junio C Hamano, Eric Sunshine
On Wed, Aug 09, 2023 at 02:15:28PM +0100, Phillip Wood wrote:
> Hi Torsten
>
> Thanks for working on this. I've cc'd Junio for his unpack_trees()
> knowledge.
Thanks Eric for the review.
Hej Phillip,
I have been playing around with the whole thing some time.
At the end I had a version, which did fiddle the information
that we are doing a `git stash` (and not any other operation)
into entry.c, and all test cases passed.
So in principle I can dig out all changes, polish them
and send them out, after doing cleanups of course.
(And that could take a couple of days, or weeks ;-)
My main question is still open:
Is it a good idea, to create a "helper file" ?
The naming can be discussed, we may stick the date/time
into the filename to make it really unique, or so.
Reading the different reports and including own experience,
I still think that a directory called ".deleted-by-user"
or ".wastebin" or something in that style is a good idea.
What do others think ?
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v1 1/1] git stash needing mkdir deletes untracked file
2023-08-09 13:15 ` Phillip Wood
2023-08-09 18:47 ` Torsten Bögershausen
@ 2023-08-09 20:57 ` Junio C Hamano
2023-08-15 9:16 ` Phillip Wood
1 sibling, 1 reply; 15+ messages in thread
From: Junio C Hamano @ 2023-08-09 20:57 UTC (permalink / raw)
To: Phillip Wood; +Cc: tboegi, git, friebetill
Phillip Wood <phillip.wood123@gmail.com> writes:
> Although this change is framed in terms of changes to "git stash push"
> I think the underlying issue and this patch actually affects all users
> of unpack_trees(). For example if "README" is untracked then
>
> git checkout <rev> README
>
> will currently fail if <rev>:README is a blob but will succeed and
> remove the untracked file if <rev>:README is a tree.
Very true, and with an .untracked file nobody asked Git to create,
presumably? I am not sure if the updated behaviour is better than
the current behaviour.
If "silent and unconditional removal" bothers us, I wonder if it is
a lot better approach to error out and have the user sort out the
mess, which is what we usually do when it gets tempting to "move it
away with an arbitrary rename" like this patch tries to do. I
dunno.
Thanks.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v1 1/1] git stash needing mkdir deletes untracked file
2023-08-09 18:47 ` Torsten Bögershausen
@ 2023-08-15 9:15 ` Phillip Wood
2023-08-15 15:25 ` Torsten Bögershausen
2023-08-15 18:03 ` Junio C Hamano
0 siblings, 2 replies; 15+ messages in thread
From: Phillip Wood @ 2023-08-15 9:15 UTC (permalink / raw)
To: Torsten Bögershausen, phillip.wood
Cc: git, friebetill, Junio C Hamano, Eric Sunshine
Hi Torsten
Sorry for the slow reply
On 09/08/2023 19:47, Torsten Bögershausen wrote:
> On Wed, Aug 09, 2023 at 02:15:28PM +0100, Phillip Wood wrote:
>> Hi Torsten
>>
>> Thanks for working on this. I've cc'd Junio for his unpack_trees()
>> knowledge.
>
> Thanks Eric for the review.
>
> Hej Phillip,
> I have been playing around with the whole thing some time.
> At the end I had a version, which did fiddle the information
> that we are doing a `git stash` (and not any other operation)
> into entry.c, and all test cases passed.
> So in principle I can dig out all changes, polish them
> and send them out, after doing cleanups of course.
I don't think we should be treating "git stash" as a special case here -
commands like "git checkout" should not be removing untracked files
unprompted either.
> (And that could take a couple of days, or weeks ;-)
>
> My main question is still open:
> Is it a good idea, to create a "helper file" ?
> The naming can be discussed, we may stick the date/time
> into the filename to make it really unique, or so.
I think stopping and telling the user that the file would be overwritten
as we do in other cases would be better.
> Reading the different reports and including own experience,
> I still think that a directory called ".deleted-by-user"
> or ".wastebin" or something in that style is a good idea.
I can see an argument for being able to opt-in to that for "git restore"
and "git reset --hard" but that is a different problem to the one here.
Best Wishes
Phillip
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v1 1/1] git stash needing mkdir deletes untracked file
2023-08-09 20:57 ` Junio C Hamano
@ 2023-08-15 9:16 ` Phillip Wood
0 siblings, 0 replies; 15+ messages in thread
From: Phillip Wood @ 2023-08-15 9:16 UTC (permalink / raw)
To: Junio C Hamano; +Cc: tboegi, git, friebetill
On 09/08/2023 21:57, Junio C Hamano wrote:
> Phillip Wood <phillip.wood123@gmail.com> writes:
>
> If "silent and unconditional removal" bothers us, I wonder if it is
> a lot better approach to error out and have the user sort out the
> mess, which is what we usually
Yes I think that would be a better approach.
Best Wishes
Phillip
> do when it gets tempting to "move it
> away with an arbitrary rename" like this patch tries to do. I
> dunno.
>
> Thanks.
>
>
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v1 1/1] git stash needing mkdir deletes untracked file
2023-08-15 9:15 ` Phillip Wood
@ 2023-08-15 15:25 ` Torsten Bögershausen
2023-08-15 18:03 ` Junio C Hamano
1 sibling, 0 replies; 15+ messages in thread
From: Torsten Bögershausen @ 2023-08-15 15:25 UTC (permalink / raw)
To: phillip.wood; +Cc: git, friebetill, Junio C Hamano, Eric Sunshine
On Tue, Aug 15, 2023 at 10:15:37AM +0100, Phillip Wood wrote:
> Hi Torsten
>
> Sorry for the slow reply
No problem.
Thanks for the response, I think that we have an
agreement not to overwrite an untracked file, when a directory
with the same name needs to be created.
I try to come up with a patch series -
starting with the stash operation.
>
> On 09/08/2023 19:47, Torsten Bögershausen wrote:
> > On Wed, Aug 09, 2023 at 02:15:28PM +0100, Phillip Wood wrote:
> > > Hi Torsten
> > >
> > > Thanks for working on this. I've cc'd Junio for his unpack_trees()
> > > knowledge.
> >
> > Thanks Eric for the review.
> >
> > Hej Phillip,
> > I have been playing around with the whole thing some time.
> > At the end I had a version, which did fiddle the information
> > that we are doing a `git stash` (and not any other operation)
> > into entry.c, and all test cases passed.
> > So in principle I can dig out all changes, polish them
> > and send them out, after doing cleanups of course.
>
> I don't think we should be treating "git stash" as a special case here -
> commands like "git checkout" should not be removing untracked files
> unprompted either.
>
> > (And that could take a couple of days, or weeks ;-)
> >
> > My main question is still open:
> > Is it a good idea, to create a "helper file" ?
> > The naming can be discussed, we may stick the date/time
> > into the filename to make it really unique, or so.
>
> I think stopping and telling the user that the file would be overwritten as
> we do in other cases would be better.
>
> > Reading the different reports and including own experience,
> > I still think that a directory called ".deleted-by-user"
> > or ".wastebin" or something in that style is a good idea.
>
> I can see an argument for being able to opt-in to that for "git restore" and
> "git reset --hard" but that is a different problem to the one here.
>
> Best Wishes
>
> Phillip
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v1 1/1] git stash needing mkdir deletes untracked file
2023-08-15 9:15 ` Phillip Wood
2023-08-15 15:25 ` Torsten Bögershausen
@ 2023-08-15 18:03 ` Junio C Hamano
1 sibling, 0 replies; 15+ messages in thread
From: Junio C Hamano @ 2023-08-15 18:03 UTC (permalink / raw)
To: Phillip Wood
Cc: Torsten Bögershausen, phillip.wood, git, friebetill,
Eric Sunshine
Phillip Wood <phillip.wood123@gmail.com> writes:
> I don't think we should be treating "git stash" as a special case here
> - commands like "git checkout" should not be removing untracked files
> unprompted either.
Yeah, I tend to agree. "git checkout branch path" should overwrite
a leftover "path" in the working tree in response to such an
explicit request, and that should equally apply for a request with
pathspec e.g. "git checkout branch .", as the latter is also an
explicit "please check out all paths out of the tree-ish of the
branch".
But "git checkout branch" in a working tree with untracked "path"
should not lose it if "branch" has it as a tracked file.
> I think stopping and telling the user that the file would be
> overwritten as we do in other cases would be better.
Yup, that is what we have done and probably one of the design
choices that made us successful.
>> Reading the different reports and including own experience,
>> I still think that a directory called ".deleted-by-user"
>> or ".wastebin" or something in that style is a good idea.
>
> I can see an argument for being able to opt-in to that for "git
> restore" and "git reset --hard" but that is a different problem to the
> one here.
Yeah, I tend to agree. If anything, such a trash directory should
be kept out-of-line, not inside the working tree. Perhaps in $HOME
or somewhere, and not necessarily tied to the use of Git, as the way
a file gets "deleted by user" is not necessarily limited to the use
of Git.
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2023-08-15 18:04 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-07-21 17:31 Lost files after git stash && git stash pop Till Friebe
2023-07-22 21:44 ` Torsten Bögershausen
2023-07-23 10:01 ` Phillip Wood
2023-07-23 20:52 ` Torsten Bögershausen
2023-07-24 9:59 ` Phillip Wood
2023-08-08 17:26 ` [PATCH v1 1/1] git stash needing mkdir deletes untracked file tboegi
2023-08-08 18:03 ` Torsten Bögershausen
2023-08-08 19:28 ` Eric Sunshine
2023-08-09 13:15 ` Phillip Wood
2023-08-09 18:47 ` Torsten Bögershausen
2023-08-15 9:15 ` Phillip Wood
2023-08-15 15:25 ` Torsten Bögershausen
2023-08-15 18:03 ` Junio C Hamano
2023-08-09 20:57 ` Junio C Hamano
2023-08-15 9:16 ` Phillip Wood
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).