Skip to content

feat: hide .jj directory on Windows #6514

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

Natural-selection1
Copy link

Closes: #6513

Checklist

If applicable:

  • I have updated CHANGELOG.md
  • I have updated the documentation (README.md, docs/, demos/)
  • I have updated the config schema (cli/src/config-schema.json)
  • I have added tests to cover my changes

This is my first attempt to contribute to the jj project. I have read https://jj-vcs.github.io/jj/prerelease/contributing/, but it's still unclear, which caused me to encounter many issues (and I'm not sure how to resolve them) while trying to use jj to manage this contribution. If there's anything I've overlooked, any suggestions are welcome.

On macOS and Linux, directories starting with a dot (.) are hidden by default.
However, on Windows, we need to set additional attributes to achieve the same behavior.
To maintain consistency across platforms, we now hide the .jj directory on Windows as well.

Note: This change requires using unsafe code to call Windows FFI functions.
Therefore, we had to relax the unsafe code restrictions in lib/src/lib.rs.

Closes: jj-vcs#6513
@Natural-selection1 Natural-selection1 requested a review from a team as a code owner May 12, 2025 00:29
Copy link

google-cla bot commented May 12, 2025

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@martinvonz
Copy link
Member

Sounds reasonable, but do you know if Git, Mercurial, or other tools do this? Otherwise I'd be worried that this behavior will surprise users.

@Natural-selection1
Copy link
Author

Natural-selection1 commented May 12, 2025

Sounds reasonable, but do you know if Git, Mercurial, or other tools do this? Otherwise I'd be worried that this behavior will surprise users.

I used to be a Windows git user(as for other tools, I have not used). For the .git directory in Windows, it is hidden, this is the reason why I discovered this problem

image

@martinvonz
Copy link
Member

Looks like it's controlled by this Git config. (I don't think we need a config for it.)

@Natural-selection1
Copy link
Author

Git seems to have many 'quite peculiar' settings. For me, apart from:

  • CRLF/LF
  • username, email
  • SSH
    I don't think I've really encountered any others, haha.
    Regarding the issue with the current PR, I think it would be best to maintain consistency with Git's default behavior.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: please squash this into the parent commit

Copy link
Contributor

@yuja yuja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for reference, Mercurial doesn't do, and there were some push back against the feature.
https://lists.mercurial-scm.org/pipermail/mercurial-devel/2014-March/205421.html

@yuja
Copy link
Contributor

yuja commented May 12, 2025

Can you also check if the .git directory created by jj git init/clone --colocate is hidden on Windows? I couldn't find any relevant code in gitoxide. It would be odd if only .jj directory were hidden.

lib/src/lib.rs Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: per contributing guidelines you can squash the third commit into this and probably should use workspace: <title> as the topic, since feat: just means feature which isn't something used by the project.

@Natural-selection1
Copy link
Author

Can you also check if the .git directory created by jj git init/clone --colocate is hidden on Windows? I couldn't find any relevant code in gitoxide. It would be odd if only .jj directory were hidden.

Yes, your concern is valid - there is indeed an issue where the .git directory doesn't get hidden. Now I'm thinking of moving the timing of hiding the folder from when the folder is created to near the end of the command execution.

This would be implemented around lines 108-109 in this code:

do_init(
ui,
command,
&wc_path,
args.colocate,
args.git_repo.as_deref(),
)?;
let relative_wc_path = file_util::relative_path(cwd, &wc_path);
writeln!(
ui.status(),
r#"Initialized repo in "{}""#,
relative_wc_path.display()
)?;
Ok(())

And also around line 147 in this code:

if clone_result.is_err() {
let clean_up_dirs = || -> io::Result<()> {
fs::remove_dir_all(canonical_wc_path.join(".jj"))?;
if args.colocate {
fs::remove_dir_all(canonical_wc_path.join(".git"))?;
}
if !wc_path_existed {
fs::remove_dir(&canonical_wc_path)?;
}
Ok(())
};
if let Err(err) = clean_up_dirs() {
writeln!(
ui.warning_default(),
"Failed to clean up {}: {}",
canonical_wc_path.display(),
err
)
.ok();
}
}
let (mut workspace_command, default_branch) = clone_result?;
if let Some(name) = &default_branch {

I'm not very familiar with jj yet - are there any other possible scenarios/cases I should be aware of?

@martinvonz
Copy link
Member

We don't want to make .git when running e.g. jj git init --git-repo=<some existing git repo>, so I think we should instead do at the lower level here:

jj/lib/src/git_backend.rs

Lines 233 to 238 in 9d1f799

let git_repo = gix::ThreadSafeRepository::init_opts(
canonical_workspace_root,
gix::create::Kind::WithWorktree,
gix::create::Options::default(),
gix_open_opts_from_settings(settings),
)

That should cover both the jj git init and jj git clone callers.

@yuja
Copy link
Contributor

yuja commented May 13, 2025

I think ".git" directory should be (ideally) handled by gitoxide, and would have to respect the Git configuration.

An easier (but adhoc) option might be to copy the hidden attribute from colocated .git directory by abusing std::fs::set_permissions(), which calls SetFileAttributesW() under the hood. It means .jj directory would be hidden only when .git exists and was hidden.

https://doc.rust-lang.org/stable/std/fs/fn.set_permissions.html

@Natural-selection1
Copy link
Author

Natural-selection1 commented May 13, 2025

I think the ".git" directory should (ideally) be handled by gitoxide and should respect the Git configuration.

An easier (but ad-hoc) option might be to copy the hidden attribute from a colocated .git directory by abusing std::fs::set_permissions(), which calls SetFileAttributesW() under the hood. This means the .jj directory would only be hidden if .git exists and is hidden.

doc.rust-lang.org/stable/std/fs/fn.set_permissions.html


I don’t know much about gitoxide, but based on my search, it's a Rust implementation of Git. If possible, could you provide more details? :)

https://git-scm.com/docs/git-config#Documentation/git-config.txt-corehideDotFiles
On Windows, Git always hides .git. I’m not sure how gitoxide handles the visibility of .git, but for Git users, .git is always hidden. Therefore, I think directly setting .git as hidden wouldn’t conflict with Git users' expectations.

Based on the current discussion, I’ve extracted the logic for hiding .jj and .git into the following function:

#[cfg(windows)]  
pub fn hide_dotjj_and_dotgit(workspace_root: &Path) {  
    use std::ffi::CString;  
    use winapi::um::fileapi::SetFileAttributesA;  
    use winapi::um::winnt::FILE_ATTRIBUTE_HIDDEN;  
    let jj_dir = workspace_root.join(".jj");  
    let git_dir = workspace_root.join(".git");  
    let c_jj_dir = CString::new(jj_dir.as_os_str().as_encoded_bytes()).unwrap();  
    let c_git_dir = CString::new(git_dir.as_os_str().as_encoded_bytes()).unwrap();  
    #[allow(unsafe_code)]  
    unsafe {  
        if jj_dir.exists() {  
            let hide_jj_dir = SetFileAttributesA(c_jj_dir.as_ptr(), FILE_ATTRIBUTE_HIDDEN);  
            if hide_jj_dir == 0 {  
                println!("Failed to hide .jj directory");  
            }  
        }  
        if git_dir.exists() {  
            let hide_git_dir = SetFileAttributesA(c_git_dir.as_ptr(), FILE_ATTRIBUTE_HIDDEN);  
            if hide_git_dir == 0 {  
                println!("Failed to hide .git directory");  
            }  
        }  
    }  
}  

It is applied at line 121 in the following function to cover jj git init:

jj/lib/src/workspace.rs

Lines 118 to 127 in cb877a9

fn create_jj_dir(workspace_root: &Path) -> Result<PathBuf, WorkspaceInitError> {
let jj_dir = workspace_root.join(".jj");
match std::fs::create_dir(&jj_dir).context(&jj_dir) {
Ok(()) => Ok(jj_dir),
Err(ref e) if e.error.kind() == io::ErrorKind::AlreadyExists => {
Err(WorkspaceInitError::DestinationExists(jj_dir))
}
Err(e) => Err(e.into()),
}
}

And at lines 244-245 in the following function to cover jj git init --colocate and jj git clone --colocate:

jj/lib/src/git_backend.rs

Lines 222 to 245 in cb877a9

pub fn init_colocated(
settings: &UserSettings,
store_path: &Path,
workspace_root: &Path,
) -> Result<Self, Box<GitBackendInitError>> {
let canonical_workspace_root = {
let path = store_path.join(workspace_root);
dunce::canonicalize(&path)
.context(&path)
.map_err(GitBackendInitError::Path)?
};
let git_repo = gix::ThreadSafeRepository::init_opts(
canonical_workspace_root,
gix::create::Kind::WithWorktree,
gix::create::Options::default(),
gix_open_opts_from_settings(settings),
)
.map_err(GitBackendInitError::InitRepository)?;
let git_repo_path = workspace_root.join(".git");
let git_settings = settings
.git_settings()
.map_err(GitBackendInitError::Config)?;
Self::init_with_repo(store_path, &git_repo_path, git_repo, git_settings)
}

Any further thoughts?

@yuja
Copy link
Contributor

yuja commented May 13, 2025

I don’t know much about gitoxide, but based on my search, it's a Rust implementation of Git. If possible, could you provide more details? :)

https://github.com/search?q=repo%3AGitoxideLabs%2Fgitoxide%20hidedotfiles&type=code

Appears that they have a plan.

https://git-scm.com/docs/git-config#Documentation/git-config.txt-corehideDotFiles On Windows, Git always hides .git. I’m not sure how gitoxide handles the visibility of .git, but for Git users, .git is always hidden. Therefore, I think directly setting .git as hidden wouldn’t conflict with Git users' expectations.

It's unclear from the doc, but Git supports hideDotFiles=false|true|dotGitOnly. If false, .git wouldn't be hidden. (I don't have Windows machine, so the actual behavior isn't tested.)

@martinvonz
Copy link
Member

@Natural-selection1: Could you sign the Google CLA? We won't be able to accept your PR without it.

@Natural-selection1
Copy link
Author

@Natural-selection1: Could you sign the Google CLA? We won't be able to accept your PR without it.

I signed it before, but forgot to rescan as required. Sorry about that. : )

@Natural-selection1
Copy link
Author

Natural-selection1 commented May 13, 2025

I think ".git" directory should be (ideally) handled by gitoxide, and would have to respect the Git configuration.

Respecting Git settings seems unlikely, as many tools bundle their own Git. Here's a demonstration of cargo's built-in Git behavior:
image


However, after configuration, cargo new doesn't appear to comply:

image


Perhaps only respecting pre-existing .git directories would be practical. The basic logic would be:

  1. Check if the time difference between the .git directory's creation and the current moment is within 1 second
  2. If no, then .jj should match its state
  3. If yes (meaning we just created it), then both should be hidden

I wonder if this approach would be convincing?

@yuja
Copy link
Contributor

yuja commented May 13, 2025

Respecting Git settings seems unlikely, as many tools bundle their own Git.

Well, I assume gitoxide will respect Git settings when they implement the feature. And it's nice if we don't have to "fix" the .git directory attribute created by gitoxide by ourselves.

I don't know whether the .jj directory should be made hidden/visible consistently with the colocated .git directory.

  1. The easiest option is to do nothing about .jj directory (= the current behavior.)
  2. Another easy (but a bit weird depending on user's expectation) option I think is to just copy file attribute from the colocated .git directory (which would require no unsafe code, but I haven't tested.)
  3. Lastly, we can make .jj directory hidden unconditionally (= your implementation, I think.)

a demonstration of cargo's built-in Git behavior:

Maybe cargo doesn't because they still use git2/libgit2?

@Natural-selection1
Copy link
Author

  1. The easiest option is to do nothing about the .jj directory (= the current behavior).

Remember the title of our PR? It's to ensure .xx remains hidden in Windows just like it does in Mac or Linux.


  1. Another easy (but a bit weird depending on user's expectation) option I think is to just copy file attribute from the colocated .git directory (which would require no unsafe code, but I haven't tested.)
  2. Lastly, we can make .jj directory hidden unconditionally (= your implementation, I think.)

Perhaps you could take a look at my current implementation (please ignore the rough eprintln for now). You can start reading from around match dotgit_path.exists():

#[cfg(windows)]
pub fn set_dotgit_and_dotjj_visibility(workspace_root: &Path) {
    use std::ffi::CString;
    use std::time::Duration;
    use std::time::SystemTime;
    use winapi::um::fileapi::GetFileAttributesA;
    use winapi::um::fileapi::SetFileAttributesA;
    use winapi::um::winnt::FILE_ATTRIBUTE_HIDDEN;

    let dotjj_path = workspace_root.join(".jj");
    let dotgit_path = workspace_root.join(".git");
    let c_dotgit_path = CString::new(dotgit_path.as_os_str().as_encoded_bytes()).unwrap();
    fn hide_folder(path: &Path) {
        let dir_name = Path::new(path).file_name().unwrap().to_str().unwrap();
        let c_path = CString::new(path.as_os_str().as_encoded_bytes()).unwrap();
        #[allow(unsafe_code)]
        unsafe {
            if SetFileAttributesA(c_path.as_ptr(), FILE_ATTRIBUTE_HIDDEN) == 0 {
                eprintln!("Failed to hide {dir_name}");
            }
        }
    }
    fn get_folder_age(path: &Path) -> Result<Duration, io::Error> {
        SystemTime::now()
            .duration_since(fs::metadata(path)?.created()?)
            .map_err(|e| std::io::Error::new(std::io::ErrorKind::Other, e))
    }

    match dotgit_path.exists() {
        // cover `jj git init`
        // because there is no `.git` created by us just now or exists before,
        // so we hide `.jj`, just like dot_folder on Mac or Linux
        false => {
            hide_folder(&dotjj_path);
        }
        // cover `jj git init --colocate` and `jj git clone [Source] --colocate`
        true => {
            // check if it's created by us just now or is old
            let is_dotgit_exist_before = match get_folder_age(&dotgit_path) {
                Ok(duration) => duration > Duration::from_secs(1),
                Err(_) => true,
            };

            if is_dotgit_exist_before {
                // if `.git` exists before, keep `.jj` the same visibility as `.git`
                #[allow(unsafe_code)]
                unsafe {
                    if GetFileAttributesA(c_dotgit_path.as_ptr()) & FILE_ATTRIBUTE_HIDDEN != 0 {
                        hide_folder(&dotjj_path);
                    }
                }
            } else {
                // if `.git` is newly created, hide `.jj` and `.git` just like dot_folder on Mac or Linux
                hide_folder(&dotjj_path);
                hide_folder(&dotgit_path);
            }
        }
    }
}

I think the above approach might be the best solution without adding user configuration options.


  1. Another easy (but a bit weird depending on user's expectation) option I think is to just copy file attribute from the colocated .git directory (which would require no unsafe code, but I haven't tested.)

I don't think we need to be overly cautious about unsafe here - we're just calling FFI, not playing with raw pointers.


Maybe cargo doesn't because they still use git2/libgit2?

I haven't found relevant doc, but judging from cargo help new, it seems they don't even provide visibility options.

@yuja
Copy link
Contributor

yuja commented May 13, 2025

  1. The easiest option is to do nothing about the .jj directory (= the current behavior).

Remember the title of our PR? It's to ensure .xx remains hidden in Windows just like it does in Mac or Linux.

I know, but git doesn't apply this rule to e.g. .gitignore despite any dot files are "hidden" on Unix, so there may be conflicting requirements.

(Just for reference, Mercurial doesn't make .hg directory hidden. Git does for some reasons, though.)

  1. Another easy (but a bit weird depending on user's expectation) option I think is to just copy file attribute from the colocated .git directory (which would require no unsafe code, but I haven't tested.)
  2. Lastly, we can make .jj directory hidden unconditionally (= your implementation, I think.)

Perhaps you could take a look at my current implementation (please ignore the rough eprintln for now). You can start reading from around match dotgit_path.exists():

I believe we shouldn't do timestamp-based heuristics. (I suggested (2) just because it's easier to implement.)

To make both .git and .jj directories hidden when the .git directory was also created by jj, insert a relevant code to GitBackend::init_colocated() and its caller. (To be clear, I don't think we should do this for .git directory, and instead send PR/FR to gitoxide.)

BTW, you should use SetFileAttributeW instead of A. A Rust String is UTF-8, and OsString is platform-dependent bytes (iirc, it's less-strict UTF-8 on Windows), whereas the A (or ANSI) Windows API expects a string of legacy encoding (such as ISO-8859-1) depending on the configured locale. I don't know if there are better/safer Windows API wrapper.

@yuja
Copy link
Contributor

yuja commented May 14, 2025

you should use SetFileAttributeW instead of A.

Appears that Microsoft's windows-rs implements automatic conversion to the native 16-bit string type. The API is still unsafe, though.

https://microsoft.github.io/windows-docs-rs/doc/windows/Win32/Storage/FileSystem/fn.SetFileAttributesW.html

@thoughtpolice might have an opinion about the dependency.

@Natural-selection1
Copy link
Author

I've pushed a new version. Please ignore the suboptimal git commit messages for now (I'm currently having some difficulties using jj to manage the GitHub repository). If we agree this PR is mergeable, I'll reorganize the commits properly in Git : )

@Natural-selection1
Copy link
Author

Appears that Microsoft's windows-rs implements automatic conversion to the native 16-bit string type. The API is still unsafe, though.

Oh, I noticed https://crates.io/crates/windows when I was looking up information about SetFileAttributeW. winapi hasn't been updated in six years! I'll try to update the dependency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

FR: By default, hide .jj in windows
4 participants