Skip to content

Commit 3c99335

Browse files
LawnGnomeLawnGnome
committed
Initial git-cvs-fast-import release.
0 parents  commit 3c99335

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

89 files changed

+4709
-0
lines changed

‎.dockerignore

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
**/target
2+
/target
3+
4+
**/examples
5+
examples
6+
7+
**/dist
8+
dist
9+
10+
*.md
11+
12+
Dockerfile
13+
.dockerignore
14+
.gitignore
15+
.git
16+
.goreleaser
17+
.goreleaser.yml
18+
LICENSE
19+
renovate.json

‎.github/workflows/goreleaser.yaml

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
name: goreleaser
2+
3+
on:
4+
push:
5+
tags:
6+
- 'v[0-9]*'
7+
8+
permissions:
9+
contents: write
10+
11+
jobs:
12+
goreleaser:
13+
runs-on: ubuntu-latest
14+
steps:
15+
- name: Checkout
16+
uses: actions/checkout@v2
17+
with:
18+
fetch-depth: 0
19+
20+
# Unsurprisingly, we need Go to run goreleaser.
21+
- name: Set up Go
22+
uses: actions/setup-go@v2
23+
with:
24+
go-version: 1.17
25+
26+
# We need a Rust toolchain, but specifically only the
27+
# x86_64-unknown-linux-musl target.
28+
- name: Set up Rust
29+
uses: actions-rs/toolchain@v1
30+
with:
31+
toolchain: 1.55.0
32+
target: x86_64-unknown-linux-musl
33+
override: true
34+
35+
- name: Run GoReleaser
36+
uses: goreleaser/goreleaser-action@v2
37+
with:
38+
distribution: goreleaser
39+
version: latest
40+
args: release --rm-dist
41+
env:
42+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

‎.github/workflows/rust.yaml

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
on: [push, pull_request]
2+
3+
name: Rust CI
4+
5+
jobs:
6+
test:
7+
name: Test Suite
8+
runs-on: ubuntu-latest
9+
steps:
10+
- uses: actions/checkout@v2
11+
- uses: actions-rs/toolchain@v1
12+
with:
13+
profile: minimal
14+
toolchain: stable
15+
override: true
16+
- uses: actions-rs/cargo@v1
17+
with:
18+
command: test
19+
args: --all
20+
21+
fmt:
22+
name: Rustfmt
23+
runs-on: ubuntu-latest
24+
steps:
25+
- uses: actions/checkout@v2
26+
- uses: actions-rs/toolchain@v1
27+
with:
28+
profile: minimal
29+
toolchain: stable
30+
override: true
31+
- run: rustup component add rustfmt
32+
- uses: actions-rs/cargo@v1
33+
with:
34+
command: fmt
35+
args: --all -- --check
36+
37+
clippy:
38+
name: Clippy
39+
runs-on: ubuntu-latest
40+
steps:
41+
- uses: actions/checkout@v2
42+
- uses: actions-rs/toolchain@v1
43+
with:
44+
profile: minimal
45+
toolchain: stable
46+
override: true
47+
- run: rustup component add clippy
48+
- uses: actions-rs/cargo@v1
49+
with:
50+
command: clippy
51+
args: -- -D warnings

‎.gitignore

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
**/target
2+
**/.vscode
3+
Cargo.lock
4+
/dist
5+
6+
7+
# Added by cargo
8+
9+
/target
10+
.gdb_history
11+
12+
dist/

‎.goreleaser.yml

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# This approach is adapted from
2+
# https://jondot.medium.com/shipping-rust-binaries-with-goreleaser-d5aa42a46be0
3+
#
4+
# Basically, we want to use goreleaser to handle packaging and releasing
5+
# git-cvs-fast-import binaries, since it already has all the functionality
6+
# required to build the packages and create the changelog. To do that, we
7+
# provide a dummy main package that goreleaser can happily build into a no-op
8+
# binary, and then overwrite the binary with one that we build normally using
9+
# cargo.
10+
11+
project_name: git-cvs-fast-import
12+
builds:
13+
- main: .goreleaser/dummy.go
14+
goarch:
15+
- amd64
16+
goos:
17+
- linux
18+
binary: git-cvs-fast-import
19+
hooks:
20+
post: sh .goreleaser/post.sh
21+
nfpms:
22+
- package_name: git-cvs-fast-import
23+
vendor: Sourcegraph
24+
homepage: https://github.com/sourcegraph/git-cvs-fast-import
25+
maintainer: Batch Changes <[email protected]>
26+
description: A tool to import CVS repositories into Git for analysis
27+
license: Proprietary
28+
formats:
29+
- rpm
30+
- deb
31+
dependencies:
32+
- git
33+
bindir: /usr/bin
34+
checksum:
35+
name_template: 'checksums.txt'
36+
snapshot:
37+
name_template: "{{ incpatch .Version }}-next"
38+
changelog:
39+
sort: asc

‎.goreleaser/dummy.go

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
package main
2+
3+
func main() {}

‎.goreleaser/post.sh

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
#!/bin/sh
2+
3+
# This script is used by goreleaser, and shouldn't be invoked otherwise.
4+
5+
set -e
6+
set -x
7+
8+
# Perform a completely static build using musl to avoid glibc versioning issues
9+
# on older distros. This requires the x86_64-unknown-linux-musl Rust target to
10+
# be available.
11+
RUSTFLAGS='-C link-arg=-s' cargo build --release --target x86_64-unknown-linux-musl
12+
13+
# Overwrite the dummy Go binary.
14+
cp target/x86_64-unknown-linux-musl/release/git-cvs-fast-import dist/git-cvs-fast-import_linux_amd64/git-cvs-fast-import

‎Cargo.toml

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
[package]
2+
name = "git-cvs-fast-import"
3+
version = "0.1.0"
4+
edition = "2018"
5+
6+
[workspace]
7+
members = [ "comma-v", "eq-macro", "git-fast-import", "internal/process", "internal/state", "patchset", "rcs-ed" ]
8+
9+
[dev-dependencies]
10+
tokio-test = "0.4.2"
11+
12+
[dependencies]
13+
anyhow = "1.0.44"
14+
comma-v = { path = "comma-v" }
15+
flexi_logger = { version = "0.19.4", features = ["async", "colors"] }
16+
flume = "0.10.9"
17+
git-cvs-fast-import-process = { path = "internal/process" }
18+
git-cvs-fast-import-state = { path = "internal/state" }
19+
git-fast-import = { path = "git-fast-import" }
20+
log = "0.4.14"
21+
num_cpus = "1.13.0"
22+
parse_duration = "2.1.1"
23+
patchset = { path = "patchset" }
24+
rcs-ed = { path = "rcs-ed" }
25+
structopt = "0.3.23"
26+
tempfile = "3.2.0"
27+
thiserror = "1.0.29"
28+
tokio = { version = "1.12.0", features = ["io-util", "macros", "process", "rt-multi-thread", "signal", "sync", "time", "fs"] }
29+
walkdir = "2.3.2"
30+
31+
[features]

‎DEVELOPMENT.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Development
2+
3+
`git-cvs-fast-import` is a fairly standard Rust binary that uses several packages within its workspace to provide functionality. We intend to publish the packages that make sense to publish to crates.io in the future.
4+
5+
## Package structure
6+
7+
### Generic packages
8+
9+
* `comma-v`: a parser for the RCS `,v` format used by CVS.
10+
* `git-fast-import`: a client for the [`git fast-import`](https://git-scm.com/docs/git-fast-import) streaming format.
11+
* `rcs-ed`: an implementation of the subset of [`ed`](https://linux.die.net/man/1/ed) commands [used by RCS](https://www.gnu.org/software/diffutils/manual/html_node/RCS.html).
12+
13+
### Helper packages
14+
15+
* `eq-macro`: a proc macro to derive `PartialEq<[u8]>`, used internally by `comma-v`.
16+
17+
### `git-cvs-fast-import` specific packages
18+
19+
* `src`: contains the source for the `git-cvs-fast-import` binary itself, which intentionally doesn't do very much and mostly delegates to other packages.
20+
* `internal/process`: process management for `git fast-import`.
21+
* `internal/state`: state management and persistence.
22+
23+
## Releasing
24+
25+
[GoReleaser](https://github.com/goreleaser/goreleaser) is used for package building and GitHub changelog generation. A mild amount of hackery is required to make it work with a Rust program, see [`.goreleaser.yml`](.goreleaser.yml) for the gory details.
26+
27+
For maximum compatibility, binaries are built as static Linux binaries using the `x86_64-unknown-linux-musl` Rust target. Note that this means that non-Rust dependencies can only be added if they can easily be statically linked. (In practice, this hasn't been a problem thus far.)

‎Dockerfile

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
FROM rust:1.55.0-alpine@sha256:1c01fb410179b21f809ef935fd66277e964b5a8ad20431ad49b1c52b5778fd34 AS builder
2+
3+
RUN apk add --update alpine-sdk
4+
COPY --chown=nobody:nobody . /src/
5+
WORKDIR /src
6+
USER nobody:nobody
7+
RUN cargo build --release
8+
9+
FROM alpine:3.14@sha256:e1c082e3d3c45cccac829840a25941e679c25d438cc8412c2fa221cf1a824e6a
10+
11+
COPY --from=builder /src/target/release/git-cvs-fast-import /usr/local/bin/git-cvs-fast-import
12+
ENTRYPOINT ["/usr/local/bin/git-cvs-fast-import"]

‎LICENSE

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
The Sourcegraph Enterprise license (the “Enterprise License”)
2+
Copyright (c) 2021 Sourcegraph Inc.
3+
4+
With regard to the Sourcegraph Software:
5+
6+
This software and associated documentation files (the "Software") may only be
7+
used in production, if you (and any entity that you represent) have agreed to,
8+
and are in compliance with, the Sourcegraph Terms of Service, available
9+
at https://about.sourcegraph.com/terms (the “Enterprise Terms”), or other
10+
agreement governing the use of the Software, as agreed by you and Sourcegraph,
11+
and otherwise have a valid Sourcegraph Enterprise subscription for the
12+
correct number of user seats. Subject to the foregoing sentence, you are free to
13+
modify this Software and publish patches to the Software. You agree that Sourcegraph
14+
and/or its licensors (as applicable) retain all right, title and interest in and
15+
to all such modifications and/or patches, and all such modifications and/or
16+
patches may only be used, copied, modified, displayed, distributed, or otherwise
17+
exploited with a valid Sourcegraph Enterprise subscription for the correct
18+
number of user seats. Notwithstanding the foregoing, you may copy and modify
19+
the Software for development and testing purposes, without requiring a
20+
subscription. You agree that Sourcegraph and/or its licensors (as applicable) retain
21+
all right, title and interest in and to all such modifications. You are not
22+
granted any other rights beyond what is expressly stated herein. Subject to the
23+
foregoing, it is forbidden to copy, merge, publish, distribute, sublicense,
24+
and/or sell the Software.
25+
26+
The full text of this Enterprise License shall be included in all copies or
27+
substantial portions of the Software.
28+
29+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
30+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
31+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
32+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
33+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
34+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
35+
SOFTWARE.
36+
37+
For all third party components incorporated into the Sourcegraph Software, those
38+
components are licensed under the original license provided by the owner of the
39+
applicable component.

‎README.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# git-cvs-fast-import
2+
3+
`git-cvs-fast-import` provides a restartable, performant Git importer for CVS repositories, focused on providing continuously updated mirrors of CVS repositories so that they can be analysed using existing tools that can only handle Git natively, such as [Sourcegraph](https://sourcegraph.com).
4+
5+
To support continuous updates, `git-cvs-fast-import` makes some tradeoffs compared to other tools that convert CVS repositories into Git: most notably, considerably less effort is made to preserve precise history, especially on branches and tags. This is not intended to be a general purpose, one time converter when migrating from a CVS setup to Git: see [the comparison to other tools](#comparison-to-other-tools) for suggestions on what to use in that case.
6+
7+
## Installation
8+
9+
Linux binaries are provided on [the releases page](https://github.com/sourcegraph/git-cvs-fast-import/releases), including RPM and DEB packages for RHEL/CentOS and Debian/Ubuntu installs, respectively. These binaries have been tested back to CentOS 7 and Ubuntu 16.04.
10+
11+
You will also need `git` installed, as `git-cvs-fast-import` uses the [`git fast-import`](https://git-scm.com/docs/git-fast-import) command internally when operating. Any version released in the last decade should be sufficient.
12+
13+
## Usage
14+
15+
You will need access to the `CVSROOT` of the CVS repository you wish to import, as `git-cvs-fast-import` parses the RCS files in the root to import the history of each file. In practice, this means you should expect to see a tree of files ending in `,v`.
16+
17+
You will also need a valid Git repository. This means that you need to `git init` your target repository before running `git-cvs-fast-import` for the first time.
18+
19+
Full help is available through `git-cvs-fast-import --help`, but for most uses, you only need to provide the CVSROOT, Git repository, metadata store, and (optionally) the CVS directories to be imported. For example, to import the `project` and `src` directories from a CVS repository at `/cvs`, and write to a Git repository at `/git`, and store the metadata at `/tmp/import.db`, you would run the following:
20+
21+
```sh
22+
git-cvs-fast-import -c /cvs -g /git -s /tmp/import.db project src
23+
```
24+
25+
## Comparison to other tools
26+
27+
We know of three other tools that allow for CVS-to-Git conversion:
28+
29+
* [`git-cvsimport`](https://git-scm.com/docs/git-cvsimport): this ships with Git, and has support for incremental updates.
30+
* [`cvs-fast-export`](https://gitlab.com/esr/cvs-fast-export): this is a standalone tool that parses CVS repositories and exports data in the `git fast-import` stream format, but does not support incremental updates.
31+
* `cvs2git` is referenced in the `git-cvsimport` manpage, but no longer appears to have a home page.
32+
33+
We would suggest trying `cvs-fast-export` first for one-time conversions where the CVS repository will not be used thereafter, and then falling back to `git-cvsimport` if `cvs-fast-export` fails (which can happen with complex Git histories).
34+
35+
## Known issues
36+
37+
* Importing branches other than `HEAD` is currently unsupported, but is intended to be added in the medium term.
38+
* Tag history can be misleading: CVS tags are applied on a per-file basis, whereas Git tags are per-repository. As a result, `git-cvs-fast-import` makes a fake commit for each tag: this ensures that the actual content of the tag is correct, but may be misleading in terms of the history of the tag if the same CVS tag was applied to different files at different times, as commits may appear in the Git log that weren't logically part of the CVS history for a specific file.
39+
40+
## Development
41+
42+
Please refer to [`DEVELOPMENT.md`](DEVELOPMENT.md) for more detail on how this tool is structured, why some choices were made, and how to contribute.

‎comma-v/Cargo.toml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
[package]
2+
name = "comma-v"
3+
version = "0.1.0"
4+
edition = "2018"
5+
6+
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
7+
8+
[dependencies]
9+
chrono = "0.4.19"
10+
derive_more = "0.99.16"
11+
eq-macro = { path = "../eq-macro" }
12+
nom = "7.0.0"
13+
thiserror = "1.0.29"
14+
15+
[dev-dependencies]
16+
anyhow = "1.0.44"
17+
structopt = "0.3.23"

0 commit comments

Comments
 (0)