Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logging what percent of file has been read #86

Open
dpmccabe opened this issue Sep 16, 2024 · 0 comments
Open

Logging what percent of file has been read #86

dpmccabe opened this issue Sep 16, 2024 · 0 comments

Comments

@dpmccabe
Copy link

dpmccabe commented Sep 16, 2024

Hi, I'm wondering if you know of a way to print a message during block reading containing the percent of the file that's been read so far. The first thing I tried was this:

total_size_bytes = os.path.getsize(vcf_path)
memoryview_batch_size = 2**32
read_bytes = 0

with open(vcf_path, "rb") as raw:
    with bgzip.BGZipReader(raw) as f:
        while True:
            if not (d := f.read(memoryview_batch_size)):
                break

            read_bytes += len(d)
            logging.info(
                "Read {0} bytes ({1:.0%})".format(
                    read_bytes, read_bytes / total_size_bytes
                )
            )

This isn't right, though: read_bytes contains the size of the uncompressed content so my log message eventually starts printing progress beyond 100%. I've looked through the code here and the methods available to the memoryview object d but haven't found any way to determine in this loop how close each iteration is to completion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant