Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

llr calculation error #4

Open
ivllm opened this issue Nov 15, 2020 · 3 comments
Open

llr calculation error #4

ivllm opened this issue Nov 15, 2020 · 3 comments

Comments

@ivllm
Copy link

ivllm commented Nov 15, 2020

Thank you for making a useful library.
During the experiment, I found an error in the llr calculation of composite funtion.
I attached samples.
https://drive.google.com/drive/folders/1FqArlwPXnLHQ53Kn99oP24IS7XoaBCNl?usp=sharing

238.wav is well calculated by CSIG, CBAK, COVL
However, CSIG and COVL are not properly calculated(outputs are 1.0) for the 101.wav
As a result of the analysis, the llr_mean of the composite function had an 'inf' value.

So I made some corrections by comparing with matlab composite function.

I modified line181 in qualityMeasures.py as below.
clean_speech_framed=extract_overlapped_windows(clean_speech+eps,winlength,winlength-skiprate,hannWin)
processed_speech_framed=extract_overlapped_windows(processed_speech+eps,winlength,winlength-skiprate,hannWin)

Still the problem has not been solved the issue and lines 151-154 have been modified as follows.
if np.abs(E[i]) < eps:
rcoeff[i]=(R[i+1] - sum_term) / (E[i])
else:
rcoeff[i]=(R[i+1] - sum_term) / (E[i])

Then, We have a value that is almost similar to the result obtained through matlab, but it still does not exactly match (to the second decimal place).

Since my modifications have not been verified, I think further modifications will be necessary.

@schmiph2
Copy link
Owner

Hi ivllm

Thank you for testing the library and fixing the bug. What is the difference between the two statements in lines 151-154? I guess you forgot to add your bugfix in the comment? Did you do something like this?

if np.abs(E[i]) < eps:
rcoeff[i]=(R[i+1] - sum_term) / (E[i]+eps)
else:
rcoeff[i]=(R[i+1] - sum_term) / (E[i])

I will fix it as soon as possible and test it with your files (all metrics). Thanks again for reporting the bug!

@ivllm
Copy link
Author

ivllm commented Nov 17, 2020

Thanks for your reply.

I didn't forget bugfix. I'll explain in more detail for your debugging.

The original code line 151 to 154


if np.abs(E[i]) < eps:
rcoeff[i]=np.inf
else:
rcoeff[i]=(R[i+1] - sum_term) / (E[i])


When code is executed using 101.wav, denominators in line 195 has many 'nan' values.
This problem first occurs in line 187 for loop, when numFrames=186.
Then, the nan values of frac become 'inf' by line196.
The 'inf' remains at the end of the distortion array of Line 202, and llr_mean have the 'inf' value.
As a result, the outputs of CISG and COVL are 1.0.

The 'nan' value of the denominator is caused by the 'nan' value of A_clean(calculated by def lpcoff).
I couldn't analyze the lpcoff in more detail, but I thought there was a problem in line152, so I just fixed the line 152 as below.

fixed code line 151 to 154


if np.abs(E[i]) < eps:
rcoeff[i]=(R[i+1] - sum_term) / (E[i]) # Same with else condition
else:
rcoeff[i]=(R[i+1] - sum_term) / (E[i])


I hope my explanation will help you debug.
Because of your library, I have reduced sample measurement time a lot. Thank you.

@schmiph2
Copy link
Owner

Thank you, I see it now. I will change it asap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants