Update certificate dirs and file names on FreeBSD

[michael-o]

FreeBSD contains a canonical certstore managed by certctl(8) since 12.2 located in the base system (/etc/ssl), search there first. Alternatively, a user can populate a custom store in distbase (/usr/local/etc/ssl) with certctl(8) which shall be queried if the former does not exist. At last, there is a store for OpenSSL from the ports (/usr/local/openssl) outside of certctl(8)'s reach. Within these there can be also a bundle in parallel to a hashed directory.

This fixes #20 and fixes #37

FreeBSD port maintainer here.

Improvement on top of @djc's work: certctl(8)'s introduction on FreeBSD 12.2 make life much easier. CERTIFICATE_DIRS uses those two from it. Additionally from security/openssl*. /usr/local/share/certs is a source for certlctl(8) which ends up in either of them, thus would lead to duplicates. Extended CERTIFICATE_FILE_NAMES by those files created by certctl(8) from FreeBSD 15+ or ultimately for those who still install security/ca_root_nss which is included in /usr/share/certs/trusted anyway.

Sample code:

use openssl_probe::probe;
use curl::easy::Easy;

fn main() {
    let result = probe();

    match result.cert_file {
        Some(ref path) => println!("result.cert_file: {}", path.display()),
        None => println!("result.cert_file: None"),
    }

    if result.cert_dir.is_empty() {
        println!("result.cert_dir: None");
    } else {
        for path in &result.cert_dir {
            println!("result.cert_dir: {}", path.display());
        }
    }

 let mut data = Vec::new();
 let mut handle = Easy::new();
 handle.url("https://dw-eng-rsc.innomotics.net/").unwrap();
 {
     let mut transfer = handle.transfer();
     transfer.write_function(|new_data| {
         data.extend_from_slice(new_data);
         Ok(new_data.len())
     }).unwrap();
     transfer.perform().unwrap();
 }
}

Verification in a standalone application while removing and adding potential candidates:

cafe-custom-uis@deblndw013x3j:/usr/home/cafe-custom-uis/openssl-probe-tester
$ ./target/debug/openssl-probe-tester
result.cert_file: /etc/ssl/cert.pem
result.cert_dir: /etc/ssl/certs
cafe-custom-uis@deblndw013x3j:/usr/home/cafe-custom-uis/openssl-probe-tester
$ ./target/debug/openssl-probe-tester
result.cert_file: /usr/local/etc/ssl/cert.pem
result.cert_dir: /etc/ssl/certs
cafe-custom-uis@deblndw013x3j:/usr/home/cafe-custom-uis/openssl-probe-tester
$ ./target/debug/openssl-probe-tester
result.cert_file: /usr/local/share/certs/ca-root-nss.crt
result.cert_dir: /etc/ssl/certs
cafe-custom-uis@deblndw013x3j:/usr/home/cafe-custom-uis/openssl-probe-tester
$ vim ./target/debug/openssl-probe-tester
cafe-custom-uis@deblndw013x3j:/usr/home/cafe-custom-uis/openssl-probe-tester
$ vim src/main.rs
cafe-custom-uis@deblndw013x3j:/usr/home/cafe-custom-uis/openssl-probe-tester
$ ./target/debug/openssl-probe-tester
result.cert_file: None
result.cert_dir: /etc/ssl/certs
cafe-custom-uis@deblndw013x3j:/usr/home/cafe-custom-uis/openssl-probe-tester
$ ./target/debug/openssl-probe-tester
result.cert_file: None
result.cert_dir: /etc/ssl/certs
cafe-custom-uis@deblndw013x3j:/usr/home/cafe-custom-uis/openssl-probe-tester
$ ./target/debug/openssl-probe-tester
result.cert_file: None
result.cert_dir: None

thread 'main' (189247) panicked at src/main.rs:29:25:
called `Result::unwrap()` on an `Err` value: Error { description: "SSL peer certificate or SSH remote key was not OK", code: 60, extra: Some("SSL certificate OpenSSL verify result: unable to get local issuer certificate (20)") }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
cafe-custom-uis@deblndw013x3j:/usr/home/cafe-custom-uis/openssl-probe-tester
$ ./target/debug/openssl-probe-tester
result.cert_file: None
result.cert_dir: /etc/ssl/certs

Running against publically available servers as well as in-house with enterprise CA structure.

[michael-o]

Also built a custom version of uv, works flawlessly against in-house servers and public ones.

[cpu]

This seems fairly divergent from what Go is doing for FreeBSD and I suspect that project has a lot more deployment in the real world than openssl-probe. Has you or someone else from the FreeBSD community made a case in the Go issue tracker that it's incorrect/suboptimal?

[michael-o]

This seems fairly divergent from what Go is doing for FreeBSD and I suspect that project has a lot more deployment in the real world than openssl-probe. Has you or someone else from the FreeBSD community made a case in the Go issue tracker that it's incorrect/suboptimal?

Good point. This needs a clean up for FreeBSD. We have started using Go for a DB project, but aren't doing any outbound connections at the moment, therefore I didn't notice it. I am sure it will be a problem as well and needs attention. I need to create an enterprise test with https://github.com/golang-auth/go-gssapi written by @jake-scott, but won't be able to before next year.

I'd be happy to address also for the Go community because this might hit us sooner or later.

[djc]

I'm sorry, after all these iterations we (after some private discussion) still don't feel confident in these changes. We'd rather stick to the smaller set derived from what Go uses (which arguably has a much larger install base). If you have other needs, there's always the opportunity to set SSL_CERT_FILE and/or SSL_CERT_DIR.

[michael-o]

I'm sorry, after all these iterations we (after some private discussion) still don't feel confident in these changes. We'd rather stick to the smaller set derived from what Go uses (which arguably has a much larger install base). If you have other needs, there's always the opportunity to set SSL_CERT_FILE and/or SSL_CERT_DIR.

That's quite disappointing because I have put quite some effort to explain explictly every single one. If you are still afraid, we can completely narrow down to certctl(8), that is /etc/ssl and /usr/local/etc/ssl?! Setting the env vars simply does not scale if you can cover 99% of all cases with my PR, be aware that this does not only improve for the Rust ecosystem, but also for the Python ecosystem. So the improvement is huge. Please reconsider. WDYT?

[michael-o]

@djc I have pushed a simplified version.

[michael-o]

@MikaelUrankar @kevans91 @emaste @Jehops FYI

[kevans91]

@MikaelUrankar @kevans91 @emaste @Jehops FYI

It doesn't seem unreasonable to just go fix golang really quickly, if they insist. I'm not quite sure I understand the apprehension (particularly because the paths in golang come from golang folks, not from FreeBSD folks that design this stuff for their OS and the changes wouldn't impact others), but it's a valid policy and you can see why they might want that.

[michael-o]

@MikaelUrankar @kevans91 @emaste @Jehops FYI

It doesn't seem unreasonable to just go fix golang really quickly, if they insist. I'm not quite sure I understand the apprehension (particularly because the paths in golang come from golang folks, not from FreeBSD folks that design this stuff for their OS and the changes wouldn't impact others), but it's a valid policy and you can see why they might want that.

The hint with golang I wanted to fix next after this--and I agree with you, it should be after the first party (us), not a third party (golang).