Detect file type

[sudeeptarlekar]

Rustcore should have functionality to detect the file type. If file type is binary or text, and if text what is the encoding of file.

Resolves: #1841

[DmitryAstafyev]

Location

We have a nice suitable place for it application/apps/indexer/addon, but now it's just one module. I would suggest (if @marcmo agree):

• rename application/apps/indexer/addon into application/apps/indexer/addons
• current lib in application/apps/indexer/addon rename into dlt-tools and add as submodule
• same for your tool, create files-tools and add there

It will be something like:

-- addons
---- dlt-tools
------ src
------ Cargo.tolm
---- files-tools
------ src
------ Cargo.tolm

Create it as lib and add tests of course.

Artifacts for tests

Also, take into account, we have this place developing/resources. Here you can store resources for testing. You can left there some example files, which you would use for testing.

General idea

I think reading some parts of target-file and searching "specific binary or not binary" symbols is a good idea and the right direction. Just a question, would it be simple just try some chunks of file with std::str::from_utf8. Because if it isn't UTF8: you will get error or result string len will be != to the len of given byte's slice. That's just an idea, I don't know what is a better.

Good habit

As soon as it's draft, no problems of course to skip cargo clippy, but I would recommend doing it always before pushing with rake clippy:all... As usual, it gives you nice and useful recommendations about code styling and allows you to better understand rust philosophy

[DmitryAstafyev]

DO NOT MERGE in the name of PR scares me )) sounds like a bomb in our solution)) If your PR in the draft state it will not be merged - no worries )

[DmitryAstafyev]

linked to: #1921, #1841

[marcmo]

indexer/addon is a good place...no need to rename to addons

[sudeeptarlekar]

Is this good to 🚢 ? Asking as I was bit skeptical about changes...

[DmitryAstafyev]

I don't remember (please check it), but if fucntion is async, we can return just Error or throw new Error(), instead Promise.reject(Error)... but please check it.

[DmitryAstafyev]

Here is a good example to better understand promises and async stuff.

async function a() {
    return new Error('Test');
}

async function b() {
    return Promise.reject(new Error('Test'));
}

async function c() {
    throw new Error('Test');
}

async function d() {
    return new Promise(() => {
        throw new Error('Test');
    })
}

a().then((_) => {
    console.error(`[A]: CATCH: isn't handled`);
}).catch((_) => {
    console.log('[A] - ok');
});

b().then((_) => {
    console.error(`[B]: CATCH: isn't handled`);
}).catch((_) => {
    console.log('[B] - ok');
});

c().then((_) => {
    console.error(`[C]: CATCH: isn't handled`);
}).catch((_) => {
    console.log('[C] - ok');
});

d().then((_) => {
    console.error(`[D]: CATCH: isn't handled`);
}).catch((_) => {
    console.log('[D] - ok');
});

console:

[A]: CATCH: isn't handled
[C] - ok
[B] - ok
[D] - ok

Well returning new Error will not reject promise, but return result with resolve

[DmitryAstafyev]

Please make sure statements try/catch are still actual after switching to async code

[DmitryAstafyev]

Here is a good example to better understand promises and async stuff.

async function a() {
    return new Error('Test');
}

async function b() {
    return Promise.reject(new Error('Test'));
}

async function c() {
    throw new Error('Test');
}

async function d() {
    return new Promise(() => {
        throw new Error('Test');
    })
}

a().then((_) => {
    console.error(`[A]: CATCH: isn't handled`);
}).catch((_) => {
    console.log('[A] - ok');
});

b().then((_) => {
    console.error(`[B]: CATCH: isn't handled`);
}).catch((_) => {
    console.log('[B] - ok');
});

c().then((_) => {
    console.error(`[C]: CATCH: isn't handled`);
}).catch((_) => {
    console.log('[C] - ok');
});

d().then((_) => {
    console.error(`[D]: CATCH: isn't handled`);
}).catch((_) => {
    console.log('[D] - ok');
});

console:

[A]: CATCH: isn't handled
[C] - ok
[B] - ok
[D] - ok

Well returning new Error will not reject promise, but return result with resolve

[DmitryAstafyev]

Please also make sure, all places of this function's usage are checked and corrected also. It's quite radical change (sync -> async) and we should make sure - all calls of changed functions are corrected too.