Add the ability to parse workflows from AVIF images

[FerrahWolfeh]

Adds the ability to import workflows directly from avif files. Specially useful when using specific nodes that allow the user to specify the output format of the saved image or some tools that allow converting png files into AVIF while preserving their metadata.

The added code works normally as expected, though I plan to create a more robust implementation in the future by reusing some code from the isobmff.ts file.

┆Issue is synchronized with this Notion page by Unito

[christian-byrne]

Wow, a PR adding support for open, royalty-free image file format that people have requested.

And you are searching by using the binary tags instead of just decoding it all as text and searching for string matches linearly. That's seriously amazing.

Was this inspired by Pillow adding AVIF support last week?

[FerrahWolfeh]

Wow, a PR adding support for open, royalty-free image file format that people have requested.

And you are searching by using the binary tags instead of just decoding it all as text and searching for string matches linearly. That's seriously amazing.

Was this inspired by Pillow adding AVIF support last week?

Wait, did Pillow just add AVIF support? That's pretty neat!

Also, thanks for the compliment, I really appreciate that.

Currently I'm using a "brute-forcey" way of finding the exif tag inside the AVIF header. Going the binary route is actually faster and opens a way for me to correctly scan for the boxes in the future, as sometimes, the location of the mdat box and the exif header may vary.

[christian-byrne]

Wait, did Pillow just add AVIF support? That's pretty neat!

Yep. I think from python-pillow/Pillow#8858, AVIF is supported without requiring specific wheel. If you run pip install -U Pillow then start ComfyUI, AVIF images will be loadable, it's awesome

Currently I'm using a "brute-forcey" way of finding the exif tag inside the AVIF header. Going the binary route is actually faster and opens a way for me to correctly scan for the boxes in the future, as sometimes, the location of the mdat box and the exif header may vary.

I totally agree. If you look at some of our decoding implementations in https://github.com/Comfy-Org/ComfyUI_frontend/tree/main/src/scripts/metadata, you will find some lazy approaches. Your approach seems proper and should be 2-3 magnitudes of order faster. I will do some research into AVIF format, I wrote the ISOBMFF decoder somewhat recently and had a lot of fun.

[christian-byrne]

If you are able to create a simple AVIF with a workflow embedded, you can add a test case by just adding the file to this folder: https://github.com/Comfy-Org/ComfyUI_frontend/tree/main/browser_tests/assets

then just putting the filename in this list:

ComfyUI_frontend/browser_tests/tests/loadWorkflowInMedia.spec.ts

Lines 5 to 19 in 19eaf6e

	test.describe('Load Workflow in Media', () => {
	const fileNames = [
	'workflow.webp',
	'edited_workflow.webp',
	'no_workflow.webp',
	'large_workflow.webp',
	'workflow.webm',
	// Skipped due to 3d widget unstable visual result.
	// 3d widget shows grid after fully loaded.
	// 'workflow.glb',
	'workflow.mp4',
	'workflow.mov',
	'workflow.m4v',
	'workflow.svg'
	]

It will run the browser emulation and test dropping the AVIF onto the graph and loading the workflow.

[christian-byrne]

Oh sorry I forgot to mention that we generate the screenshots in the GitHub test runner automatically by adding the "New Browser Test Expectations" label. I will generate the screenshot now!

[webfiltered]

N.B. Workflow needs to be approved after adding label.

[christian-byrne]

Give us a moment to fix this, apologies.

[FerrahWolfeh]

Is there anything I need to do with the code or the snapshots? Because I'm pretty lost about what should be done on my end...

[webfiltered]

Is there anything I need to do with the code or the snapshots? Because I'm pretty lost about what should be done on my end...

Nope! Not unless you want to set your monitor to a MUCH smaller res, nuke all your settings, and upload a new screenshot? 😃

I'll just do a quick follow-up afterwards.

[webfiltered]

Oh ah, actually - could you delete the existing playwright screenshot? Will be able to merge aftter it's all tested, but we may as well not add it to the commit history seeing as it's about to be deleted.

[FerrahWolfeh]

Ok, done.

[webfiltered]

LGTM w/minor changes - thank you for this!

[christian-byrne]

Should we just merge this and add the screenshot in followup? Then again, we want the update expectations to be runnable by any contributor, especially with new PR requirements.

[webfiltered]

Was literally halfway through merging this yesterday when internet problems started.

For the root issue, the action would need to run on the fork. Which means it can't be based on a PR label. But should still be very doable.

[christian-byrne]

Okay nice, we can merge as part of 1.25 then later today!

[FerrahWolfeh]

For the root issue, the action would need to run on the fork. Which means it can't be based on a PR label. But should still be very doable.

Alright, I can do that. But which workflow in the actions menu should I run then if at all?

[webfiltered]

Alright, I can do that. But which workflow in the actions menu should I run then if at all?

We'll need to fix our workflows to be fork-friendly. We're good to merge this - just waiting for some bug fixes to go in, so this can land in the version after that.