Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Multispectral support #138

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[WIP] Multispectral support #138

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 99 additions & 4 deletions ffv1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -731,10 +731,32 @@ Parameters( ) { |
colorspace_type | ur
if (version >= 1) |
bits_per_raw_sample | ur
chroma_planes | br
log2_h_chroma_subsample | ur
log2_v_chroma_subsample | ur
extra_plane | br
if (colorspace_type < 2) { |
chroma_planes | br
log2_h_chroma_subsample | ur
log2_v_chroma_subsample | ur
extra_plane | br
} else { |
plane_count | ur
per_plane_metadata_count | ur
for( i = 0; i < plane_count; i++ ) { |
plane_type | ur
center_frequency | ur
frequency_response ( center_frequency ) |
color_transfer_characteristic | ur
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

color_transfer_characteristic is common to YUV (+colour_primaries, matrix_coefficients), shouldn't it be in a color info dedicated part e.g. at the end of the configuration box?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, the idea was that we need something to go from physical energy to binary values and that often is not linear. Maybe color_transfer_characteristic is a poor term for this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

transfer_characteristic wording would be misleading as the name is already used elsewhere (prefix 'color' would not be enough for differentiation IMO), so in that case it should be another name.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • color_translation?
  • color_conversion?
  • color_interpretation?

possibly add _characteristic to any of those?

black_point | sr
white_point | sr
unit | sr
direction_x | sr
direction_y | sr
direction_z | sr
h_subsample | ur
v_subsample | ur
for (j = 0; j < per_plane_metadata_count; j++) { |
reserved[j] | sr
} |
} |
} |
if (version >= 3) { |
num_h_slices - 1 | ur
num_v_slices - 1 | ur
Expand Down Expand Up @@ -823,6 +845,7 @@ If state_transition_delta is not present in the FFV1 bitstream, all Range coder
|-------|:------------------------|:------------------------|:------------------------|:------------------------|
| 0 | YCbCr | None | Transparency | `Plane` then `Line` |
| 1 | RGB | JPEG2000-RCT | Transparency | `Line` then `Plane` |
| 2 | Multispectral | Custom | Transparency | TODO/WIP/XXX |
| Other | reserved for future use | reserved for future use | reserved for future use | reserved for future use |

Restrictions:
Expand Down Expand Up @@ -868,6 +891,78 @@ RFC:`log2_v_chroma_subsample` indicates the subsample factor, stored in powers t
| 0 | extra `Plane` is not present |
| 1 | extra `Plane` is present |

### plane_type

`plane_type` indicates the semantic type of the plane.

|value | plane_type |
|---------|:------------------------------------------------|
| 0 | Unspecified |
| 1 | Emmissive |
| 2 | Absorptive |
| 3 | Reflective |
| 4 | Retro-Reflective |
| 5 | Mask |
| 6 | Depth |
| 7 | Motion |
| 8 | Error |
| 9 | Force Field |
| 10 | Polarization |
| 11-999 | Reserved |

### per_plane_metadata_count

`per_plane_metadata_count` allows future extension of the information we store about planes

### center_frequency

`center_frequency` indicates the center frequency in hz. Or 0 if unknown or not applicable.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit Hz

Copy link

@pjotrek-b pjotrek-b Oct 25, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for jumping in (3 years+ later) 😄 :
Does any of you have any particular real-world examples in mind (or access to them)?
Like the xrays, long radio wave images, air/vacuum/etc recordings @michaelni mentioned?

I'm wondering if it'd make sense not to raw-parametrize this, but rather define/offer presets which can be tuned in the code, rather than as arguments? Then access these interpretation options by a given name shortcut?
(If I understood the subject alright)

I assume that any area of expertise that would create such recordings will very likely have some sets of "common parameters" - therefore domain-specific "insider terms" for those imaging types.

Like FFmpeg having "dvd_pal" containing a set of sane/valid parameters, we could have something like "xray_10nm" or "10nm-water-hot". Dunno. Something like that?

Wouldn't this also allow arbitrary choice of data-types (per imaging-type) for proper calculations?
Integers, fractions, etc - "different strokes for different (imaging type) folks" - so to say 😜

If I'm right, then this is tricky to handle in a Standards paper definition... But if x264 can be a standard and do presets, so could ffv1, I presume? 😉

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need samples there should be lots of public stuff from astronomy and satellites. For xrays and MRT the word to search for is probably dicom image. One source for public domain samples would be any images you own the rights to, like from a medical exam. I didnt really try to find free proper raw sample images for these things previously ...


### black_point

`black_point` Value repesenting 0% brightness, -1 if unknown

### white_point

`white_point` Value repesenting 100% brightness, -1 if unknown

### unit

`unit` TODO, we need a way to tie the white point to some physical counterpart like energy when this information is known.
0 if unknown.

### direction_x, direction_y, direction_z

`direction_*` indicates the direction of motion for planes representing motion,
the direction of Polarization for planes which can be polarized. 0,0,0 when
unknown or not applicable.

### color_transfer_characteristic

`color_transfer_characteristic`

|value | color_transfer_characteristic |
|---------|:--------------------------------------------------------------------------------------|
|0 | Reserved |
|1 | ITU-R BT709 / BT1361 |
|2 | Unspecified |
|3 | Reserved |
|4 | Gamma 2.2 also ITU-R BT470M / ITU-R BT1700 625 PAL & SECAM |
|5 | Gamma 2.8 also ITU-R BT470BG |
|6 | SMPTE 170M also ITU-R BT601-6 525 or 625 / ITU-R BT1358 525 or 625 / ITU-R BT1700 NTSC|
|7 | SMPTE 240M |
|8 | Linear transfer characteristics |
|9 | Logarithmic transfer characteristic (100:1 range) |
|10 | Logarithmic transfer characteristic (100 * Sqrt(10) : 1 range) |
|11 | IEC 61966-2-4 |
|12 | ITU-R BT1361 Extended Colour Gamut |
|13 | IEC 61966-2-1 (sRGB or sYCC) |
|14 | ITU-R BT2020 for 10-bit system |
|15 | ITU-R BT2020 for 12-bit system |
|16 | SMPTE ST 2084 for 10-, 12-, 14- and 16-bit systems |
|17 | SMPTE ST 428-1 |
|18 | ARIB STD-B67, known as "Hybrid log-gamma" |

### num_h_slices

`num_h_slices` indicates the number of horizontal elements of the slice raster.
Expand Down