Missing metadata for MP4

[cjee21]

When video from Android phone is re-encoded with FFmpeg, MediaInfo does not show some metadata such as the location and video encoder for MP4 files. The metadata is present in the file and is shown by FFmpeg/FFprobe or ExifTool. For MKV files, MediaInfo does show such info.

Attached are two samples in a ZIP file.
sample.mp4 created with

ffmpeg -i "input.mp4" -t 1 -c:v av1_qsv -preset veryslow -global_quality:v 25 -c:a libopus -b:a 192k sample.mp4

sample_mdta.mp4 created with

ffmpeg -i "input.mp4" -t 1 -c:v av1_qsv -preset veryslow -global_quality:v 25 -c:a libopus -b:a 192k -movflags use_metadata_tags sample_mdta.mp4

For both, Lavc61.31.100 av1_qsv is missing for video encoder. For sample.mp4, location information is missing.

I think FFmpeg did not include Lavc61.31.100 libopus for audio encoder in the metadata although it is shown in output metadata during trascoding as both FFmpeg and ExifTool did not show this when the output file is read. Same with the creation_time of video and audio stream.

sample.zip

[cjee21]

Just came across this issue that looks similar: MediaArea/MediaInfo#453

Side note: on the other hand, there are some mp4 tags in other files that are shown by MediaInfo but not by FFprobe or Mp3tag.

[cjee21]

Made new sample files and updated the posts above after doing more tests.

[cjee21]

Here is another sample straight from a Samsung phone: samsung_sample.zip

For this one, com.android.capture.fps is not shown by MediaInfo and also(upon looking further, it is already handled here) ExifTool has Samsung Model : SM-M325FV. In this sample, Samsung Model may seem similar to Performer that MediaInfo does show but for other Samsung phones, it is different. For example:

Author                          : Galaxy S23 Ultra
Samsung Model                   : SM-S918B

This is not very important though.

[cjee21]

Location from iPhone video found at drewnoakes/metadata-extractor#515 (comment) is not shown by MediaInfo too. Seems both FFmpeg and iOS are using loci atom for location information and it is formatted as a combination of variable-length null-terminated strings and fixed-length bytes.

From sample.mp4 attached in first post:

[mov,mp4,m4a,3gp,3g2,mj2 @ 0000016dd0dbaa00] type:'loci' parent:'udta' sz: 35 97 124

40267   Location Information (35 bytes)
40267    Header (8 bytes)
40267     Size:                               35 (0x00000023)
4026B     Name:                               loci
4026F    Version:                             0 (0x00)
40270    Flags:                               0 (0x000000)
40273    Data:                                (23 bytes)

[cjee21]

Regarding the encoder information, it is in MediaTrace but not shown by MediaInfo output.
From sample.mp4:

3FBF2        Compressor name size:            21 (0x15)
3FBF3        Compressor name:                 Lavc61.31.100 av1_qsv

[cjee21]

This does get Writing library : Lavc61.31.100 av1_qsv showing in MediaInfo output but I do not know if it is the correct way.

diff --git a/Source/MediaInfo/Multiple/File_Mpeg4_Elements.cpp b/Source/MediaInfo/Multiple/File_Mpeg4_Elements.cpp
index 77e7d921d..437b6d98b 100644
--- a/Source/MediaInfo/Multiple/File_Mpeg4_Elements.cpp
+++ b/Source/MediaInfo/Multiple/File_Mpeg4_Elements.cpp
@@ -6279,6 +6279,7 @@ void File_Mpeg4::moov_trak_mdia_minf_stbl_stsd_xxxxVideo()

     int16u Width, Height, Depth, ColorTableID;
     int8u  CompressorName_Size;
+    Ztring  CompressorName;
     bool   IsGreyscale;
     Skip_B2(                                                    "Version");
     Skip_B2(                                                    "Revision level");
@@ -6296,12 +6297,12 @@ void File_Mpeg4::moov_trak_mdia_minf_stbl_stsd_xxxxVideo()
     {
         //This is pascal string
         Skip_B1(                                                "Compressor name size");
-        Skip_UTF8(CompressorName_Size,                          "Compressor name");
+        Get_UTF8(CompressorName_Size, CompressorName,           "Compressor name");
         Skip_XX(32-1-CompressorName_Size,                       "Padding");
     }
     else
         //this is hard-coded 32-byte string
-        Skip_UTF8(32,                                           "Compressor name");
+        Get_UTF8(32, CompressorName,                            "Compressor name");
     Get_B2 (Depth,                                              "Depth");
     if (Depth>0x20 && Depth<0x40)
     {
@@ -6339,6 +6340,7 @@ void File_Mpeg4::moov_trak_mdia_minf_stbl_stsd_xxxxVideo()
             CodecID_Fill(Codec, Stream_Video, StreamPos_Last, InfoCodecID_Format_Mpeg4);
         Fill(Stream_Video, StreamPos_Last, Video_Codec, Codec, true);
         Fill(Stream_Video, StreamPos_Last, Video_Codec_CC, Codec, true);
+        Fill(Stream_Video, StreamPos_Last, Video_Encoded_Library, CompressorName);
         if (Codec==__T("drms"))
             Fill(Stream_Video, StreamPos_Last, Video_Encryption, "iTunes");
         if (Codec==__T("encv"))

[JeromeMartinez]

This does get Writing library : Lavc61.31.100 av1_qsv showing in MediaInfo output but I do not know if it is the correct way.

It is in theory but IIRC there is a lot of garbage in this MP4 field in lot of files, need to check... Additionally there may be conflict with other places having the writing library.
I'll do a pass with all the files I have.

[cjee21]

Additionally there may be conflict with other places having the writing library.

I did a quick test with x264 encoded mp4 with FFmpeg. MediaInfo shows x264 core 164 r3204 373697b with or without this patch while FFmpeg and ExifTool shows Lavc61.31.100 libx264.

For MKV, also MediaInfo only shows x264 core 164 r3204 373697b while FFmpeg shows Lavc61.31.100 libx264.
For audio part of MKV:
MediaInfo:

Writing application                      : Lavc61.31.100
Writing library                          : libVorbis (Reducing Environment) (20200704 (Reducing Environment))

FFmpeg:

    Metadata:
      ENCODER         : Lavc61.31.100 libvorbis

[JeromeMartinez]

while FFmpeg and ExifTool shows Lavc61.31.100 libx264

Because they check only the container, not the video bitstream. In practice here Lavc is not the compressor, it is an intermediate layer forwarding the content to the compressor (x264).
x264 core 164 r3204 373697b is more useful (bugs are likely in the compressor) than Lavc61.31.100, while still being useful (so I display it in the Writing application).
Not so easy and lot of corner cases...

[cjee21]

In practice here Lavc is not the compressor, it is an intermediate layer forwarding the content to the compressor

That's true but in the case of MKV made with FFmpeg using an encoder that does not write name to stream, MediaInfo does show Writing library : Lavc61.31.100 av1_qsv. So this patch does make MediaInfo output consistent between MP4 and MKV (show only stream data if available else show container one) at least in the case of files made with FFmpeg.

---

The Writing application field for MKV only appears for libVorbis it seems. Not shown for x264. For libopus stream just shows Writing library : Lavc61.31.100 libopus.

[cjee21]

Regarding the loci location metadata, I managed to parse it. I've put it here: https://github.com/cjee21/MediaInfoLib/tree/metadata-tests. I have not checked or tested thoroughly so not sure if it is good quality or robust enough.

Using test file from iPhone as linked above...

MediaInfo:

General
Complete name                            : IMG_3613.MP4
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (isom/mp41/mp42)
File size                                : 1.52 MiB
Duration                                 : 3 s 635 ms
Overall bit rate mode                    : Variable
Overall bit rate                         : 3 518 kb/s
Frame rate                               : 30.000 FPS
Recorded date                            : 2020-11-20T15:04:02+0100
Encoded date                             : 2020-11-20 14:04:36 UTC
Tagged date                              : 2020-11-20 14:04:37 UTC
Recorded location                        : +52.36929 +006.67609 +10.53499

MediaTrace:

000DAF   Location Information (35 bytes)
000DAF    Header (8 bytes)
000DAF     Size:                               35 (0x00000023)
000DB3     Name:                               loci
000DB7    Version:                             0 (0x00)
000DB8    Flags:                               0 (0x000000)
000DBB    Language:                            21956 (0x55C4)
000DBD    LocationString:
000DBE    Role:                                0 (0x00)
000DBF    Longitude:                           437524 (0x0006AD14)
000DC3    Latitude:                            3432074 (0x00345E8A)
000DC7    Altitude:                            690421 (0x000A88F5)
000DCB    Body:                                earth
000DD1    Notes:

[JeromeMartinez]

Regarding the loci location metadata, I managed to parse it.

great, please a PR.

[cjee21]

great, please a PR.

Made the PR. Now remains #2175 (comment) and the not so important #2175 (comment) for this issue.

---

Also do you want a PR to handle com.android.version since com.android.capture.fps is already handled? If so let me know what name to map it to. "Android version"?

[cjee21]

I managed to parse all the missing metadata mentioned in this issue so I just made another PR containing everything I have done. The changes work fine on all files I tested so far but I am still open to changes :)

[cjee21]

Found more yet-to-be-mapped metadata from iPhone videos:

com.apple.quicktime.make                 : Apple
com.apple.quicktime.model                : iPhone 12 Pro
com.apple.quicktime.software             : 14.1
com.apple.quicktime.creationdate         : 2020-11-06T17:19:58-0500