Saturday, August 1, 2009

AVCHD timecodes revealed

When playing with my new Canon HF200 camera, I got curious where the recording time (and date) is hidden in the AVCHD format.

The first idea was the SEI pic timing message of the H.264 stream. I already parse it for getting information whether pictures are frame- or field coded. So I extended my code to parse the timecode in HH:MM:SS:FF format, only to find out, that this info isn't present at all in my files :(

Googling for more informations about that, I found that nobody knows how to get the recording time and even professional programs fail to display it. But some very few programs do, so we know that it must be coded in the transport stream itself (and not in the other files written by the camera).

Finally I found this perl script, which extracts the date and time from canon mts files. It's a pretty simple implementation: It scans the multiplexed transport stream for a particular bit-pattern and then extracts the data. The script works for Canon-files but fails e.g. for Panasonic files.

Then I found where exactly the information is located: A H.264 stream has SEI (supplemental enhancement information) messages, which can contain additional (e.g. timing) information. For each SEI message the parser can obtain the message type (an integer) and the size of the message in bytes. AVCHD files have SEI messages of type 5, which means "user data unregistered" (== proprietary extension). The H.264 standard says, that these messages start with a 16 byte GUID followed by the payload.

Now take a look at the hexdump of such an SEI message:
17 ee 8c 60 f8 4d 11 d9 8c d6 08 00 20 0c 9a 66 ...`.M...... ..f
4d 44 50 4d 09 18 02 20 09 08 19 01 01 25 45 70 MDPM... .....%Ep
c7 f2 ff ff 71 ff ff ff ff 7f 00 00 65 84 e0 10 ....q.......e...
11 30 02 e1 07 ff ff ff ee 19 19 02 00 ef 01 c0 .0..............
00 00 ..
From this I found the following structure:
  • The GUID is the first 16 bytes. It's always the same for the info we want, but I found other SEI messages of type 5 with different GUIDs in AVCHD files.
  • 4 characters "MDPM". They occur in all files I looked at.
  • An unknown byte (0x09, other vendors have other values)
  • The byte 0x18 (probably indicating that year and month follow)
  • An unknown byte (0x02, other vendors have other values)
  • The year and month in 3 BCD coded bytes: 0x20 0x09 0x08
  • The byte 0x19 (probably indicating that day and time follow)
  • An unknown byte (0x01, other vendors have other values)
  • The day, hour, minute and second as 4 BCD encoded bytes (0x01 0x01 0x25 0x45)
In this case, I extract the recording time "2009-08-01 01:25:45" (which is correct).

The remainder of the SEI is completely unknown, but I'm sure if someone would figure out the complete data structure (including the unknown bytes), one might be able to extract other interesting informations.

These messages are present for almost all frames, but I plan to read them only from the first frame because the following ones are redundant.

Next project will be to clean up the parsing code in gmerlin-avdecoder and make the timecode actually appear along with the first decoded frame.


Pepper said...

I've managed to decode much of this information for a number of different cameras. I'm still working on decoding some fields that I don't know. Particularly the ExposureTime (ID 0x7f) is giving me trouble because I know it is there but it seems to be in different formats for different cameras. For your HF200, the mystery 0x09 indicates there are 9 tags in the data to follow. Each tag begins with a 1-byte tag ID number, followed by 4 bytes of data. The date/time value is found in tags with ID 0x18 and 0x19. See the H.264 tags here for more details.

Pepper said...

Also, you mention an unknown byte of 0x01 found before the time value. This is not unknown, and is of course the day as you also mention. I don't know the value of the unknown byte before the year (0x02 in your example) -- if anyone knows what this is I would like to hear about it.

burkhard said...

Thanks a lot for this info, that makes perfect sense to me. Also you are right about the last unknown 0x01, it's of course the day. Must have been a copy & paste error :)

pcoud said...

Interesting stuff... I'm also digging into my MTS files from a panasonic GH1 in hope to find time code info, no luck so far :(
And the same for associated CPI (clipinfo) files.
Could it be that Panasonic is not using BCD encoding ? Or worse, does not provide timing info in the GH1 streams ?

burkhard said...

I could look into GH1 streams if I had one. Can you upload it somewhere?

BTW an update on AVCHD timecodes is here:

pcoud said...
This comment has been removed by the author.
pcoud said...

Here is a short clip (15MB):

(capture date is 2010-02-03)

I finally found some timestamps in the the playlist (MPL) files, but I'm still wondering whether they are also embedded in the transport stream produced by the GH1. Thanks for the info on AVCHD timecode!

pcoud said...

Just looked at some GH1 .MTS, I cannot find anywhere the MDPM signature nor the 16 byte GUID signature.

burkhard said...

Your GH1 file contains only SEI messages of types 0 (buffering period) and 1 (pic timing). If these files contain the recording date/time, they are well hidden :)

Does the camera have an AVCHD logo?

pcoud said...

Yes, the GH1 feature the AVCHD logo:

May be a way to put some pressure on Panasonic ;-)

burkhard said...

I don't have the AVCHD specs, so I don't know if the timecodes are optional or required.

Will said...

Do you know if the timecode increments per frame or if all frames just have the timecode/datecode of the instant the recording started? I.E. If you started recording at 8:03:02 and you advanced 60 frames into the recording of a 1080p source would you find 8:03:03?

burkhard said...

They are increased.

I think the time display of the camera is simply updated each time a new timecode is found in the stream.

burkhard said...

Please put further comments under the newer post