Skip to content

Add AVS3 audio and video#474

Open
paulhiggs wants to merge 22 commits into
gpac:mainfrom
paulhiggs:add-AVS
Open

Add AVS3 audio and video#474
paulhiggs wants to merge 22 commits into
gpac:mainfrom
paulhiggs:add-AVS

Conversation

@paulhiggs
Copy link
Copy Markdown
Contributor

Description

The PR adds support for the relevant sample types for AVS3 audio and video.

AVS3 video files are available at DVB - https://dvb.org/specifications/verification-validation/avs3-test-content/
AVS3 audio files are also available at DVB - https://dvb.org/avs3-audio-test-content/

See also #456

Copy link
Copy Markdown
Member

@DenizUgur DenizUgur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @paulhiggs, thank you so much for patience and for your contribution. Aside from the comments I've already left below, I have some requests:

  • Although avs-common.ts is a nice addition, I don't think that should be part of the library. Take avcC for example, we only use the primitive js types. Which many libraries would expect to see. If the intent behind the classes inside avs-common.ts is purely for aesthetics then we should remove it.
  • As for the BitBuffer, we should either incorportate it inside DataStream class or make it accessible through DataStream. I'm just thinking out loud here but maybe we can call getBitReader and access the methods of BitReader from there. BitReader (or BitBuffer as you named) can request bytes from DataStream. What do you think?

Comment thread src/boxes/av3c.ts
Comment thread demo/boxHtmlTable.js
Comment thread .vscode/settings.json
Comment thread src/BitBuffer.ts Outdated
@paulhiggs
Copy link
Copy Markdown
Contributor Author

paulhiggs commented Aug 12, 2025

On the classes defined in avs-common.ts: I took this approach as the semantif definition (and to a degree the representation of th value as binary, decimal, hexadecimal) is guided in the parsing process, so its the best place to determine how the output should be formatted while still retaining the possibility to ready the original value back for future use in write(). avcC can probably use decimal typing as elements in the ISO/IEC 14496-15 AVCDecoderConfigurationRecord and ISO/IEC 14496-10 syntax and semantics only use decimal values so they are likely more familiar to the user of mp4box.js.
Presenting in mp4box.js a decimal value when it is defined in hexadecimal or binary notation in the underlying specification just makes for another manual activity for the user.

@paulhiggs
Copy link
Copy Markdown
Contributor Author

If the intent behind the classes inside avs-common.ts is purely for aesthetics then we should remove it.

Yes, they are for aestetic purposes - describing the semantic value of the field in the box. If these are not of interest to the project, I will remove them.

@DenizUgur
Copy link
Copy Markdown
Member

DenizUgur commented Aug 12, 2025

If the intent behind the classes inside avs-common.ts is purely for aesthetics then we should remove it.

Yes, they are for aestetic purposes - describing the semantic value of the field in the box. If these are not of interest to the project, I will remove them.

They could be useful in the filereader demo, but probably not so much useful for the users of this library. As those libraries would be much more interested in accesing the values of these fields directly rather than the human-readable format of the field.

That's my opinion though. I just don't want to complicate it more than necessary.

@DenizUgur
Copy link
Copy Markdown
Member

DenizUgur commented Sep 23, 2025

Hi @paulhiggs, would you like to finalize this PR? The only thing blocking this is the BitBuffer class. To summarize my requests for that class:

  • Extend DataStream for the existing getUintXX methods and endianness
  • Consistent camel case for the public methods
  • Constructor should have a single parameter with MultiBufferStream | DataStream type
  • Rename to BitStream to continue the ...Stream naming
  • A warning when DataStream is used but BitStream isn't on a byte boundary. This is useful when we use BitStream when parsing a box but forgot to align at the end. For this, we would need a state variable on DataStream.

Also reminding my previous comment for the avs-common file

  • Although avs-common.ts is a nice addition, I don't think that should be part of the library. Take avcC for example, we only use the primitive js types. Which many libraries would expect to see. If the intent behind the classes inside avs-common.ts is purely for aesthetics then we should remove it.

@paulhiggs
Copy link
Copy Markdown
Contributor Author

paulhiggs commented Sep 26, 2025 via email

@DenizUgur
Copy link
Copy Markdown
Member

We can add bit-level read/write methods to DataStream if that's easier. In a way that would be a more seamless integration.

@paulhiggs
Copy link
Copy Markdown
Contributor Author

We can add bit-level read/write methods to DataStream if that's easier. In a way that would be a more seamless integration.

It would be great if you can add functions to peek(n) and read(n) bits from the datastream as well as functions to byte_align() in both read and write operations. I would certainly use them.

@DenizUgur
Copy link
Copy Markdown
Member

Okay, I won't have time for this for the next couple weeks, so I'm not fully sure when I can look at this. If you want to take a stab at this, I would certainly help you finalize it though.

@paulhiggs
Copy link
Copy Markdown
Contributor Author

One reason for handling this is a seperate buffer is that it allows the various bitwise representations to be gathered, the overall length to be determined and then that length (in bytes) written before the bitwise values.
As far as I have seen, TSduck has the best approach to this.

@DenizUgur
Copy link
Copy Markdown
Member

You could still do that within DataStream. You could have a buffer only used for operations within a byte.

If you want to, you can consolidate all "low-level" bitwise operations to a seperate file (like _rdb) if you don't want to clutter DataStream. Then either call from or bind from (so that you can use this) DataStream them with your buffer.

The advantage I see with this is that when you continue using other byte-aligned read/write operations, warning the user then aligning and flushing that buffer would be easier (as they are co-located).

So, long story short:

  • Remove all redundant stuff that's available in DataStream
  • Create a new member variable in DataStream for the bitwise buffer
  • Perform all bit operations using that buffer.
  • Every call to DataStream.memcpy or DataStream.(get|write)* should check that buffer and flush it out.

How's that sound?

@paulhiggs
Copy link
Copy Markdown
Contributor Author

Okay, I won't have time for this for the next couple weeks, so I'm not fully sure when I can look at this. If you want to take a stab at this, I would certainly help you finalize it though.

Did you look into how bit-level read/peek/write can be achieved, as well as a "post write" setting of the number of bytes written?
My current preference is to

  1. keep the BitBuffer implementation in the AVS codebase, then
  2. develop and test bit level read/write/align methods on existing DataStream/Bitstream, then
  3. revise the AVS related code (and others too) to use the bit level functions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants