Content for  TS 23.333  Word version:  17.1.0

The MRFC and the MRFP may support the Multi-stream Multiparty Conferencing Media Handling (MMCMH) using "simulcast" and "RTP-level pause and resume" functionality as specified in TS 26.114 Annex S. Usage of simulcast RTP streams is negotiated via SDP offer/answer exchange through media level SDP attributes:

• simulcast stream description ("a=simulcast" attribute defined in RFC 8853); and
• restriction identifier ("a=rid" attribute defined in RFC 8851).

When using simulcast, the "a=rid" identification of simulcast formats shall be unique within a media clause ("m=" line). SDP simulcast negotiation decides which simulcast formats, if any, are sent between the conference participant and the MRFP. Furthermore, the media level SDP attribute "a=content" (defined in RFC 4796) is used to indicate content of RTP stream. The main video SDP "m=" line is indicated by the "a=content:main" under that "m=" line. The screenshare video is indicated as a separate SDP video "m=" line, identified by "a=content:slides" under that "m=" line. The screenshare video "m=" line shall be listed after the main video "m=" line. The "thumbnail" video is defined as unidirectional video which can be offered under the main video "m=" line with a smaller resolution then a main video or as a separate "m=" line that is not the first video "m=" line in the SDP, and that is also not identified with any "a=content:main" or "a=content:slides". The main audio SDP "m=" line shall be the first "m=" line in an SDP offer, to increase the probability that it is accepted by a non-MMCMH capable conference participant. Support for multiple, simultaneous audio streams is indicated in SDP as separate audio "m=" lines. The number of supported channels in multi-channel audio is indicated per audio stream through the SDP "m=" line <encoding parameters>, with the default being a single channel when <encoding parameters> is omitted. The additional audio "m=" lines shall be listed after the main audio "m=" line. "RTP-level pause and resume" functionality shall be supported by conference participants supporting MMCMH feature. The following RTCP feedback "CCM PAUSE-RESUME" messages are defined in RFC 7728:

• "PAUSE" message to request an RTP stream sender to pause an RTP stream;
• "RESUME" message to request an RTP stream sender to resume an RTP stream;
• "REFUSED" message to notify an RTP stream receiver that a pause or resume request is not accepted; and
• "PAUSED" message to inform an RTP stream receiver that an RTP stream is paused.

Usage of "RTP-level pause and resume" functionality is negotiated via SDP offer/answer exchange through an extension of RTCP feedback capability attribute "a=rtcp-fb" (defined in RFC 4585) to include request for pause and resume, as specified in RFC 7728. The use of the "RTP-level pause and resume" functionality can be restricted according to a "config" pause attribute (defined in RFC 7728), including not using it at all if that is the SDP offer/answer negotiation outcome. Floor Control function using BFCP (as specified in clause 5.14) to control main video and screenshare video shall be supported by conference participants supporting MMCMH feature. Examples of media portions from possible SDP offers and SDP answers related to a multi-stream multiparty conference establishment using simulcast and RTP-level pause and resume signalling are given in TS 26.114 Annex T. If the MRFC and the MRFP support the MMCMH feature, the requirements and procedures in the following clauses apply.

The MRFC shall support separating simulcast formats through referencing separate RTP payload types for each simulcast format, i.e., using the "pt=" parameter on the "a=rid" line (RFC 8851). The MRFC and the MRFP may support restrictions parameters defined in RFC 8851. The MRFP:

• shall support sending at least two thumbnails;
• may support sending any number of additional thumbnails;
• shall support receiving at least one thumbnail-sized simulcast format of the main video;
• may support receiving also other simulcast formats;
• shall support the BFCP floor control server role as specified in clause 5.14;
• shall support the "RTP-level pause and resume" functionality corresponding to "config=2" (as defined in RFC 7728, i.e. sending of "PAUSE", "RESUME" and "PAUSED", and reception of "PAUSED" and "REFUSED" RTCP feedback "CCM PAUSE-RESUME" messages); and
• should support the full "RTP-level pause and resume" functionality corresponding to "config=1" (as defined in RFC 7728, i.e. sending and receiving of all RTCP feedback "CCM PAUSE-RESUME" messages).

If the MRFC receives an SDP offer containing:

• an "a=simulcast" attribute; and
• the corresponding "a=rid" lines with a "pt" parameter defining the simulcast stream identification,

under an "m=" line then, in accordance with local configuration and SDP simulcast negotiation results on other call legs, the MRFC shall:

• if the MRFC supports the "Compact Concurrent Codec Negotiation and Capabilities" function specified in clause S.5.7 of TS 26.114 and received the session level "a=ccc_list" attribute (defined in clause S.5.7.2 of TS 26.114) within the SDP offer, use the information from the received "a=ccc_list" attribute in addition to the local configuration and SDP simulcast negotiation results on other call leg when deciding which media stream configuration to select towards the conference participant and when requesting the MRFP to reserve resources towards this conference participant;
• select the payload types (codecs and codec configurations) that it accepts to use for the accepted "m=" lines;
• select simulcast formats i.e. the MRFC shall:
  1. verify the "pt" parameter from the "a=rid" line against the list of payload types listed on the "m=" line:
     1. if payload type is not listed on the "m=" line, remove the payload type from the set of values in the "pt" parameter; and
     2. if no values are left in the "pt=" parameter after this processing, then remove the "a=rid" line;
  2. if the "a=rid" line contains a restrictions parameter which the MRFC does not support and if the "direction" field is "recv", remove the "a=rid" line; and
  3. change the directionality of the selected simulcast formats i.e. for the selected "a=simulcast" and "a=rid" attributes "send" from the received SDP offer becomes "recv" and "recv" becomes "send" in the SDP answer;
• if the "thumbnail" video descriptions are included in the SDP offer, based on the MRFP capabilities accept all or some of the offered the "thumbnail" video "m=" lines and change the directionality of the accepted "thumbnail" video "m=" lines from "recvonly" to "sendonly";
• if a video "m=" line with the "a=content:slides" attribute is included in the SDP offer and if the MRFP supports the screenshare video, accept the screenshare video "m=" line;
• when requesting the MRFP to configure resources towards the conference participant, for the accepted "m=" lines:
  1. if the participant is the first conference participant, request the MRFP to create a new context and include a "MMCMH policy" information element to indicate to the MRFP that interconnection of video media streams with different StreamIDs is allowed in the context used for MMCMH and that the MRFP shall mix media streams autonomously based on policies for MMCMH. Within the "MMCMH policy" information element the MRFC:
     1. shall indicate with the value "MMCMH basic policy" that the MRFP shall not send media streams received on a termination towards that termination and that the MRFP shall forward a received media stream of a particular media type (i.e. audio, main video or screenshare video) only towards outgoing media streams of the same media type. The MRFP shall select the video streams to be sent to a conference participant from among the videos received from the other conference participants in such a way that from each other conference participant at most one main video is sent to this conference participant and at most one screenshare video stream is sent to this conference participant;
     2. may indicate with the value "Voice activity detected video" that the MRFP shall detect voice activity on audio streams and that the MRFP shall forward the main video received from the active speaker (i.e. from the media sender from which an audio stream is received where voice activity is currently detected) to all other conference participant;
     3. may indicate with the value "Voice activity detected audio" that the MRFP shall detect voice activity on audio streams;
     4. may indicate with the value "Mix audio" that the MRFP shall mix all the received audio streams from all other conference participants in the context and send the resulting audio stream(s) to each conference participant;
     5. may indicate with the value "BFCP audio" that if the MRFP receives BFCP messages, the MRFP shall select received audio streams to forward or mix based on these BFCP messages;
     6. may indicate with the value "BFCP video" that if the MRFP receives BFCP messages, the MRFP shall select received video streams to forward or mix based on these BFCP messages; and
     7. may indicate with the value "BFCP screenshare" that if the MRFP receives BFCP messages, the MRFP shall select received screenshare streams to forward or mix based on these BFCP messages;
  2. include all accepted media streams sent to or received from the conference participant into the same termination;
  3. include selected payload types for each accepted the "m=" line;
  4. include a "Simulcast desc" information element with the selected simulcast streams for the accepted "m=" line and request the MRFP to configure termination with a simulcast capability;
  5. include a "Simulcast format" information element with the accepted "a=rid" lines for the accepted "m=" line;
  6. if the "a=content" attribute was received in the SDP offer, include a "Stream content" information element to indicate content of stream;
  7. if the "a=rtcp-fb" line(s) with a "CCM" parameter (as defined in RFC 5104) and a "pause" CCM parameter (as defined in RFC 7728) was received in the SDP offer:
     1. include a "CCM pause-resume" information element to indicate to the MRFP which RTCP feedback "CCM PAUSE-RESUME" messages the MRFP may send to the conference participant;
     2. if a "nowait" pause attribute was received in the SDP offer, shall include a "nowait" pause attribute in the "CCM pause-resume" information element to indicate to the MRFP that exchange of the RTCP feedback "CCM PAUSE-RESUME" messages will be point-to-point and no hold-off period will therefore be necessary when resuming paused streams;
     3. include a "Autonomous request" information element to indicate that the MRFP is allowed to autonomously send RTCP feedback CCM PAUSE and RESUME messages; and
     4. include a "Autonomous response" information element to indicate that the MRFP is allowed to autonomously send RTCP feedback CCM PAUSED and REFUSED messages; and
  8. if the BFCP media stream was included in the SDP offer, apply additional specific procedures for BFCP in clause 5.14.5;
• include the accepted simulcast streams in the SDP answer;
• include the "thumbnail" video "m=" lines in the SDP answer;
• include the screenshare video "m=" line in the SDP answer;
• if the "a=rtcp-fb" line with a "pause" CCM parameter was included in the SDP offer:
  1. include the "a=rtcp-fb" line with a "pause" CCM parameter in the SDP answer;
  2. if the "nowait" pause attribute was received in the SDP offer, include the "nowait" pause attribute in the SDP answer; and
  3. if the MRFP does not support the full "RTP-level pause and resume" functionality corresponding to "config=1" include the "config" pause attribute in the SDP answer with the value set according to the MRFP capability ("config=2") and the SDP offer/answer rules specified in RFC 7728;
• if an offered BFCP media stream was accepted, include in the SDP answer:
  1. an "a=floorctrl:s-only" attribute to indicate that the MRFP will act as a floor control server;
  2. an "a=confid" attribute indicating the conference identity to be used by BFCP signalling;
  3. an "a=userid" attribute indicating the user identity to be used by BFCP signalling;
  4. "a=floorid" attribute line(s) containing the BFCP floor identity(ies) to be used to control acceped audio and/or video media streams;
  5. "a=label" attribute line(s) indicating which of the accepted audio and/or video media streams will be BFCP controlled;
  6. an "a=connection:new" attribute to request a new TCP connection; and
  7. an "a=setup" attribute to indicate which end point will act as a TCP client; and
• include in the SIP message containing the SDP answer an "isfocus" media feature tag, defined in RFC 3840.

If the MRFP supports the "Compact Concurrent Codec Negotiation and Capabilities" function specified in TS 26.114 and if the MRFC received the session level "a=ccc_list" attribute within the SDP offer, then when requesting the MRFP to configure resources towards the conference participant, the MRFC may provide to the MRFP a "Concurrent Codec Capabilities" information element to indicate the concurrent codec capabilities of the conference participant.

If the MRFC receives a trigger to add a new participant in the MMCMH conference, then in accordance with local configuration and SDP simulcast negotiation results on other call legs, the MRFC shall decide which media stream configuration to offer to a new conference participant. If the MRFC supports the "Compact Concurrent Codec Negotiation and Capabilities" function specified in clause S.5.7 of TS 26.114 and received the "a=ccc_list" attribute (defined in clause S.5.7.2 of TS 26.114) within the 200 (OK) final response to the OPTIONS request, the MRFC shall use the information from the received "a=ccc_list" attribute in addition to the local configuration and SDP simulcast negotiation results on other call legs when deciding which media stream configuration to offer to a new conference participant and when requesting the MRFP to reserve resources towards this conference participant. The MRFC:

• should offer an audio media stream as simulcasted media stream;
• should offer a main video media stream as simulcasted media stream;
• shall decide if a "screenshare" video media stream will be offered;
• shall decide the number of "thumbnail" video media streams that will be offered in a sending direction towards the conference participant;
• should offer a BFCP media stream to control the audio and the main video media streams, and, if a "screenshare" video media stream is offered, to control the "screenshare" video media stream and should assign separate floor identities for each of these streams;
• for each offered audio and video media stream, shall select the payload types (codecs and codec configurations);
• for each offered simulcasted media stream, shall select a simulcast configuration (the number of RTP media streams which will be offered in a sending and in a receiving direction towards the conference participant) i.e. shall create the "a=simulcast" attribute, and shall select the corresponding simulcast formats i.e. shall create the corresponding "a=rid" lines with the "pt" parameter defining the simulcast stream identification; and
• for each offered audio and video media stream, shall decide if the "RTP-level pause and resume" functionality corresponding to "config=1" or "config=2" (see Figure 7 in RFC 7728) will be offered.

The MRFC shall then, in accordance with the selected media stream configuration, request the MRFP to reserve resources towards the conference participant. The MRFC shall:

• if the participant is the first conference participant, request the MRFP to create a new context and include a "MMCMH policy" information element to indicate to the MRFP that interconnection of video media streams with different StreamIDs is allowed in the context used for MMCMH and that the MRFP shall mix RTP media streams autonomously based on the MMCMH policies. Within the "MMCMH policy" information element the MRFC:
  1. shall indicate with the value "MMCMH basic policy" that the MRFP shall not send media streams received on a termination towards that termination and that the MRFP shall forward a received media stream of a particular media type (i.e. audio, main video or screenshare video) only towards outgoing media streams of the same media type. The MRFP shall select the video streams to be sent to a conference participant from among the videos received from the other conference participants in such a way that from each other conference participant at most one main video is sent to this conference participant and at most one screenshare video stream is sent to this conference participant;
  2. may indicate with the value "Voice activity detected video" that the MRFP shall detect voice activity on audio streams and that the MRFP shall forward the main video received from the active speaker (i.e. from the media sender from which an audio stream is received where voice activity is currently detected) to all other conference participant;
  3. may indicate with the value "Voice activity detected audio" that the MRFP shall detect voice activity on audio streams;
  4. may indicate with the value "Mix audio" that the MRFP shall mix all the received audio streams from all other conference participants in the context and send the resulting audio stream(s) to each conference participant;
  5. may indicate with the value "BFCP audio" that if the MRFP receives BFCP messages, the MRFP shall select received audio streams to forward or mix based on these BFCP messages;
  6. may indicate with the value "BFCP video" that if the MRFP receives BFCP messages, the MRFP shall select received video streams to forward or mix based on these BFCP messages; and
  7. may indicate with the value "BFCP screenshare" that if the MRFP receives BFCP messages, the MRFP shall select received screenshare streams to forward or mix based on these BFCP messages;
• include all offered media streams to that conference participant into the same termination;
• request the local IP address and the port for each offered audio and video media stream;
• include selected payload types for each offered audio and video media stream;
• include a "Stream content" information element with the value "main" for the main video media stream;
• if the "screenshare" video media stream is offered, include a "Stream content" information element with the value "slides";
• for each offered simulcast media stream:
  1. include a "Simulcast desc" information element with the selected simulcast configuration (containing the "a=simulcast" attribute) and request the MRFP to configure the termination with the simulcast capability; and
  2. include a "Simulcast format" information element with selected simulcast formats (the corresponding "a=rid" lines) to indicate to the MRFP the simulcast stream identification and which payload type to reserve for the particular simulcast format;
• for each audio and video media stream offered with the "RTP-level pause and resume" functionality:
  1. include a "CCM pause-resume" information element to indicate to the MRFP which RTCP feedback "CCM PAUSE-RESUME" messages the MRFP may send to the end user;
  2. include a "nowait" pause attribute in the "CCM pause-resume" information element to indicate to the MRFP that exchange of the RTCP feedback "CCM PAUSE-RESUME" messages will be point-to-point and no hold-off period will therefore be necessary when resuming paused streams;
  3. include an "Autonomous request" information element to indicate that the MRFP is allowed to autonomously send RTCP feedback CCM PAUSE and RESUME messages; and
  4. include an "Autonomous response" information element to indicate that the MRFP is allowed to autonomously send RTCP feedback CCM PAUSED and REFUSED messages; and
• apply additional specific procedures for BFCP in clause 5.14.5.

Upon receipt of a confirmation from the MRFP about local IP address and port and reserved resources for each selected media stream, the MRFC shall include in the SIP request an "isfocus" media feature tag (defined in RFC 3840), and the SDP offer with the selected audio, video and BFCP media streams. The MRFC shall include in the SDP offer:

• selected payload types for each offered audio and video media stream;
• an "a=content:main" attribute under the main video "m=" line;
• if the "screenshare" video media stream is offered, an "a=content: slides" attribute under the "screenshare" video "m=" line;
• for each offered simulcast media stream, an "a=simulcast" attribute and related "a=rid" attribute lines with a "pt" parameter defining the simulcast stream identification under the corresponding "m=" line;
• "a=label" attribute line(s) indicating which of the offered audio and/or video media streams will be BFCP controlled;
• for each offered "thumbnail" video media stream, an "a=sendonly" attribute under the corresponding "m=" line;
• for each offered audio and video media stream, an "a=rtcp-fb" line with a "pause" CCM parameter, a "nowait" pause attribute and a "config" pause attribute; and
• under BFCP over TCP "m=" line:
  1. an "a=floorctrl:s-only" attribute to indicate that the MRFP will act as a floor control server;
  2. an "a=confid" attribute indicating the conference identity to be used by BFCP signalling;
  3. an "a=userid" attribute indicating the user identity to be used by BFCP signalling;
  4. "a=floorid" attribute line(s) containing the BFCP floor identity(ies) to be used to control offered audio and/or video media streams;
  5. an "a=connection:new" attribute to request a new TCP connection; and
  6. an "a=setup" attribute to negotiate which end point will act as a TCP client;

Upon receipt of the corresponding SDP answer, the MRFC shall request the MRFP to modify termination towards the conference participant accordingly. The MRFC shall provide to the MRFP:

• the remote IP address and the port for each accepted media stream;
• payload types for each accepted audio and video media stream;
• for each accepted simulcast media stream:
  1. the "Simulcast desc" information element with the accepted simulcast configuration (containing the "a=simulcast" attribute); and
  2. the "Simulcast format" information element with accepted simulcast formats (the corresponding "a=rid" lines); and
• if the "a=rtcp-fb" line(s) with a "CCM" parameter and a "pause" CCM parameter was included in the SDP answer for the accepted audio or video media stream:
  1. the "CCM pause-resume" information element;
  2. if a "nowait" pause attribute was included in the SDP answer, the "nowait" pause attribute in the "CCM pause-resume" information element;
  3. the "Autonomous request" information element; and
  4. the "Autonomous response" information element.

If the MRFP supports the "Compact Concurrent Codec Negotiation and Capabilities" function specified in TS 26.114 and if the MRFC received the "a=ccc_list" attribute from the conference participant then when requesting the MRFP to modify termination towards the conference participant, the MRFC may provide to the MRFP a "Concurrent Codec Capabilities" information element to indicate the concurrent codec capabilities of the conference participant.

Upon reception of a request from the MRFC to configure resources towards the conference participant, the MRFP:

• shall assign the requested resources:
  1. shall configure termination to handle simulcast RTP streams; and
  2. shall start monitoring voice activity for audio media streams when instructed by the MRFC;
• if "Concurrent Codec Capabilities" information element was received from the MRFC, shall store the concurrent codec capabilities of the conference participant i.e. the maximum number of concurrent encoders and decoders the conference participant can operate and take these constraints into account when selecting the media streams to/from a conference participant;
• shall select the media source (among the received RTP streams) to be sent as one of the RTP streams, and then among the available simulcast streams the MRFP shall select for the most appropriate one to the sent to the particular conference participant, as specified in RFC 8853; and
• based on the value of the received "CCM pause-resume" information element, may send the RTCP "CCM PAUSE" messages towards the conference participant following the rules specified in RFC 7728.

Upon reception of a request from the MRFC to create a context with an "MMCMH policies" capability, the MRFP shall create that context and shall configure it according to value(s) received within the "MMCMH policy" information element. The MRFP shall handle media streams placed into the context as follows:

• The MRFP shall not send media streams received from a conference participant towards that conference participant.
• The MRFP shall forward a received media stream of a particular media type (i.e. audio, video or screenshare) only towards outgoing media streams of the same media type.
• The MRFP should forward the received audio stream of the active speaker (i.e. the audio stream where voice activity is detected) to all other conference participants. If simulcasted audio streams are received from the active speaker, the MRFP should select for each other conference participant an audio encoding among the received audio simulcast formats that is supported at the termination towards that participant to avoid transcoding.
• Alternatively, the MRFP may mix all the received audio streams from all other conference participants in the context and send the resulting audio stream(s) to a conference participant. If two audio streams were reserved towards a conference participant, the MRFP may distribute the received audio stream from each other conference participant in a specific way to render a stereo impression.
• The MRFP shall select the video streams to be sent to a conference participant from among the videos received from the other conference participants in such a way that:
  • from each other conference participant at most one main video is sent to this conference participant; and
  • at most one screenshare video stream is sent to this conference participant.
  If simulcasted video streams are received from a participant, the MRFP should select for each other conference participant a video encoding among the received video simulcast formats that is supported at the termination towards that participant to avoid transcoding.
• The MRFP should forward the main video received from the active speaker (i.e. from the media sender from which an audio stream is received where voice activity is currently detected) to all other conference participant. If several video streams are simulcasted from the active speaker, the MRFP should select for each other conference participant the simulcast format that matches the configured encoding and resolution of the main video stream towards that conference participant to avoid transcoding.
• The MRFP should forward the main video of the previous speaker (i.e. received from the media sender from which an audio stream was received where the most recent past voice activity has been detected) to the active speaker (i.e. towards the media receiver associated with the media sender from which an audio stream is received where voice activity is currently detected). If several video streams are simulcasted from the previous speaker, the MRFP should select the simulcast format that matches the configured encoding and resolution of the main video stream towards the active speaker to avoid transcoding.
• The MRFP should forward received thumbnail video streams from the most recent previous speaker(s) (i.e. from the media sender(s) from which audio stream(s) was/were received where the most recent past voice activities have been detected). If several video streams are simulcasted from a previous speaker, the MRFP should select for each other conference participant the simulcast format that matches the configured encoding and resolution of a thumbnail video stream towards that conference participant to avoid transcoding.
• In order to avoid a too frequent switching of video images, the MRFP should wait for a short period when detecting voice activity from a new source before switching the video image.
• If the MRFC receives RTCP feedback about increased packet loss from a media receiver, the MRFP should reduce the number of video streams sent towards that media receiver and select only video streams with lower resolution (e.g. thumbnail video streams); the MRFP should select video streams received from the most recent speaker(s) (i.e. from the media sender(s) from which audio stream(s) are received where the most recent voice activities are or have been detected).
• If BFCP is configured at the MRFP and the MRFP receives BFCP messages, the MRFP should select received streams to forward or mix based on these BFCP messages.
• If the MRFP does not pass a received media stream to any conference participant, based on any of the criteria above, and the "RTP-level pause resume" capability was configured for that media stream, the MRFP should signal to the sender of that media stream to pause sending that media stream in accordance with RFC 7728.
• If the MRFP has previously signalled to a sender to pause sending a media stream and decides to pass that media stream to some conference participant(s), based on any of the criteria above, the MRFP shall signal to the sender to resume sending that media stream in accordance with RFC 7728.