1.前言

本文介绍webrtc的API层整体结构和通话的api流程， 适合作为对webrtc有一定认知但是想研究源码的第一篇文章，推荐piasy的webrtc源码导读13以及webrtc源码导读10，本文实际就是在它们的基础上结合了webrtc源码上的example分析

2.正文

2.1 关键类介绍

2.1.1 PeerConnection

最核心的类，没有之一，所有的功能接口都由它提供，围绕它转，提供了下列类型的接口:

• 流相关: stream 和 track

  stream接口现已被标准弃用，mdn的上说addstream会导致后续的track变化感知不敏感

  track代表媒体轨，比如麦克风音轨，摄像头画轨等

local_streams() 
remote_streams()
AddStream()
RemoveStream()
AddTrack()
RemoveTrack()
RemoveTrackNew()

• 传输相关: transceiver, sender, receiver

  一个tranceiver包含了一个sender(编码，发送), receiver(解码接收)。addtrack的时候就是将track添加到transceiver的过程， 一个 sender 至多有一个要发送的 track一个要接收的 track 有一个 receiver，tranceiver使用mid作为标识。

AddTransceiver()
GetTransceivers() 
CreateSender() 
GetSenders() 
GetReceivers() 

• SDP和ICE相关

local_description()
remote_description()
current_local_description()
current_remote_description()
pending_local_description()
pending_remote_description()
RestartIce()
CreateOffer()
CreateAnswer()
SetLocalDescription()
SetRemoteDescription()
AddIceCandidate()
RemoveIceCandidates()

• 其它

SetBitrate()
SetAudioPlayout()
SetAudioRecording()
LookupDtlsTransportByMid()
GetSctpTransport()
signaling_state()
ice_connection_state()
standardized_ice_connection_state()
peer_connection_state()
ice_gathering_state()

2.1.2 PeerConnectionFactory

提供peerconnection，track，audio/video source的创建接口

CreatePeerConnection()
CreateLocalMediaStream()
CreateAudioSource()
CreateVideoTrack()
CreateAudioTrack()

2.2 通话的流程

2.2.1 流程概览

PeerConnectionFactory::CreatePeerConnection()  //  创建Pc
	
PeerConnection::CreateAudioTrack() // 创建音轨
PeerConnection::AddTrack()		   // 添加音轨
PeerConnection::CreateVideoTrack() // 创建视轨
PeerConnection::AddTrack()		   // 添加视轨

PeerConnection::CreateOffer()	   		// 创建offer sdp
PeerConnection::SetLocalDescription()	// 设置local sdp
send sdp to peer...						// 发送给对端
-----------------------------------------------------------
receive remote sdp...					// 收到对端的sdp
PeerConnection::SetRemoteDescription()	// 设置remote sdp



OnIceCandidate()					// ice candidate收集到了
send candidate to peer...			// 发送给对端
------------------------------------------------------------
receive remote candidate...			// 收到对端的ice candidate
PeerConnection::AddIceCandidate()	// 设置到pc中

2.2.2 创建PeerConnection

在InitializePeerConnection()函数中首先调用 webrtc::CreatePeerConnectionFactory()创建PeerConnectionFactory, 注意，创建的参数中需要提供[Audio|Video] [Encoder|Decoder]Factory()以提供视频编解码器

bool Conductor::InitializePeerConnection() {
  RTC_DCHECK(!peer_connection_factory_);
  RTC_DCHECK(!peer_connection_);

  // 创建PC factory
  peer_connection_factory_ = webrtc::CreatePeerConnectionFactory(
      nullptr /* network_thread */, nullptr /* worker_thread */,
      nullptr /* signaling_thread */, nullptr /* default_adm */,
      webrtc::CreateBuiltinAudioEncoderFactory(),
      webrtc::CreateBuiltinAudioDecoderFactory(),
      webrtc::CreateBuiltinVideoEncoderFactory(),
      webrtc::CreateBuiltinVideoDecoderFactory(), nullptr /* audio_mixer */,
      nullptr /* audio_processing */);

  if (!peer_connection_factory_) {
    main_wnd_->MessageBox("Error", "Failed to initialize PeerConnectionFactory",
                          true);
    DeletePeerConnection();
    return false;
  }

  // 创建PC
  if (!CreatePeerConnection(/*dtls=*/true)) {
    main_wnd_->MessageBox("Error", "CreatePeerConnection failed", true);
    DeletePeerConnection();
  }

  AddTracks();

  return peer_connection_ != nullptr;
}

随后在CreatePeerConnection()中创建PeerConnection, 创建的时候可以放RTCConfiguration 用来做当前PC进行一些选项配置： dtls, audio, video, jitter, ice等等（FEC的配置不在此处，在创建PeerConncetionFactory中）

bool Conductor::CreatePeerConnection(bool dtls) {
  RTC_DCHECK(peer_connection_factory_);
  RTC_DCHECK(!peer_connection_);

  webrtc::PeerConnectionInterface::RTCConfiguration config;
  config.sdp_semantics = webrtc::SdpSemantics::kUnifiedPlan;
  config.enable_dtls_srtp = dtls;
  webrtc::PeerConnectionInterface::IceServer server;
  server.uri = GetPeerConnectionString();
  config.servers.push_back(server);

  // 创建pc
  peer_connection_ = peer_connection_factory_->CreatePeerConnection(
      config, nullptr, nullptr, this);
  return peer_connection_ != nullptr;
}

如此，PC就创建完毕了

2.2.3 设置track

创建完PC后就开始添加track，在函数AddTracks()中

void Conductor::AddTracks() {
  if (!peer_connection_->GetSenders().empty()) {
    return;  // Already added tracks.
  }

  // 创建音轨
  rtc::scoped_refptr<webrtc::AudioTrackInterface> audio_track(
      peer_connection_factory_->CreateAudioTrack(
          kAudioLabel, peer_connection_factory_->CreateAudioSource(
                           cricket::AudioOptions())));
  // 添加音轨
  auto result_or_error = peer_connection_->AddTrack(audio_track, {kStreamId});
  if (!result_or_error.ok()) {
    RTC_LOG(LS_ERROR) << "Failed to add audio track to PeerConnection: "
                      << result_or_error.error().message();
  }

  // 创建视频源
  rtc::scoped_refptr<CapturerTrackSource> video_device =
      CapturerTrackSource::Create();
  if (video_device) {
    // 创建视轨
    rtc::scoped_refptr<webrtc::VideoTrackInterface> video_track_(
        peer_connection_factory_->CreateVideoTrack(kVideoLabel, video_device));
    main_wnd_->StartLocalRenderer(video_track_);

    // 添加视轨
    result_or_error = peer_connection_->AddTrack(video_track_, {kStreamId});
    if (!result_or_error.ok()) {
      RTC_LOG(LS_ERROR) << "Failed to add video track to PeerConnection: "
                        << result_or_error.error().message();
    }
  } else {
    RTC_LOG(LS_ERROR) << "OpenVideoCaptureDevice failed";
  }

  main_wnd_->SwitchToStreamingUI();
}

audio_track(音轨)的创建使用的audio_source(音频源)来自于peer_connection_factory_->CreateAudioSource(options) ，options可以配置是否做增益，噪声消除等，但是audio_source的这种创建方式就导致我们似乎无法自定义一个音频源？ 创建完的````audio_track通过AddTrack()``的方式添加到pc中

而video_track(视轨)的创建使用的video_source来自于自定义的CapturerTrackSource，不妨来看看，这个类的弯弯绕绕比较多, 如下，在Create() 中调用VideoCaptureFactory::CreateDeviceInfo()获取所有的视频捕获设备的信息，然后创建了一个VcmCapturer, 最后使用VcmCapturer创建CapturerTrackSource；CapturerTrackSourceoverride了source()函数， 调用者调用source()获取到video_source源也就是VcmCapturer, VcmCapturer override了OnFrame() 用来提供帧

class VcmCapturer : public TestVideoCapturer,
                    public rtc::VideoSinkInterface<VideoFrame> {
 public:
  static VcmCapturer* Create(size_t width,
                             size_t height,
                             size_t target_fps,
                             size_t capture_device_index);
  virtual ~VcmCapturer();
  // 提供帧
  void OnFrame(const VideoFrame& frame) override;

 private:
  VcmCapturer();
  bool Init(size_t width,
            size_t height,
            size_t target_fps,
            size_t capture_device_index);
  void Destroy();

  rtc::scoped_refptr<VideoCaptureModule> vcm_;
  VideoCaptureCapability capability_;
};


class CapturerTrackSource : public webrtc::VideoTrackSource {
 public:
  static rtc::scoped_refptr<CapturerTrackSource> Create() {
    const size_t kWidth = 640;
    const size_t kHeight = 480;
    const size_t kFps = 30;
    std::unique_ptr<webrtc::test::VcmCapturer> capturer;
    // 获取所有的视频捕获设备
    std::unique_ptr<webrtc::VideoCaptureModule::DeviceInfo> info(
        webrtc::VideoCaptureFactory::CreateDeviceInfo());
    if (!info) {
      return nullptr;
    }
    int num_devices = info->NumberOfDevices();
    for (int i = 0; i < num_devices; ++i) {
      // 创建一个VcmCapturer， VcmCapturer继承了VideoSinkInterface
      capturer = absl::WrapUnique(
          webrtc::test::VcmCapturer::Create(kWidth, kHeight, kFps, i));
      if (capturer) {
        // 使用video capturer创建一个CapturerTrackSource
        return new rtc::RefCountedObject<CapturerTrackSource>(
            std::move(capturer));
      }
    }

    return nullptr;
  }

 protected:
  explicit CapturerTrackSource(
      std::unique_ptr<webrtc::test::VcmCapturer> capturer)
      : VideoTrackSource(/*remote=*/false), capturer_(std::move(capturer)) {}

 private:
 // 提供源
  rtc::VideoSourceInterface<webrtc::VideoFrame>* source() override {
    return capturer_.get();
  }
  std::unique_ptr<webrtc::test::VcmCapturer> capturer_;
};

然后通过video_source创建的video_track也通过AddTrack() 的方式添加到peerconnection中

2.2.4 设置SDP

SDP的流程由于涉及到双方，所以放一个图直接说明

作为发起端， PeerConnection执行 CreateOffer()

void Conductor::ConnectToPeer(int peer_id) {
  RTC_DCHECK(peer_id_ == -1);
  RTC_DCHECK(peer_id != -1);

  if (peer_connection_.get()) {
    main_wnd_->MessageBox(
        "Error", "We only support connecting to one peer at a time", true);
    return;
  }

  if (InitializePeerConnection()) {
    peer_id_ = peer_id;
    // 创建offer
    peer_connection_->CreateOffer(
        this, webrtc::PeerConnectionInterface::RTCOfferAnswerOptions());
  } else {
    main_wnd_->MessageBox("Error", "Failed to initialize PeerConnection", true);
  }
}

offer创建完成后通过异步回调的方式调用Conductor::OnSuccess()；

调用SetLocalDescription()可以设置本地SDP ，同时把创建的SDP转成json发送给远端

void Conductor::OnSuccess(webrtc::SessionDescriptionInterface* desc) {
  //设置本地SDP
  peer_connection_->SetLocalDescription(
      DummySetSessionDescriptionObserver::Create(), desc);

  std::string sdp;
  desc->ToString(&sdp);

  // For loopback test. To save some connecting delay.
  if (loopback_) {
  	//如果是本地测试，remote answer sdp 和当前的是一样的
  	//可以提前设置？
  	
    // Replace message type from "offer" to "answer"
    std::unique_ptr<webrtc::SessionDescriptionInterface> session_description =
        webrtc::CreateSessionDescription(webrtc::SdpType::kAnswer, sdp);
    peer_connection_->SetRemoteDescription(
        DummySetSessionDescriptionObserver::Create(),
        session_description.release());
    return;
  }

  Json::StyledWriter writer;
  Json::Value jmessage;
  jmessage[kSessionDescriptionTypeName] =
      webrtc::SdpTypeToString(desc->GetType());
  jmessage[kSessionDescriptionSdpName] = sdp;
  SendMessage(writer.write(jmessage));
}

过一段时间后，对端会发来remote sdp， 通过SetRemoteDescriptioin()设置好远端的sdp，对于发起端而言，SDP就设置完毕了

void Conductor::OnMessageFromPeer(int peer_id, const std::string& message) {
  .....

  Json::Reader reader;
  Json::Value jmessage;
  if (!reader.parse(message, jmessage)) {
    RTC_LOG(WARNING) << "Received unknown message. " << message;
    return;
  }
 std::string type_str;
 std::string json_object;

  rtc::GetStringFromJsonObject(jmessage, kSessionDescriptionTypeName,
                               &type_str);
  if (!type_str.empty()) {
    // remote sdp come
    if (type_str == "offer-loopback") {
      // This is a loopback call.
      // Recreate the peerconnection with DTLS disabled.
      if (!ReinitializePeerConnectionForLoopback()) {
        RTC_LOG(LS_ERROR) << "Failed to initialize our PeerConnection instance";
        DeletePeerConnection();
        client_->SignOut();
      }
      return;
    }
    absl::optional<webrtc::SdpType> type_maybe =
        webrtc::SdpTypeFromString(type_str);
    if (!type_maybe) {
      RTC_LOG(LS_ERROR) << "Unknown SDP type: " << type_str;
      return;
    }
    webrtc::SdpType type = *type_maybe;
    std::string sdp;
    if (!rtc::GetStringFromJsonObject(jmessage, kSessionDescriptionSdpName,
                                      &sdp)) {
      RTC_LOG(WARNING) << "Can't parse received session description message.";
      return;
    }
    webrtc::SdpParseError error;
    std::unique_ptr<webrtc::SessionDescriptionInterface> session_description =
        webrtc::CreateSessionDescription(type, sdp, &error);
    if (!session_description) {
      RTC_LOG(WARNING) << "Can't parse received session description message. "
                          "SdpParseError was: "
                       << error.description;
      return;
    }
    RTC_LOG(INFO) << " Received session description :" << message;
    // 设置remote sdp
    peer_connection_->SetRemoteDescription(
        DummySetSessionDescriptionObserver::Create(),
        session_description.release());
    if (type == webrtc::SdpType::kOffer) {
      // 如果是offer类型， 创建一个answer作为回应
      peer_connection_->CreateAnswer(
          this, webrtc::PeerConnectionInterface::RTCOfferAnswerOptions());
    }
  }
  .....
  
}

而对于被发起端而言，仍然是通过OnMessageFromPeer( )接收到远端的offer sdp，如上，然后通过SetRemoteDescription()设置好该SDP后，使用CreateAnswer()创建answer，answer创建完毕之后会触发Conductor::OnSuccess()，需要将本地生成的SDP进行SetLocalDescription()，然后发送给对端。

2.2.5 设置 ICE

调用了接口SetLocalDescription()后，底层开始收集ICE Candidate，收集完成后回调Conductor::OnIceCandidate(), 解析之后写成json的形式发送给对方

void Conductor::OnIceCandidate(const webrtc::IceCandidateInterface* candidate) {
  RTC_LOG(INFO) << __FUNCTION__ << " " << candidate->sdp_mline_index();
  // For loopback test. To save some connecting delay.
  if (loopback_) {
    if (!peer_connection_->AddIceCandidate(candidate)) {
      RTC_LOG(WARNING) << "Failed to apply the received candidate";
    }
    return;
  }

  Json::StyledWriter writer;
  Json::Value jmessage;
  
  // 写成json 发送给对方
  jmessage[kCandidateSdpMidName] = candidate->sdp_mid();
  jmessage[kCandidateSdpMlineIndexName] = candidate->sdp_mline_index();
  std::string sdp;
  if (!candidate->ToString(&sdp)) {
    RTC_LOG(LS_ERROR) << "Failed to serialize candidate";
    return;
  }
  jmessage[kCandidateSdpName] = sdp;
  SendMessage(writer.write(jmessage));
}

对端在收到candidate之后，就执行AddCandidate()将candidate添加到PeerConnection中

void Conductor::OnMessageFromPeer(int peer_id, const std::string& message) {
	....
	
  std::string type_str;
  std::string json_object;

  rtc::GetStringFromJsonObject(jmessage, kSessionDescriptionTypeName,
                               &type_str);
  if (!type_str.empty()) {
    // handle sdp
    .....
  } else {
    // handle candidate
    std::string sdp_mid;
    int sdp_mlineindex = 0;
    std::string sdp;
    if (!rtc::GetStringFromJsonObject(jmessage, kCandidateSdpMidName,
                                      &sdp_mid) ||
        !rtc::GetIntFromJsonObject(jmessage, kCandidateSdpMlineIndexName,
                                   &sdp_mlineindex) ||
        !rtc::GetStringFromJsonObject(jmessage, kCandidateSdpName, &sdp)) {
      RTC_LOG(WARNING) << "Can't parse received message.";
      return;
    }
    webrtc::SdpParseError error;
    // 创建candidate
    std::unique_ptr<webrtc::IceCandidateInterface> candidate(
        webrtc::CreateIceCandidate(sdp_mid, sdp_mlineindex, sdp, &error));
    if (!candidate.get()) {
      RTC_LOG(WARNING) << "Can't parse received candidate message. "
                          "SdpParseError was: "
                       << error.description;
      return;
    }
    // 添加candidate
    if (!peer_connection_->AddIceCandidate(candidate.get())) {
      RTC_LOG(WARNING) << "Failed to apply the received candidate";
      return;
    }
    RTC_LOG(INFO) << " Received candidate :" << message;
  }
}

3.REF

1.WebRTC Native 源码导读（十四）：API 概览

2.WebRTC Native 源码导读（十）：视频数据 native 层之旅