CrugoWebRTC.md

Crugo WebRTC - Phase 1

For phase 1, only 2 users can video / audio chat at the same time in Crugo. This will eventually be developed to allow multiple user video / audio chat.

Brief Description - UX / UI

User A goes to the direct tag of User B and presses the call button. User A is presenting with a calling UI. User B received that call and is shown the being called UI. User B Then presses answer. At this stage both User A and User B are taken into the video chat UI. WebRTC does its thing and the call is initialised.

How it Works

Requesting a call

User A requests to start a call with user B. User A emits the following socket message to the server.

Socket message = “webrtc.call” Start

{
"caller": "myUserID"
”tag”: tagID, 
“receiver”: personWhoImCallingId, 
“payload”: { 
	“type”: “call” 
	"action": "start"
	} 
}

When this socket is sent to the server, the client listens for inCall boolean on the emit acknowledge. If the user is in the User busy step, the socket isnt sent to user B and the User busy is show by user A

Expected result User A: UI showing user 1 calling user B with the option to hang up

Expected result User B: UI Showing user 1 calling user B with the options to cancel or accept

Socket message = “webrtc.call” Cancel

{
"caller": "myUserID"
”tag”: tagID, 
“receiver”: personWhoImCallingId, 
“payload”: { 
	“type”: “call” 
	"action": "stop"
	} 
}

Expected result User A: UI Dismisses, A new chat message for missed call is posted in the tag

Expected result User B: UI Dismisses, A new chat message for missed call is posted in the tag

User B Received that call, and chooses whether or not to answer.

Socket Message to answer = “webrtc.answer”

{
"caller": "personWhoStartedTheOriginalCallId"
”tag”: tagID, 
“receiver”: "myUserID", 
“payload”: { 
		“type”: “answer”, 
		“action”: accept 
	} 
}

Expected result User A: UI dismisses and present the video calling View

Expected result User B: UI dismisses and present the video calling View

After the call is accepted, both user A and user B should now be in the User Busy state so that they cant receive calls from any other users

Socket Message to decline = “webrtc.answer”

{
"caller": "personWhoStartedTheOriginalCallId"
”tag”: tagID, 
“receiver”: "myUserID", 
“payload”: { 
		“type”: “answer”, 
		“action”: decline
	}
}

Expected result User A: UI displays User 2 busy

Expected result User B: UI dismisses

(Both users now taken to full video chat UI and WebRTC is started)

Once both users are in the video calling UI, one of the two users is responsble for calling the other. This happens with the following

Socket Message to start call = “webrtc.message”

{
“payload”: { 
		“type”: “sdp-offer”, 
		“data”: {
			"type": "#sdp_candidate#",
			"sdp": "#sdp_descrption#"
		},
		"from": "#user_ID#",
		"to": "#person_to_call_ID#",
		"callerID": "#the_id_of_the_user_making_the_call#"
		
	}
}

This message will be sent over the socket and sent to the other user. At which point that will send back an answer.

Socket Message to answer call = “webrtc.message”

{
“payload”: { 
		“type”: “sdp-answer”, 
		“data”: {
			"type": "#sdp_candidate#",
			"sdp": "#sdp_descrption#"
		},
		"from": "#user_ID#",
		"to": "#person_to_call_ID#",
		"callerID": "#the_id_of_the_user_making_the_call#"
		
	}
}

Once the call has been answers and throughout the offer / answer phase, Ice candidates are also exchanged across the socket. The format for these are below.

Socket Message to ice candidate = “webrtc.message”

{
“payload”: { 
		“type”: “ice”, 
		“data”: {
			"candidate": "#candidarte_sdp#",
			"sdpMid": "#candidate_sdpMid#",
			"sdpMLineIndex": "#candidate_sdpMLineIndex#"
		},
		"from": "#user_ID#",
		"to": "#person_to_call_ID#",
		"callerID": "#the_id_of_the_user_making_the_call#"
		
	}
}

Once the call has been accepted and the ice candidates have been exchanged, the call will begin.

During the call we have an additional socket message than can be sent to inform the other user of changes. This is used for when one of the user mutes their video and when the orientation of one of the devices changes.

Socket Message to send options = “webrtc.message”

{
“payload”: { 
		“type”: “options”, 
		“data”: {
			"video": "#video_muted#",
			"isPortrait": "#is_device_portrait#",
		},
		"from": "#user_ID#",
		"to": "#person_to_call_ID#",
		"callerID": "#the_id_of_the_user_making_the_call#"
		
	}
}

Throughout the call, both clients also ping the server every 30 seconds

This is an empty socket messages with key = "webrtc.incall"

Finally, to hang up the call one of the user sends a socket message to end the call

Socket Message to send options = “webrtc.message”

{
“payload”: { 
		“type”: “hang”, 
		"from": "#user_ID#",
		"to": "#person_to_call_ID#",
		"callerID": "#the_id_of_the_user_making_the_call#"
		
	}
}

Once the hang is received by the other client, the ui is dismissed. At this point we expect there to be a new chat message in the tag showing the length of the call. Also both users should now be out of the busy state.

https://www.draw.io/?lightbox=1&highlight=0000ff&edit=_blank&layers=1&nav=1&title=Untitled%20Diagram.xml#R7Vxbc6M2GP01fmwGgYTtx9za7UzbyWxm2%2BaRGNlmg5EH5HW8v74CxEUXHBKEbGe6LzEfQkhH57tK7MS73bz%2Blgbb9Z8kxPHEdcLXiXc3cV0AgMv%2B5JIDlzjzWSlZpVHIZY3gMfqJq4ZcuotCnAkNKSExjbaicEGSBC%2BoIAvSlOzFZksSi2%2FdBiusCB4XQaxK%2F4lCui6lM9dv5F9wtFpXbwb%2BvLzzHCxeVinZJfx9E9dbFv%2FK25ug6otPNFsHIdm3RN79xLtNCaHlr83rLY5zcCvYyud%2B7bhbjzvFCe31AORL9SOId7gaczEyeqjQKOaD8yeciXezX0cUP26DRX53zwjAZGu6idkVYD95dzil%2BLVzUKCeKuMQJhtM0wNrwh%2BowOHsQdX1vlmKKeKydWsZkMOFAV%2F%2BVd11AwH7wVHoQASA80dEA4ivAQRCE4BUqLbmj0OmLvySpHRNViQJ4vtGeiMi1EKDzTg9%2FMvlxcVTfnGF2OV3TOmBW4NgRwkTNb3%2FQchWQDQfxHE82ZjJLl3wVnOuvTRIV5g3m8%2F1uKc4Dmj0Q%2Bx%2FGIoqrZ6YiXP9mA3lJox%2BsJ8rWkyuFD2nsoS9Q2gnLQqjFxWxTnEW%2FQyeiwY53FsSJbSYBLqZoDsmCeJolTDBggGHUybIaRoxQ3jNb2yiMCwWNA6ecXxTm7dbEpO0eG9l4I4QnZtpPpLG%2BgkL5jj6peBdOVdA0AGulL0Xinf8kEPQ9PqLK3bqih2Q5TJjVJHXuR5dz6Wfj6NAhc40KnQSBQLOzJYGVcFBS4P%2BIp9OCzoMUsVX50pkrG9EDQAUOoUjaEFFntbq3eyyg%2Bpa12TzvMusuFWARL%2FqOqpjnWn86syAW60iYkNG4TWiLZvArp5Oby6mGn%2FrW7IW1ctbdPuWMd12netcFYM4Zr7X4SLVlzZAAzsB3lRkItDEvLWsTUXfSIQ3NcnFkVk1NGLTW0AohdiuHDqXLOZPDbOEc4Waj2uWBDZ0LOyi8%2B33s6NlnaBboSVARk3kyQwhcJDGEk4tWcL67S2%2B3UXZJsqy8yAZ9E5JMmg2OOd%2BGLS8cItvJ8l7eYguxe3QWtxevb7Fv%2FtNlNveW%2BaGo2RVjHfxkoeYA7Jh5dHcqGbc2ZdvycneK5W2rQJAUgFvatXOusoCNbHSm%2BlVRlPygqtMKCFJriDLKI4lUf8kS4e3AZCRBLIGY9cbC2N3pCKadV8GPI0xAfaKAJ7C1a94gdk7Mm5Q2J9lSjZNOKWS%2BNTqjTQF3BHVGyqQqVxMwut836RR1zDI1jVEOuqBieLH9FmoY5hsXRWnFpj63YFSNjBRcOeSqZaLAeUElERB6ciDUkdev4yDrVNwaDXj5azOAQNpwHDqHB%2BXO7C9I7RnP8oRf7h8Ci4oOwVgaIGjA2TJgtTFUPPpKZ%2FCcXtB0yhIVp3e2qz5RG%2F6bd3ml4kiHahKoZfBvaFeuIN7joS%2F7JUMcs81mvJfJt7AtYc3UHOz6yTb53GTHDOdsC7v%2B6rKj1WXB9BopD4yBbuANEzBmYysSZVXN8Z5eaBionO9WOAtVSlpOYz3Jb3sm6QjI7Q0WwztUacSU0sbZOZxhhDte0MZ3t%2FXq2WQuyg7o0KpCz9IQDMVDDUs7NDLC64RASnW8TUQQw3CrhGEVWf8FX%2FPzzd%2BIoTlaN4qwt7bvqYDcdu67knOxm7JyFO3jFTv079k1FkWOuZsPuhEqiXuc8rPTsUIVhtw9elNJHbRt2IEJeOEehYB3lsxkvfCqwpPZwVI3r0Q2w%2BvAHmq63lIcYYTcRPp7wjvNftALSe%2BJdvd9kz3gHy5HqhJekbUePVQwieEWMpq5jYRhj3OlY9Zhh9iU3UHP72TGlUfiGs5l21hX6MKJLXrm%2Bq%2B16j6kpGcv2VUp0fbDzeqUDWqncfhLjjeRDJRoKr04237QvUY4nnllEiOKazGmchTTaCBQ7CTczkCy0%2FZSJYTdXzpYL6kAdWM8ktQHoXZqtSzWOGFM5F2rsYVj1bhrSj%2B%2F4Gk7vWRQqU6JbBjFoACgNVap3M1m9aCB5xGbAq58zqZxbB3WA%2B5im70%2BsjmcuIB%2BfyGpzJ7tPoTUo%2FiFB8Bfh54oVTe051kHA%2Feyy1bQc3HMp2Bgp0UC0llICTb9r4pFpJSmbHqVkiumYLjKZYyQbH9e1Msdtl8uF82b%2F57BO%2F%2BPw%3D%3D