This Django project, based on ROMP, WebSocket, Channels, WebRTC and Three.js, does two things:
- Use a webcam to drive a single 3D character in real time in a web page.
- Multiple people join the same chatroom, communicating with each other and seeing the 3D characters of themselves and others. Each part supports multiple 3D characters driven by multiple people.
The principle of the project can be find in the following paper: WOC: A Handy Webcam-based 3D Online Chatroom
Go to webcam module to have a fast try. You can drive a 3D character with your webcam.
Go to Characters module. Then you can chooese your favourite 3D character and start chatting with others as shown below.
In this module, you can chat with others using your 3D characters. (Your video will not transmit to others. The images and videos are just for the sake of showing the effect.)
WOC.mp4
# conda env
conda create -n woc python=3.9
conda activate woc
# pytorch-gpu
conda install pytorch==1.11.0 torchvision cudatoolkit=11.3 -c pytorch
# simple romp
pip install norfair==1.0.0
pip install --upgrade setuptools numpy cython
pip install --upgrade simple-romp==1.0.5
romp --mode=video -i=/path/to/video.mp4 -o=/path/to/results # test if romp run successfully
# django
pip install django==3.2 channels 'uvicorn[standard]' channels_redis
sudo apt install docker
sudo docker run -p 6379:6379 -d redis:5
# download characters under the static folder
cd static
wget https://github.com/yanchxx/WOC/releases/download/v1.0/characters.zip
# nginx
sudo apt install nginx
sudo vi /etc/nginx/nginx.conf # add a server under http { ... } in the file
sudo nginx -s reload# /etc/nginx/nginx.conf
server {
listen 80;
server_name localhost;
charset utf-8;
location /static {
alias /path/to/your/static; # /home/yanch/WOC/static
}
location / {
proxy_pass http://127.0.0.1:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}uvicorn cdwd.asgi:application --workers 3 --port 8000Now you can visit http://localhost to see the website.
First, enable SSL in the nginx configuration file.
# /etc/nginx/nginx.conf
server {
listen 443 ssl;
server_name www.your_domain_name.com your_domain_name.com;
ssl_certificate /path/to/your/cert
ssl_certificate_key /path/to/your/key
ssl_session_timeout 5m;
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
charset utf-8;
location /static {
alias /path/to/your/static;
}
location / {
proxy_pass http://127.0.0.1:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}Second, run the coturn server to enable the WebRTC connection.
sudo apt install coturn
sudo vi /etc/turnserver.conf
sudo systemctl start coturn# /etc/turnserver.conf
realm=your_domain_name.com
server-name=your_domain_name.com
user=username:password
cert=/path/to/your/cert
pkey=/path/to/your/key
listening-port=3478
tls-listening-port=5349
fingerprint
lt-cred-mech
total-quota=100
stale-nonce=600
cipher-list="ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384"
proc-user=turnserver
proc-group=turnserver
log-file=/var/log/turnserver/turnserver.log
simple-logModify the value of config.iceServers at line 312 of the chatroom.js.
Now you can visit https://your_domain_name.com to see the website.