Skip to content

Instantly share code, notes, and snippets.

@msabwat
Last active June 6, 2018 21:13
Show Gist options
  • Save msabwat/413e7e77cf8a142f8f104e1805547f36 to your computer and use it in GitHub Desktop.
Save msabwat/413e7e77cf8a142f8f104e1805547f36 to your computer and use it in GitHub Desktop.
Welcome file

Magic pointer

This is a draft for a solution that enables a mobile device to control a mouse pointer. For convenience we will set the server as the phone, and the client as the device with the mouse.

Getting the device orientation

We have a webpage that we will use in the device to get the rotation around two axes. We use an EventListener for deviceorientation.

Alt

function getData()
    {
        var dx = Math.round(data.beta); //Rotation around the x axis
        var dy = Math.round(data.gamma); //Rotation around the y axis
        var msg =
            {
                delta_x: dx,
                delta_y: dy,
                id: clientID,//enable device/user verification?
                date: Date.now(),
            };
...
// Send the data to the client
...
        window.requestAnimationFrame(getData);
    }

It is advised not to directly update the data from the Event Listener.

==Issue== It creates lag if I want to bind the sensor to a viewable object on screen.

    window.addEventListener('deviceorientation', function(event)
                            {
                                data = event;
                                getData();
                            });

Sources:

https://developer.mozilla.org/en-US/docs/Web/API/DeviceMotionEvent

https://developers.google.com/web/fundamentals/native-hardware/device-orientation/

Sending the sensor data to the client

We will be using the Websockets protocol, with SSL.

Server side

url = protocol + "//" + window.location.host + window.location.pathname;
var ws = new WebSocket(url);

protocol set to "wss:" because we will be setting ssl up in the client.

To send the message, we add this next to where the data is updated :

ws.addEventListener('open', function(event)
                            {
                                ws.send(JSON.stringify(msg));
                            }
                           );

Client side

We launch can launch a node script or python script to create the connection, and receive all the messages sent by the server. the pointer won't be hooked directly to the sensor. I think we could set a range to watch for changes in dx/dy and update the pointer position only we enter that range. Refresh rate should be between 60 Hz and 165 Hz. This will be covered in the next section.

import logging
import asyncio
import websockets
import json
import ssl
import pathlib

log = logging.getLogger('websockets')
host = "localhost"
port = 443 #default for https/wss

async def hello(websocket, path):
    while 42: # while connection open, server will be closed later (server side)
        message = await websocket.recv()
        message = json.loads(name) # name is a dict.
        print(f"{name}")

ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ssl_context.load_cert_chain(pathlib.Path(__file__).with_name(host+'.pem'))

# Enable the connection
start_server = websockets.serve(hello, host, port, ssl=ssl_context)

log.setLevel(logging.INFO)
log.addHandler(logging.StreamHandler())

asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()

Sources:

http://websockets.readthedocs.io/en/stable/

https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_client_applications

Hacking the mouse pointer (not tested yet)

==Issue== I get the error below when I try the websocket connection outside the localhost.

OSError: [Errno 49] error while attempting to bind on address

I think it has something to do with the deployment method (bitballoon doesn't allow port 443 ? or refuses the first handshake?)

This can be done, with a C program (or python) using the uinput kernel module. We can open a file descriptor and use system calls to write the Mouse position directly.

Sources :

https://www.kernel.org/doc/html/v4.12/input/uinput.html

https://unix.stackexchange.com/questions/422698/how-to-set-absolute-mouse-cursor-position-in-wayland-without-using-mouse

http://tjjr.fi/sw/python-uinput/

Handling sensor changes (TODO)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment