Bobbie Smulders Freelance Software Developer

Remote Phone Control

After reading an article on Hack a Day about a machine that automatically pushes cellphone buttons, I set myself the challenge to create a more efficient way to tackle this problem.

My starting point would be the same: Have a camera monitor the phone’s screen, and a computer to control the phone’s keypad.

The difference I wanted to make was sending the key-presses using an RS232 or Bluetooth serial connection. This would eliminate the need for external hardware pressing the buttons, or the need for altering the cellphone by hot-wiring the buttons. This was a very simple task, all I needed was the AT Commands Online Reference. The phone I used (Sony Ericsson T610) supports the AT+CKPD command (Control Keypad), thus giving me the power to emulate the entire keypad over a serial connection.

The second difference would be on the graphical user interface. At first, I thought it would be best to control the phone using the PC’s keyboard, but whilst developing I found out it would be better to be able to click the phone’s actual buttons using your mouse. I rewrote the key-press part, so when a mouse-click on the video-stream is registered, it translates the mouseX and mouseY to a certain button on the phone, and sends the corresponding key-press using the Bluetooth connection.

The end result: A live video-stream of the phone on your computer and being able to use your mouse to click the phone’s buttons within this stream. A demonstration:

Download source code (for Processing)

Note: The original cellphone button pusher also is an amazing piece of art. I get why the over-engineered method is chosen, and it is quite astonishing how it all worked out. This article is purely to demonstrate how it can be done in a simpler way.