Menu

Subscribe

Using a Phantom 2 DJI Controller in AirSim

Flying a drone in AirSim requires a controller. One could use a simple game controller (Playstation or Xbox both work, as per these instructions), but this is no ideal solution. I have tried this multiple times myself: these controllers are really 'noisy' and make it very difficult to fly a drone carefully. Great tools to test the entire AirSim environment and for some first flights, but as soon as you are ready for 'serious work' you should really consider getting an RC controller of sorts. Not only are they a lot more precise, the longer sticks also offer a wider range to more subtly control the AirSim drone.

Seeing as you are interested in drone simulations, odds are that you have a DJI controller lying around. Wouldn't it be great to use that as a controller for AirSim? My thought exactly when i looked at the Phantom 2 controller (NDJ6) I had lying around. After that thought, I found this very interesting Github project called mDjiController by Matsemann. It does exactly what I hoped it would do: read the serial output from the Phantom 2 controller and send it to vJoy, simulating a joystick in Windows.

In what follows, I will describe how I currently run my setup on a Windows machine. This guide assumes you have already configured a working Unreal project with the AirSim plugin. If this is not the case, the AirSim documentation provides an excellent starting point when it comes to installation or setting up your first project. Linux users: I have not tested my setup in a Linux environment, but there's a good chance you can get it working just as easily as on Windows. If you do, please use the comment section to keep us all updated!

Driver installation

First off, we'll need the vJoy device driver, a piece of sofware that describes itself as 'bridging the gap between any device that is not a joystick and an application that requires a joystick'. Grab the installer from the official website and get it installed on your machine.

Next, let's get the software we need to talk to our 'not a joystick device'. This task is fulfilled by the simplified, wannabe driver [sic] mDjiController, found on GitHub and written by Matsemann. I made a humble contribution to this project to unleash the full potential of the Phantom 2 controller, adding functionality for all the buttons and camera scroll wheel. You can opt to git clone the repository and build the driver yourself, or simply grab the 64-bit Windows executable from release v1.0 on my fork.

To connect the controller to our computer over USB, we'll need the DJI USB driver. If previous link should stop working, you can find it on the Phantom 2 downloads page.

To wrap up, a final piece of software we need to get is QGroundControl. This application will bridge the gap between our simulated joystick and the simulated PX4 flight controller. Get the installer for your platform and install the software if you don't already have it. Granted, we need a lot of different things installed but fear not: we're getting close to flying fun!

Trying it out in AirSim

Ok, time to get everything rolling in AirSim. There is a specific order in which the different components required must be launched, in order for everything to work correctly. Having the mDjiController running before starting QGroundControl for instance, will prevent QGroundControl from starting at all (see screenshot below). I reported this erroneous behaviour in a GitHub issue, no response from their team so far though.

In any case, this is the specific procedure that is guaranteed to successfully start the entire setup.

  • Launch Unreal Editor project
  • SITL users: start the Posix PX4 simulation
  • Start the Unreal Project in 'AirSimGameMode'
  • Start QGroundControl
  • Initialize mDjiControl driver
  • Fly!

Ok, in what follows we'll see how previous steps look like at the hand of actual screen shots of the different applications. First off, let's start our Unreal Editor AirSim project. I'm assuming you have followed the excellent documentation available at the AirSim Github page to set up a custom Unreal project, so I won't go over that here. Let's open said project, you'll notice I already have the Game Mode set to AirSimGameMode.

The next step is optional for those that are using a PX4 connected to their machine over USB. I'm not as fortunate to have one lying around, so I fire up my Bash On Windows (Windows users on a version prior to Windows 10 will have to run Ubuntu in VirtualBox or a different virtual machine), navigate to the folder where you have compiled the PX4 stack and execute following command:

./buildposixsitl_default/src/firmware/posix/px4 ./posix-configs/SITL/init/ekf2/iris

There, be it SITL or HITL user: we both have a PX4 stack waiting for input from AirSim ;)

On to the next: let's start the Unreal AirSim simulation, and verify that it connects correctly to the PX4.

Ow, so close now! Let's fire up QGroundControl and verify it is connected (you should see the extended menu when you click the configuration cogwheels up top).

Now is he time to start the mDjiController mDji.exe executable. First specify the COM port your controller is connected to (you can find that out in the Device Manager screen). Some configurable options come up, just hit enter twice to confirm the default values unless you want to run a custom vJoy configuration.

If all goes well, the driver should connect to the DJI controller over the serial port and start outputting values as can be seen below. Don't you just love a terminal blasting digits at high rate? If you didn't accept the default values and disabled logging one step ago maybe it's not for you ;)

In QGroundControl, let's select the Joystick page.

This is where you can calibrate and configure your controller. As can be seen from the screenshot, my controller still needs to be calibrated (all dots should be in the middle when the sticks are released). The calibration process is pretty straightforward.

Second item to be noticed here, is the Button actions submenu. This is where we can define what the Phantom 2 toggle levers will trigger in the PX4 flight controller. I have mine set up so that the left toggle arms or disarms the drone, and the right controls the different flight modes.

Believe it or not: time to fly! Arm the drone using your newly configured controller and have at it. Isn't this environment breathtakingly beautiful?

Conclusion

In this article, I have illustrated how to use a Phantom 2 controller to fly a drone in an AirSim simulation on a Windows machine. I haven't tested yet if the proposed mDjiController also works in a Linux environment. If you happen to test this, please use the comment section below to report your findings.

If anything was unclear or incomplete, please let me know. If you liked this article and it helped you in your journeys, I'd be happy to find out! I will update the article with every suggestion that comes up.

As a side remark: I have only verified the proposed approach with a DJI Phantom 2 controller. If other DJI controllers can be used the same way, I cannot say yet. I also own a DJI Matrice 100 controller that I am trying to set up, without success yet however. If you do find something out, we have created an issue here to join efforts in getting these or the M600 controllers working too. I have contacted DJI to find out if we can get stick values over USB, let's see what they report back.