Modified FreeFlight 2.4.10 APK with game controller support

iOS Apps and Games for the AR.Drone. General Android discussions.

What should be the X button "secret feature"?

absolute control toggle
3
13%
return home
13
54%
map view
0
No votes
flat trim
0
No votes
emergency
7
29%
other (post your suggestion below)
1
4%
 
Total votes: 24

Darklo
Just flips over
Posts: 59
Joined: 04 Jul 2013, 21:01
Drone Type: AR.Drone 2

Modified FreeFlight 2.4.10 APK with game controller support

Post by Darklo » 08 Feb 2015, 19:58

TL/DR version:
FreeFlight 2.4.10 app modified with controller support and hover lock toggle, plus several fixes.


Get the APK here: viewtopic.php?f=75&t=8262&p=81189#p81188

(updated 13th April 2017)



The only reason Parrot keeps selling ARD 2.0 at this late stage in its lifecycle is because it fills the price gap between the Bebop and Rolling Spider.
It should be pretty obvious that there aren't going to be any more updates, neither firmware nor an app.
Director Mode on Android has been "coming soon" for what, 16 months now? At this point it should be rather "not coming, ever", and the FreeFlight app is now more or less in abandonware territory. But even though there are a number of excellent alternatives, if it could be improved, it could offer a lot of value.

So I thought it would be a good idea for the community to take over the development of the FreeFlight app and fix some of the longstanding bugs and introduce a couple of new features. Thanks to the amazing tools — smali and apktool — we're now able to freely modify the app and do (almost) anything. Editing smali code is very different from normal Java programming, and is a lot like assembly — you deal with registers instead of variables and control flow is affected by goto's and if-zero or if-not-zero jumps. But you can learn a lot about internal workings of Dalvik VM, and I actually had a lot of fun breaking the app apart and modifying it to my liking.
  • One feature that is an absolute must is hardware controller support (see controller mapping below). As it turns out, FreeFlight fully supports game controllers, as long as it runs on Nvidia Shield. There is literally a function isNvidiaShield() that is called every time a joystick or button event is processed:

    Code: Select all

        private boolean isNvidiaShield() {
            return Build.MODEL.equals("SHIELD") ? true : false;
        }
    As you probably guessed, if your device doesn't happen to be Shield, joystick movements and button presses are simply ignored. Well, that is simply ridiculous and our device choices should not be limited by Parrot's exclusive partnerships.
    Fortunately, it's also ridiculously easy to patch. The easy way, of course, would be to write "return true", but that wouldn't be very interesting. Better solution is to detect whether controller is plugged in when app is launched. Even better is to watch for hardware changes at runtime and modify the return value when controller is connected.
    As two added bonuses, you get a different splash screen if you start the app with controller; the touch screen controls react to controller presence and disappear as necessary (it's only a transparency change, so it's still possible to fly using touchscreen with controller connected).
  • Even if you have a Shield, you've probably noticed that joystick deadzone is very large (0.3 out of 1). If you push the stick slightly, nothing happens. If you push more, the drone suddenly lurches forward. As a result, controlling the drone becomes more difficult than it should be.
    Again, this is easy to patch. In this version hardcoded deadzone is removed, and the code relies on the value provided by Android. In addition, joystick response curves are now non-linear, improving precision at low speeds.
  • Another issue that also affects Shield owners is that the left stick is disabled in accelerometer mode (when you press and hold the shoulder button). This is correct if pitch/roll stick is on the left, but in left-handed mode this prevents you from controlling yaw and altitude, so this artificial limitation is also patched out.
  • Unlike touchscreen controls, where you can keep the finger on the roll/pitch button as the drone flies level, there is no equivalent for hardware controller. Well, here is a first: FreeFlight app with hover lock toggle! Click the pitch/roll thumbstick (left or right, depending on your settings) and the drone will no longer overcorrect as you release the stick, instead it will continue to glide smoothly. Click the stick again to turn hover lock back on. You'll see a brief message "Hover ON" (or "OFF"), so it's easy to tell which mode is active.

Here is a complete list of changes:
  1. increased max zoom in map downloader from 16x to 19x
  2. reduced joystick dead zone from 0.3 to 0.075
  3. increased param:video:codec_fps from 25 to 30 and param:video:max_bitrate from 1500 to 4000
  4. keep gas/yaw stick active in accelerometer mode (when shoulder button is pressed, in left-handed mode)
  5. added listener to detect controller presence and toggle controller mode (isNvidiaShield()) at runtime
  6. restored data collection opt-out setting
  7. added hover toggle to pitch/roll thumb stick
New in 20150312 version:
  1. fixed blurry photos in the photo/video viewer
  2. improvements to no hover mode
  3. fixed controller not working after being disconnected and reconnected again
  4. better handling of left-handed mode
  5. despaghettification of joystick code
  6. nonlinear joystick response curves
  7. removed hardcoded joystick deadzone, now the code uses only the value provided by Android
  8. added combined yaw toggle to gas/yaw thumb stick
  9. fixed restarting of com.parrot.freeflight.academy.services.restoremedia.RestoreFlightMediaService when exiting the app using back button

Controller mapping:
Since the Shield controller uses standard gamepad layout, every compatible controller should work. I tested with a Wikipad.
nvidia-shield-mapping copy.jpg
Please note that if you use "left-handed mode" (equivalent to RC Mode 2), both sticks and shoulder buttons are swapped. In that case, L1 (left shoulder) switches cameras and R1 (right shoulder) enables accelerometer. The easier way to think is that whatever stick you use to pitch/roll your drone, the shoulder button on the same side switches accelerometer control.
Triggers are unaffected, LT is video and RT is photo.

You may notice in the above image that Parrot labeled the X button as a "secret feature" in their original blog post. Well, the Shield owners have discovered that the button does in fact nothing. And now after decompiling the code, I can conclusively say that there is indeed no secret feature.
Although it's a small thing, Parrot lied to the customers. The good thing is that we can implement whatever we like, so I invite everyone to vote what the secret feature should be!


Patch notes:
As I don't have access to Parrot's private key, the APK is signed with the key from official SDK. As a result, instead of replacing the official app with patched version, you'll have to uninstall the official app first.

If you want to patch the APK yourself and make sure that I didn't bundle any malware, you can. The attached zip contains the shell script that does decompilation, patching, rebuilding and signing the APK. You should place the original named com.parrot.freeflight.apk in the same folder.

Requirements:
  • Linux or OS X (Windows users will need to get patch.exe from diffutils and translate the shell script to a .bat file)
  • Oracle JDK (JRE won't work)
  • apktool 2.0.0rc4
  • zipalign tool from Android SDK (not strictly necessary, but recommended by Google)
You do not have the required permissions to view the files attached to this post.
Last edited by AR.Tommy on 13 Apr 2017, 13:51, edited 3 times in total.

EricTN
Ready for take off
Posts: 26
Joined: 08 Feb 2015, 14:14
Drone Type: AR.Drone 2

Re: Modified FreeFlight 2.4.10 APK with full controller supp

Post by EricTN » 20 Feb 2015, 14:19

Good job, that sounds really awesome!
You should continue work with patching this app, e.g. make control smoother when on joystick mode (if you know what i mean)!
Is it compatible with the ps3 controller??

Darklo
Just flips over
Posts: 59
Joined: 04 Jul 2013, 21:01
Drone Type: AR.Drone 2

Re: Modified FreeFlight 2.4.10 APK with full controller supp

Post by Darklo » 21 Feb 2015, 00:38

Hey Eric, I'm glad you like it :D

Regarding ps3 controller, I don't own one, so I can't say for sure. I remember reading somewhere that your device needs to be rooted, but that information might be outdated. Basically, if your controller works in other apps or games natively (i.e., without emulating touchscreen), it should work. (It would work with touchscreen emulation too, but then the official app would work just as well).

Making control smoother is at the top of my list. I'm thinking of using getFlat() method to determine deadzone instead of using a hardcoded value and making the output non-linear. It should improve precision and smoothness at low speeds.
This picture should explain it better:
joystick_curves.gif
Figure 1: joystick response as implemented by Parrot
Figure 2: what it should look like
You do not have the required permissions to view the files attached to this post.

EricTN
Ready for take off
Posts: 26
Joined: 08 Feb 2015, 14:14
Drone Type: AR.Drone 2

Re: Modified FreeFlight 2.4.10 APK with full controller supp

Post by EricTN » 21 Feb 2015, 20:38

I can use my ps3 controller w/bluetooth as a native gamepad, by using Sixaxis Controller from play store, but yes, you need root for this!
Also i can remap the buttons via the app!
I think the ios version of freeflight already has this feature built in!

Carl190_1
Just flips over
Posts: 73
Joined: 07 Apr 2013, 20:51
Drone Type: AR.Drone 2

Re: Modified FreeFlight 2.4.10 APK with full controller supp

Post by Carl190_1 » 21 Feb 2015, 22:14

Great I'm really happy to see this work of yours. Thank you.
I just received my Nvidia Shield yesterday, purchased used online.
Spent yesterday working with trying to learn how to use it as a flight controller since I already have a AR Drone Version 2.0 and am planning to get a Bebop.
Biggest problem I had was finding a picture of the mapping for the Shield. Had to screen capture their ad just as you posted here to use.
Today flew first time with Shield indoors in living room. Indoor hull, placed clear plastic floor protector on carpet so sonar would bounce and white paper plate underneath so bottom camera
would have some contrast reference points. Flew great. Quit after just one minute.

Questions:
1. I don't get/understand this Accelometer mode; was it ever explained anywhere. I can never find any help from Parrot!

2. So I just un-install the existing Parrot apk and install yours; Right?

Thanks,
Carl

Darklo
Just flips over
Posts: 59
Joined: 04 Jul 2013, 21:01
Drone Type: AR.Drone 2

Re: Modified FreeFlight 2.4.10 APK with full controller supp

Post by Darklo » 22 Feb 2015, 00:33

1. Accelerometer mode is just like control with a normal phone/tablet where you direct the drone by tilting your device, but instead of touching the screen, you hold the shoulder button. Of course, you can still use touchscreen controls too, but that can be really unwieldy on a Shield.

The difference between the official and my patched version is that for some unknown reason Parrot decided to disable both joysticks in accelerometer mode, so you can't control yaw/altitude and pitch/roll simultaneously. I fixed that "feature" and now yaw/altitude stick can be used while tilting the whole device to control roll and pitch.

2. Yes, that's it.

Carl190_1
Just flips over
Posts: 73
Joined: 07 Apr 2013, 20:51
Drone Type: AR.Drone 2

Re: Modified FreeFlight 2.4.10 APK with full controller supp

Post by Carl190_1 » 22 Feb 2015, 02:10

O.K. found that mode in Quick Start Guide. I just never tried it.

I thought that I had just recently read while doing my research on the Nvidia Shield that it didn't have the required device/sensor for that action.

Just Goggled to verify: 3-axis gyro & 3-axis accelerometer.
It has both! My bad (but that's good!).

EricTN
Ready for take off
Posts: 26
Joined: 08 Feb 2015, 14:14
Drone Type: AR.Drone 2

Re: Modified FreeFlight 2.4.10 APK with full controller supp

Post by EricTN » 22 Feb 2015, 12:44

Darklo wrote:Hey Eric, I'm glad you like it :D

Regarding ps3 controller, I don't own one, so I can't say for sure. I remember reading somewhere that your device needs to be rooted, but that information might be outdated. Basically, if your controller works in other apps or games natively (i.e., without emulating touchscreen), it should work. (It would work with touchscreen emulation too, but then the official app would work just as well).

Making control smoother is at the top of my list. I'm thinking of using getFlat() method to determine deadzone instead of using a hardcoded value and making the output non-linear. It should improve precision and smoothness at low speeds.
This picture should explain it better:
joystick_curves.gif
Figure 1: joystick response as implemented by Parrot
Figure 2: what it should look like
That's what i mean!
(sorry, i'm not good with that kind of stuff)
just.png
You do not have the required permissions to view the files attached to this post.

EricTN
Ready for take off
Posts: 26
Joined: 08 Feb 2015, 14:14
Drone Type: AR.Drone 2

Re: Modified FreeFlight 2.4.10 APK with full controller supp

Post by EricTN » 22 Feb 2015, 18:58

I flew w/my ps3 controller , and when i pressed the L3 button (click on the left joypad, to turn out ground stabiliation) my drone freaked out and started to spin. After a 360° everything was fine, stabilisation was disabled. I love the nVidia shield startup image, and physical controller gives me more free space at the screen, so i can easily keep an eye on the altitude and speed, and makes me feel more confident!

Carl190_1
Just flips over
Posts: 73
Joined: 07 Apr 2013, 20:51
Drone Type: AR.Drone 2

Re: Modified FreeFlight 2.4.10 APK with full controller supp

Post by Carl190_1 » 05 Mar 2015, 18:07

Is there a way of noting in your patch file that this is a patched file; not the original so I won't forget?

This would be great to post on other Parrot AR Drone forums.

Could you also program a patch file for the Parrot Bebop FreeFlight 3 APK for the Shield?

Thanks,
Carl

Darklo
Just flips over
Posts: 59
Joined: 04 Jul 2013, 21:01
Drone Type: AR.Drone 2

New update 20150313

Post by Darklo » 13 Mar 2015, 21:44

New update
Download APK here: https://db.tt/3dLyJPHi


List of changes:
  1. fixed blurry photos in the photo/video viewer
  2. improvements to no hover mode
  3. fixed controller not working after being disconnected and reconnected again
  4. better handling of left-handed mode
  5. despaghettification of joystick code
  6. nonlinear joystick response curves
  7. removed hardcoded joystick deadzone, now the code uses only the value provided by Android
  8. added combined yaw toggle to gas/yaw thumb stick (*)
  9. fixed restarting of com.parrot.freeflight.academy.services.restoremedia.RestoreFlightMediaService when exiting the app using back button
So turns out previously I didn't fully understand the original joystick code for Shield. It was such a spaghettified mess that was not only difficult to understand, but in several places was just wrong. For example, if you pressed and held the accelerometer shoulder button, it would disable the left stick whether you used left-handed mode or not (previously I incorrectly said that the throttle/yaw stick was disabled). I don't know whether it was written by Nvidia's or Parrot's engineers, but they all should have known better.
Also, they in fact used the deadzone (or "flat area" as Google documentation calls it — the range of joystick values near centre where it's considered to be zero) provided by Android (InputDevice.MotionRange.getFlat() method), but on top of that they added their own deadzone of 0.3. It appears what they intended to do was "snapping" to axes, so, for example, if joystick position is 0.5 forward and 0.25 left, it ignores the latter value and the drone goes straight forward. However, it also caused all values ≤ 0.3 to be ignored, effectively adding another deadzone on top of system provided one.

In my previous patch I only changed the 0.3 value to 0.075, but didn't account for the system provided value (for example, on my Wikipad it is 0.125 for both analogue sticks). The new code handles it better and in addition you get non-linear response curves, so the controls are less sensitive near the centre.
Here's a pretty graph of both old and new for comparison:
Image

(*) Also in this version I added combined yaw mode that is toggled by clicking the gas/yaw stick. As Parrot describes it in the documentation, "This mode includes a complex hybridation of the phi parameter to generate complete turns (phi+yaw)." And "This is intended to be an easier control mode for racing games." It was in fact already half-enabled when moving both sticks at the same time, but the corresponding configuration flag for it was not turned on.
Give it a try if you're feeling adventurous, but in my short tests so far it sometimes yawed in the wrong direction, so it's disabled by default.
You do not have the required permissions to view the files attached to this post.
Last edited by Darklo on 13 Mar 2015, 23:58, edited 1 time in total.

Darklo
Just flips over
Posts: 59
Joined: 04 Jul 2013, 21:01
Drone Type: AR.Drone 2

Re: Modified FreeFlight 2.4.10 APK with full controller supp

Post by Darklo » 13 Mar 2015, 22:58

EricTN wrote:I flew w/my ps3 controller , and when i pressed the L3 button (click on the left joypad, to turn out ground stabiliation) my AR.Drone freaked out and started to spin. After a 360° everything was fine, stabilisation was disabled.
Hm. The 360° does sound like magnetometer calibration to me. The drone sometimes does it automatically even if you don't click the button in settings.

Try the new version, it has some improvements to "no hover" mode.
Carl190_1 wrote:Is there a way of noting in your patch file that this is a patched file; not the original so I won't forget?

This would be great to post on other Parrot AR AR.Drone forums.

Could you also program a patch file for the Parrot Bebop FreeFlight 3 APK for the Shield?

Thanks,
Carl
My first thought was to simply increment the version, like 2.4.11, but what if Parrot suddenly releases an update? It would be really confusing.
Though it should be relatively easy to add a small string somewhere indicating it's a patched version, but I was too focused on the code and haven't looked into it :)

As for FreeFlight 3, sorry, but I can't do anything without a Bebop to test with. What exactly would you like to be changed? Have you tried talking to Parrot about it?

Carl190_1
Just flips over
Posts: 73
Joined: 07 Apr 2013, 20:51
Drone Type: AR.Drone 2

Re: Modified FreeFlight 2.4.10 APK with full controller supp

Post by Carl190_1 » 14 Mar 2015, 01:14

Thanks for considering my suggestion about naming.

Confused by this:"yawed in the wrong direction, so it's disabled by default." So you have permanently disabled this?
and this: "After a 360° everything was fine, stabilisation was disabled."

Answer re Bebop.
I'm assuming that Nvidia Shield will react poorly to this flat or dead zone in FreeFlight 3.0 also.
Although they may have written a new program it seems that anything learned in FreeFlight 2.4.10 or with AR Drone 2.0 was throw out the window.
I fact I think that there have been several post to that effect.

Thanks for the great work, Parrot could use a few more like you!
Carl

Darklo
Just flips over
Posts: 59
Joined: 04 Jul 2013, 21:01
Drone Type: AR.Drone 2

Re: Modified FreeFlight 2.4.10 APK with full controller supp

Post by Darklo » 14 Mar 2015, 07:34

Carl190_1 wrote:Confused by this:"yawed in the wrong direction, so it's disabled by default." So you have permanently disabled this?
Sorry, I should have explained it better but didn't want to go too deep into technical details.

There is a "progressive command" AT*PCMD that is sent repeatedly to the drone with the pitch/roll/yaw/vertical speed and a few other parameters. One of the parameters is a bitfield that changes the way the drone interprets the commands. For example, bit 0 is what controls the hovering mode (1 - normal flight, 0 - ignore other parameters and hover). Bit 1 controls combined yaw. In Parrot's code it was already turned on whenever both sticks were applied and the variable combinedYawEnabled was true (it was by default).
That's what I meant by saying that combined yaw was already half-enabled. The only part missing was the bit 1 of configuration setting "control:control_level", which controls whether the drone accepts or ignores the bit 1 of the flag of the progressive command above. Parrot's developer guide states that both bits should be on for the combined yaw feature to work. So I added code that does that and also toggles the combinedYawEnabled variable when the yaw stick is clicked.
However, then I noticed that with both sticks applied the drone yawed in the wrong direction — the yaw stick was right but the drone turned left. So I set combinedYawEnabled to false by default, but you can turn it back to true by clicking the yaw stick.

So basically l I did exactly what was in the official developer documentation, but the result was unexpected. More testing is definitely needed, so as I mentioned, turn combined yaw on only if you're feeling adventurous.
Carl190_1 wrote:and this: "After a 360° everything was fine, stabilisation was disabled."
That was said by EricTN, not me, regarding hover lock toggle, a different feature.
Carl190_1 wrote:Answer re Bebop.
I'm assuming that Nvidia Shield will react poorly to this flat or dead zone in FreeFlight 3.0 also.
Although they may have written a new program it seems that anything learned in FreeFlight 2.4.10 or with AR AR.Drone 2.0 was throw out the window.
I fact I think that there have been several post to that effect.
To be honest, I haven't followed discussions about Bebop, but I don't remember Shield even being mentioned as a supported device. I think that what they learned from Shield and FreeFlight 2.4.10 actually served as basis for the SkyController.
Of course, please correct me if I'm wrong.
Carl190_1 wrote:Thanks for the great work, Parrot could use a few more like you!
Thank you for the kind words, they encourage me to continue this work!

Carl190_1
Just flips over
Posts: 73
Joined: 07 Apr 2013, 20:51
Drone Type: AR.Drone 2

Re: Modified FreeFlight 2.4.10 APK with game controller supp

Post by Carl190_1 » 14 Mar 2015, 12:40

"but you can turn it back to true by clicking the yaw stick." Wouldn't it be better to permanetly disable this feature for now?
Someone would have to remember to not accidentally click switch.

I fly my Parrot Bebop using the Nvidia Shield Portable. Here is a couple of sites:

http://forum.parrot.com/usa/viewtopic.php?id=29190

http://www.rcgroups.com/forums/showthre ... 04&page=10

Also there is one at GeForce I think my be the owner of Nvidia.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest