The Elusive Open Source: How Does Google Gradually Exert Full Control Over Android?

Google’s iron grip on Android: Controlling open source by any means necessary

“Open-source” is like a genie; once it's out, getting it back into the bottle isn't easy. How does Google control an open-source platform? Although Google has been trying every possible way to weaken the value of the open-source codebase, upgrading apps and making them closed-source isn’t the only strategy in Google's playbook.

For most OEM brands and third-party app developers, choosing the closed-source version of Google Android has become an "offer they can't refuse." The high-quality API resources provided by Google have ensnared OEMs and developers in a cycle of mutual dependence, uniting them around Google. Any forked version of Android (Android Forker) struggles to break through, and anyone who violates Google's rules inevitably faces punishment.

Six years ago, in November 2007, the Android Open Source Project (AOSP) was born. Just six months before that, the first iPhone had just debuted under great anticipation, marking the beginning of a new era for smartphones. Although Google was merely a partner at the app layer for the iPhone at the time, it clearly understood what it meant for Apple to dominate the smartphone world. As Vic Gundotra recalled, Andy Rubin once said:

If Google did nothing, we would be forced to accept a very scary future — a world without choices: one person, one company, one phone, one carrier.

Google feared that Apple might eventually rule the entire mobile world. Therefore, when Google had no foothold in the mobile space, Android emerged as an open-source project as a temporary measure to counter Apple.

Back then, Google felt satisfied with even a small share of the market. Thus, Google decided to contribute Android and use it as a Trojan horse to embed Google services everywhere. The rationale was that if Apple ever blocked Google Search, users could lose their search capabilities on desktops as well. Android was essentially a defensive trench in front of the "castle" of Google Search, ensuring the preservation and growth of Google's online assets in the mobile space.

However, times have changed. Android's global market share has soared from zero to nearly 80%. Android may have won the war of smartphones, but "Android's victory" doesn’t equate to "Google's victory." Since Android is open-source, it doesn’t belong solely to Google—anyone capable can develop a new version.

The struggles of Windows Phone and BlackBerry 10 in the mobile market tell us that having apps is king. Android's advantage in installed base means it's a platform with millions of apps. If another player develops a new operating system based on Android, it will naturally be compatible with millions of apps; all they need is to build their own app store. If another company creates a better version of Android, it could threaten Google's current dominance in smartphones. The biggest danger for Google is the emergence of a superior alternative version of Android.

Some companies are attempting to erase Google's footprint from Android, with Amazon's Kindle Fire (Mojito) being the most prominent example. Amazon removed all Google add-ons, building its own app store, content store, browser, cloud storage, and email. The entire Chinese market has also filtered out the Google portion of Android, as most Google services don't work in mainland China anyway. In both cases, Google gets no compensation from its Android usage.

When you have nothing, there's nothing to lose—this was true for Android at the start. But now, maintaining openness and compatibility isn't so easy. Android has grown from being Google's umbrella of protection into a mobile asset that requires Google's protection. Mobile is the future of the internet, and controlling the world's largest mobile platform comes with obvious benefits. However, "open-source" is like a genie—once released, it's not easy to put it back in the bottle. The question arises: how do you control an open-source project?

Google has always taken precautions against various Android alternatives. The Android people know consists of two parts: the open-source component AOSP, which forms the foundation of Android, and the closed-source component, the Google app suite. Although Google will neither go fully open nor fully closed, it is doing everything it can to increase control over the entire open-source project. The company's main strategy is to consolidate more and more apps under the closed-source “Google” umbrella.

Closure is a silent movement.

The closed-source Google apps have always existed. Initially, they were mainly mobile clients for Google online services such as Gmail, Maps, Talk, and YouTube. When Android had no market share, Google opened up the rest of Android based on these client apps. Now that Android has become a mobile powerhouse, it feels it should strengthen control over its open-source code.

For some apps, Google will still treat them as open-source components, but once proprietary versions are released, the AOSP versions of those apps will stop functioning. Fewer open-source codes mean that Google's competitors have to do more work to fill the gaps. While you can't kill an open-source app, you can render it obsolete by upgrading to a closed-source version. Every time Google updates or releases a new app on Play Store, it marks the end of the corresponding open-source version.

We'll start with the Search app, which perfectly illustrates the consequences when Google duplicates AOSP functionality.

In August 2010, Google launched Voice Actions. At the same time, it introduced "Google Search" to the Android Market, then running Froyo (Android 2.2). The image above shows the latest version of AOSP Search and Google Search running on Android 4.3. Yes, AOSP Search is still stuck at the Android 2.2 level, while Google Search has integrated voice, audio search, text-to-speech conversion, and the personal assistant service Google Now. Meanwhile, the AOSP version remains stuck with basic web and local searches.

At the 2010 Google I/O conference, Google first demonstrated its cloud music service, coincidentally freezing the AOSP version of the Music app at that moment. To this day, it's still a Froyo app. Besides the music store and subscription options, Play Music has already integrated Google's cloud music service, undergone multiple user interface revisions, and supports Equalizer and Chromecast. It's hard to believe they were once the same thing.

Google Calendar is the latest Google app to enter the closed-source category. Google's statement to the Android community was quite interesting: the new calendar app will soon be available for download on Play Store! There will be more features! (Oh, here comes the closure again.)

Even the keyboard couldn't escape this fate. A few months ago, Google added swipe input to its virtual keyboard. Guess where the source code is? Not in AOSP. The image above shows different settings options for the two keyboards. The Google Keyboard has a swipe input option, while AOSP does not—immediately after the release of Google Keyboard, the AOSP version was abandoned.

The Camera and Gallery are actually part of the same APK. The AOSP version is called "Gallery2.apk," while the Google version is named "GalleryGoogle.apk." As shown in the image, Photospheres are exclusive to the Google version—this innovative camera mode is unavailable in AOSP, as is Google+ Photos. Normally, the cloud-based Google+ Photos should appear alongside the local gallery.

Here, we should commend Google. Although AOSP hasn't incorporated these new features, the latest design elements from Android 4.3 have been included in the Android source code.

Although it hasn't been released yet, SMS will be the next app to exit. While many welcome the integration of SMS into Google Hangouts, competing with iMessage, it also means moving Android's SMS app into the closed-source app realm. Once Google integrates SMS, it's likely that within one or two Android updates, the SMS app will no longer be the default option—similar to how Chrome replaced the previous browser (even though Chrome remains open-source).

When Hangouts truly integrates SMS, the AOSP version of the messaging app will be completely abandoned, and the SMS app itself is nearing retirement (it hasn't seen significant updates since Android 4.0). This means: the open-source messaging app has come to an end.

The next piece of meat on the chopping block is likely the open-source Gallery. In leaked images of KitKat, there's a new icon called "Google Photos." Although we haven't seen Google Photos before, its icon resembles the current "G+ Photos." It seems the AOSP Gallery is destined to die, replaced by a closed-source app tied to Google+. This represents the ultimate explanation of Google's new independent kingdom.

Locking down OEM manufacturers

Although Google has been weakening the value of the open-source codebase by upgrading apps and making them closed-source, this isn't the only way Google wins this game. Even if a more powerful version of Android suddenly appeared, it would be hard for it to gain widespread support from manufacturers. In a competitive market, securing an OEM partnership isn't difficult, but Google is making it increasingly harder.

Google's control in the mobile space primarily stems from its app suite—Gmail, Maps, Google Now, Hangouts, YouTube, and Play Store. These are Android's killer apps, and manufacturers of all sizes want them on their devices. However, these apps aren't open-source and must be licensed from Google, creating a situation reminiscent of a scene from *The Godfather*—"an offer you can't refuse."

While this isn't a strict requirement, joining the Open Handset Alliance (OHA) makes life much easier when obtaining Google licenses. OHA is an alliance of companies that agree to adhere to Google's Android platform, stipulating that no member can produce related Android devices without Google's permission. Joining OHA is akin to signing a contract of servitude, meaning their devices cannot run other versions of Android.

Acer was punished for adopting Alibaba's Aliyun OS (a forked version of Android). Upon learning this, Google immediately cut off its access to Google apps. Google even published an official blog post explaining:

"Although Android is open to everyone, only devices compatible with Android can benefit from the complete Android ecosystem. Any member of the Open Handset Alliance should commit to building a unified Android platform—not a series of incompatible versions."

This made life difficult for Amazon, the lone "heretic" Android device brand in the Western world. Because Kindle OS is an incompatible version, no major OEM manufacturer can produce Kindle Fire devices for Amazon. Thus, when Amazon sought its next tablet manufacturer, it had to bypass a long list of companies including Acer, Asus, Dell, Foxconn, Fujitsu, HTC, Huawei, Kyocera, Lenovo, LG, Motorola, NEC, Samsung, Sharp, Sony, Toshiba, and ZTE. Currently, Amazon has outsourced all its Kindle device orders to Quanta Computer, a notebook computer manufacturer. Perhaps this is Amazon's reluctant choice.

This means any OEM that strays will face the kiss of death—being kicked out of the Android camp. Cutting ties with Google is terrifying for any OEM, making the choice of Google Android a one-way street.

Any OEM hoping to obtain Google Apps licenses must undergo Google's so-called "compatibility test." Compatibility ensures that apps from the Play Store can run on specific branded devices. "Compatibility" has a special meaning for Google; internally, engineers refer to it as "a lock to make OEMs obedient." Although Google has developed automated tools to detect device "compatibility," OEMs still need private emails with Google to secure access to Google apps. These agreements are mostly reached behind closed doors.

Furthermore, any OEM that obtains Google apps licenses must take them all—if you want Gmail and Maps, you also get Google Play Services, Google+, and whatever else Google deems fit to include in the package. Skyhook, a location-based WiFi service provider, encountered significant resistance when developing a location service for the Android platform. If an OEM device includes Skyhook's service, Google can't collect user location data, which is obviously disadvantageous for Google, so Skyhook was deemed "incompatible." Skyhook subsequently sued Google, and the case remains unresolved.

Shadow software

For most OEMs, surviving outside the Google ecosystem is a pipe dream. One way to stay independent while not offending the big boss, Google, is to provide a full suite of shadow versions of Google apps, though this is often criticized as "bloatware."

Samsung is a typical example. It has its own account system, cloud synchronization, and app store, along with a full suite of Google app alternatives like Internet, Email, and Calendar. These applications are still based on AOSP, but Samsung has been providing its own upgrades to users for a long time.

Pre-installing two calendar apps on a device might seem silly and redundant, but many OEMs view it as a Plan B to hedge against Google apps—just in case something goes wrong, they have a fallback. If Google plays unfairly and forces them out, the company at least has something to show potential consumers and gather valuable feedback. Why not?

Although this burdens and confuses users, for certain core applications, a few users might prefer the OEM's version. Samsung appears ready to jump ship at any time, but creating a shadow app suite is a limited move to break free from the Google ecosystem. What OEMs truly value about Android is the vast selection of third-party applications. Google is aware of this weakness and is working to increase the dependency of the entire app ecosystem on itself.

Locking down third-party apps

Play Services is one of Google's key weapons against forked versions of Android. As a closed-source Google app, it is authorized to OEMs along with the Google Apps package. Any feature moved from "normal" Android to Google Play Services means transitioning from open-source to closed-source. This move not only teases users with exclusive monopolized features but also aims to control third-party app developers through API licensing.

Leaving Google's app ecosystem seems straightforward: build your own app store, convince developers to distribute apps there, and then you can operate independently. However, Google is actively increasing the dependency of third-party apps on its platform. Developers choosing to build apps for "compatible" devices are thriving, while those outside the Google Android ecosystem are struggling. Essentially, Google is transforming the "Android App Ecosystem" into the "Google Play Ecosystem."

If you use any Google API, and try to run the app on Kindle or other AOSP versions, surprise! It will crash. With an 80% global market share, Android developers care about simplifying the development process, ensuring smooth operation, and reaching more users—all of which Google APIs can easily solve. The downside is that your app becomes dependent on devices authorized by Google Apps.

Google Maps API

Accessing Google Maps provides the right to use Google map data, which is convenient for weather or travel app development. The only problem is that this part of Google services is not part of the open-source Android services. Choosing the Maps API inherently means selecting Google-compatible devices as your development platform.

As a result, Amazon had no choice but to use Nokia's licensed map data and clone a set of Google Maps APIs. The company even provides a dedicated page to tell developers how to migrate apps from Google Maps. Google excels at optimizing its own ecosystem, which indirectly increases the difficulty for external ecosystems to survive. To run smoothly on Kindle, you must be compatible with two different map APIs.

This puts forked versions of Android in a difficult position. Here, Amazon either has to pay Nokia's service license fees annually or develop its own maps from scratch. Worse still, Amazon must keep pace with Google's rhythm: Amazon's Maps API supports Google Maps API v1, but if a developer needs new features from Maps v2 API, Amazon has a lot of work to do.

Google Cloud Messaging

Google Cloud Messaging (GCM) is the simplest and most effective way to push notifications on the Android platform, but it will never appear in AOSP versions. In 2013 at the I/O conference, it was integrated into Play Services. GCM mainly helps developers synchronize and push instant messages across platforms.

Location APIs

While the Google Maps API may only apply to a niche group of apps, for whatever reason, more and more apps need embedded message push functionality. This is why Amazon had to replicate this feature too. Its forked version is called "Amazon Device Messaging," supporting only Amazon devices. Similar to the Maps API situation, Amazon still has to do extra work but accepts the reality of a smaller user base. All functions of GCM may not be fully replicated in the Amazon version, adding to Amazon's workload.

In 2013 at the Google I/O conference, Google revamped the Android Location API and integrated it into Google Play Services. In other words, Android's latest location services are now closed-source. If previous examples are any indication, the old open-source geolocation services will be left to fend for themselves. New features include Fused Location Provider (allegedly using a new location algorithm), Geofencing, and Activity Recognition—the former provides location-based activity recommendations, and the latter combines accelerometer data with sophisticated algorithms to determine whether the user is walking, cycling, or driving—all without turning on GPS.

Since both Maps API and GCM rely on Google servers, standalone apps have good reasons to integrate them. However, throughout the entire geolocation service, Google's influence is omnipresent. Currently, there are two ways to obtain geolocation information services: one is to get energy-efficient and high-quality closed-source services from Google; the other is to choose subpar, power-hungry open-source services.

In-app purchases

The most effective in-app purchases on Android undoubtedly occur in the Google Play Store. If a developer chooses Kindle or develops apps in China, they have to look elsewhere. This again proves that to leave Google's Android, one must continuously replicate its services. Amazon has introduced the Amazon In-App Purchasing API. Even Samsung has resisted, having similar moves two years ago.

Play Games

Play Games is another exclusive API that solves a series of problems for mobile developers, allowing them to easily incorporate user accounts, leaderboards, point management, cloud saves, and multiplayer mechanisms. Its biggest advantage is cross-platform operation, of course excluding AOSP platforms. This is another API that third-party apps depend on and forked Android platforms must replicate. Amazon has a set of APIs called "GameCircle," but its functionality does not overlap with Play Games, so developers choosing Amazon games must additionally develop a completely independent multiplayer module.

Locking developers through iOS

One clever aspect of Google is that over 90% of its APIs support the iOS platform. From a developer's perspective, would you consider using Google's APIs? Google's solutions are often top-notch in usability, functionality, and ease of use; they support the two major platforms, meaning choosing Google's APIs can cover the majority of potential users. The only drawback is that they're incompatible with forked Android versions, but any forked Android version represents a small number of target devices you care about.

Perhaps most developers will embrace Google APIs, but they must answer this question: how will they handle Kindle and other Android versions? Developers have the autonomy to choose alternative API solutions, but these alternatives might be outdated, incompatible, or lacking in functionality. Product-focused developers usually abandon these niche Android forks outright, saving themselves unnecessary workloads.

Samsung's struggle

Let's explain why Amazon can survive independently from Google while Samsung cannot. Amazon may be a Google API replication machine, but Samsung falls short in this regard. Any speculation about Samsung breaking away from Google's ecosystem is premature unless you see it licensing map data or developing a cloud-based messaging API.

Amazon indeed strives, but it was born on the internet. Servers and software services are its forte, so developing a suite of cloud services isn't groundbreaking. Samsung, however, is an electronics company—it lacks the genetic makeup for cloud infrastructure and API development. Therefore, while Amazon can quickly establish itself as a follower of Google's cloud platform within a few years, Samsung continues to struggle.

Samsung has made some progress, as mentioned earlier, by launching its own in-app purchase SDK. Interestingly, it also has an advertising SDK, but it hasn't made much money. On the contrary, Google supports ads across Android, iOS, forked Android, and even Windows Phone.

Open-source that's out of reach

Any company challenging Google Android must replicate all the services mentioned in this article. Even then, it only equals Google Android. You still need to give users a compelling reason to abandon Google Android for yours.

Google has established a self-contained system; its foundational cloud services and Maps are free to use. Any company in need is bound to use Google services. Amazon might be an exception, but compare this: Google can monetize Maps through ad sales, while Amazon must pay Nokia annually for its users. This is the predicament faced by any forked Android version.

Even if a company produces an amazing forked version of Android, it must face the fact that almost all OEMs have signed contracts with Google. For OEMs, leaving Google for another forked Android entails far greater risks than benefits.

Although Android is open-source, it's a kind of "out-of-reach" open-source. Everywhere, without Google's protection, utilizing Android encounters numerous obstacles. Violating Google's rules means watching the world collapse around you.