AndroidX, App Bundle and Profiled AOT for Xamarin Android

Recently I tweeted about some early observations on some updates I made for an Android project that's currently in the App Store. What was so amazing about this is that the App Bundle was showing a download size that was roughly around 18% of what the traditional APK per ABI would produce.

The truth is some of these numbers are a little misleading. For instance while Google shows a download size around 4.3 - 4.8MB, in practice I really am seeing around a 7MB download from Google Play. While it is about 50-60% larger than what I see the download should be from the Release Dashboard in Google Play, it's still only around 29% of the size of the existing stable release that customers are downloading right now. In some respects with Unlimited data being widely available in major markets like the US, and very large internal storage being available on most devices it's still very attractive to ship smaller app packages to customers. Surprisingly it's really not that difficult to update your project to use all of this, though the CI/CD gets a little trickier.

Android App Bundle

I'm not going to go into detail about the Android App Bundle as I'm sure you can find plenty of other articles that will educate you more on what they are. For the purposes of this post I am simply going to define them as an optimization of a single APK that contains the necessary resources for all of your target ABI's. In the case of the app that I referenced in my Twitter post, the generated Android App Bundle was about 10MB which you may notice by itself was less than half the size of the existing APK size of the app in Google Play. So how do you update your Xamarin.Android project to start generating an .aab? Well for starters your build environment will need to be using Xamarin Android 9.4 or later. At the moment the hosted agents in Azure DevOps / App Center do not have that out of the box. 

It really couldn't be easier to generate the .aab though as you simply need to add a single property to the csproj for your Xamarin.Android project.

<PropertyGroup>
  <AndroidPackageFormat>aab</AndroidPackageFormat>
</PropertyGroup>

Note that while this snippet shows the AndroidPackageFormat property being declared in a root PropertyGroup (without conditions), you could put this in a PropertyGroup that is conditioned for Release or Store builds if you still want to generate an APK for Debug or Release and then ship an .aab to Google Play. It's also important to note here that before you can upload an .aab to Google Play you must delegate signing to Google. If you have not previously done so you will need to export your keystore with the private key from Android Studio so that you can upload it to Google and enable Android App Bundles for your app.

Critical For Distribution

It is also worth noting here that as a gotcha when you version your builds the Version Name does not matter to Google. So if you're at version 2.0 and you release 2.1 even after rolling it out, it does not mean that your users will be able to download it. Google Play is very dependent on the Version Code. For those who have uploaded multiple APK's to Google Play you've probably given your build a Version Code like 98 only to see each APK show something like 200098, 300098, 400098, 500098 like shown in the picture I posted on Twitter. This becomes very important because if you notice the build of the Android App Bundle in that same picture shows the Version Code as 123. In order to download this update we had to offset our builds by 500000 so that the next build was 500124 which was therefore recognized as being a newer version than 500098 that was currently available for download.

AndroidX (Android JetPack)

I'm not even going to pretend I fully understand everything around AndroidX. I do however like some of the promises that it's supposed to simplify the dreaded Android Support libraries. While the process I'm going to outline here is going to show you how to manually migrate to AndroidX I should add that there is hope on the horizon to make this a little easier.

We will be shipping a migration wizard type experience in 16.3 which will do all the dirty work for you
- Jonathan Dick (via Xamarin.Android Gitter)

Before you start, if your project is still using packages.config to manage it's NuGet references be sure to use the migration wizard in Visual Studio to migrate to PacakgeReference. Projects using PackageReference will have a much easier time migrating as you only need to install the Top Level dependencies and not the entire dependency chain. To start open the NuGet Package Manager for your Xamarin.Android project. Be sure to enable preview packages and install the latest Xamarin.AndroidX.Migration package. Once you've installed that and rebuild you should start getting build errors. As you scroll through the various build errors that appear, you'll see each error lists a current Android Support package and the corresponding AndroidX counterpart. The important thing to consider here is that you may see some crazy number like 27 packages that need to be installed. This doesn't accurately represent what the top level packages are for your project. As an example here for the Moment app update we only had to install 3 AndroidX packages directly.

  • Xamarin.AndroidX.Legacy.V4
  • Xamarin.Google.Android.Material
  • Xamarin.AndroidX.Browser

If you see the first two listed here chances are you should install them right away as they'll bring down your dependencies really quick. Don't be afraid to go to NuGet.org and look at the dependencies of each of the AndroidX packages you need to install. It will help you identify which ones you need to reference specifically and which you'll get transitively. 

You can read more on AndroidX from Jon Douglas on the official Xamarin blog.

Startup Tracing

AOT tends to both solve and cause a lot of problems. One of the problems that many people do not realize that they are causing for themselves is app size bloat. While AOT will speedup performance on Android, it also is likely to double the size of your app. Startup Tracing or Profiled AOT is a newer thing from the Xamarin team promising to keep the app bloat from AOT down while optimizing the AOT around startup performance where people tend to be the most frustrated. I should probably start by saying before you use AOT or Profiled AOT, do not do this for a Debug build. Doing so may encourage behavior that is not overly productive from day drinking to banging your head on the desk asking where you went wrong in life. The answer of course was using AOT in a Debug build. 

To use Profiled AOT (which is a great thing in a Release build) it really couldn't be easier. Similar to the Android App Bundle it simply requires a new property be added. The property should only be added to a Property Group intended for Release or the Store.

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' "> 
    <AndroidEnableProfiledAot>true</AndroidEnableProfiledAot> 
</PropertyGroup>

Again you can read more on Startup Tracing from Jon Douglas on the official Xamarin Blog.

Secret Sauce

Since everyone thinks there is some secret sauce I'll say there's not exactly any secret sauce... you have what you need already. That is of course until you get to the topic of CI/CD. Right now basically every tool I might rely on is pretty far behind on supporting Android App bundles.

App Center

At the moment there isn't really anything you can do with App Bundles... in fact App Center has been pretty limiting for Android developers for a while with the fact they only supported a single artifact for distribution making it hard to generate a single APK per ABI... fast forward and they still don't have Android App Bundle support at all though the team says it should be available this month.

While you probably could install Boots and generate the .aab the problem here is that App Center wouldn't be able to do anything with the artifact and you'd probably fail the build.

Azure Pipelines

The Xamarin.Android build task is utterly useless. I eventually gave up on it and just wrote a simple PowerShell script to generate my .aab. Luckily with Azure Pipelines you have full control over what you want to have available as an artifact so this made it really easy to grab the .aab later on so I could do something with it. What I ended up with was something like this in my Android build stage. This downloads my Keystore from Secure Storage, then does a NuGet restore (via MSBuild), builds and signs the Android app all in one single step. It's worth noting here that since Xamarin.Android already signs the generated package/bundle with a default keystore and Google Play requires the ability to use your keystore to sign the apks they generate from the App Bundle, you probably don't need the complexity of signing.

- task: vs-publisher-473885.motz-mobi[email protected]1
  displayName: 'Bump Android Versions in AndroidManifest.xml'
  inputs:
    sourcePath: pathTo/AwesomeApp.Android/Properties/AndroidManifest.xml
    versionName: '1.1.0'
    versionCode: $(Build.BuildId)
    versionCodeOffset: 500000

- script: sudo $AGENT_HOMEDIRECTORY/scripts/select-xamarin-sdk.sh 5_18_1
  displayName: 'Select Xamarin SDK version'

- task: [email protected]
  displayName: Install Latest Android SDK
  inputs:
    uri: https://aka.ms/xamarin-android-commercial-d16-2-macos

- task: [email protected]
  name: androidKeyStore
  inputs:
    secureFile: $(KeystoreFileName)

- powershell: |
   if($env:SYSTEM_DEBUG -eq $true)
   {
     $extraArgs = '/bl:android.binlog'
   }
   $keystorePath = $env:KeystoreFilePath
   $keystoreName = $env:KeystoreName
   $keystorePassword = $env:KeystorePassword
   $project = $env:AndroidProjectPath
   $outputDirectory = "$($env:BUILD_BINARIESDIRECTORY)/$($env:BuildConfiguration)"
   Write-Host "Output Path = $outputDirectory"
   msbuild $project /t:SignAndroidPackage /p:Configuration=$($env:BuildConfiguration) /p:OutputPath=$outputDirectory /restore /p:AndroidKeyStore=true /p:AndroidSigningKeyStore=$keystorePath /p:AndroidSigningStorePass=$keystorePassword /p:AndroidSigningKeyAlias=$keystoreName /p:AndroidSigningKeyPass=$keystorePassword $extraArgs
  displayName: Build & Generate AppBundle
  env:
    AndroidProjectPath: 'pathTo/AwesomeApp.Android.csproj'
    Secret_AppCenterSecret: ${{ parameters.appcenterKey }}
    KeystoreFilePath: $(androidKeyStore.secureFilePath)
    KeystoreName: $(KeystoreName)
    KeystorePassword: $(KeystorePassword)

- task: [email protected]
  displayName: 'Publish BinLog'
  inputs:
    targetPath: 'android.binlog'
    artifactName: android-binlog
  condition: and(failed(), eq(variables['system.debug'], true))

- task: [email protected]
  displayName: 'Publish Package Artifacts'
  inputs:
    targetPath: '$(Build.BinariesDirectory)'
    artifactName: ${{ parameters.artifactName }}
  condition: eq(variables['system.pullrequest.isfork'], false)

 

Builds though are only the first part of it. This is where we kind of lose here. I'm sure if I spent some time I could write a script to handle this but for now manual uploads are where we're at. From Azure Pipelines there are two methods I tend to rely on to upload my artifacts to Google Play.

  • App Center Distribution task: Well as we already discussed App Center doesn't support .aab so we're out of luck there at the moment.
  • Google Play task: Since whoever at Microsoft is responsible for it doesn't seem to be responding to the community... I'm just not sure what to say on this front.

The important thing is that we do have a generated artifact though and with that we can at least manually upload the artifact to Google Play.

Developer Toolkit for Visual Studio Mac

Several years ago when I was still just a web developer wanting to break into mobile development, I asked myself how does anybody do this? You have to learn Java for Android, Objective-C or Swift for iOS.... of course then I learned about Xamarin. Without a doubt Xamarin makes the tedious tasks of mobile app development far easier by centralizing your code in one common language, and even further with Xamarin Forms by abstracting the UI into reusable code. That doesn't mean that creating a new app is by any means easy. In fact setting up a new app can take a lot of time.

Nearly a year and a half ago I introduced the Prism QuickStart Templates which were the first .NET Standard templates using the new project format available for Xamarin apps. The project took on a life of it's own and was loved by many even despite it's limited availability in the CLI. As I set out to bring the templates into Visual Studio for Mac, it again took on a life of its own. A number of developers and MVP's were gracious enough to give me their feedback on things they would like to see, and while it may have delayed my ability to release, what we have today is simply stunning.

Prism Template Studio and Developer Toolkit

Ok I admit it, it's a mouthful, and if you have a better name feel free to tweet it to @DanJSiegel. Why the mouthful, because it is absolutely jammed pack with so many tools, so many helpers, and so many templates, that every time I explain it someone asks, "well what about...", and I keep either responding yeah it does that too... or yeah we could add that. It's probably that second one that has admittedly  generated the most delay in getting this out. Whether you use vanilla Xamarin Forms or Prism you'll want to install the Prism Template Studio and Developer Toolkit. 

Templates

As the name suggests it contains the a Template Pack. This Template Pack isn't quite like anything you've seen before. There are 14 new project templates that ship in this Template Pack, including 7 projects for Unit and UI Testing, 3 more for building Prism Modules, and another 3 for Prism Applications, plus a new basic Xamarin Forms project template.

Each of the templates bring something special for different developers. You can still go with the traditional flat "Official" template, or one includes PropertyChanged.Fody with projects and tests separated into src and tests folders. You can also take advantage of the powerhouse QuickStart Template or the App Center Connected App. Both of these provide the to setup a project in VSTS and automatically configure a Build in App Center.

Tools

To start there is some integrated tooling for all of your Xamarin projects to enable support for the Mobile.BuildTools, you can connect an existing project to an app in App Center, and even get some quick links to the Prism Docs, GitHub issues, and StackOverflow. Over time you can expect to see additional tooling for App Center, and refinements to do more with VSTS and better expand on your ability to get started with Unit and UI Tests.

Get started today by making sure Visual Studio Mac is up to date, and then simply install the Prism Template Studio and Developer Toolkit from the Extension Manager.

Secure App Builds with AppCenter

AppCenter has been touted as this wonderful new service from Microsoft. It's supposed to make it easier to build, test and distribute our apps. While there is a lot I love about AppCenter, the simplification of the Build pipeline over VSTS always seemed to be problematic to me. For years I have had a pet peeve that developers often check things into source control that should never be checked in. Sometimes it's simply a backend URL, other times it could be a Client ID. These sorts of things should never be checked in, and should be injected as part of the Build pipeline. The problem is that when you have only a very simple process in place for creating a new build it opens you up to make these sorts of poor decisions with your code base.

For those who are familiar with AppCenter you may be familiar with the fact that you can add scripts to your projects:

  • appcenter-post-clone.sh (Bash for iOS & Android)
#!/usr/bin/env bash

# Example: Clone a required repository
git clone https://github.com/example/SomeProject

# Example: Install App Center CLI
npm install -g appcenter-cli
  • appcenter-pre-build.sh (Bash for iOS & Android)
#!/usr/bin/env bash

# Example: Change bundle name of an iOS app for non-production
if [ "$APPCENTER_BRANCH" != "master" ];
then
    plutil -replace CFBundleName -string "\$(PRODUCT_NAME) Beta" $APPCENTER_SOURCE_DIRECTORY/MyApp/Info.plist
fi
  • appcenter-post-build.sh (Bash for iOS & Android)
#!/usr/bin/env bash

HOCKEYAPP_API_TOKEN={API_Token}
HOCKEYAPP_APP_ID={APP_ID}

# Example: Upload master branch app binary to HockeyApp using the API
if [ "$APPCENTER_BRANCH" == "master" ];
then
    curl \
    -F "status=2" \
    -F "[email protected]$APPCENTER_OUTPUT_DIRECTORY/MyApps.ipa" \
    -H "X-HockeyAppToken: $HOCKEYAPP_API_TOKEN" \
    https://rink.hockeyapp.net/api/2/apps/$HOCKEYAPP_APP_ID/app_versions/upload
else
    echo "Current branch is $APPCENTER_BRANCH"
fi

Simplifying Builds

While ridiculously powerful in what you can do with these scripts (I've been told you can install pretty much anything you want), this seems like far too much effort. I'll even admit that while I'm quite capable of writing whatever scripts I want, beyond a POC, I have never, and have no plans of writing scripts, to get my projects building on AppCenter. There are some things, that shouldn't require lots of work, just to make builds work from one project to another. This is where the Mobile.BuildTools comes in. The Mobile.BuildTools is an easy to use NuGet package. It simply adds tooling for MSBuild and has no binaries that are injected into your application. Because of this it can be used anywhere that you have MSBuild including Visual Studio, Visual Studio Mac, Visual Studio Code, or any Build Host including AppCenter. I have often referenced it as "DevOps in a box", or at least in a NuGet. I should probably add here, that it's called Mobile.BuildTools not just because you can use it on Mobile Apps. You can use this on literally ANY .NET Project, on ANY Platform supported by MSBuild.

Setting Up Your Application

After installing the NuGet, we can add a JSON file to our PCL or .NET Standard named secrets.json, like the following:

{
  "AppCenter_iOS_Secret": "{Your Secret Here}",
  "AppCenter_Android_Secret": "{Your Secret Here}"
}

Note that I've given it the rather long name here for illustrative purposes and to make it easier to read, but ultimately you can add as many keys as you want with whatever name you want. At build this will then automatically generate a Secrets class in your project's Helpers namespace. This will regenerate on each build, so any changes should be made to the JSON file and not the C# file. You should also add secrets.json and Secrets.cs to your .gitignore. If you're using my template packs, this is already done for you. 

I've gotten the question, what happens when I have some value that isn't a string? That's a fantastic question, and the Mobile.BuildTools has you covered there as well with support for string, int, double, and bool data types.

Protecting Your App Manifest

Sometimes secrets don't limit themselves to some constant value in our App. Sometimes we need to declare something sensitive in either our Info.plist or AndroidManifest.xml. This creates an issue for us as well. We obviously need to be able to test locally, but again we need to ensure we don't check these files in with those sensitive values. The Mobile.BuildTools again have your back. By adding these files to our .gitignore we keep from checking in sensitive values. In truth most of these manifests really aren't all that sensitive. To make it simple though we can simply check in a tokenized version of these manifests. Note that by default tokens should are delimited by double dollar signs before and after the variable name such as $$AppCenterSecret$$.

| - MyProject.sln
| - build/
| - | - AndroidTemplateManifest.xml
| - | - BuildTemplateInfo.plist

Sample BuildTemplateInfo.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>CFBundleDisplayName</key>
    <string>AppCenter.DemoApp</string>
    <key>CFBundleName</key>
    <string>AppCenter.DemoApp</string>
    <key>CFBundleIdentifier</key>
    <string>com.appcenter.demoapp</string>
    <key>CFBundleShortVersionString</key>
    <string>1.0</string>
    <key>CFBundleVersion</key>
    <string>1.0</string>
    <key>LSRequiresIPhoneOS</key>
    <true/>
    <key>MinimumOSVersion</key>
    <string>8.0</string>
    <key>UIDeviceFamily</key>
    <array>
        <integer>1</integer>
        <integer>2</integer>
    </array>
    <key>UILaunchStoryboardName</key>
    <string>LaunchScreen</string>
    <key>UIRequiredDeviceCapabilities</key>
    <array>
        <string>armv7</string>
    </array>
    <key>UISupportedInterfaceOrientations</key>
    <array>
        <string>UIInterfaceOrientationPortrait</string>
        <string>UIInterfaceOrientationLandscapeLeft</string>
        <string>UIInterfaceOrientationLandscapeRight</string>
    </array>
    <key>UISupportedInterfaceOrientations~ipad</key>
    <array>
        <string>UIInterfaceOrientationPortrait</string>
        <string>UIInterfaceOrientationPortraitUpsideDown</string>
        <string>UIInterfaceOrientationLandscapeLeft</string>
        <string>UIInterfaceOrientationLandscapeRight</string>
    </array>
    <key>XSAppIconAssets</key>
    <string>Assets.xcassets/AppIcon.appiconset</string>
    <key>CFBundleURLTypes</key>
    <array>
        <dict>
            <key>CFBundleURLSchemes</key>
            <array>
                <string>appcenter-$$AppCenterSecret$$</string>
            </array>
        </dict>
    </array>
</dict>
</plist>

Setting Up AppCenter For Builds

Keep in mind here that our code now relies on a class called Secrets with several constants that exist nowhere in our codebase, and we have an iOS project without an Info.plist and an Android project without the AndroidManifest.xml... AppCenter offers us the ability to easily add Environment Variables. And this is where the build tools will be able to pick up what we need. 

The Mobile.BuildTools are highly configurable based on preferences, but by default a common project type such as PCL or .NET Standard library will look for any variables prefixed with Secret_, while platform targets look for this with their name like iOSSecret_ or DroidSecret_. Setting any of these in the build will generate a secrets.json and resulting Secrets class in the target project. I did of course say that we have a tokenized version of our manifests which have both the wrong name and tokens that need to be replaced. Just like with the secrets though the variables need to be prefixed, however the default prefix for this is simply Manifest_.  

Get Started

To get started install the Mobile.BuildTools into your project. These tools are free and open source, with documentation on additional configuration options that can be found on GitHub. If you have any issues or have a great idea that you would like to see added to the tools open an issue.

Want to see more? Be sure to check out the demo app.