Getting Started with Azure Pipelines for Xamarin Developers

DevOps for Xamarin apps is a rather large topic. Rather than trying to go A-Z in one bite, I thought it might make more sense to divide this up into bite sized chunks. In this first article we'll take a look at how to get started with Azure DevOps (aka Azure Pipelines... aka I've lost track of what we're supposed to call it anymore)… Obviously the first thing you'll need to do is to create a new Azure DevOps organization (assuming you don't already have one). If you don't you'll want to head over to https://dev.azure.com and create one. You should see a dialog similar to this.

After creating your new organization you'll need to setup a project. This project could be whatever you need. For the purposes of this article we'll call it' the Xamarin Mobile project and make it private.

Once you've created the project you should see something like this:

But we're still not done yet. As you'll notice there's a repos section in here. We could choose to use the FREE private git hosting in Azure DevOps. But if our code lives somewhere else such as GitHub we can keep using Azure DevOps for our CI/CD pipelines. If that's the case you may want to head to the Project Settings. All of the Azure DevOps services will be on by default, however you can choose what you want to use to narrow it down and focus the view on only what you'll be using.

Remember that we've just set up a brand new Azure DevOps organization so we only have what's available out of the box, and that's really not sufficient for Xamarin Developers.

Marketplace for Azure DevOps

To really light up Azure DevOps and make sure that we have the full power we need for our Xamarin CI Builds you'll want to head over to https://marketplace.visualstudio.com and make sure you're looking at the Azure DevOps tab. You may find a number of great extensions for 

The first two extensions you'll want to install I honestly cannot figure out why they are not included out of the box with Azure DevOps particularly since they're released by Microsoft.

  • Apple App Store: As the name implies this will give us an ability to deploy directly to the App Store and makes it very easy to get builds into Test Flight
  • Google Play: Again this extension will make it very easy to integrate and deploy directly to Google Play. We can deploy to an internal testers group, alpha, beta, or production as we see fit. NOTE: this extension does not currently support the new Android App Bundles (.aab)

The next two extensions are from our great friends on the on the Xamarin team but they are published individually and not by Microsoft as an organization.

  • Mobile App Tasks for iOS and Android: This great extension from James Montemagno makes it very easy to provide a unique version for each build of our app. This is particularly important for those times where we may need to upload version 1.5 of our app to the app store 10 times to get through QA, and then the app review. By ensuring we have a unique Build number on each build of version 1.5 we can continue to upload new artifacts for testers and eventual release.
  • Boots: Boots is an awesome .NET CLI Tool from Jonathan Peppers. Thanks to a little help from Peter Collins on the Xamarin team, it's been made even easier to use with this great extension. With this simple utility you can provide a download link for any Xamarin SDK such as an Alpha or nightly build of Mono, Xamarin.iOS or Xamarin.Android and it will download and install the SDK so that it's ready to use for your build.

Installing the Extensions

It really couldn't be easier to install the extensions. You can navigate directly to each extension in the marketplace by clicking on the links above. Then you will want to click the Get it free button. When you are getting ready to install the extension you'll see a screen like the one below. Be sure to take note of the organization as it may not have the organization selected for which you want to install to.

Secrets and Secure Files

Now that we have the extensions set up, lets head back into Azure DevOps and click on the Pipelines, and then the Library. The Library will give us an easy to manage location for our resources.

To get started round up the Android keystore, and any provisioning profile and signing certificate you will need for your Android and iOS apps. Select the Secure Files tab and then start uploading your secure files. Note that there is a certain amount of insanity in this step as you just uploaded files that are currently unavailable to be used. You will need to open each file that you have just uploaded. You will see something like the following. Notice that the toggle switch for all of the pipelines to use this file is currently off, you will need to toggle it to the on position and then hit save.

Now we'll want to head back to the first tab we landed on and start adding some variable groups. How this looks for you may differ slightly based on your needs. In general though I tend to end up with 3 Variable Groups like the following:

  • Android-Signing
    • AndroidKeystoreFileName
    • AndroidKeystorePassword
    • AndroidKeystoreAlias
  • iOS-Signing
    • iOSDevelopmentCertificate
    • iOSDevelopmentPassword
    • iOSDevelopmentProvisioningProfile
    • iOSDistributionCertificate
    • iOSDistributionPassword
    • iOSDistributionProvisioningProfile
  • MyAppSecrets
    • AppCenterKey_Android_QA
    • AppCenterKey_Android_Store
    • AppCenterKey_iOS_QA
    • AppCenterKey_iOS_Store

Again you will want to be sure that each variable group is allowing access to all pipelines otherwise the variables will not be available for the build you will setup next.

Variable Group Take Aways

  • Android app signing is very different than for iOS. There is no need to track separate keystores for QA and Production.
  • iOS App Signing is very dependent on how to intend to consume the app. If you plan on side loading the app outside the App Store/Test Flight you will need a Development certificate. If you're ok with doing QA through Test Flight it may be worth it to simplify and use a single production certificate. That said you should change variables used such as where analytics are tracked, backend etc so you should be careful not to release QA builds into the wild where a customer could potentially use it.
  • You should have processes in place that ensure there is some manual validation that can occur before your app ends up on the Store.
  • You should also separate where Analytics/Crash Diagnostics are going between Stage and Production. While App Center is rather horrible at this currently, creating separate apps in App Center to track different environments can be a helpful technique in easily identifying Development noise verses what your customers are really doing and experiencing.
  • You are building the iOS and Android apps in more than one step. There is no reason to leak the Android app secret in the iOS build or lead the iOS app secret in the Android build... just provide the one you need at build.

Next Steps

In the next post we'll look at how we integrate all of this into a build using YAML and how we organize it.

Developer Toolkit for Visual Studio Mac

Several years ago when I was still just a web developer wanting to break into mobile development, I asked myself how does anybody do this? You have to learn Java for Android, Objective-C or Swift for iOS.... of course then I learned about Xamarin. Without a doubt Xamarin makes the tedious tasks of mobile app development far easier by centralizing your code in one common language, and even further with Xamarin Forms by abstracting the UI into reusable code. That doesn't mean that creating a new app is by any means easy. In fact setting up a new app can take a lot of time.

Nearly a year and a half ago I introduced the Prism QuickStart Templates which were the first .NET Standard templates using the new project format available for Xamarin apps. The project took on a life of it's own and was loved by many even despite it's limited availability in the CLI. As I set out to bring the templates into Visual Studio for Mac, it again took on a life of its own. A number of developers and MVP's were gracious enough to give me their feedback on things they would like to see, and while it may have delayed my ability to release, what we have today is simply stunning.

Prism Template Studio and Developer Toolkit

Ok I admit it, it's a mouthful, and if you have a better name feel free to tweet it to @DanJSiegel. Why the mouthful, because it is absolutely jammed pack with so many tools, so many helpers, and so many templates, that every time I explain it someone asks, "well what about...", and I keep either responding yeah it does that too... or yeah we could add that. It's probably that second one that has admittedly  generated the most delay in getting this out. Whether you use vanilla Xamarin Forms or Prism you'll want to install the Prism Template Studio and Developer Toolkit. 

Templates

As the name suggests it contains the a Template Pack. This Template Pack isn't quite like anything you've seen before. There are 14 new project templates that ship in this Template Pack, including 7 projects for Unit and UI Testing, 3 more for building Prism Modules, and another 3 for Prism Applications, plus a new basic Xamarin Forms project template.

Each of the templates bring something special for different developers. You can still go with the traditional flat "Official" template, or one includes PropertyChanged.Fody with projects and tests separated into src and tests folders. You can also take advantage of the powerhouse QuickStart Template or the App Center Connected App. Both of these provide the to setup a project in VSTS and automatically configure a Build in App Center.

Tools

To start there is some integrated tooling for all of your Xamarin projects to enable support for the Mobile.BuildTools, you can connect an existing project to an app in App Center, and even get some quick links to the Prism Docs, GitHub issues, and StackOverflow. Over time you can expect to see additional tooling for App Center, and refinements to do more with VSTS and better expand on your ability to get started with Unit and UI Tests.

Get started today by making sure Visual Studio Mac is up to date, and then simply install the Prism Template Studio and Developer Toolkit from the Extension Manager.

Secure App Builds with AppCenter

AppCenter has been touted as this wonderful new service from Microsoft. It's supposed to make it easier to build, test and distribute our apps. While there is a lot I love about AppCenter, the simplification of the Build pipeline over VSTS always seemed to be problematic to me. For years I have had a pet peeve that developers often check things into source control that should never be checked in. Sometimes it's simply a backend URL, other times it could be a Client ID. These sorts of things should never be checked in, and should be injected as part of the Build pipeline. The problem is that when you have only a very simple process in place for creating a new build it opens you up to make these sorts of poor decisions with your code base.

For those who are familiar with AppCenter you may be familiar with the fact that you can add scripts to your projects:

  • appcenter-post-clone.sh (Bash for iOS & Android)
#!/usr/bin/env bash

# Example: Clone a required repository
git clone https://github.com/example/SomeProject

# Example: Install App Center CLI
npm install -g appcenter-cli
  • appcenter-pre-build.sh (Bash for iOS & Android)
#!/usr/bin/env bash

# Example: Change bundle name of an iOS app for non-production
if [ "$APPCENTER_BRANCH" != "master" ];
then
    plutil -replace CFBundleName -string "\$(PRODUCT_NAME) Beta" $APPCENTER_SOURCE_DIRECTORY/MyApp/Info.plist
fi
  • appcenter-post-build.sh (Bash for iOS & Android)
#!/usr/bin/env bash

HOCKEYAPP_API_TOKEN={API_Token}
HOCKEYAPP_APP_ID={APP_ID}

# Example: Upload master branch app binary to HockeyApp using the API
if [ "$APPCENTER_BRANCH" == "master" ];
then
    curl \
    -F "status=2" \
    -F "[email protected]$APPCENTER_OUTPUT_DIRECTORY/MyApps.ipa" \
    -H "X-HockeyAppToken: $HOCKEYAPP_API_TOKEN" \
    https://rink.hockeyapp.net/api/2/apps/$HOCKEYAPP_APP_ID/app_versions/upload
else
    echo "Current branch is $APPCENTER_BRANCH"
fi

Simplifying Builds

While ridiculously powerful in what you can do with these scripts (I've been told you can install pretty much anything you want), this seems like far too much effort. I'll even admit that while I'm quite capable of writing whatever scripts I want, beyond a POC, I have never, and have no plans of writing scripts, to get my projects building on AppCenter. There are some things, that shouldn't require lots of work, just to make builds work from one project to another. This is where the Mobile.BuildTools comes in. The Mobile.BuildTools is an easy to use NuGet package. It simply adds tooling for MSBuild and has no binaries that are injected into your application. Because of this it can be used anywhere that you have MSBuild including Visual Studio, Visual Studio Mac, Visual Studio Code, or any Build Host including AppCenter. I have often referenced it as "DevOps in a box", or at least in a NuGet. I should probably add here, that it's called Mobile.BuildTools not just because you can use it on Mobile Apps. You can use this on literally ANY .NET Project, on ANY Platform supported by MSBuild.

Setting Up Your Application

After installing the NuGet, we can add a JSON file to our PCL or .NET Standard named secrets.json, like the following:

{
  "AppCenter_iOS_Secret": "{Your Secret Here}",
  "AppCenter_Android_Secret": "{Your Secret Here}"
}

Note that I've given it the rather long name here for illustrative purposes and to make it easier to read, but ultimately you can add as many keys as you want with whatever name you want. At build this will then automatically generate a Secrets class in your project's Helpers namespace. This will regenerate on each build, so any changes should be made to the JSON file and not the C# file. You should also add secrets.json and Secrets.cs to your .gitignore. If you're using my template packs, this is already done for you. 

I've gotten the question, what happens when I have some value that isn't a string? That's a fantastic question, and the Mobile.BuildTools has you covered there as well with support for string, int, double, and bool data types.

Protecting Your App Manifest

Sometimes secrets don't limit themselves to some constant value in our App. Sometimes we need to declare something sensitive in either our Info.plist or AndroidManifest.xml. This creates an issue for us as well. We obviously need to be able to test locally, but again we need to ensure we don't check these files in with those sensitive values. The Mobile.BuildTools again have your back. By adding these files to our .gitignore we keep from checking in sensitive values. In truth most of these manifests really aren't all that sensitive. To make it simple though we can simply check in a tokenized version of these manifests. Note that by default tokens should are delimited by double dollar signs before and after the variable name such as $$AppCenterSecret$$.

| - MyProject.sln
| - build/
| - | - AndroidTemplateManifest.xml
| - | - BuildTemplateInfo.plist

Sample BuildTemplateInfo.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>CFBundleDisplayName</key>
    <string>AppCenter.DemoApp</string>
    <key>CFBundleName</key>
    <string>AppCenter.DemoApp</string>
    <key>CFBundleIdentifier</key>
    <string>com.appcenter.demoapp</string>
    <key>CFBundleShortVersionString</key>
    <string>1.0</string>
    <key>CFBundleVersion</key>
    <string>1.0</string>
    <key>LSRequiresIPhoneOS</key>
    <true/>
    <key>MinimumOSVersion</key>
    <string>8.0</string>
    <key>UIDeviceFamily</key>
    <array>
        <integer>1</integer>
        <integer>2</integer>
    </array>
    <key>UILaunchStoryboardName</key>
    <string>LaunchScreen</string>
    <key>UIRequiredDeviceCapabilities</key>
    <array>
        <string>armv7</string>
    </array>
    <key>UISupportedInterfaceOrientations</key>
    <array>
        <string>UIInterfaceOrientationPortrait</string>
        <string>UIInterfaceOrientationLandscapeLeft</string>
        <string>UIInterfaceOrientationLandscapeRight</string>
    </array>
    <key>UISupportedInterfaceOrientations~ipad</key>
    <array>
        <string>UIInterfaceOrientationPortrait</string>
        <string>UIInterfaceOrientationPortraitUpsideDown</string>
        <string>UIInterfaceOrientationLandscapeLeft</string>
        <string>UIInterfaceOrientationLandscapeRight</string>
    </array>
    <key>XSAppIconAssets</key>
    <string>Assets.xcassets/AppIcon.appiconset</string>
    <key>CFBundleURLTypes</key>
    <array>
        <dict>
            <key>CFBundleURLSchemes</key>
            <array>
                <string>appcenter-$$AppCenterSecret$$</string>
            </array>
        </dict>
    </array>
</dict>
</plist>

Setting Up AppCenter For Builds

Keep in mind here that our code now relies on a class called Secrets with several constants that exist nowhere in our codebase, and we have an iOS project without an Info.plist and an Android project without the AndroidManifest.xml... AppCenter offers us the ability to easily add Environment Variables. And this is where the build tools will be able to pick up what we need. 

The Mobile.BuildTools are highly configurable based on preferences, but by default a common project type such as PCL or .NET Standard library will look for any variables prefixed with Secret_, while platform targets look for this with their name like iOSSecret_ or DroidSecret_. Setting any of these in the build will generate a secrets.json and resulting Secrets class in the target project. I did of course say that we have a tokenized version of our manifests which have both the wrong name and tokens that need to be replaced. Just like with the secrets though the variables need to be prefixed, however the default prefix for this is simply Manifest_.  

Get Started

To get started install the Mobile.BuildTools into your project. These tools are free and open source, with documentation on additional configuration options that can be found on GitHub. If you have any issues or have a great idea that you would like to see added to the tools open an issue.

Want to see more? Be sure to check out the demo app.

Xamarin DevOps In A Box

Several Months ago I set out to make some of the most powerful Xamarin Project Templates. I've gotten a lot of feedback on the Prism QuickStart Templates and how they have accelerated Mobile Development for Developers. One of the features that has really caught the attention of so many developers is the Application Secrets generation. Mobile Apps so often have sensitive information such as Client Id's, or builds that require some minor changes such as pointing to one backend API for Development, another for Staging, and yet another for Production. The custom tasks that have been included in the QuickStart Templates have been helping developers for months to more easily handle these tasks.

Over time as I've made changes and improvements I've come to realize that it has left existing projects in a state where they have been unable to take advantage of changes. There has also been the fact that while they have been tied exclusively to the Prism QuickStart Templates though there is nothing about these great Build Tasks that are tied in any way to Prism. These wonderful Build Props and Build Targets have been separated out into an easy to install NuGet. Since this only contains build props and targets it adds nothing to the size of assemblies, but it does make your DevOps a whole lot easier.

Build Props

While many developers may not currently be utilizing many Build Props, the Mobile.BuildTools adds a number of properties that help you determine what type of project is building and on what type of Host. This could for example better assist you in developing Build tasks that only execute on Windows or Mac, determine if PowerShell is installed. You can also easily determine what platform the project is. We'll take a look at an example below.

Build Secrets

Modern apps are full of Client Id's, and Secrets that it can be maddening for Security. After all how do you develop an app that requires this type of sensitive information without compromising security by checking code into Source Control that contains our Id's and Secrets? Better yet how can we develop better CI/CD pipelines that are customized for an environment such as Development, Staging, or Production? This is where the Secrets Task shines. By simply including a file named secrets.json in your project root, the Secrets Task will generate a Secrets class for you. This enables you to ignore both the Secrets.cs and secrets.json files in your .gitignore. This frees your Build Server to generate your secrets.json and the rest is handled for you.

Templating Project Manifests

Sometimes our DevOps process requires having flexibility across our app manifests. While I am sure there are a multitude of reasons for why you might have this requirement the two most common use cases I see are:

  • I am developing an Application that must be tweaked for multiple customers and deployed to the App Store for each (i.e. a Banking App)
  • My application requires a setting in my manifest that exposes a Client ID or some other sensitive piece of information (i.e. I am using the Microsoft.Identity.Client for Azure Active Directory)

This is another area where the Mobile.BuildTools really shines by allowing you to include safe to check in Manifest Templates which will then be appropriately copied to your iOS or Android project. As I mentioned before this is just a sample of how the Build Props can better assist your DevOps. Each Copy task is restricted so that the AndroidManifest.xml is only copied when IsAndroidProject is true, and the Info.plist is only copied when IsiOSProject is true. 

FAQ

Q. What happens if I don't have a secrets.json included in my Project?
A. Nothing. The Task will safely execute, not having found a secrets.json file and will finish without creating anything.

Q. Can I name secrets.json something else?
A. Yes, it is a configurable Property. You can add JsonSecretsFileName to the PropertyGroup of your Project with the file name it should look for.

Q. Can you have secrets in more than one Project?
A. Yes, you can have secrets in as many projects as you want. Again the Task will only generate the Secrets class if you have a secrets.json present.

Q. The idea of a Tokenized Manifest sounds cool, but I don't need it. Can I still use the BuildTools?
A. Yes! As long as the AndroidManifest.xml or Info.plist is present when the Build is started it will not copy them over. Otherwise it would get really annoying during development to get some Tokenized manifest constantly undoing your changes.

Q. How do I find out more about setting up the Manifest Templates? Is is customizable?
A. You can find out more on GitHub. You can override the default variables to change the location of the templates, and even the Template Names. 

Q. Am I able to swap out the appxmanifest on UWP? 
A. UWP is not currently handled by default, however you can easily add support by adding ManifestDestinationPath and TemplateManifestPath to the PropertyGroup in your UWP Project.

Q. Does it work on with Visual Studio and Visual Studio for Mac?
A. Yes it work on both Mac and Windows. As part of migrating this out of the templates, the tasks have been upgraded to compiled tasks meaning it works with MSBuild without any additional requirement for PowerShell.

Getting Started

For existing QuickStart Template Projects, you will need to delete the Directory.build.props and Directory.build.targets. You can also delete the PowerShell scripts. To get started, simply install the Mobile.BuildTools NuGet into any project you need to generate App Secrets or an iOS/Android project that you want to be able to swap out the Manifest for.