An extension library for triggering screenshots/marking test steps when running XCUITests in Visual Studio Mobile Center / Xamarin Test Cloud. By default, test steps are automatically marked at the conclusion of each test method (which also triggers a screenshot); this library allows you to explicitly declare where you want to mark your test steps / trigger screenshots.

• Requirements
  • Using
  • Building
• Installation
  • Cocoapods
  • Carthage
  • Building from source
• Usage
  • Objective-C
  • Swift
• Preparing Your Application Bundles
• Uploading Your Tests
• Known Issues

Requirements

Using

VSMobileCenterExtensions.framework is built for iOS 9.0 or later.

Building

VSMobileCenterExtensions has been tested on OSX El Capitan (10.11.5) and Xcode 8.2. However, it should work on newer versions of Xcode/OSX. Feel free to file an issue if you have trouble with a newer OS / Xcode configuration.

In order to run the unit tests, you will also need to change the project's codesign info locally to point to a certificate installed on your own machine.

If you want to install the xcpretty gem (see building from source below), you will also need a valid ruby installation. Ruby 2.2.1 or higher is recommended.

Installation

The extension can be added to your Xcode XCUITest target by using a dependency manager (Cocoapods or Carthage) or by manually including it in the target.

Building from source

Optional: It is recommended that you install the xcpretty gem for nicer output:

$ gem install xcpretty

1) Run:

$ make

This will build VSMobileCenterExtensions.framework inside of the build/Release-iphoneos folder.

2) Copy VSMobileCenterExtensions.framework into your application's project folder.

3) In Xcode, in the 'Build Phases' tab of your UI Test target (not your main application target), add the VSMobileCenterExtensions.framework in the 'Link Binary With Libraries' phase.

4) In the same tab, add the VSMobileCenterExtensions.framework to your 'Copy Files' phase.

You may need to create one if you don't already have one. To do so, click the '+' sign on the top left of the pane:

Once you have a 'Copy Files' phase, click the '+' button on the bottom left of the phase's pane to add a new file.

Click 'Add Other...' and navigate to the framework:

Make sure that the 'Copy Files' phase's destination is set to 'Frameworks'.

When you're all done, your Build Phases pane should resemble the following:

Usage

The VSMobileCenterExtensions framework exposes a label functionality that you can use to trigger a screenshot and mark a particular point in your UI Test with a label of your choosing.

Objective-C

In Objective-C, the method is called +[MCLabel label:] or label() for short. It accepts a format string and arguments just like NSLog():

#import <VSMobileCenterExtensions/VSMobileCenterExtensions.h>

- (void)testExample {
    //Some test logic...

    [MCLabel label:fmt, ...];
    // or
    label(fmt, ...);

    //More test logic...
}

Swift

In Swift the function is called MCLabel.labelStep() and it accepts a string, and an optional vaList if you want to use an Objective-C NSLog()-style format string:

import VSMobileCenterExtensions

class MyTestCase: XCTestCase {
    func textExample() {
        //Some test logic...

        MCLabel.labelStep(label)
        //or
        MCLabel.labelStep(fmt, args: getVaList([ arg, ... ]))

        //More test logic...
    }
}

Preparing Your Application Bundles

In order to run a test in Xamarin Test Cloud or Mobile Center, you will need to build your application and XCUITest runner bundles. To do this, run the following command from the root of your application project directory:

$ xcrun xcodebuild build-for-testing -configuration Debug -workspace YOUR_WORKSPACE -sdk iphoneos -scheme YOUR_APPLICATION_SCHEME -derivedDataPath .

This will build your Application and your XCUITest-Runner into a local directory called Build (specifically, the bundles are in Build/Products/Debug-iphoneos/).

YOUR_WORKSPACE should point to a .xcworkspace file, likely titled PROJECT_NAME.xcworkspace. YOUR_APPLICAITON_SCHEME should be the scheme you use to build your application. By default it is usually the name of your application. If you are unsure, you can run

$ xcrun xcodebuild -list

to see a list of valid schemes. For more information about Xcode schemes, see the Apple Developer Documentation.

Uploading Your Tests

First make sure you have the xtc uploader tool by following the installation instructions.

If you do not have an existing device key ready, you can generate one by following the new test run dialog in Test Cloud. On the final screen, extract only the device key from the generated command.

To upload your tests, run the following command:

$ xtc xcuitest <api-key> --devices <selection> --user <email> --workspace Build/Products/Debug-iphoneos

Note: If you are having trouble targeting the xtc command, try executing with the fully qualified path to the package

Known Issues

When performing gestures in XTC/Mobile Center Test, you may see an error message like the following:

UI Testing Failure - Failed to scroll to visible (by AX action) Button
...
Error -25204 performing AXAction 2003

Presently, the issue not fully understood and believed to originate in XCTest.framework. However, evidence suggests that one possible cause is related to XCUITest not being able to 'see' the element in the hierarchy when the gesture is invoked.

While not foolproof, as a potential workaround and general improvement to test stability, we recommend adapting the following scaffolding code to your gestures invocation (example is for a tap gesture):

Objective-C

- (void)waitAndTap:(XCUIElement *)button {
    NSPredicate *pred = [NSPredicate predicateWithFormat:@"exists == 1 && hittable == 1"];
    [self expectationForPredicate:pred
              evaluatedWithObject:button
                          handler:nil];
    [self waitForExpectationsWithTimeout:180.0 handler:nil];
    [button tap];
}

Swift

func waitAndTap(element: XCUIElement) {
    let predicate = NSPredicate(format: "exists == 1 && hittable == 1")
    expectation(for: predicate, evaluatedWith: element)
    waitForExpectations(timeout: 5 /*Or a larger value if necessary*/)
    element.tap()
}

You would then invoke waitAndTap instead of tap to ensure that the element in question is in a hittable state.

Note that in XTC/Mobile Center Test, this issue appears to only be prevelant on iPhone 7 devices.

Xcode 8.3 and Swift

If you are building Swift XCUITests using Xcode >= 8.3, you may encounter a build error related to bitcode. As a workaround, you can disable bitcode in your XCUITest target. To do this, go to Build Settings, search for ENABLE_BITCODE and set the value to NO for the test target. You should not need to change the setting for the App target.