Part 1. The problem
I am an iOS developer. And I love cats tests. Sometimes they love me back but recently with Swift Concurrency coming into my life, our relations went downhill.
The project is in limbo between the past and the future. Meaning we don't have SwiftUI everywhere in the project but we already use async/await in certain places, mostly in network services. It leads to cases where business entities have to call services via Task. Consider the following example where I fetch cats. The more the fluffier!
The view model that provides a count of cats in its state
final class ViewModel {
private(set) var state = State.loading
// Service gives us cats
private let catService: CatServicing
// Dependency injection in init
init(catService: CatServicing = CatService()) {
self.catService = catService
}
func start() {
requestCats()
}
private func requestCats() {
state = .loading
Task { @MainActor in
// To invoke async code, I have to either make the func async, or dispatch a Task
let catsResult = await catService.fetch()
switch catsResult {
case let .success(cats):
state = .loaded(catCount: cats.count)
case .failure:
state = .error(action: { [unowned self] in requestCats() })
}
}
}
}
extension ViewModel {
enum State {
case loading, error(action: () -> Void), loaded(catCount: Int)
}
}
CatService implementation is particularly sophistocated
struct Cat {}
protocol CatServicing {
// I use good old result because, in the future, I want to handle cat-errors like cat-overflow
func fetch() async -> Result<[Cat], Error>
}
final class CatService: CatServicing {
func fetch() async -> Result<[Cat], Error> {
try? await Task.sleep(for: .milliseconds(500))
return .success([Cat(), Cat()])
}
}
Let's write a test for the view model! First of all, we need a mocked CatService that we will inject later. For that, I use SwiftMockGeneratorForXcode. Btw, it was a great generator but it cannot handle async/await anymore. Still saves some time
@testable import TestsExample
final class DummyCatService: CatServicing {
var invokedFetch = false
var invokedFetchCount = 0
var stubbedFetchResult: Result<[Cat], Error>!
func fetch() async -> Result<[Cat], Error> {
invokedFetch = true
invokedFetchCount += 1
return stubbedFetchResult
}
}
Now writing the test, I provide the whole implementation just for reference.
final class ViewModelTest: XCTestCase {
private var sut: ViewModel!
private var catService: DummyCatService!
override func setUp() {
super.setUp()
catService = .init()
// Inject dependency
sut = .init(catService: catService)
}
func testServiceIsInvokedOnStart() {
// given
// Provide a stubbed response
catService.stubbedFetchResult = .success([Cat()])
// when
// Start the fetch
sut.start()
// then
// Expect the catService to be called
expect(self.catService.invokedFetchCount).to(equal(1))
}
}
What can go wrong? Everything! The lines are called in such order
// Test invokes start from the ViewModel
sut.start()
// The only invokation inside of ViewModel.start()
--> requestCats()
// In requestCats(), first we toggle the state
----> state = .loading
// Then in the same function we invoke Task to fetch things from the service
----> Task { @MainActor in
// The execution comes back to test. It validates the condition and fails miserably
expect(self.catService.invokedFetchCount).to(equal(1))
// P.S. How do normal people write down the call hierarhcy? I struggled a lot with it
The reason for the failure: lines are executed synchronously one by one. In its execution way, it comes to the Task. Contents of this Task are not invoked right away, rather they are dispatched. Meaning the block with let catsResult = await catService.fetch() will get executed but sometime later.
That means that all the nested synchronous instructions from sut.start() were executed and the execution continues from where it left off. The next line after sut.start is expect(self.catService.invokedFetchCount).to(equal(1)). Obviously, the condition validation fails. Task will still be executed but we've already left the local context of the test function. It's too late, cat
Let's now explore the bad ways of testing this code!
Part 2. Solutions
Bad way #1. Nimble's toEventually
A common approach for those who use Nimble. It's a bit like Bezoar
- "Just shove a toEventually down their throats."
expect(self.catService.invokedFetchCount).toEventually(equal(1))
Works fine.
Pros:
It's simple and takes little code
It needs little support and little knowledge
Cons:
It is flaky aka non-deterministic. One never knows if a certain lag will hit the
toEventuallytimeout. It heavily relies on the passage of time. How long the condition will take to become true is getting challenging to predict.Debugging tests that run in multiple threads is tricky. Add to that background threads executing something when the new test has already started. A nightmare.
Hard to test a reverse condition,
toAlways. When something is true but might occasionally become false time plays an even more critical role.toEventuallyis seeking the "right" answer in a given time window. What if we would want to test that state switches fromerrortoloadingwhen the cat request is done again? The initial state of the view model isloadingand it won't even change by the time we arrive to condition validation. Hello, false-positiveExecution time goes up due to waiting
What if the test needs the
sutto be in a certain state? In our example, we might want to check if the request is fired after failure. For that, we need to get the updated state. And it is not possible unless the task has been completed! We have to wait for the state even in thegivenpart and only then execute the test further.
// extra conformance just for tests to know what to wait for
expect(self.sut.state).toEventually(equal(.error))
guard case let .error(action) = sut.state else {
fail("Incorrect state")
return
}
// The actual when
action()
// finally, the then
expect(self.catService.invokedFetchCount).toEventually(equal(2))
Bad way #2. XCTest's expectation
We can utilise XCTestExpectation to know when exactly the Task was called! For that, we need just a little: make the dummy service publish when its stubbed function has been called
final class DummyCatService: CatServicing {
// now the property is published
@Published private(set) var invokedFetch = false
// ...
func fetch() async -> Result<[Cat], Error> {
// We could avoid using defer but I want to bring it as close to scope end as possible
defer {
invokedFetch = true
}
invokedFetchCount += 1
return stubbedFetchResult
}
}
Then our test grows up
func testServiceIsInvokedOnStartWithWait() {
// given
// we will wait for this expectation
let taskExpectation = XCTestExpectation()
catService.$invokedFetch
.filter { $0 }
.sink { _ in
// When the fetch func was invoked dummy fulfills the expectation
taskExpectation.fulfill()
}
.store(in: &cancellable)
catService.stubbedFetchResult = .success([Cat()])
// when
// start() asks dummy for the result, the sink closure above is executed
sut.start()
// And the wait is fulfilled, we can proceed to the assertion below
wait(for: [taskExpectation])
// then
expect(self.catService.invokedFetchCount).to(equal(1))
}
Pros:
- We are testing exactly when something is expected, not just probing the
catwater at arbitrary points in time.
Cons:
Still flaky, what if waiting takes more than one second? How large should be the timeout?
Still a long waiting time.
A lot of code.
What if the invoke called multiple times just with the delay?
It does not work in more complex examples. E.g. in the code snippet below the expectation immediately continues with
guard case let.
func testServiceIsInvokedAgainOnRetryWithWaitForExpectation() {
// given
let taskExpectation = XCTestExpectation()
catService.$invokedFetch
.filter { $0 }
.sink { _ in
taskExpectation.fulfill()
}
.store(in: &cancellable)
catService.stubbedFetchResult = .failure(NSError(domain: "cat tests", code: 636174))
sut.start()
wait(for: [taskExpectation])
guard case let .error(action) = sut.state else {
// early exit here. Expectation is fulfilled before ViewModel has managed to process the result of await
fail("Incorrect state")
return
}
// ...
}
The reason for that lies in the DummyService. It roughly looks so:
We are on the main thread, instructions are led one by one
deferin Dummy is called before returning the stubbed valuedeferchanges the value ofinvokedFetchand thus fulfils the expectationThe next instruction is
guard case let– we still haven't returned the value from Dummy.
So the awaiting finishes only when we have already failed the test. Such a misfortune.
let catsResult = await catService.fetch()
switch catsResult {
// ...
We could wrap the fulfilment in a Task in the test or the dummy and even add a sleep but then it's what we are trying to avoid, no?
.sink { _ in
Task {
// try? await Task.sleep(for: .microseconds(100))
taskExpectation.fulfill()
}
}
Even though it does not work, I loved playing with the idea and want to credit the source where I saw it the first time: https://stackoverflow.com/a/74052023/6624900
Bad way #3. Custom Task wrap
I love stubbing everything. If I could persuade teammates to stub an Int, I would do it. Just for the sake of chaos.
So why not to stub Task?
// Updated ViewModel.swift
func startWithTasker() {
requestCatsWithTasker()
}
private func requestCatsWithTasker() {
state = .loading
// Tasker is just a wrapper around Task that.. calls Task!
tasker.run { @MainActor [weak self] in
// We are also now notified about self being captured
guard let self else {
return
}
let catsResult = await catService.fetch()
switch catsResult {
case let .success(cats):
state = .loaded(catCount: cats.count)
case .failure:
state = .error(action: { self.requestCatsWithTasker() })
}
}
}
protocol Tasking {
func run(priority: TaskPriority?, operation: @escaping () async -> Void)
}
extension Tasking {
func run(operation: @escaping () async -> Void) {
run(priority: nil, operation: operation)
}
}
struct Tasker: Tasking {
func run(priority: TaskPriority? = nil, operation: @escaping () async -> Void) {
// As soon as run is called Tasker executes the Task. We don't need such a struct per se, it's required only for testing
Task(priority: priority) {
await operation()
}
}
}
DummyTasker then looks like that
final class DummyTasker: Tasking {
var invokedRun = false
var invokedRunCount = 0
var invokedRunParameters: (priority: TaskPriority?, operation: () async -> Void)?
var invokedRunParametersList = [(priority: TaskPriority?, operation: () async -> Void)]()
func run(priority: TaskPriority?, operation: @escaping () async -> Void) {
invokedRun = true
invokedRunCount += 1
invokedRunParameters = (priority, operation)
invokedRunParametersList.append((priority, operation))
}
}
And our test then looks rather simple
func testServiceIsInvokedOnStart() async{
// given
// Good old stubbed response
catService.stubbedFetchResult = .success([Cat()])
// when
// The same start function just with Tasker. Of course, it should not be called this way in production
sut.startWithTasker()
// DummyTasker got the operation and saved it in invokedRunParameters
// Now we simply invoke it on the same thread we are on. It is an absolutely synchronous operation
await tasker.invokedRunParameters?.operation()
// then
// Validating the condition
expect(self.catService.invokedFetchCount).to(equal(1))
}
Pros:
Only testing values when we expect them to change
It's all synchronous for the first time in the history of the Cativerse 🌏🐈
No extra execution time
Debugging is simple
Summarising all that – we stop testing parallelism per se, we mock it
Cons:
Taskis an essential part of SwiftToo many things that
Taskcan do are left out and need manual implementationExtra effort for supporting the wrapper
Managing
selfcapturing (Btw,Taskalso captures but the compiler does not show the warning. So maybe it's a pro after all)The more complex test is still complex, we need to call
asker.invokedRunParameters?.operation()after each time tasker is accessed. So we know the internal implementation details. Not as much of a black box testing really
func testServiceIsInvokedAgainOnRetryWithTasker() async {
// given
catService.stubbedFetchResult = .failure(NSError(domain: "cat tests", code: 636174))
sut.startWithTasker()
// First invoke the operation to let ViewModel "consume" error response and react to it
await tasker.invokedRunParameters?.operation()
guard case let .error(action) = sut.state else {
fail("Incorrect state")
return
}
// when
action()
// Then awaiting for the next request
await tasker.invokedRunParameters?.operation()
// then
expect(self.catService.invokedFetchCount).to(equal(2))
}
Part 3. Instead of a summary
In the end, I am torn between toEventually and Tasker. Both have downsides and some upsides. I could summarise it with a short quote from the early 19th century:
💩🐿️
it seems the tests will never be as easy as before – unless we find a better approach. Which we as a society usually do :)
Part 4. Other options
- We could also make the view model work in async/await mode and extract
Taskto controllers or whatever is up the hierarchy. It's not always possible but might make sense to move the concurrency managing code to the view because we cannot test it there and can live happily. Haha what am I saying
P.S.
Do you like any of the approaches? Which one is the winner in this worst-of-the-worst chart? Do you have better approaches and how do you test your async code in general?