Asserting state with #count on in Swift Testing – Donny Wals

Testing easy boolean situations with #count on

The commonest approach that you just’re in all probability going to be utilizing #count on is to make it possible for sure situations are evaluated to betrue. For instance, I’d need to take a look at that the operate beneath truly returns 5 every time I name it.

func returnFive() -> Int {
  return 0
}

After all this code is slightly bit foolish, it would not actually try this a lot, however you might think about {that a} extra difficult piece of code would should be examined extra totally.

Since I have never truly applied my returnFive operate but, it simply returns 0. What I can do now could be write a take a look at as proven beneath.

@Check func returnFiveWorks() async throws {
  let functionOutput = Incrementer().returnFive()
  #count on(5 == functionOutput)
}

This take a look at goes to check that once I name my operate, we get quantity 5 again. Discover the road the place it says #count on(5 == functionOutput).

That’s an assertion.

I’m making an attempt to claim that 5 equals the output of my operate through the use of the #count on macro.

When our operate returns 5, my expression (5 == functionOutput) evaluated to true and the take a look at will go. When the expression is false, the take a look at will fail with an error that appears a bit like this:

Expectation failed: 5 == (functionOutput → 0)

This error will present up as an error on the road of code the place the expectation failed. That signifies that we are able to simply see what went mistaken.

We are able to present extra context to our take a look at failures by including a remark. For instance:

@Check func returnFiveWorks() async throws {
  let functionOutput = Incrementer().returnFive()
  #count on(5 == functionOutput, "returnFive() ought to all the time return 5")
}

If we replace our exams to look slightly bit extra like this, if the take a look at fails we’ll see an output that is a little more elaborate (as you possibly can see beneath).

Expectation failed: 5 == (functionOutput → 0)
returnFive() ought to all the time return 5

I all the time like to put in writing a remark in my expectations as a result of this may present slightly bit extra context about what I anticipated to occur, making debugging my code simpler in the long term.

Typically talking, you are both going to be passing one or two arguments to the count on macro:

1. The primary argument is all the time going to be a Boolean worth
2. A remark that will likely be proven upon take a look at failure

So within the take a look at you noticed earlier, I had my comparability between 5 and the operate output inside my expectation macro as follows:

5 == functionOutput

If I had been to alter my code to appear like this the place I put the comparability outdoors of the macro, the output of my failing take a look at goes to look slightly bit totally different. Here is what it’s going to appear like:

@Check func returnFiveWorks() async throws {
  let functionOutput = Incrementer().returnFive()
  let didReturnFive = 5 == functionOutput
  #count on(didReturnFive, "returnFive() ought to all the time return 5")
}

// produces the next failure message:
// Expectation failed: didReturnFive
// returnFive() ought to all the time return 5

Discover how I am not getting any suggestions proper now about what may need gone mistaken. I merely get a message that claims “Expectation failed: didReturnFive” and no context as to what precisely may need gone mistaken.

I all the time suggest making an attempt to place your expressions contained in the count on macro as a result of that’s merely going to make your take a look at output much more helpful as a result of it’s going to examine variables that you just inserted into your count on macro and it’ll say “you anticipated 5 however you’ve got acquired 0”.

On this case I solely know that I didn’t get 5, which goes to be lots more durable to debug.

We are able to even have a number of variables that we’re utilizing inside count on and have the testing framework inform us about these as properly.

So think about I’ve a operate the place I enter a quantity and the quantity that I need to increment the quantity by. And I count on the operate to carry out the maths increment the enter by the quantity given. I may write a take a look at that appears like this.

@Check func incrementWorks() async throws {
  let enter = 1
  let incrementBy = 2
  let functionOutput = Incrementer().increment(enter: enter, by: incrementBy)
  #count on(functionOutput == enter + incrementBy, "increment(enter:by:) ought to add the 2 numbers collectively")
}

This take a look at defines an enter variable and the quantity that I need to increment the primary variable by.

It passes them each to an increment operate after which does an assertion that checks whether or not the operate output equals the enter plus the increment quantity. If this take a look at fails, I get an output that appears as follows:

Expectation failed: (functionOutput → 4) == (enter + incrementBy → 3)
increment(enter:by:) ought to add the 2 numbers collectively

Discover how I fairly conveniently see that my operate returned 4, and that’s not equal to enter + increment (which is 3). It is actually like this stage of element in my failure messages.

It’s particularly helpful while you pair this with the take a look at arguments that I lined in my put up on parameterized testing. You possibly can simply see a transparent report on what your inputs had been, what the output was, and what could have gone mistaken for every totally different enter worth.

Along with boolean situations like we’ve seen up to now, you would possibly need to write exams that examine whether or not or not your operate threw an error. So let’s check out testing for errors utilizing count on subsequent.

Testing for errors with #count on

Generally, the objective of a unit take a look at is not essentially to examine that the operate produces the anticipated output, however that the operate produces the anticipated error or that the operate merely would not throw an error. We are able to use the count on macro to claim this.

For instance, I may need a operate that throws an error if my enter is both smaller than zero or bigger than 50. Here is what that take a look at may appear like with the count on macro:

@Check func errorIsThrownForIncorrectInput() async throws {
  let enter = -1
  #count on(throws: ValidationError.valueTooSmall, "Values lower than 0 ought to throw an error") {
    attempt checkInput(enter)
  }
}

The syntax for the count on macro while you’re utilizing it for errors is barely totally different than you would possibly count on based mostly on what the Boolean model regarded like. This macro is available in numerous flavors, and I favor the one you simply noticed for my basic goal error exams.

The primary argument that we go is the error that we count on to be thrown. The second argument that we go is the remark that we need to print every time one thing goes mistaken. The third argument is a closure. On this closure we run the code that we need to examine thrown errors for.

So for instance on this case I am calling attempt checkInput which signifies that I count on that code to throw the error that I specified as the primary argument in my #count on.

If every part works as anticipated and checkInput throws an error, my take a look at will go so long as that error matches ValidationError.valueTooSmall.

Now as an instance that I by accident throw a distinct error for this operate the output will look slightly bit like this

Expectation failed: anticipated error "valueTooSmall" of kind ValidationError, however "valueTooLarge" of kind ValidationError was thrown as a substitute
Values lower than 0 ought to throw an error

Discover how the message explains precisely which error we obtained (valueTooLarge) and the error that we anticipated (valueTooSmall). It is fairly handy that the #count on macro will truly inform us what we obtained and what we anticipated, making it simple to determine what may have gone mistaken.

Including slightly remark similar to we did with the Boolean model makes it simpler to motive about what we anticipated to occur or what might be taking place.

If the take a look at doesn’t throw an error in any respect, the output would look as proven beneath

ExpectMacro.swift:42:3: Expectation failed: an error was anticipated however none was thrown
Values lower than 0 ought to throw an error

This error fairly clearly tells us that no error was thrown whereas we did count on an error to be thrown.

However what if we need to be certain that a sure operate name doesn’t throw an error?

For one, you possibly can simply name that operate in your take a look at:

@Check func noErrorIsThrown() async throws {
  attempt checkInput(5)
}

That is nice, however while you’re working with parameterized exams you’d have to put in writing two logic paths in your take a look at. It isn’t an enormous deal however we are able to do higher.

You possibly can examine for By no means being thrown to examine that no error is thrown in your take a look at. Here is how that appears:

@Check func noErrorIsThrown() async throws {
  let enter = 5
  #count on(throws: By no means.self, "Values between 0 and 100 needs to be okay") {
    attempt checkInput(enter)
  }
}

There may be conditions the place you do not actually care concerning the precise error being thrown, however simply that an error of a particular kind was thrown. For instance, I won’t care that my “worth too small” or “worth too massive” error was thrown, however I do care that the kind of error that acquired thrown was a validation error. I can write my take a look at like this to examine for that.

@Check func errorIsThrownForIncorrectInput() async throws {
  let enter = -1
  #count on(throws: ValidationError.self, "Values lower than 0 ought to throw an error") {
    attempt checkInput(enter)
  }
}

As a substitute of specifying the precise case on validation error that I count on to be thrown, I merely go ValidationError.self. It will permit my take a look at to go when any validation error is thrown. If for no matter motive I throw a distinct form of error, the take a look at would fail.

There is a third model of count on in relation to errors that we may use. This one would first permit us to specify a remark like we are able to in any count on. We are able to then go a closure that we need to execute (e.g. calling attempt checkInput) and a second closure that receives no matter error we obtained. We are able to carry out some checks on that after which we are able to return whether or not or not that was what we anticipated.

For instance, if in case you have a bit extra difficult setup the place you are throwing an error with an related worth you would possibly need to examine the related worth as properly. Here is what that might appear like.

@Check func errorIsThrownForIncorrectInput() async throws {
  let enter = -1
  #count on {
    attempt checkInput(enter)
  } throws: { error in 
    guard let validationError = error as? ValidationError else {
      return false
    }

    change validationError {
    case .valueTooSmall(let margin) the place margin == 1:
      return true
    default:
      return false
    }
  }
}

On this case, our validation logic for the error is fairly fundamental, however we may increase this in the actual world. That is actually helpful when you may have an advanced error or difficult logic to find out whether or not or not the error was precisely what you anticipated.

Personally, I discover that usually I’ve fairly easy error checking, so I’m typically utilizing the very first model of count on that you just noticed on this part. However I’ve undoubtedly dropped all the way down to this one once I wished to examine extra difficult situations to find out whether or not or not I acquired what I anticipated from my error.

What you want is, after all, going to rely by yourself particular scenario, however know that there are three variations of count on that you need to use when checking for errors, and that all of them have type of their very own downsides that you just would possibly need to have in mind.

In Abstract

Normally, I consider testing libraries by how highly effective or expressive their assertion APIs are. Swift Testing has performed a extremely good job of offering us with a fairly fundamental however highly effective sufficient API within the #count on macro. There’s additionally the #require macro that we’ll discuss extra in a separate put up, however the #count on macro by itself is already an effective way to begin writing unit exams. It offers lots of context about what you are doing as a result of it is a macro and it’ll increase into much more data behind the scenes. The API that we write is fairly clear, fairly concise, and it is highly effective to your testing wants.

Be certain to take a look at this class of Swift testing on my web site as a result of I had lots of totally different posts with Swift testing, and I plan to increase this class over time. If there’s something you need me to speak about by way of Swift testing, ensure you discover me on social media, I’d love to listen to from you.