Using Newman's built-in reporters

Reporting on tests in Newman

Newman is a powerful tool for running tests, but the point of having test automation isn't just so that you can run tests. The point of test automation is to be able to improve a product's quality based on the results you get. You need to know when tests fail so that you can follow up on those failures and see if they represent issues in the product. Effective test automation also requires paying attention to the tests themselves. You want to run tests that are effective and useful, and not just keep running a test that isn't telling you anything. In order to do all of this, you need test reporting. This section will help you understand how reporting works in Newman. You will learn how to use the built-in reporters that Newman has. You will also learn how to install and use community build reporters, and I will even walk you through an example that shows you how you can build your own Newman test reporter.

Using Newman's built-in reporters

You need reports on failed tests and summaries that show you how well your tests are doing. When you ran Newman earlier, you could see some reporting that was provided at the command line. These summaries are helpful when you are running locally, and Newman has several built-in reporter options that you can use to control that report's output. You can specify which reporter you want to use with the -r flag. If you do not specify a reporter to use, Newman defaults to using the cli reporter, but there are four other built-in reporters that you can use. The other available reporters are the json,

junit, progress, and emojitrain reporters.

The json and junit reporters will each save the report to a file. The json reporter saves the data in JSON format in a .json file, while the junit reporter saves the data in a JUnit-compatible .xml file. JUnit is a testing framework for the Java programming language. The JUnit XML format can be parsed by different programs, including build systems such as Jenkins, to display nicely formatted test results. The progress reporter will give you a progress bar in the console while the tests run. Although I have mentioned it here for completeness, the emojitrain reporter isn't something you will ever use. It prints out the progress as an emoji, but it is hard to get an emoji to display correctly in a Command Prompt, and there isn't any added value over what you can see with the progress reporter. You can use multiple reporters at once if you want. So, for example, you could use both the CLI and progress reporter at the same time by specifying them as a comma-separated list after the -r flag, like this:

-r cli,progress