Test recipe
Please download the pre-compiled binary from Latest Release. If you are using Windows, please download the zip file like tbx-xx.x.xxx-win.zip
. Then, extract the archive and place tbx.exe
on the Desktop folder.
The watermint toolbox can run from any path in the system if allowed by the system. But the instruction samples are using the Desktop folder. Please replace the path if you placed the binary other than the Desktop folder.
This document uses the Desktop folder for command example.
Windows:
cd $HOME\Desktop
.\tbx.exe dev test recipe
macOS, Linux:
$HOME/Desktop/tbx dev test recipe
Note for macOS Catalina 10.15 or above: macOS verifies Developer identity. Currently, tbx
is not ready for it. Please select “Cancel” on the first dialogue. Then please proceed “System Preference”, then open “Security & Privacy”, select “General” tab.
You may find the message like:
“tbx” was blocked from use because it is not from an identified developer.
And you may find the button “Allow Anyway”. Please hit the button with your risk. At second run, please hit button “Open” on the dialogue.
DIRECT
if you want to skip setting proxy.Report file path will be displayed last line of the command line output. If you missed the command line output, please see path below. [job-id] will be the date/time of the run. Please see the latest job-id.
OS | Path pattern | Example |
---|---|---|
Windows | %HOMEPATH%\.toolbox\jobs\[job-id]\reports |
C:\Users\bob.toolbox\jobs\20190909-115959.597\reports |
macOS | $HOME/.toolbox/jobs/[job-id]/reports |
/Users/bob/.toolbox/jobs/20190909-115959.597/reports |
Linux | $HOME/.toolbox/jobs/[job-id]/reports |
/home/bob/.toolbox/jobs/20190909-115959.597/reports |
Recipe test result
The command will generate a report in three different formats. result.csv
, result.json
, and result.xlsx
.
Column | Description |
---|---|
path | Path to the recipe |
name | Name of the recipe |
skip | True if the test skipped |
timeout_enabled | True if timeout mode enabled |
use_mock | True if use mock mode enabled |
timeout | True if the test timeout |
duration | Test duration in milliseconds |
no_error | True if no error received |
error | Error |
If you run with -budget-memory low
option, the command will generate only JSON format report.
In case of a report becomes large, a report in .xlsx
format will be split into several chunks like follows; result_0000.xlsx
, result_0001.xlsx
, result_0002.xlsx
, …