Firebird automated testing site
Alexey Kovyazin recently shared some interesting news:
FYI – http://firebirdtest.com
This website is a frontend of FBT + OLTP+automated scripts, supported by Pavel Zotov and some IBSurgeon employees.
It is running FBT tests on Windows server (8Gb, 4cores) for SuperServer and Classic for 3.0 and 4.0, and OLTP-EMUL for SuperServer 3.0/4.0.
Tests are triggered automatically, when new build appear at
As a result of test run, the report is generated.
P – passed (ok)
Failed test due to failed assert is marked as F (failed), pink
Failed due to runtime marked as E (error), yellow.
Failed due to Firebird crash – F red/violet
For each error there is a hint (mouseover) with details of the error.
After click on the cell, the new page will be opened with more details and history of runs.
- Total results for each kind of outcome and server mode
- ERRORS MAP: distribution of Firebird errors which occured during runs
- FRESH FAILS: tests which failed for LAST execution only
- FRESH ERRORS: tests which raised run-time error for LAST execution only
- STILL FAILS: tests which fail for at least two last runs
- Has fails: tests with at least one fail for previous runs
- Has runtime errors: tests with at least one runtime error for previous runs
- FRESH PASSED: successful tests with previous outcome = FAIL/ERROR/SKIPPED or absent
- All tests: outcomes – all results for last 30 runs
- All tests: A. Skipped: tests which was intentionally excluded from execution
Firebird OLTP Emul results
OLTP Emul simulates high load OLTP aplication, and uses many
sophisticated SQLs in multi-thread environment.
Our plan for the future:
1) Publish sources of automated scripts to GitHub (need to clean them a bit)
2) Perform Linux tests and include report results to the frontend
3) improve web representation – https certificate, design, etc
4) Migrate to the new FBT test framework when its adoption will be completed
We invite all interested to join the effort.
Pavel Zotov is a coordinator of the project – email@example.com.