14 Commits

Author SHA1 Message Date
Csaba Osztrogonác
8964a2bd18
Optimize test262 runner (#4120)
Changes:
- Add new option to run-tests.py: --test262-test-list to run selected tests only
- Fix exclude list updater accordingly
- Run ESNext tests on GitHub CI in two batches to decrease runtime

JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác csaba.osztrogonac@h-lab.eu
2020-08-13 13:47:14 +02:00
Csaba Osztrogonác
401b36d406
Make test262 output less verbose (#4108)
Changes:
- Progress meter is precise and less verbose
- Omit list of failures, test262.report file contains it

JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác csaba.osztrogonac@h-lab.eu
2020-08-04 16:54:06 +02:00
Csaba Osztrogonác
f1eba9f826
Make Test262 results less verbose (#4090)
Changes:
- Don't dump each test output to stdout
- Upload test outputs as artifact to GitHub

JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác csaba.osztrogonac@h-lab.eu
2020-08-03 10:40:26 +02:00
Csaba Osztrogonác
40ad8c6e45
Add --test262-esnext option to run-tests.py (#4027)
Changes:
- Imported and unified test262 test harness for ES2015 and ESNext
- Simplified runner scripts accordingly
- Run tests on CI to be able detect regressions and progressions too

JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác csaba.osztrogonac@h-lab.eu
2020-07-31 15:52:45 +02:00
Csaba Osztrogonác
fee3c295a2
Fix test262 excludelist updater (#4023)
The previous fix was buggy, header and footer shouldn't be duplicated.

JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác csaba.osztrogonac@h-lab.eu
2020-07-20 19:09:44 +02:00
Csaba Osztrogonác
b162e27418
Fix test262 excludelist updater (#4021)
tools/run-tests.py --test262-es2015 update shouldn't
remove comments and newlines in the excludelist file.

JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác csaba.osztrogonac@h-lab.eu
2020-07-20 13:47:45 +02:00
Csaba Osztrogonác
89342ea503
Make test262-es2015 output more verbose (#3924)
It makes easier to debug unexpected failures on the CI.

JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
2020-06-30 12:47:54 +02:00
Csaba Osztrogonác
c55a0baaf8
Improve test262 ES2015 runner (#3892)
Changes:
 - Add options to the test runner to be able to update exclude list
 - Update the exclude list
 - Make Travis CI signal if the exclude list isn't up-to-date

JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
2020-06-17 15:42:51 +02:00
Csaba Osztrogonác
ab3461b63d
Fix tools/run-tests.py --test262-es2015 (#3803)
The output of the test262 runner changed between ES5.1 and ES6,
our script should respect it to signal regressions in return code too.

JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
2020-05-28 14:30:49 +02:00
Csaba Osztrogonác
f0d443daec
Add --test262-es2015 option to run-tests.py (#3667)
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
2020-05-18 16:43:17 +02:00
Csaba Osztrogonác
44b1b9855d Make run-tests --jerry-tests --jerry-test-suite work on Windows too (#3260)
Changes:
* Bash based runners/run-test-suite.sh runner replaced with a python runner
* Common util functions moved to util.py
* Fixed EOL issues in tests directory

JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
2019-11-06 15:12:50 +01:00
Csaba Osztrogonác
2b3faf683d Make all test262 tests pass on Windows (#3157)
Changes:
- Implemented jerry_port_get_local_time_zone_adjustment on Windows
- Implemented jerry_port_get_current_time on Windows
- Run test262 tests on Windows in PST/PDT timezone

JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
2019-09-26 14:31:22 +02:00
Csaba Osztrogonác
137dd8d1f5 Fix run-test-suite-test262.py with python3 (#3139)
subprocess.Popen needs universal_newlines=True parameter
to open the file in text mode instead of binary mode.
With this fix readline() returns str instead of bytes.

JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
2019-09-23 22:06:02 +02:00
Csaba Osztrogonác
3e661c0c5a Make run-tests --test262 work on Windows too (#3029)
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
2019-09-04 10:29:21 +02:00