We introduce setup_stdio function to setup stdout/stderr properly.
For python <-> python pipe, we always use 'utf8'/'ignore' encoding for not lost characters.
For tty <-> python, we using native encoding with xmlcharrefreplace to encode, to preserve maximal information.
For python <-> native program, we use naive encoding with 'ignore' to not cause error
update_exclude_list with binary mode so that on win32 would not generate \r\n
run-test-suite.py: Handling skiplist properly on win32
Fixes#4854
Fixes test262-harness.py complain cannot use a string pattern on a bytes-like object with running test262 with python3
For reading/writing to file, we use 'utf8' /'ignore' encoding for not lost characters.
For decoding from process stdout, using native encoding with decoding error ignored for not lost data.
Execute commands also ignore errors
Fixes#4853
Fixes running test262-esnext failed with installed python3.9 on win32 with space in path
Fixes#4852
support both / \ in --test262-test-list arg
On win32.
python tools/run-tests.py --test262-es2015=update --test262-test-list=built-ins/decodeURI/
python tools/run-tests.py --test262-es2015=update --test262-test-list=built-ins\decodeURI\
should be both valid,
currently only --test262-test-list=built-ins\decodeURI\ are valid.
Support snapshot-tests-skiplist.txt on win32 by use os.path.normpath
Guard run-tests.py with timer.
All run-tests.py are finished in 30 minutes in normal situation.
May change the timeout by command line option
Move Windows CI to github actions
Define TERM colors for win32 properly
flush stderr.write stdout.write
On CI, the stderr are redirect to stdout, and if we don't flush stderr and stdout,
The output from stderr/stdout would out of sync.
`Testing new Date(-8640000000000000) and fixes date for win32`
So that the CI can passed
if sys.version_info.major >= 3: remove, as we do not support python2 anymore
Fixes compiling warnings/errors for mingw/gcc
JerryScript-DCO-1.0-Signed-off-by: Yonggang Luo luoyonggang@gmail.com
Update code to conform to the newer version of pylint available in
ubuntu-22.04, with few exceptions:
- disabled `import-outside-toplevel` for `main()` in
`jerry_client.py`
- disabled `consider-using-with` for the logfile of `TestSuite` in
`test262-harness.py` as using `with` is not practical in that case
Update test262-harness.py to use argparse instead of the now deprecated
optparse
Rename variables in jerry_client_main.py that redefined python builtins
or shadowed variables from an outer scope
Update python files to use f-stirngs
Add minimum python versions (3.6 and 3.8) to the CI jobs: without it the
default python version did not support the `with` statement for
`subprocess.Popen` used in `build.py` on macos, or in some cases f-stirngs
Remove `from __future__` imports that are no-ops in python 3
Remove shebang from non executable files
Re-enable most pylint checkers, except `missing-docstring`
JerryScript-DCO-1.0-Signed-off-by: Máté Tokodi mate.tokodi@szteszoftver.hu
Update debugger and test262 tests to python3
Disable CI jobs that need ubuntu-18.04 (EOL) until they are updated
JerryScript-DCO-1.0-Signed-off-by: Máté Tokodi matet@inf.u-szeged.hu
- remove all '#JERRY_ESNEXT' macro
- remove 5.1 build profile, update test runner accordingly (Note: all builtins are turn on by default)
- move tests from tests/jerry/esnext into tests/jerry, concatenate files with same names
- add skiplist to some snapshot tests that were supported only in 5.1
- fix doxygen issues that were hidden before (bc. of es.next macro)
Co-authored-by: Martin Negyokru negyokru@inf.u-szeged.hu
JerryScript-DCO-1.0-Signed-off-by: Adam Szilagyi aszilagy@inf.u-szeged.hu
ES2015 features are already covered by ES.Next tests,
there is no need to run duplicated and outdated tests.
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác csaba.osztrogonac@h-lab.eu
- Fix isValidIntegerIndex usage in IntegerIndexed objects, [[Get]], [[Set]], [[DefineOwnProperty]] internal methods.
- The test262 revision is updated to the latest master hash.
- The PR is not separated into individual PRs since the standard change couldn't be tested without the new test262 revision.
JerryScript-DCO-1.0-Signed-off-by: Adam Szilagyi aszilagy@inf.u-szeged.hu
It fixes the following test on Windows:
built-ins/Function/prototype/toString/line-terminator-normalisation-LF.js
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác csaba.osztrogonac@h-lab.eu
Now the test-runner will check the jerry (server) return code.
Fixed do_eval_at TC crash if jerry was build with --debug option.
JerryScript-DCO-1.0-Signed-off-by: bence gabor kis kisbg@inf.u-szeged.hu
The --test262-object option should be passed independently,
otherwise os.path.dirname(args.engine) in tools\runners\run-test-suite-test262.py
makes no sense
JerryScript-DCO-1.0-Signed-off-by: Yonggang Luo luoyonggang@gmail.com
This makes have option to running test262-es2015 and test262-esnext with debug version
of jerryscript.
On OSX, the $RUNNER -q --jerry-tests are in dead loop, disable it first.
Increase TEST262_CASE_TIMEOUT to 180, so that --build-debug can pass the tests
without timeout
An side effect of this are
```
<test id="built-ins/decodeURIComponent/S15.1.3.2_A2.5_T1.js"><reason></reason></test>
<test id="built-ins/decodeURI/S15.1.3.1_A2.5_T1.js"><reason></reason></test>
```
passed
JerryScript-DCO-1.0-Signed-off-by: Yonggang Luo luoyonggang@gmail.com
When running tests262 cases, the jerry program may exit with error code other than 1,
that's means a crash or JERRY_ASSERT triggered and should be resolved, so dump the test
name and stderr/stdout for tracking those crash in CI.
related issue: #4463
JerryScript-DCO-1.0-Signed-off-by: Yonggang Luo luoyonggang@gmail.com
get_platform_cmd_prefix implement multiple times in multiple function.
Also add function get_python_cmd_prefix, call to python script always
add python command so that on windows, the python script doesn't have
executable permission also can be called.
JerryScript-DCO-1.0-Signed-off-by: Yonggang Luo luoyonggang@gmail.com
Since ES2018 iterator's next method is called once during the prologue of iteration,
rather than during each step. The test is incorrect and stuck in an infinite loop.
https://github.com/tc39/test262/pull/1248 fixed the test and it passes on test262-esnext.
Removing test/language/statements/for-of/iterator-next-reference.js from test262-es2015
is necessary, because it won't pass ever and only slow down testing.
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác csaba.osztrogonac@h-lab.eu
This patch disables automatic detection of module code, and instead
requires the user to explicitly specify whether to parse a source
as a module or as a script.
To achieve this the jerry_parse API function now takes a new option
which signals that the source should be parsed as a module.
JerryScript-DCO-1.0-Signed-off-by: Dániel Bátyai daniel.batyai@h-lab.eu
Use the platform independent python threading.Timer instead of the unix only timeout tool.
The timeout error are like the following things:
```
C:\Users\lygstate>timeout -version
Error: The specified timeout (/T) value is invalid. The valid range is from -1 to 99999 seconds.
C:\Users\lygstate>timeout 0 python
Error: invalid syntax. The default option does not allow more than '1' times.
Type "TIMEOUT /?" to learn how to use it.
```
JerryScript-DCO-1.0-Signed-off-by: Yonggang Luo luoyonggang@gmail.com
Less operator is not supported between int and None in Python 3. Fix
that Python 2 legacy in test262-harness.py.
JerryScript-DCO-1.0-Signed-off-by: Yonggang Luo luoyonggang@gmail.com
built-ins/RegExp/CharacterClassEscapes tests need more memory than 512Kb.
Additionally increase the timeout to make all tests pass on GitHub CI.
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác csaba.osztrogonac@h-lab.eu
This PR aims to avoid serializing the exclude_list
before each test_case.
test-262-esnext execution takes only ~2 mins (8-thread).
JerryScript-DCO-1.0-Signed-off-by: Adam Kallai kadam@inf.u-szeged.hu
Changes:
- Add new option to run-tests.py: --test262-test-list to run selected tests only
- Fix exclude list updater accordingly
- Run ESNext tests on GitHub CI in two batches to decrease runtime
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác csaba.osztrogonac@h-lab.eu
Changes:
- Progress meter is precise and less verbose
- Omit list of failures, test262.report file contains it
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác csaba.osztrogonac@h-lab.eu
Changes:
- Don't dump each test output to stdout
- Upload test outputs as artifact to GitHub
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác csaba.osztrogonac@h-lab.eu
Changes:
- Imported and unified test262 test harness for ES2015 and ESNext
- Simplified runner scripts accordingly
- Run tests on CI to be able detect regressions and progressions too
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác csaba.osztrogonac@h-lab.eu
text parameter of Popen introduced in Python 3.7,
we should use the equivalent universal_newlines.
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
Changes:
- Add options to the test runner to be able to update exclude list
- Update the exclude list
- Make Travis CI signal if the exclude list isn't up-to-date
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
The output of the test262 runner changed between ES5.1 and ES6,
our script should respect it to signal regressions in return code too.
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
With this option you can call a function after the user script and promises have ran, to be able to do assertions that are executed just before the process would exit.
JerryScript-DCO-1.0-Signed-off-by: Daniel Balla dballa@inf.u-szeged.hu
Changes:
- Crashing jerry / jerry-snapshot is always "FAIL",
only the 1 return code is accepted as expected fail.
- "FAIL (XPASS)" string is only used if the test marked
as expected fail, but the return code is 0.
- Simple "FAIL" string is used if the test marked as expected
fail, but the return code is not 0 or 1. (crash)
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
Changes:
- Use list comprehension instead of filter in get_tests
to ensure returning list instead of iterable object
- Fix unicode decoding issues with Popen call
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
Changes:
* Bash based runners/run-test-suite.sh runner replaced with a python runner
* Common util functions moved to util.py
* Fixed EOL issues in tests directory
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
Changes:
- Implemented jerry_port_get_local_time_zone_adjustment on Windows
- Implemented jerry_port_get_current_time on Windows
- Run test262 tests on Windows in PST/PDT timezone
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
subprocess.Popen needs universal_newlines=True parameter
to open the file in text mode instead of binary mode.
With this fix readline() returns str instead of bytes.
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
Changes:
* Bash based unittest runner replaced with a python runner
* Typo fixed in doctest cmake build system (python executable)
* run-tests.py prints error message if build fails
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
The test runner used `&>>` to channel stderr to stdout and append
all that to a file. However, that's Bash 4 syntax, which is not
available everywhere, e.g., on MacOS. Bash pre-4 syntax requires
the classic `>>file 2>&1` notation.
This commit rewrites the test runner to use the portable syntax.
Moreover, it also rewrites `&>` to `>file 2>&1` for the sake of
consistency.
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss akiss@inf.u-szeged.hu
The previous jerry_port interface did not allow timezones to be handled correctly,
even if the host system was up to the task. This PR changes the jerry_port interface
to allow a completely correct implementation to exist, and provides one (for Linux/BSD
systems) in default-date.c.
Fixes#1661
JerryScript-DCO-1.0-Signed-off-by: crazy2be crazy1be@gmail.com
Previously, these would run for several minutes without printing
any progress. Now, at least print the number of tests executed.
JerryScript-DCO-1.0-Signed-off-by: crazy2be crazy1be@gmail.com
The only such test was a legacy of the long removed compact
profile, which is also removed.
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss akiss@inf.u-szeged.hu
