With this option you can call a function after the user script and promises have ran, to be able to do assertions that are executed just before the process would exit.
JerryScript-DCO-1.0-Signed-off-by: Daniel Balla dballa@inf.u-szeged.hu
Changes:
- Crashing jerry / jerry-snapshot is always "FAIL",
only the 1 return code is accepted as expected fail.
- "FAIL (XPASS)" string is only used if the test marked
as expected fail, but the return code is 0.
- Simple "FAIL" string is used if the test marked as expected
fail, but the return code is not 0 or 1. (crash)
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
Changes:
- Use list comprehension instead of filter in get_tests
to ensure returning list instead of iterable object
- Fix unicode decoding issues with Popen call
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
Changes:
* Bash based runners/run-test-suite.sh runner replaced with a python runner
* Common util functions moved to util.py
* Fixed EOL issues in tests directory
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
Changes:
- Implemented jerry_port_get_local_time_zone_adjustment on Windows
- Implemented jerry_port_get_current_time on Windows
- Run test262 tests on Windows in PST/PDT timezone
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
subprocess.Popen needs universal_newlines=True parameter
to open the file in text mode instead of binary mode.
With this fix readline() returns str instead of bytes.
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
Changes:
* Bash based unittest runner replaced with a python runner
* Typo fixed in doctest cmake build system (python executable)
* run-tests.py prints error message if build fails
JerryScript-DCO-1.0-Signed-off-by: Csaba Osztrogonác oszi@inf.u-szeged.hu
The test runner used `&>>` to channel stderr to stdout and append
all that to a file. However, that's Bash 4 syntax, which is not
available everywhere, e.g., on MacOS. Bash pre-4 syntax requires
the classic `>>file 2>&1` notation.
This commit rewrites the test runner to use the portable syntax.
Moreover, it also rewrites `&>` to `>file 2>&1` for the sake of
consistency.
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss akiss@inf.u-szeged.hu
The previous jerry_port interface did not allow timezones to be handled correctly,
even if the host system was up to the task. This PR changes the jerry_port interface
to allow a completely correct implementation to exist, and provides one (for Linux/BSD
systems) in default-date.c.
Fixes#1661
JerryScript-DCO-1.0-Signed-off-by: crazy2be crazy1be@gmail.com
Previously, these would run for several minutes without printing
any progress. Now, at least print the number of tests executed.
JerryScript-DCO-1.0-Signed-off-by: crazy2be crazy1be@gmail.com
The only such test was a legacy of the long removed compact
profile, which is also removed.
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss akiss@inf.u-szeged.hu
With this feature the use can restart the actual debug session
(similar to the multiple source context reset) within a client.
JerryScript-DCO-1.0-Signed-off-by: Imre Kiss kissi.szeged@partner.samsung.com
- Add `--install` option to `tools/build.py`.
- Make use of `--install` in `tools/run-tests.py` by testing the
installed the executables instead of those in the build tree.
Related changes:
- Collect unit test binaries in the `tests` subdir of the build
tree instead of `bin`.
- The `ls`-based collection of the unit test binaries had some
shortcomings hitherto unrevealed (it didn't filter for files so
it could potentially "collect" dictionaries, too), which has now
been replaced with a more stable `find`-based solution.
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss akiss@inf.u-szeged.hu
If the result of a build is not directly executable on the host
system or needs an execution runtime for any other reason (e.g.,
cross-compiled ARM binaries on Intel, emscripten-generated JS
files, binaries built with Valgrind support) then the current test
runners cannot work with them. This patch makes test runners
execute binaries via a runtime given in the `$RUNTIME` environment
variable (if it is specified).
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss akiss@inf.u-szeged.hu
Also remove eval context support. It provides no practical advantage.
JerryScript-DCO-1.0-Signed-off-by: Zoltan Herczeg zherczeg.u-szeged@partner.samsung.com
In one of the previous PR we have encountered a problem,
where the Travis cuts off the test execution
as the generated log file is too big.
By adding a quiet mode for the test runners, we will
only report the failing tests.
JerryScript-DCO-1.0-Signed-off-by: Peter Gal pgal.u-szeged@partner.samsung.com
The test262 python script is not python 3 compatible
so we force it to be executed with python2.
Additionally print out the reason if the execution
failed.
JerryScript-DCO-1.0-Signed-off-by: Peter Gal pgal.u-szeged@partner.samsung.com
Now there is a snapshot tool which can merge snapshots
but was unable to generate snapshots by itself.
This change moves the snapshot generation utilities
to the snapshot tool.
JerryScript-DCO-1.0-Signed-off-by: Peter Gal pgal.u-szeged@partner.samsung.com
Whit this enhancement the debugger can able handle more than one source file across the new source wait mode.
This feature can be used by the python client with the --client-source [paths] switch.
The client will store every source path, when the debugger send a signal about the waiting status, then the client will send one file from the list.
JerryScript-DCO-1.0-Signed-off-by: Imre Kiss kissi.szeged@partner.samsung.com
With this feature the debugger webIDE and the python client can able to send a source code to the debugger while that is running in wait mode.
This feature can be activated with the --debugger-wait-source switch and the debugger will wait for the source messages.
If every message part are received the debugger will continue the exectuion with the initalized options.
JerryScript-DCO-1.0-Signed-off-by: Imre Kiss kissi.szeged@partner.samsung.com
There is no need to create error code named subfolders since
every failing tests return with the same code.
JerryScript-DCO-1.0-Signed-off-by: Robert Sipka rsipka.uszeged@partner.samsung.com
First issue was that `diff -u0` was not supported on OSX, thus got
replaced by `-U0`. Then, it turned out that test outcome was
determined based on diff's stdout, which was empty even though the
exit code was non-0 - this got changed, too.
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss akiss@inf.u-szeged.hu
On OSX, the traditional GNU `timeout` command is available as
`gtimeout`. So, this commit adds some logic to the test runners to
try and find the right command name on the host.
Moreover, the commit also unifies the timeout mechanism across all
test runners by rewriting occurrences of `ulimit`-based timeouting
to use the `[g]timeout` command.
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss akiss@inf.u-szeged.hu
Those tests are related the actual daylight saving adjustment.
Related issue: #1661
JerryScript-DCO-1.0-Signed-off-by: László Langó llango.u-szeged@partner.samsung.com
This makes the code more robust and error prone, since certain
messages cannot be received in certain situations, e.g. an eval
command during garbage collection.
JerryScript-DCO-1.0-Signed-off-by: Zoltan Herczeg zherczeg.u-szeged@partner.samsung.com
Add `--skip-list` option to tools/run-tests.py.
Additional modifications:
- The test `parser-oom.js` doesn't work when system-allocator is enabled,
hence we skip it in these jobs.
- The sanitizer-tasks become mandatory.
JerryScript-DCO-1.0-Signed-off-by: Zsolt Borbély zsborbely.u-szeged@partner.samsung.com
The debugger supports setting breakpoints, execution control (step, next, continue)
and getting backtrace. The communication is WebSocket-based, so a browser can
communicate with JerryScript without any intermediate application.
JerryScript-DCO-1.0-Signed-off-by: Zoltan Herczeg zherczeg.u-szeged@partner.samsung.com
JerryScript-DCO-1.0-Signed-off-by: Levente Orban orbanl@inf.u-szeged.hu
Since the project is now hosted at the JS Foundation we can move to unified copyright notices for the project.
Starting with this commit all future contributions to the project should only carry the following copyright notice (except for third-party code which requires copyright information to be preserved):
"Copyright JS Foundation and other contributors, http://js.foundation" (without the quotes)
This avoids cluttering the codebase with contributor-specific copyright notices which have a higher maintenance overhead and tend to get outdated quickly. Also dropping the year from the copyright notices helps to avoid yearly code changes just to update the copyright notices.
Note that each contributor still retains full copyright ownership of his/her contributions and the respective authorship is tracked very accurately via Git.
JerryScript-DCO-1.0-Signed-off-by: Tilmann Scheller t.scheller@samsung.com
The manual of `mktemp` states that "TEMPLATE must contain at least
3 consecutive 'X's in last component." Linux implementation seems
to be relaxed about this and accepts and rewrites X's even inside
the template, but mktemp of OSX is more strict and handles trailing
X's only. This patch makes sure that mktemp templates work on both
OS's.
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss akiss@inf.u-szeged.hu
We removed that implementation where the build directory isn't set up to build with exactly one
configuration of the project but potentially several variants: the same build directory
can/must be used for debug and release builds, for full or compact profile versions, etc.
So we reworked the CMakeLists, and now one build dir deal with exactly one configuration
of the project's libraries and tools.
JerryScript-DCO-1.0-Signed-off-by: Zsolt Borbély zsborbely.u-szeged@partner.samsung.com
JerryScript-DCO-1.0-Signed-off-by: Robert Sipka rsipka.uszeged@partner.samsung.com
* Improve the name of the temporary snapshot files in test runner:
Until now, test runner used a completely random filename for the
temporary snapshot file when testing (saving to and executing from)
snapshots. This patch makes the temp file contain part of the name
of the original test file in order to make reading the logs (and
thus, identifying failing tests) easier.
* Add `--snapshot` to summary info of test runner if testing snapshot
support:
This helps reading the logs, especially when multiple tests run in
parallel and get their output mixed. E.g., on the CI, all tests
write the console at the same time and messages get interleaved.
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss akiss@inf.u-szeged.hu
When running multiple test suites in parallel -- e.g., as it
happens on the CI --, the names of the executed commands and the
results of the executions can/do get printed far from each other,
interrupted by other prints. This can make the reading of the
output hard. This patch prints each line with one echo command,
which makes the interruption of the line less likely.
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss akiss@inf.u-szeged.hu
Both JS test suite and unit test logs can grow large as Makefile
invokes run-test-suite.sh and run-unittests.sh with absolute paths
to engines and test directories, which get then printed quite
often. This patch adds code to the runner scripts to determine the
longest directory path common to the current working directory, the
invoked script, the test directory, and the engine (in case of JS
tests). Then, when a path is to be printed, this common path
component is skipped.
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss akiss@inf.u-szeged.hu
* First of all, remove the counter-intuitive "OUT_DIR" second
argument of run-test-suite.sh. This, way, the invocation of the
script becomes easier to remember:
`tools/runners/run-test-suite.sh <engine> <testsuite>`
However, this also means that all output files (lists of
executed, passed, and failed tests) are generated in the current
working directory.
* Align the behaviour of run-unittests.sh with the above, i.e.,
don't try to guess where to put output files but write them in
the CWD.
* Adapt Makefile to the change in the use of the test runner
scripts: create and change to "check" directories before invoking
test runner scripts.
Extras:
* tools/runners/run-test-suite.sh collected fail tests from
directories twice. This does no harm but is inefficient, thus
removing.
* tools/runners/run-test-suite.sh was too permissive on the
contents of test suite list files. Better to accept those lines
only which really contain paths to JS test files.
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss akiss@inf.u-szeged.hu
* Removed unused or unnecessary parts from various make files
* Eliminated lots of duplications from Makefile with the help of
macros.
* Split tools/precommit.sh to its independent components:
* the part that checks the existence of the "signed off" text in
commit messages
(this on got factored out to tools/check-signed-off.sh),
* the part that uses vera++ for style checking (this one got
factored out to tools/check-vera.sh),
* the part that invokes targets in the cmake-generated build
directory, and
* the part that performs various tests (these latter two got
moved into the Makefile).
* Moved the functionality of precommit-full-testing.sh into the
Makefile, too.
* Added ninja build system support (e.g., `make NINJA=1`).
* Updated leading documentation comments (they were somewhat
stale).
* Tried to keep the target names exactly the same as they were --
almost succeeded... (some changes are intentional, and are
subject to personal preferences).
* Simplified console output of `make precommit`
* Unified test runner scripts and their output format
* Eliminated nothing-to-stdout everything-to-log-file policy:
info is printed to stdout and it is the caller's
responsibility to redirect it to a file if needed.
* Also applied some renaming and coding style unification to
the scripts.
* Merged the functionality of tools/runners/run-test-suite-jerry*.sh
into the Makefile
* Merged everything related to a test suite execution in a single
script.
* The new script also allows to specify pass and xfail tests in
a single list file, which was not possible hitherto.
* Also, the paths of the test cases given in a file are
interpreted relative to their container files.
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss akiss@inf.u-szeged.hu
- introducing TEST_RANDOMIZE macro for randomization of source data used in unit tests;
- replacing assert with JERRY_ASSERT;
- renaming test_* to test-*.
JerryScript-DCO-1.0-Signed-off-by: Ruben Ayrapetyan r.ayrapetyan@samsung.com
Using the find program with an argument "*" could lead to problems.
If there are ".js" files in the executing directory then the
shell replaces the asterisk with that filenames.
The fix is simple: quote the argument which has the asterisk.
JerryScript-DCO-1.0-Signed-off-by: Peter Gal pgal.u-szeged@partner.samsung.com
