13 Commits

Author SHA1 Message Date
Jonas Jenwald
9c857166cc Avoid downloading test PDFs multiple times
It's somewhat common for multiple test-cases to use the same PDF, for example tests for both `eq` and `text` with one PDF.
Currently the logic in `downloadManifestFiles` doesn't handle duplicate test PDFs, which leads to wasted time/resources by downloading the same PDF more than once. Naturally the effect of this is especially bad when downloading all linked PDFs.

Total number of test PDFs downloaded:
 - 507, with `master`.
 - 447, with this patch.
2026-03-25 08:18:05 +01:00
Jonas Jenwald
3e0571cd9c Download test PDFs with the Fetch API
Using the Fetch API simplifies and shortens the `downloadFile` function considerably, since among other things it handles redirects[1] by default.

Also, the regular expression in `downloadManifestFiles` can be replaced with a simple string function now.

---

[1] Implementations of the Fetch API should already prevent e.g. redirect loops and limit the total number of redirects allowed.
2026-03-24 21:01:42 +01:00
Nicolò Ribaudo
2f2d5c9e27
Add script to check license headers 2026-03-04 10:40:39 +01:00
Tim van der Meij
e8bbb60869
Merge pull request #19582 from dhdaines/consume_response
Be sure to consume responses in case of error in downloading test files (issue 19580)
2025-03-02 15:48:26 +01:00
David Huggins-Daines
962f972aea Be sure to consume responses in case of error in downloading test files (issue 19580)
As mentioned in https://nodejs.org/docs/latest-v20.x/api/http.html#httpgeturl-options-callback
if a request results in an error, you still need to consume the data, or call
`response.resume()`, or your code will hang waiting for the request to close.
2025-03-01 07:53:56 -05:00
Jonas Jenwald
d1b2476c14 Write string-data into the .error-file created for broken test-manifest links (issue 19579)
This appears to have broken in PR 17431, since prior to that the `downloadFile` function returned a string (via a callback) on failure and now we're instead rejecting with an Error.
2025-02-28 17:05:15 +01:00
Nicolò Ribaudo
202b26487f
Do not stringify errors when logging them
Converting errors to string drops their stack trace, making it more
difficult to debug their actual reason. We can instead pass the error
objects as-is to console.warn/error, so that Firefox/Chrome devtools
will show both the stack trace of the console.warn/error call, and the
original stack trace of the error.

This commit also enables the `unicorn/no-console-spaces` ESLint rule,
which avoids accidental extra spaces when passing multiple parameters to
`console.*` methods.
2024-12-06 14:47:22 +01:00
Tim van der Meij
f9a0d4efe8
Modernize the rewriteWebArchiveUrl test helper function
This commit changes the code to use a template string and to use `const`
instead of `var`. Combined with the previous commits this allows for
enabling the ESLint `no-var` rule for this file now.
2023-12-17 21:53:50 +01:00
Tim van der Meij
0a10a7b57b
Modernize the downloadManifestFiles/ensurePDFsDownloaded test helper functions
The test helper code largely predates the introduction of modern
JavaScript features and should be refactored to improve readability.
In particular callbacks make the code harder to understand and maintain.

This commit:

- replaces the callback argument with returning a promise;
- replaces the recursive function calls with a simple loop;
- uses `const`/`let` instead of `var`;
- uses arrow functions for shorter code;
- uses template strings for shorter string formatting code.
2023-12-17 21:53:50 +01:00
Tim van der Meij
86bee4409a
Modernize the downloadFile test helper function
The test helper code largely predates the introduction of modern
JavaScript features and should be refactored to improve readability.
In particular callbacks make the code harder to understand and maintain.

This commit:

- replaces the callback argument with returning a promise;
- uses `const` instead of `var`;
- uses arrow functions for shorter code;
- uses template strings for shorter string formatting code;
- uses `Array.includes` for shorter response code checking code.
2023-12-17 21:53:50 +01:00
Tim van der Meij
ac5667166e
Modernize the verifyManifestFiles test helper function
The test helper code largely predates the introduction of modern
JavaScript features and should be refactored to improve readability.
In particular callbacks and recursive function calls make the code
harder to understand and maintain.

This commit:

- replaces the callback argument with returning a promise;
- replaces the recursive function calls with a simple loop;
- uses `const`/`let` instead of `var`;
- uses template strings for shorter string formatting code;
- improves the error messages to have more details.
2023-12-03 20:42:50 +01:00
Tim van der Meij
c9a923066b
Modernize the calculateMD5 test helper function
The test helper code largely predates the introduction of modern
JavaScript features and should be refactored to improve readability.
In particular callbacks and recursive function calls make the code
harder to understand and maintain.

This commit:

- replaces the callback argument with returning a promise;
- uses `const` instead of `var`;
- uses arrow functions for shorter code;
- uses template strings for shorter string formatting code;
- improves the error messages to have more details.
2023-12-03 20:42:45 +01:00
Jonas Jenwald
a209ce811d [ESM] Convert *most* of test-folder to use standard modules 2023-07-08 13:13:04 +02:00