Robin Malfait 2a403267d8
Oxide (#10252)
* temporarily disable workflows

* add oxide

Our Rust related parts

* use oxide

- Setup the codebase to be able to use the Rust parts based on an
  environment variable: `OXIDE=1`.
- Setup some tests that run both the non-Rust and Rust version in the
  same test.
- Sort the candidates in a consistent way, to guarantee the order for
  now (especially in tests).
- Reflect sorting related changes in tests.
- Ensure tests run in both the Rust and non-Rust version. (Some tests
  are explicitly skipped when using the Rust version since we haven't
  implemented those features yet. These include: custom prefix,
  transformers and extractors).
  - `jest`
  -`OXIDE=1 jest`

* remove into_par_iter where it doesn't make sense

* cargo fmt

* wip

* enable tracing based on `DEBUG` env

* improve CI for the Oxide build

* sort test output

This happened because the sorting happens in this branch, but changes
happened on the `master` branch.

* add failing tests

I noticed that some of the tests were failing, and while looking at
them, it happened because the tests were structured like this:

```html
    <div
      class="
        backdrop-filter
        backdrop-filter-none
        backdrop-blur-lg
        backdrop-brightness-50
        backdrop-contrast-0
        backdrop-grayscale
        backdrop-hue-rotate-90
        backdrop-invert
        backdrop-opacity-75
        backdrop-saturate-150
        backdrop-sepia
      "
    ></div>
```

This means that the class names themselves eventually end up like this: `backdrop-filter-none\n`
-> (Notice the `\n`)

/cc @thecrypticace

* fix range to include `\n`

* Include only unique values for tests

Really, what we care about most is that the list contains every expected candidate. Not necessarily how many times it shows up because while many candidates will show up A LOT in a source text we’ll unique them before passing them back to anything that needs them

* Fix failing tests

* Don’t match empty arbitrary values

* skip tests in oxide mode regarding custom separators in arbitrary variants

* re-enable workflows

* use `@tailwindcss/oxide` dependency

* publish `tailwindcss@oxide`

* drop prepublishOnly

I don't think we actually need this anymore (or even want because this
is trying to do things in CI that we don't want to happen. Aka, build
the Oxide Rust code, it is already a dependency).

* WIP

* Defer to existing CLI for Oxide

* Include new compiled typescript stuff when publishing

* Move TS to ./src/oxide

* Update scripts

* Clean up tests for TS

* copy `cli` to `oxide/cli`

* make CLI files TypeScript files

* drop --postcss flag

* setup lightningcss

* Remove autoprefixer and cssnano from oxide CLI

* cleanup Rust code a little bit

- Drop commented out code
- Drop 500 fixture templates

* sort test output

* re-add `prepublishOnly` script

* bump SWC dependencies in package-lock.json

* pin `@swc` dependencies

* ensure to install and build oxide

* update all GitHub Workflows to reflect Oxide required changes

* sort `content-resolution` integration tests

* add `Release Insiders — Oxide`

* setup turbo repo + remote caching

* use `npx` to invoke `turbo`

* setup unique/proper package names for integration tests

* add missing `isomorphic-fetch` dependency

* setup integration tests to use `turborepo`

* scope tailwind tasks to root workspace

* re-enable `node_modules` cache for integration tests

* re-enable `node_modules` cache for main CI workflow

* split cache for `main` and `oxide` node_modules

* fix indent

* split install dependencies so that they can be cached individually

* improve GitHub actions caching

* use correct path for oxide node_modules (crates/node)

* ensure that `cargo install` always succeeds

cargo install X, on CI will fail if it already exists.

* figure out integration tests with turbo

* tmp: use `npm` instead of `turbo`

* disable `fail-fast`

This will allow us to run integration tests so that it still caches the
succesful ones.

* YAML OH YAML, Y U WHITESPACE SENSITIVE

* copy the oxide-ci workflow to release-oxide

* make `oxide-ci` a normal CI workflow

Without publishing

* try to cache cargo and node_modules for the oxide build

* configure turbo to run scripts in the root

* explicitly skip failing test for the Oxide version

* run oxide tests in CI

* only use build script for root package

* sync package-lock.json

* do not cache node_modules for each individual integration

* look for hoisted `.bin`

* use turbo for caching build tailwind css in integration tests

* Robin...

* try to use the local binary first

* skip installing integration test dependencies

Should already be installed due to workspace usage

* Robin...

* drop `output.clean`

* explicitly add `mini-css-extract-plugin`

* drop oxide-ci, this is tested by proxy

* ensure oxide build is used in integration tests

This will ensure the `@tailwindcss/oxide` dependency is available
(whether we use it or not).

* setup Oxide shim in insiders release

* add browserslist dependency

* use `install:all` script name

Just using `install` as a script name will be called when running
`npm install`.
Now that we marked the repo as a `workspace`, `npm install` will run
install in all workspaces which is... not ideal.

* tmp: enable insiders release in PRs

Just to check if everything works before merging. Can be removed once
tested.

* don't cache node_modules?

I feel there is some catch 22 going on here.
We require `npm install` to build the `oxide/crates/node` version.
But we also require `oxide/crates/node` for the `npm install` becaus of
the dependency: `"@tailwindcss/oxide": "file:oxide/creates/node"`

* try to use `oxide/crates/node` as part of the workspace

* let's think about this

Let's try and cache the `node_modules` and share as much as possible.
However, some scripts still need to be installed specific to the OS.

Running `npm install` locally doesn't throw away your `node_modules`,
so if we just cache `node_modules` but also run `npm install` that
should keep as much as possible and still improve install times since
`node_modules` is already there.

I think.

* ensure generated `index.js` and `index.d.ts` files are considered outputs

* use `npx napi` instead of `napi` directly

* include all `package-lock.json` files

* normalize caching further in all workflows

* drop nested `package-lock.json` files

* `npm uninstall mini-css-extract-plugin && npm install mini-css-extract-plugin --save-dev`

* bump webpack-5 integration tests dependencies

* only release insiders on `master` branch

* tmp: let's figure out release insiders oxide

* fix little typo

* use Node 18 for Oxide Insiders

* syncup package-lock.json

* let's try node 16

Node 18 currently fails on `Build x86_64-unknown-linux-gnu (OXIDE)`
Workflow.

Install Node.JS output:

```
Environment details
Warning: /__t/node/18.13.0/x64/bin/node: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /__t/node/18.13.0/x64/bin/node)
/__t/node/18.13.0/x64/bin/node: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by /__t/node/18.13.0/x64/bin/node)
/__t/node/18.13.0/x64/bin/node: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by /__t/node/18.13.0/x64/bin/node)
/__t/node/18.13.0/x64/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /__t/node/18.13.0/x64/bin/node)
/__t/node/18.13.0/x64/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /__t/node/18.13.0/x64/bin/node)
/__t/node/18.13.0/x64/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /__t/node/18.13.0/x64/bin/node)

Warning: node: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by node)
node: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by node)
node: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by node)
node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by node)
node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by node)
node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by node)
```

* bump some Node versions

* only release oxide insiders on `master` branch

* don't cache `npm`

* bump napi-rs

Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2023-01-13 12:22:00 +01:00

288 lines
8.0 KiB
JavaScript

let fs = require('fs')
let path = require('path')
let { cwd } = require('./cwd.js')
let { writeConfigs, destroyConfigs } = require('./config.js')
let $ = require('../../execute')
let { css } = require('../../syntax')
let { readOutputFile } = require('../../io')({
output: 'dist',
input: '.',
})
// Write default configs before running tests and remove them afterwards
beforeAll(() => writeConfigs())
afterAll(() => destroyConfigs())
// Create a symlink at ./src/link that points to ./src/resolved and remove it afterwards
beforeAll(() =>
fs.promises.symlink(
path.resolve(__dirname, '../src/resolved'),
path.resolve(__dirname, '../src/link')
)
)
afterAll(async () => {
try {
await fs.promises.unlink(path.resolve(__dirname, '../src/link'))
} catch {}
})
// If we've changed directories reset the cwd back to what it was before running these tests
afterEach(() => cwd.unwind())
async function build({ cwd: cwdPath } = {}) {
let inputPath = path.resolve(__dirname, '../src/index.css')
let outputPath = path.resolve(__dirname, '../dist/main.css')
await cwd.switch(cwdPath)
// Note that ./tailwind.config.js is hardcoded on purpose here
// It represents a config but one that could be in different places
await $(`postcss ${inputPath} -o ${outputPath}`, {
env: { NODE_ENV: 'production' },
cwd: cwdPath,
})
return {
css: await readOutputFile('main.css'),
}
}
it('looks in the CWD by default', async () => {
await writeConfigs({
both: {
content: {
files: ['./src/real/yes.html'],
},
},
})
let result = await build({ cwd: path.resolve(__dirname, '..') })
expect(result.css).toMatchCss(css`
.content-\[real-static-positive\] {
--tw-content: real-static-positive;
content: var(--tw-content);
}
`)
result = await build({ cwd: path.resolve(__dirname, '../src') })
expect(result.css).toMatchCss(``)
})
it('looks in the CWD for non-config-relative paths', async () => {
await writeConfigs({
both: {
// Turn it on by default (eventual v4 behavior)
experimental: { relativeContentPathsByDefault: true },
// But then disable it anyway
content: {
relative: false,
files: ['./src/real/yes.html'],
},
},
})
let result = await build({ cwd: path.resolve(__dirname, '..') })
expect(result.css).toMatchCss(css`
.content-\[real-static-positive\] {
--tw-content: real-static-positive;
content: var(--tw-content);
}
`)
result = await build({ cwd: path.resolve(__dirname, '../src') })
expect(result.css).toMatchCss(``)
})
it('can look for content files relative to the config', async () => {
await writeConfigs({
both: {
content: {
relative: true,
files: ['./real/yes.html'],
},
},
})
// Here `./real` doesn't exist next to the config in the root directory
let result = await build({ cwd: path.resolve(__dirname, '..') })
expect(result.css).toMatchCss(css``)
// But here it `./real` does exist next to the config in the `./src` directory!
result = await build({ cwd: path.resolve(__dirname, '../src') })
expect(result.css).toMatchCss(css`
.content-\[real-static-positive\] {
--tw-content: real-static-positive;
content: var(--tw-content);
}
`)
})
it('it handles ignored globs correctly when not relative to the config', async () => {
await writeConfigs({
both: {
content: {
relative: false,
files: [
'./src/real/yes.html', // Scanned + static
'./src/real/*.js', // Scanned + dynamic
'!./src/real/no.js', // Ignored + static
'!./src/real/no-*.js', // Ignored + dynamic
],
},
},
})
let result = await build({ cwd: path.resolve(__dirname, '..') })
expect(result.css).toMatchCss(css`
.content-\[real-dynamic-positive\] {
--tw-content: real-dynamic-positive;
content: var(--tw-content);
}
.content-\[real-static-positive\] {
--tw-content: real-static-positive;
content: var(--tw-content);
}
`)
// But here it `./real` does exist next to the config in the `./src` directory!
result = await build({ cwd: path.resolve(__dirname, '../src') })
expect(result.css).toMatchCss(``)
})
it('it handles ignored globs correctly when relative to the config', async () => {
await writeConfigs({
both: {
content: {
relative: true,
files: [
'./real/yes.html', // Scanned + static
'./real/*.js', // Scanned + dynamic
'!./real/no.js', // Ignored + static
'!./real/no-*.js', // Ignored + dynamic
],
},
},
})
let result = await build({ cwd: path.resolve(__dirname, '..') })
expect(result.css).toMatchCss(``)
// But here it `./real` does exist next to the config in the `./src` directory!
result = await build({ cwd: path.resolve(__dirname, '../src') })
expect(result.css).toMatchCss(css`
.content-\[real-dynamic-positive\] {
--tw-content: real-dynamic-positive;
content: var(--tw-content);
}
.content-\[real-static-positive\] {
--tw-content: real-static-positive;
content: var(--tw-content);
}
`)
})
it('it can resolve symlinks for files when not relative to the config', async () => {
await writeConfigs({
both: {
content: {
relative: false,
files: [
'./src/real/yes.html', // Scanned + static
'./src/real/*.js', // Scanned + dynamic
'./src/link/yes.html', // Scanned + static + symlinked
'./src/link/*.js', // Scanned + dynamic + symlinked
'!./src/real/no.js', // Ignored + static
'!./src/real/no-*.js', // Ignored + dynamic
'!./src/link/no.js', // Ignored + static + symlinked
'!./src/link/no-*.js', // Ignored + dynamic + symlinked
],
},
},
})
let result = await build({ cwd: path.resolve(__dirname, '..') })
expect(result.css).toMatchCss(css`
.content-\[real-dynamic-positive\] {
--tw-content: real-dynamic-positive;
content: var(--tw-content);
}
.content-\[real-static-positive\] {
--tw-content: real-static-positive;
content: var(--tw-content);
}
.content-\[resolved-dynamic-positive\] {
--tw-content: resolved-dynamic-positive;
content: var(--tw-content);
}
.content-\[resolved-static-positive\] {
--tw-content: resolved-static-positive;
content: var(--tw-content);
}
`)
// But here it `./real` does exist next to the config in the `./src` directory!
result = await build({ cwd: path.resolve(__dirname, '../src') })
expect(result.css).toMatchCss(``)
})
it('it can resolve symlinks for files when relative to the config', async () => {
await writeConfigs({
both: {
content: {
relative: true,
files: [
'./real/yes.html', // Scanned + static
'./real/*.js', // Scanned + dynamic
'./link/yes.html', // Scanned + static + symlinked
'./link/*.js', // Scanned + dynamic + symlinked
'!./real/no.js', // Ignored + static
'!./real/no-*.js', // Ignored + dynamic
'!./link/no.js', // Ignored + static + symlinked
'!./link/no-*.js', // Ignored + dynamic + symlinked
],
},
},
})
let result = await build({ cwd: path.resolve(__dirname, '..') })
expect(result.css).toMatchCss(``)
// But here it `./real` does exist next to the config in the `./src` directory!
result = await build({ cwd: path.resolve(__dirname, '../src') })
expect(result.css).toMatchCss(css`
.content-\[real-dynamic-positive\] {
--tw-content: real-dynamic-positive;
content: var(--tw-content);
}
.content-\[real-static-positive\] {
--tw-content: real-static-positive;
content: var(--tw-content);
}
.content-\[resolved-dynamic-positive\] {
--tw-content: resolved-dynamic-positive;
content: var(--tw-content);
}
.content-\[resolved-static-positive\] {
--tw-content: resolved-static-positive;
content: var(--tw-content);
}
`)
})