Commit Graph
460 Commits
Author SHA1 Message Date
Pierre Tachoire 79b4604f2b apply nav burst to the first requests only and set it to 10 by default 2026-09-10 08:17:02 +02:00
Pierre Tachoire ce0474d81f adjust cooldown intervals to land on 10 req/s 2026-09-10 08:17:01 +02:00
Pierre Tachoire d846042e5a catch error when observing the rate limit 2026-09-10 08:17:01 +02:00
Pierre Tachoire 1ac49bc8ae network: per-host pressure in the navigation rate limiter
The interval between two top-level navigations to the same host now
grows with the number of navigations sent to it: one extra base interval
every burst*10 navigations, capped at 60x. Pressure cools down by one
unit for every 10 base intervals the host is left alone.
--http-nav-delay is now a minimum, not a fixed spacing.
2026-09-10 08:16:53 +02:00
Karl Seguin 33ddbdf0fc Merge pull request #3466 from lightpanda-io/locale-timezone
cli: add --locale and --timezone, derive language signals from one value
2026-09-10 08:06:39 +08:00
Adrià Arrufat 37bf7b68c1 cli: one Accept-Language parser for the header and navigator.languages
navigator.languages now lists the Accept-Language tags in order, which is
Chrome's contract, instead of a second derivation from the locale tag that
disagreed with the header (--locale de-DE sent de-DE,de,en but reported
["de-DE","de"]). HttpHeaders.AcceptLanguage owns both shapes and is also
the CDP override type.

ICU canonicalizes a BCP 47 tag read from LC_ALL itself, script subtag
included, so the POSIX id conversion is gone; it dropped the script and
turned zh-Hans-TW into Traditional Chinese.

Also: the CDP handler keeps validateUserAgent's verdict instead of scanning
for Mozilla twice, the override is cleared unconditionally on context
teardown instead of through a flag, and the flags are sentinel strings so
Platform passes them to setenv without copying.
2026-09-09 18:07:03 +02:00
Adrià Arrufat c55afc1df9 cli: add --locale and --timezone, derive language signals from one value
navigator.language was hard-coded to en-US and Accept-Language was a
constant, while Intl, toLocaleString and Date followed the host process
environment. On a de_DE host a page saw navigator.language === "en-US"
next to German number formatting, a mismatch fingerprinting scripts look
for, and the same page rendered differently across machines.

Follow Chrome's --lang rule: one configured tag drives navigator.language(s),
the Accept-Language header and ICU's default locale. --locale defaults to
en-US, so Intl is now en-US on every host instead of whatever LANG says.
--timezone sets the IANA zone Date and Intl use; absent, the host zone stays.

Both are applied by writing LC_ALL and TZ before V8 initializes ICU, which
reads them lazily. Platform.init is the first call in App.init, before any
thread exists, so setenv is safe there.

CDP Emulation.setUserAgentOverride.acceptLanguage, which Playwright sends
for its locale option, now overrides the header and navigator.languages
for the browser context's lifetime, mirroring the user agent override, and
applies even when the Mozilla user agent is refused.

Emulation.setLocaleOverride and setTimezoneOverride stay no-ops: changing
ICU's defaults at runtime needs new zig-v8-fork bindings.
2026-09-09 17:49:02 +02:00
Karl Seguin 66af1dc58d webapi: Origin header + request header validation
Driven by a handful of /fetch/ WPT tests, three changes:

1 - Prevent libcurl from auto-inserting a 'application/x-www-form-urlencoded"
    content type for types we really have no content-type for.

2 - Include origin header in all requests that should have it. This is something
    CorsGate was doing in most cases, but cors can be disabled, so the logic
    is now moved to HttpClient.

3 - Expands on the header guard added in  https://github.com/lightpanda-io/browser/pull/3374/
    Adds more modes and more header check. Request.init also uses the header
    guard now
2026-09-09 14:44:48 +08:00
Karl Seguin b633dc4b16 webapi: improve various WPT fetch apis
Headers strip whitespace and guard against invalid characters

Headers iterator sorts and combines PER step, so that mutations are picked up.
Not the most efficient, but this is a short list, and how often are these being
iterated?

XMLHttpRequest: has its own extra header validation

Mime support for multiple Content-Type headers (or a header with multiple values)
last value wins.

Add BufferSource js bridge type that accepts various types -> []const u8 (at the
cost of losing the actual type). Useful in fetch, where various types can be a
body, but we only care about the underlying bytes (e.g. we didn't support A
rrayBufferView before this)

Refactored response body getters so that they all go through the same consume
and resolve logic
2026-09-08 13:12:20 +08:00
Halil Durak 7ad73d0a6d changes after rebase 2026-09-07 11:14:15 +03:00
Halil Durak 860c4950e2 first 128 tokens cached at init (common), rest streamed by each engine
Eager tokenization is ~7% of a ~2.1 µs match. A lazy fill saved nothing because every request reaches the first engine. Streaming from scratch per engine cost +10% capped and +16% uncapped, and stayed +10% even with exception gating. The 128 cap changed 1 verdict in 242,908. So we went with hybrid approach: no token is lost, and 97% of URLs still pay one tokenization.
2026-09-07 11:01:03 +03:00
Halil Durak fb58901604 pass *const Request where possible
* remove unnecessary `pub` marking
* metrics tracking cosmetic filters separately
2026-09-07 11:01:03 +03:00
Halil Durak 0a6d43c2bb HttpClient.isAdBlocked should just delegate Adblocker.isBlocked 2026-09-07 11:00:47 +03:00
Halil Durak 486c888080 drop is_subframe and notification 2026-09-07 11:00:47 +03:00
Halil Durak e696a29da7 metrics: make adblocker verdicts measurable 2026-09-07 11:00:05 +03:00
Halil Durak 085b4836ac Adblocker: parser semantics: follow uBO's behavior
* `@@…$important` -> `error.InvalidOption`
* `||host`, `||host^`, `||host|` (and bare `host|`/`|host` lines) all read as "hostname or subdomain" and land in the trie when option-free.
* Wildcard trimming now follows uBO's pointless-wildcard rules.
2026-09-07 10:57:36 +03:00
Halil Durak b60f790a03 Adblocker: changes on request construction & tokenization
* Engine.Request.fromHttp(req, source_url, buffers) now builds the adblock request straight from HttpClient.Request.
* The URL is tokenized once per request (hashed into the Request, shared by all engines); capped at 128 tokens (same as adblock-rust).
* Document hostname longer than 253 bytes now skips adblocking.
2026-09-07 10:57:36 +03:00
Halil Durak 6a85b1f386 handle all switch cases (emerged after rebase to main) 2026-09-07 10:57:36 +03:00
Halil Durak 53f513f1f8 remove unnecessary pub marking 2026-09-07 10:57:35 +03:00
Halil Durak 8aa0dda4b3 integrate request engine to Adblocker, apply changes required for filter list in HttpClient 2026-09-07 10:57:35 +03:00
Halil Durak 0b2f14194e AdBlocker: initial implementation for request engine
`Parser`: don't drop cosmetic lines

introduce url pattern matcher

yet-another-url-parser
2026-09-07 10:57:11 +03:00
Karl Seguin a5e27869d3 chore: remove unused imports 2026-09-05 17:51:47 +08:00
Muki Kiboigo 34b743fbd6 origin is tainted on cross origin redirects 2026-09-04 07:02:01 -07:00
Muki Kiboigo 54518383c0 enforce cors response on redirects as well 2026-09-04 07:00:03 -07:00
Muki Kiboigo 0610d5ecd1 collapse isCrossOriginModeAllowed check in pipeline 2026-09-04 07:00:02 -07:00
Muki Kiboigo 0aeba826b4 use credentials_mode instead of cookie bool 2026-09-04 07:00:02 -07:00
Muki Kiboigo 1f9342e92f add credentials to preflight key for cors 2026-09-04 07:00:02 -07:00
Muki Kiboigo 868882b0c8 add origin conditionally on no_cors 2026-09-04 07:00:02 -07:00
Muki Kiboigo 827d0f0598 only check cross origin mode on obey cors 2026-09-04 07:00:02 -07:00
Muki Kiboigo 8a8bb3814f fix test running 2026-09-04 07:00:01 -07:00
Muki Kiboigo addba12426 add CORS metrics 2026-09-04 07:00:01 -07:00
Muki Kiboigo 15d27902ba use experimental features flag instead of obey cors 2026-09-04 07:00:01 -07:00
Muki Kiboigo 1fe8456cbd add modes to the tests 2026-09-04 07:00:01 -07:00
Muki Kiboigo 12ed38dfdd better no cors opaque behavior 2026-09-04 07:00:00 -07:00
Muki Kiboigo d1f4605459 non-default credentials and request mode 2026-09-04 07:00:00 -07:00
Muki Kiboigo b0fffe693a more comprehensive cors singleflight key 2026-09-04 06:59:40 -07:00
Muki Kiboigo 2b06583662 authorization header doesnt accept wildcard 2026-09-04 06:59:40 -07:00
Muki Kiboigo 8ee714418d proper safelist checking for CORS headers 2026-09-04 06:59:40 -07:00
Muki Kiboigo feebb889ad cors check before cache check 2026-09-04 06:59:40 -07:00
Muki Kiboigo 89df63e956 safelisted methods always pass in CORS 2026-09-04 06:59:39 -07:00
Muki Kiboigo c16a3b3585 fix warn log on preflight blocked 2026-09-04 06:59:39 -07:00
Muki Kiboigo da00a90c02 set cors cross origin on transfer after no-cors check 2026-09-04 06:59:39 -07:00
Muki Kiboigo d3c0291bd1 add request mode for Fetch 2026-09-04 06:59:39 -07:00
Muki Kiboigo ddfa034310 add credentials_mode for proper CORS credentials handling 2026-09-04 06:59:39 -07:00
Muki Kiboigo f31b32ac4e don't store network in CorsGate 2026-09-04 06:59:38 -07:00
Muki Kiboigo cf37a94f16 get rid of blocked on CorsGate check result 2026-09-04 06:59:38 -07:00
Muki Kiboigo b976894315 properly hook up single flight for CorsGate 2026-09-04 06:59:38 -07:00
Muki Kiboigo 977874ce5f initial preflighting 2026-09-04 06:59:38 -07:00
Muki Kiboigo f428dc8e52 add origin header in CorsGate on cross origin 2026-09-04 06:59:38 -07:00
Muki Kiboigo 4e79f24814 basic validation without preflighting 2026-09-04 06:59:38 -07:00