Commit Graph

80 Commits

Author SHA1 Message Date
Simon
b3ff0a1a25 Merge pull request #373 from tageniu/feat/dark-mode-detect
Implement TODO: enhance dark mode detection
2026-06-09 20:18:33 +08:00
64JohnLee
7b623c5b22 chore(page-controller): remove accidental debug console.log calls
Two debug log statements were left in production code:

- `isScrollableElement()` logged `scrollData!!!` for every scrollable
  element found during DOM tree construction. Because the DOM is rebuilt
  on every agent action, this fired repeatedly and also triggered
  unnecessary JSON serialisation of scroll metrics on the hot path.

- `SimulatorMask.dispose()` logged 'dispose SimulatorMask' every time
  the highlight overlay was torn down.

Neither had a structured prefix or was gated behind a debug flag.
Removing both silences console noise for end users and removes the
serialisation overhead in the scroll-detection hot path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 16:47:04 +08:00
Simon
c6f09375f8 fix(controller): isMainContentDark too aggressive 2026-04-20 20:31:50 -04:00
tageniu
cc33297cc4 fix(controller): check html and body data attrs independently
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-20 20:31:45 -04:00
Shijie Sun
8556745645 feat(page-controller): enhance dark mode detection
Resolve the @TODO in checkDarkMode.ts by adding 3 new detection
strategies and expanding data-attribute coverage:

- Check data-color-mode, data-bs-theme, data-color-scheme attributes
- Read CSS color-scheme property and <meta name="color-scheme"> tag
- Inspect background of SPA containers (#app, #root, #__next, etc.)
- Detect light text color as a dark-theme signal
2026-04-20 20:14:07 -04:00
Simon
a1b7684bf2 Merge pull request #418 from lgy2020/fix/simulator-mask-raf-leak
fix(mask): dispose 后 requestAnimationFrame 循环未停止导致内存泄漏
2026-04-13 20:14:32 +08:00
Simon
da67f3b07e chore(controller): .disposed guard 2026-04-13 20:13:41 +08:00
Simon
ca197d14e9 fix: dom typing 2026-04-12 03:16:39 +08:00
Simon
4d27d49752 refactor(setup): upgrade to TypeScript 6 with source-first monorepo resolution 2026-04-12 02:04:21 +08:00
liuguiyuan
9104064e8c fix(mask): stop requestAnimationFrame loop on dispose to prevent memory leak
The #moveCursorToTarget() method recursively schedules itself via
requestAnimationFrame, creating a continuous animation loop for the
AI cursor. However, dispose() only removes the DOM wrapper element
without stopping this loop, causing:

- CPU waste: rAF callback continues executing every frame (~60fps)
  after the mask is disposed, performing unnecessary calculations
  on a detached cursor element.
- Resource leak: Each SimulatorMask instance creates an unrecoverable
  animation loop that persists for the lifetime of the page.
- Console noise: style assignments to removed DOM nodes may produce
  browser warnings.

Fix: Add a #disposed boolean flag, checked at the top of
#moveCursorToTarget() to short-circuit the recursion. Set the flag
to true in dispose() before removing DOM elements.

Changes:
- Add #disposed field (default false)
- Guard #moveCursorToTarget() with early return when #disposed
- Set #disposed = true in dispose() before cleanup
2026-04-08 18:53:42 +08:00
Simon
4272939217 fix(controller): treat interactive with aria as distinct 2026-04-03 20:21:11 +08:00
Lubrsy
148bdb6839 fix: recognize role="listitem" as interactive element (#203)
* fix: recognize role="listitem" as interactive element
* fix: DISTINCT_INTERACTIVE_ROLES

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Simon <10131203+gaomeng1900@users.noreply.github.com>
2026-04-03 20:09:48 +08:00
Simon
7e9027167d feat(controller): add keepSemanticTags config to keep the semantic structure of the page 2026-04-03 19:01:54 +08:00
Simon
0ca1c8de0b feat(controller): consider hasScrollbarSignal when detecting scrollable 2026-04-03 16:09:35 +08:00
Simon
85a33ac1a4 feat(controller): improve scroll action 2026-04-02 22:05:47 +08:00
Simon
8159aa58a6 Merge branch 'main' into fix/scroll-direction-pixels 2026-04-02 18:31:56 +08:00
Simon
2bdb3be81c Merge pull request #356 from lgy2020/fix/aria-attribute-detection
fix(isInteractiveCandidate): use hasAttribute with known aria list to detect aria- attributes
2026-04-02 17:41:36 +08:00
Simon
bde630f55d chore(controller): add @edit mark 2026-04-02 17:38:02 +08:00
Simon
3efef0ec42 fix(controller): clean up INTERACTIVE_ARIA_ATTRS 2026-04-02 17:33:27 +08:00
Simon
2b20b48dff chore(controller): reuse pointer xy 2026-03-31 20:27:04 +08:00
Simon
296459924a feat(controller): enhance click action with elementFromPoint 2026-03-31 20:02:39 +08:00
Simon
8eee3b27e2 feat(controller): fix SimulatorMast mem leak; add passthrough events 2026-03-31 19:59:57 +08:00
Simon
32d6f0c74b fix(controller): click action robust 2026-03-31 17:41:49 +08:00
liuguiyuan
f2b6c9dfd2 fix: use hasAttribute with known aria list for interactive candidate detection
Replace broken el.hasAttribute("aria-") with a curated list of 27
aria attributes checked via hasAttribute. Each check is O(1).

WAI-ARIA 1.2 defines ~50 aria attributes total per MDN.
Of these ~27 appear on interactive elements such as buttons,
inputs, sliders, and dialogs. The remaining ~20 are structural
container attributes like aria-live, aria-colcount, and
aria-rowspan that only appear on non-interactive containers.
Checking them would not change results.
2026-03-31 00:57:44 +08:00
Matt Van Horn
005bc8ab44 fix(page-controller): apply scroll direction to pixels parameter
Two bugs in the scroll direction logic:

1. Vertical scroll with `pixels` ignores the `down` boolean because the
   `??` operator bypasses the direction multiplier when pixels is provided.
   Fix: move the direction multiplier outside the `??` so it applies to
   both the pixels and numPages paths.

2. Horizontal scroll with `pixels` applies direction twice - once in
   PageController.ts and again in actions.ts - causing a double negation
   that reverses the intended direction. Fix: remove the redundant
   direction logic from actions.ts since PageController already signs
   the scroll amount.

Also removes the now-unused `down` and `right` parameters from the
scrollVertically() and scrollHorizontally() action functions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 08:38:32 -07:00
zfangqijun
cd2d33a9f6 chore(page-controller): export actions as internal methods 2026-03-21 20:17:01 +08:00
Simon
eeb5b6a5af fix(clickElement): reorder iframe scroll and click actions 2026-03-21 02:18:42 +08:00
Simon
ad31e2b750 chore: clarify current type handling in isHTMLElement 2026-03-21 02:16:51 +08:00
Simon
e98d80b6a0 fix(PageController): same-origin iframe actions 2026-03-21 01:46:09 +08:00
Simon
3459836a14 fix(PageController): lint error 2026-03-20 16:10:43 +08:00
Simon
b7b5b6db30 Merge pull request #319 from alibaba/fix/missing-pointerout
fix(PageController): add `mouseleave` event
2026-03-20 16:05:37 +08:00
Simon
05d16313c7 fix(PageController): add mouseleave event 2026-03-20 15:54:36 +08:00
Simon
a3a96d85d5 Merge pull request #210 from voidborne-d/fix/contenteditable-fallback
fix: add execCommand fallback for contenteditable input
2026-03-20 15:38:06 +08:00
Simon
c89042f142 chore: wording 2026-03-20 15:36:10 +08:00
voidborne-d
2e18bd862d refactor: use const for planASucceeded, clarify LinkedIn comment 2026-03-20 07:08:45 +00:00
Simon
53628532df Merge pull request #251 from linked-danis/pr5-scrollintoview 2026-03-16 18:45:19 +08:00
linked-danis
89546887bd fix: type-safe scrollIntoViewIfNeeded
Add proper interface for WebKit extension method.

Changes:
- Add ScrollableElement interface
- Use typeof check instead of 'as any' cast
2026-03-13 14:22:47 +01:00
linked-danis
6491118cde refactor: SimulatorMask use CSS classes
Replace inline style.display with CSS class toggling.

Changes:
- Add .visible class to CSS module
- Use classList.add/remove instead of style.display

(cherry picked from commit 33465bbf520b65908c18d8022f259803253a7621)
2026-03-13 20:44:34 +08:00
Simon
1628d48c97 chore: align doc styles 2026-03-12 02:20:24 +08:00
d 🔹
2f92a9cb32 fix: add execCommand fallback for contenteditable input (#168)
When typing into contenteditable elements (e.g. LinkedIn post editor),
the synthetic event approach (Plan A) may fail silently — the events
fire but the editor's internal state doesn't update, leaving the
element empty.

This adds an automatic fallback: after Plan A, we verify the text was
actually inserted by checking element.innerText. If it wasn't, we
fall back to execCommand('insertText') which integrates natively with
most rich-text editors including LinkedIn, Quill, and Slate.js.

The fallback uses proper Selection/Range API to select-all before
replacing, and preserves the undo stack since execCommand is handled
by the browser natively.

Fixes #168
2026-03-11 17:07:22 +00:00
caibing
de3a6e4660 fix: extract attributes for heuristically-detected interactive elements
Elements detected as interactive via heuristic methods (cursor:pointer
style, interactive class names, event listeners) had empty attributes
because `isInteractiveCandidate()` was used as the gate for attribute
extraction. This function only recognizes standard HTML tags and ARIA
attributes, missing heuristic detections.

After interactivity is confirmed by `isInteractiveElement()`, backfill
attributes for elements that were missed. This ensures
`includeAttributes` (e.g. `['class']`) works correctly for all
interactive elements, not just semantically standard ones.

Closes #124

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 16:38:36 +08:00
Simon
4d931b4430 chore: comments; spell check 2026-03-10 21:02:48 +08:00
JasonOA888
6054ca1217 docs(page-controller): document contenteditable limitations and execCommand fallback
## Changes

Based on @gaomeng1900's comprehensive testing feedback:

1. **Document known limitations**
   - Slate.js and Draft.js do not work with synthetic events
   - Draft.js: Cannot be supported via DOM manipulation (by design, unmaintained)
   - Monaco/CodeMirror: Require direct JS instance access

2. **Clarify tested editors**
   - Works: Quill, LinkedIn, simple contenteditable editors
   - Does not work: Slate.js, Draft.js

3. **Preserve execCommand as fallback**
   - execCommand works better for LinkedIn, Quill, and Draft.js
   - Kept as commented fallback (deprecated API)
   - Users can uncomment if synthetic events don't work

Co-authored-by: gaomeng1900 <gaomeng1900@users.noreply.github.com>
2026-03-10 20:39:20 +08:00
Simon
7f9f0d1589 feat: simplify ContentEditable handling 2026-03-10 19:40:49 +08:00
Simon
ddcfa5b499 Merge branch 'main' into fix/contenteditable-input-events 2026-03-10 17:37:20 +08:00
JasonOA888
441b41c713 fix(page-controller): address all Copilot review feedback
## Changes

1. **Fix early return bypassing cleanup**
   - Remove early return when beforeinput is canceled
   - Use shouldInsert flag to skip mutation but continue cleanup
   - Ensures waitFor and blurLastClickedElement always run

2. **Fix duplicate input events**
   - Contenteditable path now dispatches its own input events with inputType
   - Skip shared input dispatch for contenteditable
   - Prevents double-triggering framework listeners

3. **Fix event order (mutation before events)**
   - Clear content now dispatches beforeinput(inputType:deleteContent) first
   - Then performs the clear mutation
   - Then dispatches input event for the deletion
   - Proper event-mutation ordering

4. **Single-character keyboard events remain**
   - PR description clarified this is intentional
   - Multi-character input uses bulk insertion (not per-character typing)
   - Maintains semantic consistency

5. **Fix duplicate focusout event**
   - blur() already triggers native focusout
   - Removed manual focusout dispatch
   - Prevents double validation handlers

All changes follow Copilot's suggested fixes.
2026-03-10 16:03:25 +08:00
fancy
1c354ab5d3 refactor(page-controller): remove viewportExpansion constants module 2026-03-10 15:34:24 +08:00
JasonOA888
2d055d3909 style: fix Prettier formatting 2026-03-10 15:10:17 +08:00
fancy
16da7d936d fix(page-controller): honor viewportExpansion in DOM extraction 2026-03-10 12:28:54 +08:00
JasonOA888
efe08f445f fix(page-controller): address Copilot review feedback
## Changes

1. **Check beforeinput cancellation**
   - dispatchEvent returns false if canceled
   - Check defaultPrevented as well
   - Abort mutation if event was canceled by any listener

2. **Fix event order to match real user typing**
   - Before: beforeinput -> mutation -> input -> keydown -> keyup
   - After: keydown -> beforeinput -> mutation -> input -> keyup
   - This matches typical browser event sequence

3. **Fix blur event semantics**
   - blur doesn't bubble; focusout does
   - Call editableElement.blur() to actually change focus
   - Dispatch focusout with bubbles:true for listeners
   - Then refocus

4. **Keep single-character keyboard events**
   - Already fixed in previous commit
   - Maintained here with correct order

All changes follow Copilot's suggested fixes.
2026-03-10 12:02:01 +08:00