Commit Graph

128 Commits

Author SHA1 Message Date
d43e2025b2 Fix process not exiting after run, detect closed job listings
- All entry points with log tee now call logStream.end() + process.exit()
  (log stream kept event loop alive, blocking next cron run)
- easy_apply: detect "no longer accepting applications" and similar closed
  listing text before reporting as unsupported

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 12:19:00 -08:00
a498f49b95 Fix critical import bugs in linkedin.mjs and wellfound.mjs, clean up status/setup
- linkedin.mjs: LINKEDIN_LINKEDIN_MAX_SEARCH_PAGES → LINKEDIN_MAX_SEARCH_PAGES
  (typo would crash searcher at module load — constant doesn't exist)
- wellfound.mjs: WELLFOUND_WELLFOUND_MAX_INFINITE_SCROLL → WELLFOUND_MAX_INFINITE_SCROLL
  (same — doubled prefix crashes at import)
- status.mjs: remove double-counting ATS from both queue and log
- setup.mjs: replace PM2 instructions with OpenClaw crons

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 12:15:52 -08:00
51ca354c52 Audit fixes: remove dead code, fix run timeout bug, add log tee to all entry points
- Remove unused APPLY_PRIORITY array (replaced by score-based sort)
- Fix run timeout only breaking inner loop — now breaks outer platform loop too
- Remove dead lastProgress variable in easy_apply step loop
- Add stdout/stderr log tee to job_searcher, job_filter, telegram_poller

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 12:13:01 -08:00
8db00d94a5 Fix summary showing all zeros: count unhandled statuses, clean up format
- Default case in handleResult now increments skipped_other
- Summary only shows non-zero categories (cleaner output)
- Applied count always shown

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:54:06 -08:00
7c9de1af4a AI fallback for unknown form fields: ask Claude before Telegram
Every unknown required field now goes through AI before falling back to
Telegram. Claude sees the question + all saved answers + profile, and
either recognizes it as a variation of a saved answer or generates a new
one. AI answers are auto-saved to answers.json so the same question is
a free pattern match next time. Telegram is now last resort (no API key).

Flow: pattern match (free) → AI (smart) → auto-save → Telegram (human)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:52:39 -08:00
3fc5c38df7 Add clear status logging for needs_answer and honeypot exits
Log lines now explicitly say STOPPING with reason when the modal is
dismissed due to unknown questions or honeypots. handleResult logs
the full flow: paused → generating AI answer → sent to Telegram →
will retry after reply. Prevents claw from misreading as a hang/timeout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:47:40 -08:00
0920554dad Add Telegram answer learning: poller + applier safety net
- New lib/telegram_answers.mjs: shared module that polls Telegram getUpdates,
  matches replies to needs_answer jobs, saves to answers.json, flips job to new
- telegram_poller.mjs: lightweight cron script (every minute via OpenClaw)
- Applier also processes replies at start of each run as safety net
- sendTelegram now returns message_id, stored on job for reply matching
- User replies "ACCEPT" to use AI answer, or types their own
- Answers persist in answers.json and apply to ALL future jobs
- Also includes: selectOptionFuzzy, multiple dialog handling, browser
  recovery, answers reload, per-job timeout bump to 10min

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:38:37 -08:00
14cf9a12c1 Reliability improvements: click retry, resume selection, answer loop, browser recovery
- Easy Apply click: click found element directly, retry with force if modal doesn't open
- Resume: select first radio if none checked, fall back to file upload
- AI answers: inject stored answers into formFiller on needs_answer retry
- Answers persistence: reload answers.json before each job for Telegram replies
- Browser recovery: detect dead page, create fresh browser session
- Multiple dialogs: findApplyModal() tags the right dialog among cookie banners etc.
- Select matching: case-insensitive fuzzy match with substring fallback
- dismissModal: scope Discard scan to dialog elements only
- Label dedup: normalize whitespace, fix odd-length edge case
- no_modal status: explicit handleResult case
- Per-job timeout: 10 minutes (was 3)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:30:09 -08:00
7f8cc3658e Fix submit verification: wait for modal detach instead of fixed sleep
Replace fixed 2.5s sleep + single check with event-driven waitForSelector
(state: 'detached', timeout: 8s). If modal persists, check for success
indicators (success text, Submit button gone) before marking incomplete.
Fixes false 'incomplete' status when LinkedIn shows post-submit dialog.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:23:03 -08:00
51a4231b5d fix: stuck detection uses progress bar value, not heading
Heading is always "Apply to <Company>" on every step — comparing it
caused false stuck detection on step 2. Now:
- Progress bar selector finds <progress> elements (no explicit role)
- Stuck detection re-reads progress AFTER clicking Next to see if it changed
- Threshold raised to 3 same-progress clicks before exiting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:08:35 -08:00
00bcfaca53 feat: include select options in unknown field reports and Telegram messages
When a required select has no answer, the unknown field now includes
the available options. Telegram notification shows them so the user
knows exactly what to reply with.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:06:38 -08:00
2c04504ad8 fix: report unfilled required selects as unknown fields
Non-EEO required selects with no answer were silently ignored, causing
infinite loops when LinkedIn blocked page advancement. Now reported as
unknown fields so the applier can exit with needs_answer status.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:05:54 -08:00
5aed8eb404 fix: use evaluateHandle to find Continue link in frames
page.$() and frame.$() can't find the <a> in LinkedIn's shadow DOM,
but frame.evaluateHandle with document.querySelectorAll can.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:00:46 -08:00
1e67c930db fix: search frames for Continue /apply/ link when page.$() misses it
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:59:41 -08:00
46c84bf28a fix: click parent <a> link for Continue, not child <span>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:57:35 -08:00
4c85a88902 fix: stuck loop on unfilled selects, Continue button detection
- Stuck detection after clicking Next: if heading+progress unchanged 2x,
  exit with 'stuck' status instead of looping forever
- Select dropdowns: treat "Select an option" as unfilled (LinkedIn's
  placeholder has a truthy value that was skipping the fill logic)
- Continue button fallback: detect draft "Continue" span via apply URL
  pattern when Easy Apply button not found

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:51:26 -08:00
5e4a0c6599 fix: always discard on modal dismiss, support "Continue applying" button
- dismissModal now always waits for Discard confirmation after closing
  (previously returned early after Dismiss click, leaving draft saved)
- LINKEDIN_APPLY_BUTTON_SELECTOR matches both "Easy Apply" and
  "Continue applying" (shown when a previous draft exists)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:34:19 -08:00
3c46de1358 fix: shadow DOM support — LinkedIn modal is inside shadow root
LinkedIn renders Easy Apply modal inside shadow DOM. document.querySelector()
in evaluate() cannot pierce shadow DOM, but Playwright's page.$() can.

easy_apply.mjs:
- Replaced all frame.evaluate(document.querySelector) with ElementHandle ops
- findModalButton uses modal.$$() + btn.evaluate() instead of evaluateHandle
- getModalDebugInfo uses modal.$eval and modal.$$() for all queries
- dismissModal scans buttons via page.$$() instead of evaluateHandle
- Removed findModalFrame (no longer needed)

form_filler.mjs:
- getLabel() walks up ancestor DOM to find labels (LinkedIn doesn't use label[for])
- Deduplicates repeated label text ("Phone country codePhone country code")
- isRequired() walks ancestors to find labels with * or required indicators

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:25:51 -08:00
df31019fd0 fix: modal frame detection — LinkedIn renders Easy Apply in /preload/ frame
LinkedIn renders the Easy Apply modal inside a /preload/ iframe, not the
main document. page.$() searches cross-frame but evaluate() only runs in
the main frame context, causing blank headings/buttons and broken navigation.

- Added findModalFrame() to locate the frame owning the modal dialog
- All evaluate/evaluateHandle calls now use modalFrame instead of page
- findModalButton() and dismissModal() updated to accept frame parameter
- formFiller.fill() receives modalFrame so container scoping works correctly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:20:58 -08:00
0df5eb1b63 debug: add modal structure diagnostics — iframe detection, child tags, html snippet
Logs iframe count, child element tags, and first 500 chars of modal innerHTML
on step 0 to diagnose why buttons/heading aren't being found.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:13:22 -08:00
e6fb380e1c fix: cover letter silent drop, submit verification, autocomplete scoping
- Text inputs matched to cover letter now report as unknown if required,
  instead of silently leaving the field empty
- Submit click now verifies modal closed before reporting success;
  returns 'incomplete' with actionable log if modal stays open
- selectAutocomplete scoped to container (modal) to avoid clicking
  wrong dropdowns from the underlying page

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:11:44 -08:00
e62756c6ca fix: robustness improvements — atomic writes, timeouts, shell injection, validation errors
- Atomic JSON writes (write-to-tmp + rename) prevent queue/log corruption
- Per-job (3min) and overall run (45min) timeouts prevent hangs
- execFileSync in ai_answer.mjs prevents shell injection with resume paths
- Validation error detection after form fill in Easy Apply modal
- Config-driven enabled_apply_types (from settings.json)
- isRequired() detects required/aria-required/label * patterns
- getLabel() strips trailing * from required field labels
- Actionable logging on failures ("Action: ..." messages)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:01:53 -08:00
a7ce119bde fix: resilient modal button detection and form filler robustness
easy_apply.mjs:
- findModalButton() uses 3-strategy detection: aria-label exact/substring,
  then exact button text match — survives LinkedIn aria-label changes
- Check order fixed: Next → Review → Submit (submit only when no forward nav)
- All queries scoped to modal + :not([disabled])
- dismissModal() with fallback chain: Dismiss → Close/X → Escape → Discard
- Uses innerText for button text (ignores hidden children)

form_filler.mjs:
- All queries scoped to container (modal when present, page otherwise)
- Radio labels use $$('label') + textContent instead of broken :has-text()
- Autocomplete uses waitForSelector instead of blind 800ms sleep
- EEO selects iterate options directly (selectOption doesn't accept regex)
- Country code check ordered before country to prevent fragile match order

constants.mjs:
- Add AUTOCOMPLETE_WAIT, AUTOCOMPLETE_TIMEOUT
- Remove unused button selectors (now handled inline by findModalButton)

ai_answer.mjs + keywords.mjs:
- Use ANTHROPIC_API_URL constant, claude-sonnet-4-6 model

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 09:54:44 -08:00
33c50178f4 debug: full-page button scan, modal heading, real progress, validation errors; bump max steps to 20 2026-03-06 17:43:03 +00:00
b79e9d0b9c feat: city autocomplete dropdown, phone country code, checkboxes, EEO selects, location/heard-about answers 2026-03-06 17:39:05 +00:00
8556208405 debug: verbose step-by-step logging in easy_apply modal flow 2026-03-06 17:27:39 +00:00
093b349aad fix: waitForSelector on modal after click instead of fixed 1.5s delay 2026-03-06 17:13:56 +00:00
8c626b5147 debug: log apply elements on skip; use state=attached for waitForSelector 2026-03-06 17:05:57 +00:00
318da35e01 fix: scroll page before waiting for Easy Apply button; extend timeout to 12s 2026-03-06 17:04:27 +00:00
45ad5d1ec9 fix: Easy Apply is an <a> not <button> — update selector; wait for element instead of fixed timeout 2026-03-06 17:01:02 +00:00
02a7501e9d fix: use aria-label selectors for LinkedIn Easy Apply button (class names are now hashed) 2026-03-06 16:53:25 +00:00
b8a0e03c75 feat: AI answer generation for unknown questions + Easy Apply only mode 2026-03-06 16:47:58 +00:00
b1528ac0ad refactor: extract magic numbers to constants, fix audit issues
- Centralize all magic numbers/strings in lib/constants.mjs
- Fix double-replaced import names in filter.mjs
- Consolidate duplicate fs imports in job_applier/job_searcher
- Remove empty JSDoc block in job_searcher
- Update keywords.mjs model from claude-3-haiku to claude-haiku-4-5
- Extract Anthropic API URLs to constants
- Convert :has-text() selectors to page.locator() API
- Fix SIGTERM handler conflict — move partial-run notification into lock.onShutdown
- Remove unused exports (LOCAL_USER_AGENT, DEFAULT_REVIEW_WINDOW_MINUTES)
- Fix variable shadowing (b -> v) in job_filter reduce callback
- Replace SKILL.md PM2 references with system cron

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 08:45:17 -08:00
37b95b6b85 feat: track token usage and estimated cost per filter run in filter_runs.json 2026-03-06 16:22:14 +00:00
3575f06018 fix: dedupeAfterFilter skips groups with unscored members — wait until all copies are scored 2026-03-06 16:00:23 +00:00
c9b527c83a feat: find-all → filter → dedup flow
- addJobs: allows same job on multiple tracks (dedup key = track::id)
- Cross-track copies get composite id (job.id_track) to avoid batch collisions
- dedupeAfterFilter(): after collect, keeps highest-scored copy per URL, marks rest as 'duplicate'
- Called automatically at end of collect phase
2026-03-06 15:55:00 +00:00
2dfadbde99 refactor: generic system prompt — dumps full JSON profiles, no hardcoded criteria 2026-03-06 11:36:05 +00:00
c88a71fc20 feat: one batch per track — separate GTM/AE batches with their own system prompts
- submitBatch → submitBatches: groups jobs by track, submits one batch each
- filter_state.json now stores batches[] array instead of single batch_id
- Collect waits for all batches to finish before processing
- Each track gets its own cached system prompt = better caching + cleaner scoring
- Idempotent collect: skips already-scored jobs
2026-03-06 11:35:15 +00:00
aadec0704b fix: AE filter criteria - don't penalize company size, SMB, or staffing agencies with real roles 2026-03-06 11:32:30 +00:00
2f05a40954 fix: addJobs always reads fresh from disk to prevent searcher clobbering filter scores 2026-03-06 11:24:05 +00:00
fac286aaeb fix: use claude-sonnet-4-6 (no date suffix) for batch API, bump max_tokens to 1024 2026-03-06 11:01:46 +00:00
0b40d284cd fix: bump max_tokens to 256, note batch API requires native model IDs 2026-03-06 10:58:11 +00:00
728e0773b9 fix: sanitize Unicode surrogates in job descriptions, handle custom_id > 64 chars 2026-03-06 10:18:54 +00:00
dbe9967713 feat: rewrite filter to use Anthropic Batch API
- Batch API = 50% cost savings vs synchronous calls
- Prompt caching on system prompt (profile + criteria shared across all jobs)
- One request per job with custom_id = job ID for result matching
- Two-phase state machine: submit → poll/collect (hourly cron safe)
- filter_state.json tracks pending batch ID between runs
- Model configurable via settings.filter.model (default: claude-sonnet-4-6)
- Telegram notifications on submit + collect
- Errors pass through — never block applications due to filter failure
- --stats flag for queue overview
2026-03-06 10:12:47 +00:00
9bf904dada feat: AI job filter — score jobs 0-10 with Claude Haiku before applying
- lib/filter.mjs: batch scoring engine (10 jobs/call, Claude Haiku)
- job_filter.mjs: standalone CLI with --dry-run and --stats flags
- Threshold configurable globally + per-search in search_config.json (filter_min_score, default 5)
- Job profiles (gtm/ae) passed as context via settings.filter.job_profiles
- Filtered jobs get status='filtered' with filter_score + filter_reason
- Filter errors pass jobs through (never block applications)
- status.mjs: added 'AI filtered' line to report
2026-03-06 10:01:15 +00:00
be20f5a4e9 feat: capture job description from right panel during search — no extra page load 2026-03-06 02:49:28 +00:00
4a2b24d562 fix: show global keyword position (2/20) not slice position (1/19) in logs 2026-03-06 02:47:39 +00:00
c01abe8884 fix: location scraper — use .tvm__text selector, also capture work_type (Remote/Hybrid/On-site) 2026-03-06 02:44:17 +00:00
97b753d401 feat: cache keywords in search_progress.json — restarts reuse same keywords, no regeneration mid-run 2026-03-06 02:24:54 +00:00
65d6d1e50c feat: per-keyword resume — restart picks up from last completed keyword, not keyword 1 2026-03-06 02:23:07 +00:00