Commit Graph

108 Commits

Author SHA1 Message Date
51a4231b5d fix: stuck detection uses progress bar value, not heading
Heading is always "Apply to <Company>" on every step — comparing it
caused false stuck detection on step 2. Now:
- Progress bar selector finds <progress> elements (no explicit role)
- Stuck detection re-reads progress AFTER clicking Next to see if it changed
- Threshold raised to 3 same-progress clicks before exiting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:08:35 -08:00
00bcfaca53 feat: include select options in unknown field reports and Telegram messages
When a required select has no answer, the unknown field now includes
the available options. Telegram notification shows them so the user
knows exactly what to reply with.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:06:38 -08:00
2c04504ad8 fix: report unfilled required selects as unknown fields
Non-EEO required selects with no answer were silently ignored, causing
infinite loops when LinkedIn blocked page advancement. Now reported as
unknown fields so the applier can exit with needs_answer status.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:05:54 -08:00
5aed8eb404 fix: use evaluateHandle to find Continue link in frames
page.$() and frame.$() can't find the <a> in LinkedIn's shadow DOM,
but frame.evaluateHandle with document.querySelectorAll can.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:00:46 -08:00
1e67c930db fix: search frames for Continue /apply/ link when page.$() misses it
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:59:41 -08:00
46c84bf28a fix: click parent <a> link for Continue, not child <span>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:57:35 -08:00
4c85a88902 fix: stuck loop on unfilled selects, Continue button detection
- Stuck detection after clicking Next: if heading+progress unchanged 2x,
  exit with 'stuck' status instead of looping forever
- Select dropdowns: treat "Select an option" as unfilled (LinkedIn's
  placeholder has a truthy value that was skipping the fill logic)
- Continue button fallback: detect draft "Continue" span via apply URL
  pattern when Easy Apply button not found

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:51:26 -08:00
5e4a0c6599 fix: always discard on modal dismiss, support "Continue applying" button
- dismissModal now always waits for Discard confirmation after closing
  (previously returned early after Dismiss click, leaving draft saved)
- LINKEDIN_APPLY_BUTTON_SELECTOR matches both "Easy Apply" and
  "Continue applying" (shown when a previous draft exists)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:34:19 -08:00
3c46de1358 fix: shadow DOM support — LinkedIn modal is inside shadow root
LinkedIn renders Easy Apply modal inside shadow DOM. document.querySelector()
in evaluate() cannot pierce shadow DOM, but Playwright's page.$() can.

easy_apply.mjs:
- Replaced all frame.evaluate(document.querySelector) with ElementHandle ops
- findModalButton uses modal.$$() + btn.evaluate() instead of evaluateHandle
- getModalDebugInfo uses modal.$eval and modal.$$() for all queries
- dismissModal scans buttons via page.$$() instead of evaluateHandle
- Removed findModalFrame (no longer needed)

form_filler.mjs:
- getLabel() walks up ancestor DOM to find labels (LinkedIn doesn't use label[for])
- Deduplicates repeated label text ("Phone country codePhone country code")
- isRequired() walks ancestors to find labels with * or required indicators

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:25:51 -08:00
df31019fd0 fix: modal frame detection — LinkedIn renders Easy Apply in /preload/ frame
LinkedIn renders the Easy Apply modal inside a /preload/ iframe, not the
main document. page.$() searches cross-frame but evaluate() only runs in
the main frame context, causing blank headings/buttons and broken navigation.

- Added findModalFrame() to locate the frame owning the modal dialog
- All evaluate/evaluateHandle calls now use modalFrame instead of page
- findModalButton() and dismissModal() updated to accept frame parameter
- formFiller.fill() receives modalFrame so container scoping works correctly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:20:58 -08:00
0df5eb1b63 debug: add modal structure diagnostics — iframe detection, child tags, html snippet
Logs iframe count, child element tags, and first 500 chars of modal innerHTML
on step 0 to diagnose why buttons/heading aren't being found.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:13:22 -08:00
e6fb380e1c fix: cover letter silent drop, submit verification, autocomplete scoping
- Text inputs matched to cover letter now report as unknown if required,
  instead of silently leaving the field empty
- Submit click now verifies modal closed before reporting success;
  returns 'incomplete' with actionable log if modal stays open
- selectAutocomplete scoped to container (modal) to avoid clicking
  wrong dropdowns from the underlying page

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:11:44 -08:00
e62756c6ca fix: robustness improvements — atomic writes, timeouts, shell injection, validation errors
- Atomic JSON writes (write-to-tmp + rename) prevent queue/log corruption
- Per-job (3min) and overall run (45min) timeouts prevent hangs
- execFileSync in ai_answer.mjs prevents shell injection with resume paths
- Validation error detection after form fill in Easy Apply modal
- Config-driven enabled_apply_types (from settings.json)
- isRequired() detects required/aria-required/label * patterns
- getLabel() strips trailing * from required field labels
- Actionable logging on failures ("Action: ..." messages)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:01:53 -08:00
a7ce119bde fix: resilient modal button detection and form filler robustness
easy_apply.mjs:
- findModalButton() uses 3-strategy detection: aria-label exact/substring,
  then exact button text match — survives LinkedIn aria-label changes
- Check order fixed: Next → Review → Submit (submit only when no forward nav)
- All queries scoped to modal + :not([disabled])
- dismissModal() with fallback chain: Dismiss → Close/X → Escape → Discard
- Uses innerText for button text (ignores hidden children)

form_filler.mjs:
- All queries scoped to container (modal when present, page otherwise)
- Radio labels use $$('label') + textContent instead of broken :has-text()
- Autocomplete uses waitForSelector instead of blind 800ms sleep
- EEO selects iterate options directly (selectOption doesn't accept regex)
- Country code check ordered before country to prevent fragile match order

constants.mjs:
- Add AUTOCOMPLETE_WAIT, AUTOCOMPLETE_TIMEOUT
- Remove unused button selectors (now handled inline by findModalButton)

ai_answer.mjs + keywords.mjs:
- Use ANTHROPIC_API_URL constant, claude-sonnet-4-6 model

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 09:54:44 -08:00
33c50178f4 debug: full-page button scan, modal heading, real progress, validation errors; bump max steps to 20 2026-03-06 17:43:03 +00:00
b79e9d0b9c feat: city autocomplete dropdown, phone country code, checkboxes, EEO selects, location/heard-about answers 2026-03-06 17:39:05 +00:00
8556208405 debug: verbose step-by-step logging in easy_apply modal flow 2026-03-06 17:27:39 +00:00
af4449feb8 docs: update scheduling to OpenClaw crons, remove system crontab instructions 2026-03-06 17:21:27 +00:00
093b349aad fix: waitForSelector on modal after click instead of fixed 1.5s delay 2026-03-06 17:13:56 +00:00
8c626b5147 debug: log apply elements on skip; use state=attached for waitForSelector 2026-03-06 17:05:57 +00:00
318da35e01 fix: scroll page before waiting for Easy Apply button; extend timeout to 12s 2026-03-06 17:04:27 +00:00
45ad5d1ec9 fix: Easy Apply is an <a> not <button> — update selector; wait for element instead of fixed timeout 2026-03-06 17:01:02 +00:00
02a7501e9d fix: use aria-label selectors for LinkedIn Easy Apply button (class names are now hashed) 2026-03-06 16:53:25 +00:00
b8a0e03c75 feat: AI answer generation for unknown questions + Easy Apply only mode 2026-03-06 16:47:58 +00:00
b1528ac0ad refactor: extract magic numbers to constants, fix audit issues
- Centralize all magic numbers/strings in lib/constants.mjs
- Fix double-replaced import names in filter.mjs
- Consolidate duplicate fs imports in job_applier/job_searcher
- Remove empty JSDoc block in job_searcher
- Update keywords.mjs model from claude-3-haiku to claude-haiku-4-5
- Extract Anthropic API URLs to constants
- Convert :has-text() selectors to page.locator() API
- Fix SIGTERM handler conflict — move partial-run notification into lock.onShutdown
- Remove unused exports (LOCAL_USER_AGENT, DEFAULT_REVIEW_WINDOW_MINUTES)
- Fix variable shadowing (b -> v) in job_filter reduce callback
- Replace SKILL.md PM2 references with system cron

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 08:45:17 -08:00
3a52bdc72e feat: dynamic lookback window — time since last run × 1.25 buffer (min 4h) 2026-03-06 16:25:09 +00:00
37b95b6b85 feat: track token usage and estimated cost per filter run in filter_runs.json 2026-03-06 16:22:14 +00:00
3575f06018 fix: dedupeAfterFilter skips groups with unscored members — wait until all copies are scored 2026-03-06 16:00:23 +00:00
c9b527c83a feat: find-all → filter → dedup flow
- addJobs: allows same job on multiple tracks (dedup key = track::id)
- Cross-track copies get composite id (job.id_track) to avoid batch collisions
- dedupeAfterFilter(): after collect, keeps highest-scored copy per URL, marks rest as 'duplicate'
- Called automatically at end of collect phase
2026-03-06 15:55:00 +00:00
2dfadbde99 refactor: generic system prompt — dumps full JSON profiles, no hardcoded criteria 2026-03-06 11:36:05 +00:00
c88a71fc20 feat: one batch per track — separate GTM/AE batches with their own system prompts
- submitBatch → submitBatches: groups jobs by track, submits one batch each
- filter_state.json now stores batches[] array instead of single batch_id
- Collect waits for all batches to finish before processing
- Each track gets its own cached system prompt = better caching + cleaner scoring
- Idempotent collect: skips already-scored jobs
2026-03-06 11:35:15 +00:00
aadec0704b fix: AE filter criteria - don't penalize company size, SMB, or staffing agencies with real roles 2026-03-06 11:32:30 +00:00
85c88f9eed fix: make filter idempotent - skip already-scored jobs on collect, exclude by filter_score on submit 2026-03-06 11:25:19 +00:00
2f05a40954 fix: addJobs always reads fresh from disk to prevent searcher clobbering filter scores 2026-03-06 11:24:05 +00:00
56eb645e73 fix: import saveQueue statically instead of dynamic import; was causing queue writes to silently fail 2026-03-06 11:22:09 +00:00
8748c0e383 fix: send telegram on SIGTERM so partial cron runs still notify when jobs found 2026-03-06 11:16:39 +00:00
d4cc5b2541 fix: only send telegram when new jobs found (>0) 2026-03-06 11:14:50 +00:00
00b0094d97 feat: always send telegram after searcher run, not just when new jobs found 2026-03-06 11:14:30 +00:00
64748d5889 fix: stamp filter_batch_id on submitted jobs; exclude already-submitted/filtered from resubmit
- Submit phase now excludes jobs with filter_batch_id set (already in a batch)
- After submitting, stamps each job with filter_batch_id = batchId
- Filtered jobs already excluded by status='filtered'
- Prevents duplicate submissions when batch errors cause state to clear without scores
2026-03-06 11:13:10 +00:00
fac286aaeb fix: use claude-sonnet-4-6 (no date suffix) for batch API, bump max_tokens to 1024 2026-03-06 11:01:46 +00:00
0b40d284cd fix: bump max_tokens to 256, note batch API requires native model IDs 2026-03-06 10:58:11 +00:00
85038b6ce1 fix: batch collect O(n²) → single queue write; correct model to claude-3-haiku-20240307
- updateJobStatus was called 4,652 times causing ~4,652 file reads/writes
- Now loads queue once, applies all updates in memory, saves once
- Model was using OpenClaw alias (sonnet-4-6) not native Anthropic ID
- Only claude-3-haiku-20240307 is available on this API key; update settings.example.json
2026-03-06 10:56:54 +00:00
17fd919356 feat: add filter section to status report 2026-03-06 10:24:22 +00:00
728e0773b9 fix: sanitize Unicode surrogates in job descriptions, handle custom_id > 64 chars 2026-03-06 10:18:54 +00:00
d610060dbb feat: persistent run history logs for searcher and filter
- search_runs.json: append-only history of every searcher run
  (started_at, finished, added, seen, platforms, lookback_days)
- search_progress_last.json: snapshot of final progress state after
  each completed run — answers 'what keywords/tracks were searched?'
- filter_runs.json: append-only history of every filter batch
  (batch_id, submitted/collected timestamps, model, passed/filtered/errors)
Fixes the 'did the 90-day run complete?' ambiguity going forward
2026-03-06 10:16:06 +00:00
dbe9967713 feat: rewrite filter to use Anthropic Batch API
- Batch API = 50% cost savings vs synchronous calls
- Prompt caching on system prompt (profile + criteria shared across all jobs)
- One request per job with custom_id = job ID for result matching
- Two-phase state machine: submit → poll/collect (hourly cron safe)
- filter_state.json tracks pending batch ID between runs
- Model configurable via settings.filter.model (default: claude-sonnet-4-6)
- Telegram notifications on submit + collect
- Errors pass through — never block applications due to filter failure
- --stats flag for queue overview
2026-03-06 10:12:47 +00:00
9bf904dada feat: AI job filter — score jobs 0-10 with Claude Haiku before applying
- lib/filter.mjs: batch scoring engine (10 jobs/call, Claude Haiku)
- job_filter.mjs: standalone CLI with --dry-run and --stats flags
- Threshold configurable globally + per-search in search_config.json (filter_min_score, default 5)
- Job profiles (gtm/ae) passed as context via settings.filter.job_profiles
- Filtered jobs get status='filtered' with filter_score + filter_reason
- Filter errors pass jobs through (never block applications)
- status.mjs: added 'AI filtered' line to report
2026-03-06 10:01:15 +00:00
ff59316abc chore: update default searcher schedule to every 12 hours 2026-03-06 09:51:57 +00:00
9c03c2661e chore: remove test file 2026-03-06 02:58:23 +00:00
7de05fdce4 chore: remove PM2 — system cron + lockfile is sufficient, no daemon needed 2026-03-06 02:57:49 +00:00