Commit Graph

52 Commits

Author SHA1 Message Date
76c9d0df31 Add S3 binary file support and resume download
- ensureLocalFile() downloads binary files (resume PDF) from S3 to temp
- Applier downloads resume from S3 before applying
- Cached in /tmp to avoid re-downloading each run

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 22:07:31 -08:00
534d318953 Make S3 the primary storage layer (not backup)
storage.mjs is now a single interface: loadJSON() and saveJSON()
route to either local disk or S3 based on settings.storage.type.
The app never touches disk/S3 directly.

- All queue/log functions are now async (saveQueue, appendLog, etc.)
- All callers updated with await
- Data validation prevents saving corrupt types (strings, nulls)
- S3 versioned bucket preserves every write
- Config: storage.type = "local" (disk) or "s3" (S3 primary)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 22:03:16 -08:00
253d1888e9 Add S3-backed storage to prevent data loss
- New lib/storage.mjs: async S3 backup on every queue/log save
- Versioned S3 bucket (claw-apply-data) keeps every revision
- Auto-restore from S3 if local file is missing or corrupt
- saveQueue/saveLog now validate data type before writing
  (prevents the exact bug that corrupted the queue)
- IAM role attached to EC2 instance for credential-free S3 access
- Config: storage.type = "local" (default) or "s3"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:56:37 -08:00
5caa063175 Show apply_url in summary for external jobs, add Ashby closed detection
External ATS jobs now show their actual apply URL instead of the
LinkedIn listing URL. Also added Ashby-specific "job not found"
text to closed detection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:08:50 -08:00
a17886e58b Fix Ashby resume upload and add validation error logging
Ashby wraps file input inside #_systemfield_resume container — search
for the actual input[type="file"] element. Also capture and log
validation errors from the page when submit returns incomplete.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:47:15 -08:00
f85c4295eb Disable unknown_external in applier for now, keep ashby only
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:23:32 -08:00
298d01ae44 Only enable ATS types that work with generic applier
Ashby and unknown_external work. Greenhouse, Lever, Jobvite have
visible CAPTCHAs. Workday requires login. Keep those disabled until
proper handlers are built.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:15:47 -08:00
260f996ebc Add generic external ATS applier
Best-effort form filler for any career page with standard HTML forms.
Handles single-page and multi-step flows, resume upload, login wall
and CAPTCHA detection. All ATS stub handlers now delegate to generic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:02:55 -08:00
3a02c40b02 Move RATE_LIMIT_COOLDOWN_MS to constants file
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:16:14 -08:00
bbe68942e1 Exclude rate-limited platforms from job selection
When LinkedIn is rate-limited, its jobs were filling the maxApps quota
but then getting skipped, leaving 0 applied. Now excludes easy_apply
jobs from selection during cooldown so Wellfound jobs get picked up.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:15:29 -08:00
cc0d15ece7 6-hour cooldown after LinkedIn Easy Apply daily limit
When rate limited, writes timestamp to data/linkedin_rate_limited_at.json.
Subsequent runs skip LinkedIn until 6 hours have passed. Other platforms
(Wellfound) continue unaffected. Cooldown file auto-deleted on expiry.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:02:24 -08:00
374b2e6640 Rate limit only stops LinkedIn, continues other platforms
Don't break the entire run when LinkedIn rate limits — skip LinkedIn
jobs and continue to Wellfound and other platforms.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 17:55:02 -08:00
331408be41 Detect LinkedIn Easy Apply daily limit and stop run
When LinkedIn shows "Easy Apply limit" message, detect it in the
modal-open check, return rate_limited status, put job back in queue,
send Telegram alert, and stop the entire run. Prevents burning retries
and wasting browser sessions when rate limited.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 17:54:45 -08:00
80d2323a37 Dynamic profile lookup from auth connection
Profile name is returned by ensureAuth() from the auth connection
(looked up by domain). No more storing profile names in settings.json
for the applier flow. createBrowser() still supports legacy platform
keys as fallback for searcher/setup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:02:19 -08:00
668a40a51a Look up auth connections by domain instead of stored ID
Connection IDs change when re-authing. Instead of storing IDs in
settings.json (which go stale), look up connections by domain at
runtime via kernel.auth.connections.list({ domain }). This keeps
the applier in sync regardless of connection recreation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 15:55:26 -08:00
59e410b9c4 Add auth health check before browser creation
- Rewrite session.mjs: check connection status via SDK before creating
  browser. If NEEDS_AUTH + can_reauth, auto re-auth with stored creds.
  If can't re-auth, send Telegram alert and skip platform.
- Wire ensureAuth() into job_applier.mjs before createBrowser()
- Jobs are returned to queue (not failed) when auth is down

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 15:45:24 -08:00
7bd91a19a0 Fix maxRetries not defined crash in handleResult
The stuck/incomplete retry logic referenced maxRetries which was only
defined in main() scope, not in handleResult(). Compute it locally.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 15:11:32 -08:00
ad281d1a9f Retry stuck/incomplete jobs instead of marking permanently failed
Stuck and incomplete jobs now get retried up to max_retries (default 2)
before being permanently marked. Honeypots are still permanent.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 14:51:20 -08:00
d2de496d27 Track apply_started_at per job and log duration on success
Adds timing data to queue entries and shows seconds in the apply log.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 14:30:11 -08:00
82fa2b3697 Rich apply summary with per-job details (title, company, link)
Each job now shows title @ company with a clickable link, grouped by
status category. Only non-empty categories are shown.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 13:30:58 -08:00
094824abb2 Add dedicated 'closed' status for listings no longer accepting applications
Distinguishes closed listings from missing apply buttons. Shows in summary
as a separate line item.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 13:23:39 -08:00
bb0c96dd3d Enable Wellfound apply: fix missing apply_type, add submit verification
- Search now sets apply_type: 'wellfound' on discovered jobs (was missing,
  so applier silently skipped all Wellfound jobs)
- Add Wellfound to DEFAULT_ENABLED_APPLY_TYPES (LinkedIn first, then Wellfound)
- Sort platform processing order: linkedin → wellfound → external
- Apply handler: add closed listing detection, submit verification,
  error handling on meta extraction

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 12:48:03 -08:00
4419363b3c Fix process exit: use process.exit() directly instead of logStream.end callback
logStream.end() callback wasn't firing reliably, leaving processes hanging.
process.exit() is synchronous and forces exit regardless of open handles.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 12:21:55 -08:00
d43e2025b2 Fix process not exiting after run, detect closed job listings
- All entry points with log tee now call logStream.end() + process.exit()
  (log stream kept event loop alive, blocking next cron run)
- easy_apply: detect "no longer accepting applications" and similar closed
  listing text before reporting as unsupported

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 12:19:00 -08:00
51ca354c52 Audit fixes: remove dead code, fix run timeout bug, add log tee to all entry points
- Remove unused APPLY_PRIORITY array (replaced by score-based sort)
- Fix run timeout only breaking inner loop — now breaks outer platform loop too
- Remove dead lastProgress variable in easy_apply step loop
- Add stdout/stderr log tee to job_searcher, job_filter, telegram_poller

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 12:13:01 -08:00
9e6b9beb17 feat: sort apply queue by score desc, recency desc 2026-03-06 20:08:54 +00:00
b7836fcec2 Tee applier stdout/stderr to data/applier.log
Writes all console output to a log file from inside the process so
logs are always available regardless of how the process is launched.
Fixes invisible output when claw redirects stdout to a file that
gets overwritten by a second lock-blocked attempt.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:59:27 -08:00
8db00d94a5 Fix summary showing all zeros: count unhandled statuses, clean up format
- Default case in handleResult now increments skipped_other
- Summary only shows non-zero categories (cleaner output)
- Applied count always shown

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:54:06 -08:00
7c9de1af4a AI fallback for unknown form fields: ask Claude before Telegram
Every unknown required field now goes through AI before falling back to
Telegram. Claude sees the question + all saved answers + profile, and
either recognizes it as a variation of a saved answer or generates a new
one. AI answers are auto-saved to answers.json so the same question is
a free pattern match next time. Telegram is now last resort (no API key).

Flow: pattern match (free) → AI (smart) → auto-save → Telegram (human)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:52:39 -08:00
3fc5c38df7 Add clear status logging for needs_answer and honeypot exits
Log lines now explicitly say STOPPING with reason when the modal is
dismissed due to unknown questions or honeypots. handleResult logs
the full flow: paused → generating AI answer → sent to Telegram →
will retry after reply. Prevents claw from misreading as a hang/timeout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:47:40 -08:00
0920554dad Add Telegram answer learning: poller + applier safety net
- New lib/telegram_answers.mjs: shared module that polls Telegram getUpdates,
  matches replies to needs_answer jobs, saves to answers.json, flips job to new
- telegram_poller.mjs: lightweight cron script (every minute via OpenClaw)
- Applier also processes replies at start of each run as safety net
- sendTelegram now returns message_id, stored on job for reply matching
- User replies "ACCEPT" to use AI answer, or types their own
- Answers persist in answers.json and apply to ALL future jobs
- Also includes: selectOptionFuzzy, multiple dialog handling, browser
  recovery, answers reload, per-job timeout bump to 10min

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:38:37 -08:00
14cf9a12c1 Reliability improvements: click retry, resume selection, answer loop, browser recovery
- Easy Apply click: click found element directly, retry with force if modal doesn't open
- Resume: select first radio if none checked, fall back to file upload
- AI answers: inject stored answers into formFiller on needs_answer retry
- Answers persistence: reload answers.json before each job for Telegram replies
- Browser recovery: detect dead page, create fresh browser session
- Multiple dialogs: findApplyModal() tags the right dialog among cookie banners etc.
- Select matching: case-insensitive fuzzy match with substring fallback
- dismissModal: scope Discard scan to dialog elements only
- Label dedup: normalize whitespace, fix odd-length edge case
- no_modal status: explicit handleResult case
- Per-job timeout: 10 minutes (was 3)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:30:09 -08:00
00bcfaca53 feat: include select options in unknown field reports and Telegram messages
When a required select has no answer, the unknown field now includes
the available options. Telegram notification shows them so the user
knows exactly what to reply with.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 11:06:38 -08:00
e62756c6ca fix: robustness improvements — atomic writes, timeouts, shell injection, validation errors
- Atomic JSON writes (write-to-tmp + rename) prevent queue/log corruption
- Per-job (3min) and overall run (45min) timeouts prevent hangs
- execFileSync in ai_answer.mjs prevents shell injection with resume paths
- Validation error detection after form fill in Easy Apply modal
- Config-driven enabled_apply_types (from settings.json)
- isRequired() detects required/aria-required/label * patterns
- getLabel() strips trailing * from required field labels
- Actionable logging on failures ("Action: ..." messages)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 10:01:53 -08:00
b8a0e03c75 feat: AI answer generation for unknown questions + Easy Apply only mode 2026-03-06 16:47:58 +00:00
b1528ac0ad refactor: extract magic numbers to constants, fix audit issues
- Centralize all magic numbers/strings in lib/constants.mjs
- Fix double-replaced import names in filter.mjs
- Consolidate duplicate fs imports in job_applier/job_searcher
- Remove empty JSDoc block in job_searcher
- Update keywords.mjs model from claude-3-haiku to claude-haiku-4-5
- Extract Anthropic API URLs to constants
- Convert :has-text() selectors to page.locator() API
- Fix SIGTERM handler conflict — move partial-run notification into lock.onShutdown
- Remove unused exports (LOCAL_USER_AGENT, DEFAULT_REVIEW_WINDOW_MINUTES)
- Fix variable shadowing (b -> v) in job_filter reduce callback
- Replace SKILL.md PM2 references with system cron

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 08:45:17 -08:00
261f5800ad fix: load API keys from .env at runtime — never embed credentials in cron payloads or source code 2026-03-06 02:04:55 +00:00
1c3e9f3561 fix: split skip counters for accurate reporting, complete status docs
- Split skipped_no_easy_apply into skipped_no_apply (no button/modal)
  and skipped_other (honeypot/stuck/incomplete) for clearer reporting
- Update Telegram summary, status.mjs display, and job_applier counters
- Add missing skipped_easy_apply_unsupported to README status table

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 17:44:25 -08:00
8212f97aba refactor: normalize apply statuses, remove dead code, fix signal handler
- lib/apply/index.mjs: add STATUS_MAP to normalize platform-specific statuses
  to generic ones (no_button/no_submit/no_modal → skipped_no_apply).
  Documented all generic statuses for AI/developer reference.
- job_applier.mjs: handleResult now handles skipped_no_apply, default case
  logs + saves instead of silently dropping
- lib/linkedin.mjs: remove dead applyLinkedIn() and detectAts(), clean imports
  (~110 lines removed). Search-only module now.
- lib/wellfound.mjs: remove dead applyWellfound(), clean imports.
  Search-only module now.
- lib/lock.mjs: fix async signal handler — shutdown handlers now actually
  complete before process.exit()
- test_linkedin_login.mjs: add try/catch/finally with proper browser cleanup
- README: update status table with all current statuses

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 17:39:48 -08:00
1bf676bb80 fix: add error logging to applier, session polling, and job card clicks
- job_applier.mjs: stack traces on per-job and browser-level errors
- session.mjs: log pending status and poll count during session refresh
- linkedin.mjs: log warning when job card click fails instead of silent catch

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 17:30:24 -08:00
35fdbc487a refactor: handler registry pattern — lib/apply/<ats>.mjs, applyToJob() routes by apply_type 2026-03-06 01:03:11 +00:00
dee6e98603 feat: searcher Phase 2 classifies apply type; applier sorts by priority; already-applied detection 2026-03-06 00:58:22 +00:00
b091473735 fix: graceful shutdown — write last-run file on SIGTERM, show interrupted state in status 2026-03-06 00:35:10 +00:00
1920df51a4 feat: rich status report — searcher/applier last run time, timeAgo, per-run metadata files 2026-03-06 00:33:03 +00:00
b496ee4a3a feat: ATS breakdown in Telegram summary after each apply run 2026-03-06 00:27:43 +00:00
58c2ad5295 feat: capture external ATS URLs + analyze_ats.mjs to rank platforms by job count 2026-03-06 00:26:28 +00:00
234820ad91 feat: lockfile to prevent parallel runs + AI keywords lib 2026-03-06 00:11:37 +00:00
a244a5fddf chore: clean up dead code, use shared loadConfig, cache queue I/O
- Remove unused readFileSync import from job_applier.mjs
- Remove unused makeJobId (dead code, nothing imports it)
- setup.mjs: use shared loadConfig instead of inline cfg()
- queue.mjs: add in-memory cache for queue and log to avoid
  redundant disk reads during a single run

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 16:06:37 -08:00
e71f940687 fix: add config validation and retry logic for failed jobs
- Add loadConfig() helper with clear errors for missing/malformed JSON
- Replace raw JSON.parse(readFileSync(...)) in both entry points
- Track retry_count on jobs; re-queue as 'new' up to max_retries (default 2)
- Add max_retries and DEFAULT_MAX_RETRIES constant

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 16:03:33 -08:00
47513e8cec fix: security/bug fixes, extract constants, remove magic values
- Remove random suffix from Wellfound job IDs (broke dedup)
- Add null coalescing to all profile field returns in form_filler
- Fix honeypot case referencing nonexistent results.skipped counter
- Remove unused makeJobId import from linkedin.mjs
- Navigate directly to job URL instead of search+click in linkedin apply
- Add Telegram notification rate limiting (1.5s between sends)
- Replace Mode B blocking sleep with --preview flag
- Add max_applications_per_run enforcement
- Remove tracked search_config.json (keep .example.json only)
- Add search_config.json to .gitignore, fix duplicate node_modules entry
- Extract all magic numbers/strings to lib/constants.mjs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 16:01:42 -08:00