Commit Graph

150 Commits

Author SHA1 Message Date
f14af48905 Route all file I/O through storage layer (S3 or disk)
- filter.mjs: loadProfile now async, uses loadJSON
- telegram_answers.mjs: answers read/write through storage layer
- status.mjs: uses initQueue + loadQueue for S3 support
- setup.mjs: await all loadConfig calls
- storage.mjs: more robust getS3Key using URL parsing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 08:25:22 -08:00
377437cf3c Fix hardcoded OpenClaw paths — use standard module resolution
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 22:51:11 -08:00
ba5cbedcf4 Make loadConfig async and route through storage layer (S3 or disk)
- loadConfig now uses loadJSON when storage is initialized
- Fix getS3Key to handle config/ and data/ paths (not just data/)
- All loadConfig calls updated to await
- settings.json still bootstraps from disk (needed to know storage type)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 22:47:07 -08:00
2e61854be5 Wrap keyword search in try/catch for resilience
Failed keywords log the error and continue to the next one.
Not marked complete, so they'll be retried on next run.
Also await async onPage callback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 22:14:28 -08:00
20e866ee31 Add ensureLoggedIn to session.mjs for searcher
Searcher creates browser first then verifies login, unlike applier
which checks auth before browser creation. Both paths now work.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 22:10:22 -08:00
76c9d0df31 Add S3 binary file support and resume download
- ensureLocalFile() downloads binary files (resume PDF) from S3 to temp
- Applier downloads resume from S3 before applying
- Cached in /tmp to avoid re-downloading each run

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 22:07:31 -08:00
534d318953 Make S3 the primary storage layer (not backup)
storage.mjs is now a single interface: loadJSON() and saveJSON()
route to either local disk or S3 based on settings.storage.type.
The app never touches disk/S3 directly.

- All queue/log functions are now async (saveQueue, appendLog, etc.)
- All callers updated with await
- Data validation prevents saving corrupt types (strings, nulls)
- S3 versioned bucket preserves every write
- Config: storage.type = "local" (disk) or "s3" (S3 primary)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 22:03:16 -08:00
253d1888e9 Add S3-backed storage to prevent data loss
- New lib/storage.mjs: async S3 backup on every queue/log save
- Versioned S3 bucket (claw-apply-data) keeps every revision
- Auto-restore from S3 if local file is missing or corrupt
- saveQueue/saveLog now validate data type before writing
  (prevents the exact bug that corrupted the queue)
- IAM role attached to EC2 instance for credential-free S3 access
- Config: storage.type = "local" (default) or "s3"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:56:37 -08:00
c78586926a Extend submit polling to ~30s for reCAPTCHA verification
Invisible reCAPTCHA + server round-trip can take 10-30s. Increased
from 6x2.5s (~15s) to 10x3s (~30s) polling window.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:45:41 -08:00
e28ccb627a Fix pressSequentially -> type() and extend submit polling to ~15s
Playwright on AWS doesn't have pressSequentially, use type() instead.
Extend submit polling from 4 to 6 iterations (~15s total) to handle
slower reCAPTCHA verification.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:39:53 -08:00
8fdc2ea7f8 Fix Ashby Yes/No button detection using field-entry containers
Use .ashby-application-form-field-entry class to find question
containers, then Playwright locators for reliable button clicking.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:35:48 -08:00
c38f80086f Handle Ashby button-style questions and Greenhouse React Selects
Ashby: detect Yes/No button questions for work auth, sponsorship,
and consent. Click appropriate button in beforeSubmit hook.

Greenhouse: use pressSequentially instead of fill() for React
Select comboboxes (Country, Location). Click the dropdown option
after typing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:32:49 -08:00
7b6ff2830c Trigger autocomplete selection for country fields
Country inputs on Greenhouse use React Select comboboxes that
need option click after typing. Add 'country' to the autocomplete
trigger list alongside city/location.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:27:05 -08:00
fa6efc3775 Fix resume upload targeting wrong file input on Ashby
Prefer resume-specific file inputs (#resume, #_systemfield_resume)
over autofill inputs. Skip autofill class inputs in fallback.
Also fix Ashby beforeSubmit to target #_systemfield_resume directly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:15:16 -08:00
0118254046 Poll for submission success instead of single check
After clicking submit, poll up to 4 times (2.5s + 3x2s = ~8.5s total)
for success text or form disappearance. Handles invisible reCAPTCHA
verification delay. Stops early on validation errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:11:50 -08:00
5caa063175 Show apply_url in summary for external jobs, add Ashby closed detection
External ATS jobs now show their actual apply URL instead of the
LinkedIn listing URL. Also added Ashby-specific "job not found"
text to closed detection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:08:50 -08:00
e4e3055bb1 Skip Phone/Name fieldsets that wrap already-handled input groups
Greenhouse wraps phone inputs in a fieldset with legend "Phone"
that gets picked up as an unanswered radio group. Skip these
since the actual phone input is handled separately.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:05:03 -08:00
c680105bc8 Fix phone country code search input being flagged as unknown
Skip inputs with placeholder="Search" or pre-filled values when
they have a "Phone" label — these are country code pickers, not
the actual phone input.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:01:53 -08:00
f586c6d091 Decouple form filler from LinkedIn modal selector
Form filler now defaults to page root instead of scoping to
[role="dialog"]. LinkedIn Easy Apply passes its modal selector
explicitly. Fixes external ATS forms being scoped to wrong
container. Also improved Greenhouse handler with targeted
resume upload and form detection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:58:51 -08:00
a17886e58b Fix Ashby resume upload and add validation error logging
Ashby wraps file input inside #_systemfield_resume container — search
for the actual input[type="file"] element. Also capture and log
validation errors from the page when submit returns incomplete.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:47:15 -08:00
ae797d73eb Refactor apply handlers: generic with extensions + domain auto-routing
Generic handler now accepts options (transformUrl, formDetector,
submitSelector, resumeSelector, beforeSubmit, verifySelector, etc.).
Each ATS handler passes its overrides instead of reimplementing.

Registry resolves handlers by: apply_type -> URL domain -> generic fallback.
New ATS handlers only need to export SUPPORTED_TYPES and an apply() that
calls genericApply with platform-specific options.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:41:54 -08:00
4f202a4e91 Build dedicated Ashby handler instead of generic delegation
Handles Ashby-specific quirks:
- Auto-appends /application to job URLs
- Targets #_systemfield_resume for resume upload (not autofill input)
- Clicks "Submit Application" specifically, avoiding "Upload file" buttons
- Checks for Ashby-specific form fields to verify submission

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:39:24 -08:00
6492444a3e Use job queue metadata instead of scraping page for title/company
Page scraping was grabbing wrong elements (e.g. "Location" instead of
company name on Ashby). Queue already has correct metadata.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:37:31 -08:00
630996f643 Use DEFAULT_MODEL constant for Anthropic API calls
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:36:44 -08:00
e1a4ccf7a0 Fix model ID to use full claude-sonnet-4-6-20251101
Short alias was returning 404 from Anthropic API.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:35:27 -08:00
a4f84cd4c3 Skip social media fields (Twitter, GitHub, etc.) in form filler
Return empty string to leave them blank without triggering needs_answer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:32:30 -08:00
73c23d9bf1 Improve generic applier: better closed/404 detection and submit selectors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:20:24 -08:00
1772229ae7 Fix CAPTCHA detection to allow invisible reCAPTCHA
Invisible reCAPTCHA (size=invisible) fires on submit and usually passes
automatically. Only block on visible CAPTCHA challenges. Also fix form
detection to work without <form> wrapper elements.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:08:44 -08:00
260f996ebc Add generic external ATS applier
Best-effort form filler for any career page with standard HTML forms.
Handles single-page and multi-step flows, resume upload, login wall
and CAPTCHA detection. All ATS stub handlers now delegate to generic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:02:55 -08:00
b19c1ee77e Add ATS patterns for grnh.se, myworkdaysite, eightfold, comeet, gem, adp, dayforcehcm
Reclassifies 58 more unknown_external jobs using URLs already in queue.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 19:58:47 -08:00
273a32e571 Add sidecar update pattern to prevent concurrent queue writes
Secondary processes (standalone classifier, ad-hoc scripts) now write
to queue_updates.jsonl via writePendingUpdate() instead of modifying
jobs_queue.json directly. Primary processes pick up updates on next
loadQueue() call using atomic rename-then-read.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 19:46:36 -08:00
de92ba85c7 Add more ATS patterns for external job classification
Adds applytojob.com (Jobvite), trakstar, resumator, breezy, jazz,
recruitee, applicantpro, paylocity, paycom, ultipro, successfactors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:56:53 -08:00
c33fb2ba0d Fix classifyExternalJobs to extract URL from redirect link
External Apply buttons are <a> tags with LinkedIn redirect URLs, not
<button> elements. Extract the real URL from the redirect's query
parameter instead of clicking and waiting for a new tab.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:51:38 -08:00
3a02c40b02 Move RATE_LIMIT_COOLDOWN_MS to constants file
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:16:14 -08:00
69eb6b124f Classify unknown_external jobs by following Apply redirects
After LinkedIn search completes, visits each unknown_external job page,
clicks the Apply button, captures the redirect URL, and matches against
known ATS patterns to identify the actual application platform.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:13:32 -08:00
dfe6810acc Add direct URL rate limit text to detection
LinkedIn shows different text on direct /apply/ URL vs button click:
- Button: "Easy Apply limit"
- Direct URL: "limit daily submissions" / "apply tomorrow"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 17:58:37 -08:00
331408be41 Detect LinkedIn Easy Apply daily limit and stop run
When LinkedIn shows "Easy Apply limit" message, detect it in the
modal-open check, return rate_limited status, put job back in queue,
send Telegram alert, and stop the entire run. Prevents burning retries
and wasting browser sessions when rate limited.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 17:54:45 -08:00
814492d9b6 Skip file upload when input only accepts images, not PDF/DOC
Some LinkedIn listings have image-only upload fields (JPG/PNG).
Don't attempt to upload a PDF resume to these inputs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 17:41:34 -08:00
38dabbd6dd Fix address hallucination and duplicate status messages
- answerFor returning '' (intentionally blank) was treated as falsy,
  falling through to AI which fabricated "123 Main Street". Now
  empty string skips the field without triggering AI or reporting unknown.
- status.mjs was printing to stdout AND sending via Telegram, causing
  OpenClaw to relay a duplicate plain-text copy. Now only prints to
  stdout when Telegram isn't configured.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:47:18 -08:00
84a98e7839 Treat number fields as required even if DOM doesn't mark them
LinkedIn validates number fields even when not marked required in the
DOM. Previously these were skipped (no AI call, no answer). Now number
fields always trigger AI fallback and are reported as unknown if empty.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:43:39 -08:00
aefd1f2023 Fix label dedup for space-separated duplicates
LinkedIn renders label text twice in nested spans, producing
"Question? Question?" instead of "Question?". The old dedup only
caught exact concatenation (ABCABC); now also handles space-separated
duplicates by comparing left/right halves at the midpoint space.

Applied to all 4 copies: extractLabel, _extractLabel, normalizeLegend,
_normalizeLegend.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:28:51 -08:00
05739a455b Fix AI hallucinating street addresses and other facts
- Add street/address pattern to answerFor() — returns profile address or empty string
- Update AI prompt: return "UNKNOWN" instead of guessing facts
- Handle UNKNOWN response by treating it as no answer (triggers Telegram ask)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:25:32 -08:00
fd30646d8a Fix number field validation: strip text from numeric answers
Number inputs (type="number") were getting answers like "5 years"
instead of "5", causing LinkedIn validation errors. Now:
- AI gets "(must be a number, no text or units)" hint
- Answers are stripped to digits before filling

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:21:37 -08:00
f248e8573f Auto-create claw-apply proxy if it doesn't exist
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:06:47 -08:00
df9c790a0e Look up proxy by name 'claw-apply' instead of stored ID
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:06:14 -08:00
80d2323a37 Dynamic profile lookup from auth connection
Profile name is returned by ensureAuth() from the auth connection
(looked up by domain). No more storing profile names in settings.json
for the applier flow. createBrowser() still supports legacy platform
keys as fallback for searcher/setup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:02:19 -08:00
668a40a51a Look up auth connections by domain instead of stored ID
Connection IDs change when re-authing. Instead of storing IDs in
settings.json (which go stale), look up connections by domain at
runtime via kernel.auth.connections.list({ domain }). This keeps
the applier in sync regardless of connection recreation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 15:55:26 -08:00
59e410b9c4 Add auth health check before browser creation
- Rewrite session.mjs: check connection status via SDK before creating
  browser. If NEEDS_AUTH + can_reauth, auto re-auth with stored creds.
  If can't re-auth, send Telegram alert and skip platform.
- Wire ensureAuth() into job_applier.mjs before createBrowser()
- Jobs are returned to queue (not failed) when auth is down

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 15:45:24 -08:00
6d69443f51 Pass placeholder format hints to AI for inputs and textareas
Captures placeholder from textareas in snapshot. When AI fallback is
used, includes placeholder as a format hint so the AI knows expected
format (dates, phone numbers, URLs, etc).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 15:12:35 -08:00
3bf1786895 Detect date format from placeholder (MM/DD/YYYY) instead of label matching
Reads input placeholder in snapshot. If it contains MM/DD/YYYY pattern,
auto-fills with today's date in the correct format. Generic — works
regardless of the field label.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 15:05:41 -08:00