Compare commits

1 Commits

Author SHA1 Message Date
Fly.io
85824a5491 New files from Fly.io Launch 2025-12-30 03:26:34 +00:00
13 changed files with 76 additions and 3025 deletions

View File

@@ -1,31 +1,3 @@
# Dependencies
node_modules node_modules
npm-debug.log npm-debug.log
yarn-error.log
pnpm-debug.log
# Environment / secrets
.env
.env.*
*.pem
*.key
*.crt
*.p12
*.pfx
secrets
credentials
# Git / VCS
.git
.gitignore
# OS / editor junk
.DS_Store .DS_Store
Thumbs.db
.vscode
.idea
# Logs / coverage
logs
*.log
c

23
.gitignore vendored
View File

@@ -1,23 +0,0 @@
# Dependencies
node_modules/
# Environment / secrets
.env
.env.*
!.env.example
!infra/.env.example
infra/.deploy-secrets
# Logs
npm-debug.log*
yarn-error.log*
pnpm-debug.log*
# Claude memory
memory/
# OS / editor junk
.DS_Store
.vscode
.idea

View File

@@ -1,55 +0,0 @@
# PostConvert
Image/PDF-to-JPEG conversion microservice. Single-file Node.js/Express app (`server.js`).
## Tech Stack
- Node.js 18+ (ES modules), Express
- Sharp (image processing), libheif-js (HEIC/HEIF via WASM)
- pdftoppm from Poppler (PDF rendering), archiver (ZIP output)
- Docker (node:20-bookworm-slim), deployed on Fly.io (sjc region)
## Endpoints
- `GET /` — web UI for stripping photo metadata (drag-and-drop, uses `/strip`)
- `GET /health` — health check
- `POST /strip` — strip metadata from image, return clean JPEG + removed metadata (JSON or raw)
- `POST /convert` — image or first PDF page → JPEG (resize/quality via headers)
- `POST /convert/pdf` — all PDF pages → JSON array of base64 JPEGs
## Auth
Bearer token via `CONVERTER_TOKEN` env var on all endpoints.
## Key Design
- Single-flight concurrency (1 conversion at a time, 429 if busy)
- Resize/quality via headers: `x-jpeg-quality`, `x-max-dimension`, `x-pdf-dpi`, `x-without-enlargement`
- Pixel limit 200M, bounded /tmp, no stack traces leaked, request timeouts
- Fly.io auto-scaling 010 machines, 2 concurrent connection hard limit
## Environment Variables
- `PORT` (default 8080)
- `CONVERTER_TOKEN` (required)
- `REQ_TIMEOUT_MS` (default 120s, range 5600s)
- `REQ_TIMEOUT_PDF_MS` (default 5m, range 10s30m)
## Development
```sh
npm install
node server.js
```
Docker:
```sh
docker build -t postconvert .
docker run -e CONVERTER_TOKEN=secret -p 8080:8080 postconvert
```
## Fly.io Deployment
Tokens are in `.env` (gitignored). Load with `source .env` or use `dotenv`.
Deploy: `FLY_API_TOKEN="$FLY_API_TOKEN" fly deploy`

16
Dockefile Normal file
View File

@@ -0,0 +1,16 @@
FROM node:20-bookworm-slim
RUN apt-get update && apt-get install -y \
libheif1 libheif-dev \
libvips libvips-dev \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY package.json package-lock.json* ./
RUN npm ci || npm i
COPY server.js ./
ENV PORT=8080
EXPOSE 8080
CMD ["node", "server.js"]

19
Dockerfile Normal file → Executable file
View File

@@ -1,20 +1,21 @@
FROM node:20-bookworm-slim FROM node:20-bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \ # Install native deps required for HEIC → JPEG
poppler-utils \ RUN apt-get update && apt-get install -y \
libheif1 libheif-dev \
libvips libvips-dev \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
WORKDIR /app WORKDIR /usr/src/app
# Prefer deterministic installs. This will fail if package-lock.json is missing. # Install node deps
COPY package.json package-lock.json ./ COPY package.json package-lock.json* ./
RUN npm ci --omit=dev RUN npm ci || npm install
# Copy app code
COPY server.js ./ COPY server.js ./
ENV PORT=8080 ENV PORT=8080
EXPOSE 8080 EXPOSE 8080
# Run as non-root (node user exists in official node images)
USER node
CMD ["node", "server.js"] CMD ["node", "server.js"]

View File

@@ -1,97 +0,0 @@
# PostConvert Robust Image & PDF → JPEG Conversion Service
PostConvert is a small, production-oriented HTTP service for converting **images and PDFs into JPEGs**, with strong emphasis on **robustness, predictable resource usage, and operational safety**.
It supports:
- JPEG → JPEG (resize + recompress)
- PNG / WebP / TIFF → JPEG
- HEIC / HEIF → JPEG (via WASM fallback)
- PDF → JPEG (first page or all pages as a ZIP)
The service is designed to run well in **containers, serverless-ish environments, and small VMs** where disk, memory, and runaway workloads matter.
---
## Primary Use Cases
- Normalize user uploads (receipts, photos, scans) into JPEG
- Resize and recompress images server-side for storage or ML pipelines
- Convert PDFs (receipts, invoices, statements) into images
- Handle HEIC uploads from iOS devices without native libheif dependencies
- Safely process untrusted user uploads with bounded resource usage
---
## Key Design Goals
- **Always JPEG out**
- **Bounded `/tmp` usage** (PDFs rendered page-by-page)
- **No stack traces leaked to clients**
- **Fast path for common formats**
- **Graceful abort on client disconnect**
- **Predictable limits** (size, pages, DPI, timeouts)
---
## Endpoints
### `POST /convert`
Converts a single image **or the first page of a PDF** to a JPEG.
#### Supported inputs
- JPEG, PNG, WebP, TIFF, etc. (anything Sharp can decode)
- HEIC / HEIF
- PDF (first page only)
#### Response
- `200 image/jpeg` on success
- JSON error on failure
---
### `POST /convert/pdf`
Converts **all pages of a PDF** into JPEGs and returns a ZIP archive.
Pages are rendered **one at a time** to keep disk usage bounded.
---
## Authentication
All endpoints require a bearer token.
```
Authorization: Bearer <CONVERTER_TOKEN>
```
---
## Image Resize & JPEG Options (Headers)
| Header | Type | Default | Description |
|------|-----|--------|-------------|
| `x-jpeg-quality` | `0100` | `100` | JPEG compression quality |
| `x-max-dimension` | px | none | Max width/height, aspect preserved |
| `x-width` | px | none | Explicit output width |
| `x-height` | px | none | Explicit output height |
| `x-fit` | enum | `inside` | `inside`, `cover`, `contain`, `fill`, `outside` |
| `x-without-enlargement` | bool | `true` | Prevent upscaling smaller images |
---
## Environment Variables
| Variable | Default | Description |
|-------|--------|------------|
| `PORT` | `8080` | Server port |
| `CONVERTER_TOKEN` | (required) | Bearer auth token |
---
## Runtime Dependencies
- Node.js 18+
- `pdftoppm` (Poppler utils) **required for PDFs**
- Sharp native dependencies (per Sharp docs)

View File

@@ -1,31 +1,23 @@
# fly.toml # fly.toml app configuration file generated for postconvert on 2025-12-30T03:26:30Z
app = "postconvert" #
primary_region = "sjc" # See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#
app = 'postconvert'
primary_region = 'sjc'
[build] [build]
dockerfile = "Dockerfile"
[[vm]]
memory = "1gb"
[http_service] [http_service]
internal_port = 8080 internal_port = 8080
force_https = true force_https = true
auto_stop_machines = 'stop'
auto_start_machines = true auto_start_machines = true
auto_stop_machines = true
min_machines_running = 0 min_machines_running = 0
max_machines_running = 2 processes = ['app']
# Allow 2 concurrent connections so /health can succeed while /convert is running. [[vm]]
# We enforce "only 1 conversion at a time" in server.js. memory = '1gb'
concurrency = { soft_limit = 2, hard_limit = 2 } cpu_kind = 'shared'
cpus = 1
grace_period = "30s" memory_mb = 1024
[[http_service.checks]]
grace_period = "10s"
interval = "15s"
method = "GET"
path = "/health"
timeout = "5s"

View File

@@ -1 +0,0 @@
CONVERTER_TOKEN=your-secret-token-here

View File

@@ -1,214 +0,0 @@
#!/bin/bash
# Deploy a project stack to Portainer and wire it into Caddy.
#
# 1. Creates (or updates) the project's own Portainer stack
# 2. Connects Caddy to the project's network via Docker API
# 3. Writes a Caddy route snippet and reloads
#
# Zero SSH required. Copy this into each project and configure the variables below.
#
# Required env:
# PORTAINER_URL — e.g. https://portainer-1.docker.pq.io
# PORTAINER_API_KEY — from docker-server setup.sh output
#
# Optional:
# ENV_FILE — path to .env file (default: ./infra/.env)
# COMPOSE_FILE — path to compose file (default: ./infra/docker-compose.yml)
#
# Usage:
# export PORTAINER_URL=https://portainer-1.docker.pq.io
# export PORTAINER_API_KEY=ptr_...
# bash infra/deploy-stack.sh
set -euo pipefail
# ── PostConvert configuration ──
STACK_NAME="${STACK_NAME:-postconvert}"
PROJECT_NETWORK="${PROJECT_NETWORK:-postconvert-net}"
CADDY_ROUTES="${CADDY_ROUTES:-$'img.pq.io {\n\treverse_proxy app:8080\n}'}"
PORTAINER_URL="${PORTAINER_URL:?Set PORTAINER_URL}"
PORTAINER_API_KEY="${PORTAINER_API_KEY:?Set PORTAINER_API_KEY}"
STACK_NAME="${STACK_NAME:?Set STACK_NAME}"
PROJECT_NETWORK="${PROJECT_NETWORK:?Set PROJECT_NETWORK}"
CADDY_ROUTES="${CADDY_ROUTES:?Set CADDY_ROUTES}"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
ENV_FILE="${ENV_FILE:-$SCRIPT_DIR/.env}"
COMPOSE_FILE="${COMPOSE_FILE:-$SCRIPT_DIR/docker-compose.yml}"
for f in "$ENV_FILE" "$COMPOSE_FILE"; do
[ -f "$f" ] || { echo "ERROR: not found: $f"; exit 1; }
done
API="$PORTAINER_URL/api"
# ── Get endpoint ID ──
echo "[$STACK_NAME] Looking up Portainer endpoint..."
ENDPOINT_ID=$(curl -s "$API/endpoints" \
-H "X-API-Key: $PORTAINER_API_KEY" | \
python3 -c "import sys,json; print(json.load(sys.stdin)[0]['Id'])")
[ -z "$ENDPOINT_ID" ] && { echo "ERROR: No Portainer endpoint found"; exit 1; }
DOCKER_API="$API/endpoints/$ENDPOINT_ID/docker"
# ── Helper: find container ID by name substring ──
find_container() {
local name="$1"
curl -s "$DOCKER_API/containers/json" \
-H "X-API-Key: $PORTAINER_API_KEY" | \
python3 -c "
import sys, json
for c in json.load(sys.stdin):
for n in c.get('Names', []):
if '/$name' in n:
print(c['Id'][:12])
sys.exit(0)
" 2>/dev/null || true
}
# ── Helper: exec in container (detached) ──
container_exec() {
local container_id="$1"; shift
local cmd_json
cmd_json=$(python3 -c "import json,sys; print(json.dumps(sys.argv[1:]))" "$@")
local exec_id
exec_id=$(curl -s -X POST "$DOCKER_API/containers/$container_id/exec" \
-H "X-API-Key: $PORTAINER_API_KEY" \
-H "Content-Type: application/json" \
-d "{\"Cmd\":$cmd_json,\"Detach\":true}" | \
python3 -c "import sys,json; print(json.load(sys.stdin)['Id'])")
curl -s -X POST "$DOCKER_API/exec/$exec_id/start" \
-H "X-API-Key: $PORTAINER_API_KEY" \
-H "Content-Type: application/json" \
-d '{"Detach":true}' > /dev/null
}
# ══════════════════════════════════════════════
# 1. Deploy project stack
# ══════════════════════════════════════════════
build_payload() {
local mode="$1"
python3 - "$COMPOSE_FILE" "$ENV_FILE" "$STACK_NAME" "$mode" <<'PYEOF'
import json, sys
compose_file, env_file, stack_name, mode = sys.argv[1:5]
with open(compose_file) as f:
compose = f.read()
import re
env_vars = []
with open(env_file) as f:
for line in f:
line = line.strip()
if not line or line.startswith('#'):
continue
key, _, value = line.partition('=')
if key:
env_vars.append({"name": key, "value": value})
# Replace env_file directives with explicit environment: blocks.
# Portainer injects stack-level env vars which docker compose substitutes
# into ${VAR} expressions — each container gets all its vars this way.
def replace_env_file(m):
indent = re.search(r'\n(\s+)env_file', m.group(0)).group(1)
lines = [f'\n{indent}environment:']
for var in env_vars:
lines.append(f'{indent} {var["name"]}: "${{{var["name"]}}}"')
return '\n'.join(lines)
compose = re.sub(r'\n\s+env_file:[^\n]*', replace_env_file, compose)
payload = {"stackFileContent": compose, "env": env_vars}
if mode == "create":
payload["name"] = stack_name
else:
payload["prune"] = True
json.dump(payload, sys.stdout)
PYEOF
}
echo "[$STACK_NAME] Checking for existing stack..."
EXISTING_ID=$(curl -s "$API/stacks" \
-H "X-API-Key: $PORTAINER_API_KEY" | \
python3 -c "
import sys, json
for s in json.load(sys.stdin):
if s['Name'] == '$STACK_NAME':
print(s['Id']); break
" 2>/dev/null || true)
if [ -n "$EXISTING_ID" ]; then
echo "[$STACK_NAME] Updating stack (ID: $EXISTING_ID)..."
build_payload update | curl -s -X PUT "$API/stacks/$EXISTING_ID?endpointId=$ENDPOINT_ID" \
-H "X-API-Key: $PORTAINER_API_KEY" \
-H "Content-Type: application/json" \
-d @- > /dev/null
else
echo "[$STACK_NAME] Creating stack..."
build_payload create | curl -s -X POST "$API/stacks/create/standalone/string?endpointId=$ENDPOINT_ID" \
-H "X-API-Key: $PORTAINER_API_KEY" \
-H "Content-Type: application/json" \
-d @- > /dev/null
fi
echo "[$STACK_NAME] Stack deployed."
# ══════════════════════════════════════════════
# 2. Connect Caddy to project network
# ══════════════════════════════════════════════
echo "[$STACK_NAME] Connecting Caddy to $PROJECT_NETWORK..."
CADDY_ID=$(find_container "shared-caddy-1")
[ -z "$CADDY_ID" ] && CADDY_ID=$(find_container "caddy")
if [ -z "$CADDY_ID" ]; then
echo "WARNING: Caddy container not found — skipping network + route setup."
else
# Wait for network to appear (stack may still be starting)
NET_ID=""
for i in $(seq 1 30); do
NET_ID=$(curl -s "$DOCKER_API/networks" \
-H "X-API-Key: $PORTAINER_API_KEY" | \
python3 -c "
import sys, json
for n in json.load(sys.stdin):
if n['Name'] == '$PROJECT_NETWORK':
print(n['Id'][:12]); break
" 2>/dev/null || true)
[ -n "$NET_ID" ] && break
sleep 1
done
if [ -z "$NET_ID" ]; then
echo "WARNING: Network $PROJECT_NETWORK not found after 30s — skipping Caddy wiring."
else
# Connect Caddy to project network (ignore error if already connected)
curl -s -X POST "$DOCKER_API/networks/$NET_ID/connect" \
-H "X-API-Key: $PORTAINER_API_KEY" \
-H "Content-Type: application/json" \
-d "{\"Container\":\"$CADDY_ID\"}" > /dev/null 2>&1 || true
echo "[$STACK_NAME] Caddy connected to $PROJECT_NETWORK."
fi
# ══════════════════════════════════════════════
# 3. Write Caddy route snippet + reload
# ══════════════════════════════════════════════
echo "[$STACK_NAME] Configuring Caddy route..."
ROUTES_B64=$(printf '%s' "$CADDY_ROUTES" | base64 | tr -d '\n')
container_exec "$CADDY_ID" sh -c "echo '$ROUTES_B64' | base64 -d > /etc/caddy/sites/$STACK_NAME.caddy"
sleep 1
container_exec "$CADDY_ID" caddy reload --config /etc/caddy/Caddyfile
echo "[$STACK_NAME] Caddy route configured."
fi
echo ""
echo "=== $STACK_NAME deployed ==="
echo "Check status at: $PORTAINER_URL"

View File

@@ -1,19 +0,0 @@
services:
app:
image: registry.docker.pq.io/postconvert:latest
restart: unless-stopped
networks:
- postconvert-net
environment:
CONVERTER_TOKEN: ${CONVERTER_TOKEN}
labels:
- "com.centurylinklabs.watchtower.enable=true"
deploy:
resources:
limits:
cpus: "1.0"
memory: 1g
networks:
postconvert-net:
name: postconvert-net

1719
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,17 +1,9 @@
{ {
"name": "postconvert", "name": "heic-converter",
"version": "1.0.0",
"description": "Robust image and PDF to JPEG conversion service with HEIC fallback and page-by-page PDF ZIP output",
"type": "module", "type": "module",
"private": true, "private": true,
"engines": {
"node": ">=18"
},
"dependencies": { "dependencies": {
"archiver": "^6.0.2",
"exif-reader": "^2.0.3",
"express": "^4.19.2", "express": "^4.19.2",
"libheif-js": "^1.18.0",
"sharp": "^0.33.5" "sharp": "^0.33.5"
} }
} }

864
server.js
View File

@@ -1,846 +1,52 @@
// server.js
import express from "express"; import express from "express";
import sharp from "sharp"; import sharp from "sharp";
import { execFile } from "child_process";
import fs from "fs/promises";
import { randomUUID } from "crypto";
import libheifModule from "libheif-js";
import exifReader from "exif-reader";
const libheif = libheifModule?.default ?? libheifModule;
const app = express(); const app = express();
// Raw binary uploads (HEIC/JPEG/etc)
app.use(express.raw({ type: "*/*", limit: "30mb" })); app.use(express.raw({ type: "*/*", limit: "30mb" }));
// Friendly "is it up" endpoints
app.get("/", (_req, res) => res.status(200).send("postconvert: ok"));
app.get("/health", (_req, res) => res.status(200).send("ok")); app.get("/health", (_req, res) => res.status(200).send("ok"));
/* ------------------------------------------------------------------ */
/* Request context / logging */
/* ------------------------------------------------------------------ */
const DEFAULT_REQ_TIMEOUT_MS = clampInt(
process.env.REQ_TIMEOUT_MS,
5_000,
10 * 60_000,
120_000
);
const DEFAULT_REQ_TIMEOUT_PDF_MS = clampInt(
process.env.REQ_TIMEOUT_PDF_MS,
10_000,
30 * 60_000,
5 * 60_000
);
app.use((req, res, next) => {
const requestId =
String(req.headers["x-request-id"] || "").trim() || randomUUID();
req.requestId = requestId;
res.setHeader("x-request-id", requestId);
const started = Date.now();
req.setTimeout(DEFAULT_REQ_TIMEOUT_MS);
res.setTimeout(DEFAULT_REQ_TIMEOUT_MS);
res.on("finish", () => {
const ms = Date.now() - started;
const len =
Number(req.headers["content-length"] || 0) ||
(req.body?.length ?? 0) ||
0;
console.log(
JSON.stringify({
requestId,
method: req.method,
path: req.originalUrl,
status: res.statusCode,
bytesIn: len,
ms,
})
);
});
next();
});
function isAborted(req, res) {
return Boolean(req.aborted || res.writableEnded || res.destroyed);
}
function sendError(res, status, code, message, requestId) {
if (res.headersSent) {
try {
res.end();
} catch {}
return;
}
res.status(status).json({ error: code, message, requestId });
}
/* ------------------------------------------------------------------ */
/* Auth */
/* ------------------------------------------------------------------ */
function requireAuth(req, res) {
const token = process.env.CONVERTER_TOKEN;
const auth = req.headers.authorization || "";
if (!token || auth !== `Bearer ${token}`) {
sendError(res, 401, "unauthorized", "Unauthorized", req.requestId);
return false;
}
return true;
}
/* ------------------------------------------------------------------ */
/* Type detection */
/* ------------------------------------------------------------------ */
function isPdfRequest(req) {
const ct = String(req.headers["content-type"] || "").toLowerCase();
const fn = String(req.headers["x-filename"] || "").toLowerCase();
return ct.startsWith("application/pdf") || fn.endsWith(".pdf");
}
function looksLikeHeic(buf) {
if (!buf || buf.length < 16) return false;
if (buf.toString("ascii", 4, 8) !== "ftyp") return false;
const brands = buf.toString("ascii", 8, Math.min(buf.length, 256));
return (
brands.includes("heic") ||
brands.includes("heif") ||
brands.includes("heix") ||
brands.includes("hevc") ||
brands.includes("hevx") ||
brands.includes("mif1") ||
brands.includes("msf1")
);
}
async function assertSupportedRaster(input) {
if (looksLikeHeic(input)) return;
try {
await sharp(input, { failOnError: false }).metadata();
} catch {
throw Object.assign(new Error("Unsupported image input"), {
statusCode: 415,
code: "unsupported_media_type",
});
}
}
/* ------------------------------------------------------------------ */
/* Options */
/* ------------------------------------------------------------------ */
function parseBool(v, fallback = false) {
if (v == null) return fallback;
const s = String(v).toLowerCase().trim();
if (["1", "true", "yes", "y", "on"].includes(s)) return true;
if (["0", "false", "no", "n", "off"].includes(s)) return false;
return fallback;
}
function parseOptions(req) {
return {
quality: clampInt(req.headers["x-jpeg-quality"], 40, 100, 85),
maxDim: clampInt(req.headers["x-max-dimension"], 500, 6000, 2000),
withoutEnlargement: parseBool(req.headers["x-without-enlargement"], true),
pdfDpi: clampInt(req.headers["x-pdf-dpi"], 72, 600, 300),
};
}
/* ------------------------------------------------------------------ */
/* Vision-safe normalization */
/* ------------------------------------------------------------------ */
function normalizeForVision(input, opts) {
const sharpInputOpts = {
failOnError: false,
limitInputPixels: 200e6,
...(opts?.raw ? { raw: opts.raw } : {}),
};
let pipeline = sharp(input, sharpInputOpts).rotate();
// Normalize into sRGB (NOT "rgb"). This avoids:
// vips_colourspace: no known route from 'srgb' to 'rgb'
pipeline = pipeline.toColorspace("srgb");
// If input has alpha, flatten to white so OCR/vision doesn't get weird transparency artifacts.
pipeline = pipeline.flatten({ background: { r: 255, g: 255, b: 255 } });
if (opts.maxDim) {
pipeline = pipeline.resize({
width: opts.maxDim,
height: opts.maxDim,
fit: "inside",
withoutEnlargement: opts.withoutEnlargement,
});
}
return pipeline
.jpeg({
quality: opts.quality,
chromaSubsampling: "4:4:4",
mozjpeg: true,
progressive: true,
})
.withMetadata(false)
.toBuffer();
}
/* ------------------------------------------------------------------ */
/* HEIC via WASM */
/* ------------------------------------------------------------------ */
function heifDisplayToRGBA(img) {
return new Promise((resolve, reject) => {
try {
const w = img.get_width();
const h = img.get_height();
const rgba = new Uint8Array(w * h * 4);
img.display({ data: rgba, width: w, height: h, channels: 4 }, () =>
resolve({ width: w, height: h, rgba })
);
} catch (e) {
reject(e);
}
});
}
async function heicToJpeg(input, opts) {
if (!libheif?.HeifDecoder) throw new Error("libheif-js unavailable");
const dec = new libheif.HeifDecoder();
const imgs = dec.decode(input);
if (!imgs?.length) throw new Error("HEIC decode failed");
const { width, height, rgba } = await heifDisplayToRGBA(imgs[0]);
// Feed Sharp raw pixel metadata so it doesn't treat the buffer as an encoded image.
return normalizeForVision(Buffer.from(rgba), {
...opts,
raw: { width, height, channels: 4 },
});
}
/* ------------------------------------------------------------------ */
/* PDF handling */
/* ------------------------------------------------------------------ */
async function pdfFirstPageToJpeg(input, opts) {
const id = randomUUID();
const pdf = `/tmp/${id}.pdf`;
const out = `/tmp/${id}.jpg`;
try {
await fs.writeFile(pdf, input);
await execFilePromise(
"pdftoppm",
["-jpeg", "-singlefile", "-r", String(opts.pdfDpi), pdf, `/tmp/${id}`],
DEFAULT_REQ_TIMEOUT_PDF_MS
);
const buf = await fs.readFile(out);
return normalizeForVision(buf, opts);
} finally {
await safeUnlink(pdf);
await safeUnlink(out);
}
}
async function pdfAllPagesToJpegs(input, opts) {
const id = randomUUID();
const pdf = `/tmp/${id}.pdf`;
const prefix = `/tmp/${id}`;
try {
await fs.writeFile(pdf, input);
await execFilePromise(
"pdftoppm",
["-jpeg", "-r", String(opts.pdfDpi), pdf, prefix],
DEFAULT_REQ_TIMEOUT_PDF_MS
);
// pdftoppm writes <prefix>-01.jpg, <prefix>-02.jpg, etc.
const dir = await fs.readdir("/tmp");
const pages = dir
.filter((f) => f.startsWith(`${id}-`) && f.endsWith(".jpg"))
.sort();
const results = [];
for (const page of pages) {
const pagePath = `/tmp/${page}`;
const buf = await fs.readFile(pagePath);
const jpeg = await normalizeForVision(buf, opts);
results.push(jpeg);
await safeUnlink(pagePath);
}
return results;
} finally {
await safeUnlink(pdf);
}
}
/* ------------------------------------------------------------------ */
/* Single-flight per machine (ONLY for /convert) */
/* ------------------------------------------------------------------ */
const MAX_CONVERT_INFLIGHT = 1;
let convertInflight = 0;
async function withConvertSingleFlight(req, res, fn) {
if (convertInflight >= MAX_CONVERT_INFLIGHT) {
res.setHeader("Retry-After", "1");
return sendError(res, 429, "busy", "Converter busy; retry shortly", req.requestId);
}
convertInflight++;
try {
return await fn();
} finally {
convertInflight--;
}
}
/* ------------------------------------------------------------------ */
/* Routes */
/* ------------------------------------------------------------------ */
/* ------------------------------------------------------------------ */
/* Web UI */
/* ------------------------------------------------------------------ */
app.get("/", (_req, res) => {
res.setHeader("Content-Type", "text/html");
res.send(`<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>PostConvert</title>
<style>
* { box-sizing: border-box; margin: 0; padding: 0; }
body { font-family: -apple-system, system-ui, sans-serif; background: #111; color: #eee;
display: flex; align-items: flex-start; justify-content: center; min-height: 100vh; padding-top: 4rem; }
.wrap { max-width: 420px; width: 100%; padding: 2rem; }
.drop { border: 2px dashed #444; border-radius: 12px; padding: 3rem 1.5rem; text-align: center;
cursor: pointer; transition: border-color .2s; }
.drop.over { border-color: #4a9eff; }
.drop p { color: #888; font-size: .95rem; }
.drop input { display: none; }
.status { margin-top: 1rem; text-align: center; font-size: .9rem; color: #888; }
.status.error { color: #f66; }
.removed { margin-top: 1.2rem; background: #1a1a1a; border: 1px solid #333; border-radius: 10px;
padding: 1rem 1.2rem; font-size: .85rem; }
.removed h3 { font-size: .8rem; text-transform: uppercase; letter-spacing: .05em; color: #666;
margin-bottom: .6rem; }
.removed table { width: 100%; border-collapse: collapse; }
.removed td { padding: .25rem 0; vertical-align: top; }
.removed td:first-child { color: #888; padding-right: .8rem; white-space: nowrap; }
.removed td:last-child { color: #eee; word-break: break-word; }
.removed .warn { color: #f90; }
.removed .sep td { padding: 0; height: 1px; }
.removed .sep hr { border: none; border-top: 1px solid #333; margin: .5rem 0; }
.overlay { position: fixed; inset: 0; background: #111; display: flex;
align-items: center; justify-content: center; z-index: 10; }
.auth { max-width: 340px; width: 100%; padding: 2rem; text-align: center; }
.auth h2 { font-size: 1.1rem; margin-bottom: 1rem; }
.auth input { width: 100%; padding: .6rem .8rem; border: 1px solid #444; border-radius: 8px;
background: #222; color: #eee; font-size: .95rem; margin-bottom: .8rem; }
.auth input:focus { outline: none; border-color: #4a9eff; }
.auth button { width: 100%; padding: .6rem; border: none; border-radius: 8px;
background: #4a9eff; color: #fff; font-size: .95rem; cursor: pointer; }
.auth button:hover { background: #3a8eef; }
.auth .err { color: #f66; font-size: .85rem; margin-top: .5rem; }
</style>
</head>
<body>
<div class="overlay" id="auth-overlay" style="display:none">
<div class="auth">
<h2>Enter access token</h2>
<input type="password" id="token-input" placeholder="Token" autocomplete="off">
<button id="token-submit">Continue</button>
<div class="err" id="token-err"></div>
</div>
</div>
<div class="wrap" id="main" style="display:none">
<div class="drop" id="drop">
<p>Drop a pdf/image here to tap to choose</p>
<input type="file" id="file" accept="image/*,application/pdf,.pdf">
</div>
<div class="status" id="status"></div>
<div class="removed" id="removed" style="display:none">
<h3>Metadata Removed</h3>
<table><tbody id="removed-body"></tbody></table>
</div>
</div>
<script>
(function() {
const KEY = "postconvert_token";
const overlay = document.getElementById("auth-overlay");
const main = document.getElementById("main");
const tokenInput = document.getElementById("token-input");
const tokenSubmit = document.getElementById("token-submit");
const tokenErr = document.getElementById("token-err");
const drop = document.getElementById("drop");
const fileInput = document.getElementById("file");
const status = document.getElementById("status");
const removedDiv = document.getElementById("removed");
const removedBody = document.getElementById("removed-body");
function showApp() { overlay.style.display = "none"; main.style.display = ""; }
function showAuth() { overlay.style.display = ""; main.style.display = "none"; tokenErr.textContent = ""; }
if (localStorage.getItem(KEY)) {
showApp();
} else {
showAuth();
}
tokenSubmit.addEventListener("click", () => {
const t = tokenInput.value.trim();
if (!t) { tokenErr.textContent = "Token required"; return; }
localStorage.setItem(KEY, t);
showApp();
});
tokenInput.addEventListener("keydown", (e) => {
if (e.key === "Enter") tokenSubmit.click();
});
function getToken() { return localStorage.getItem(KEY); }
function isPdf(file) {
return file.type === "application/pdf" || (file.name || "").toLowerCase().endsWith(".pdf");
}
function downloadBase64Jpeg(b64, name) {
const byteString = atob(b64);
const bytes = new Uint8Array(byteString.length);
for (let i = 0; i < byteString.length; i++) bytes[i] = byteString.charCodeAt(i);
const blob = new Blob([bytes], { type: "image/jpeg" });
const url = URL.createObjectURL(blob);
const a = document.createElement("a");
a.href = url;
a.download = name;
a.click();
URL.revokeObjectURL(url);
}
async function handleFile(file) {
const token = getToken();
if (!token) { showAuth(); return; }
status.className = "status";
removedDiv.style.display = "none";
removedBody.innerHTML = "";
if (isPdf(file)) {
await convertPdf(file, token);
} else {
await stripImage(file, token);
}
}
async function convertPdf(file, token) {
status.textContent = "Converting PDF…";
try {
const res = await fetch("/convert/pdf", {
method: "POST",
headers: {
"Authorization": "Bearer " + token,
"Content-Type": "application/pdf",
},
body: file,
});
if (res.status === 401) {
localStorage.removeItem(KEY);
showAuth();
tokenErr.textContent = "Invalid token. Please try again.";
return;
}
if (!res.ok) {
const err = await res.json().catch(() => ({}));
throw new Error(err.message || "PDF conversion failed");
}
const data = await res.json();
const baseName = (file.name || "document").replace(/\\.[^.]+$/, "");
for (let i = 0; i < data.pages.length; i++) {
const name = data.pages.length === 1
? baseName + ".jpg"
: baseName + "_page" + (i + 1) + ".jpg";
downloadBase64Jpeg(data.pages[i], name);
}
status.className = "status";
status.textContent = "Done — saved " + data.pages.length + " page" + (data.pages.length === 1 ? "" : "s");
} catch (e) {
status.className = "status error";
status.textContent = e.message;
}
}
async function stripImage(file, token) {
status.textContent = "Stripping metadata…";
try {
const res = await fetch("/strip", {
method: "POST",
headers: {
"Authorization": "Bearer " + token,
"Content-Type": file.type || "image/jpeg",
"Accept": "application/json",
},
body: file,
});
if (res.status === 401) {
localStorage.removeItem(KEY);
showAuth();
tokenErr.textContent = "Invalid token. Please try again.";
return;
}
if (!res.ok) {
const err = await res.json().catch(() => ({}));
throw new Error(err.message || "Strip failed");
}
const data = await res.json();
// Show removed metadata
const meta = data.metadata;
if (meta) {
const keys = Object.keys(meta);
if (keys.length > 0) {
keys.forEach(function(k) {
if (k === "Settings") {
const sep = document.createElement("tr");
sep.className = "sep";
const sc = document.createElement("td");
sc.colSpan = 2;
sc.innerHTML = "<hr>";
sep.appendChild(sc);
removedBody.appendChild(sep);
}
const tr = document.createElement("tr");
const td1 = document.createElement("td");
td1.textContent = k;
const td2 = document.createElement("td");
td2.textContent = meta[k];
if (k === "GPS Location" || k === "GPS Coordinates") td2.className = "warn";
tr.appendChild(td1);
tr.appendChild(td2);
removedBody.appendChild(tr);
});
removedDiv.style.display = "";
}
}
const name = (file.name || "photo").replace(/\\.[^.]+$/, "") + "_clean.jpg";
downloadBase64Jpeg(data.image, name);
status.className = "status";
status.textContent = "Done — saved " + name;
} catch (e) {
status.className = "status error";
status.textContent = e.message;
}
}
drop.addEventListener("click", () => fileInput.click());
fileInput.addEventListener("change", () => { if (fileInput.files[0]) handleFile(fileInput.files[0]); });
drop.addEventListener("dragover", (e) => { e.preventDefault(); drop.classList.add("over"); });
drop.addEventListener("dragleave", () => drop.classList.remove("over"));
drop.addEventListener("drop", (e) => {
e.preventDefault();
drop.classList.remove("over");
if (e.dataTransfer.files[0]) handleFile(e.dataTransfer.files[0]);
});
})();
</script>
</body>
</html>`);
});
async function extractMetadata(buf) {
const items = [];
try {
const meta = await sharp(buf, { failOnError: false }).metadata();
if (meta.exif) {
try {
const exif = exifReader(meta.exif);
// Scary / identifiable stuff first
if (exif.GPSInfo?.GPSLatitude && exif.GPSInfo?.GPSLongitude) {
const lat = exif.GPSInfo.GPSLatitude;
const lng = exif.GPSInfo.GPSLongitude;
const latRef = exif.GPSInfo.GPSLatitudeRef || "N";
const lngRef = exif.GPSInfo.GPSLongitudeRef || "E";
const latVal = typeof lat === "number" ? lat : (lat[0] + lat[1] / 60 + lat[2] / 3600);
const lngVal = typeof lng === "number" ? lng : (lng[0] + lng[1] / 60 + lng[2] / 3600);
const latFinal = latRef === "S" ? -latVal : latVal;
const lngFinal = lngRef === "W" ? -lngVal : lngVal;
let loc = `${latFinal.toFixed(4)}, ${lngFinal.toFixed(4)}`;
try {
const geoRes = await fetch(`https://nominatim.openstreetmap.org/reverse?lat=${latFinal}&lon=${lngFinal}&format=json&zoom=14`, {
headers: { "User-Agent": "PostConvert/1.0" },
signal: AbortSignal.timeout(3000),
});
if (geoRes.ok) {
const geo = await geoRes.json();
if (geo.display_name) loc = geo.display_name;
}
} catch {}
items.push(["GPS Location", loc]);
items.push(["GPS Coordinates", `${latFinal.toFixed(6)}, ${lngFinal.toFixed(6)}`]);
}
if (exif.Photo?.DateTimeOriginal) {
const d = exif.Photo.DateTimeOriginal;
items.push(["Date Taken", d instanceof Date ? d.toISOString().slice(0, 19).replace("T", " ") : String(d)]);
}
if (exif.Image?.Make && exif.Image?.Model) {
items.push(["Camera", `${exif.Image.Make} ${exif.Image.Model}`]);
} else if (exif.Image?.Model) {
items.push(["Camera", String(exif.Image.Model)]);
}
if (exif.Image?.HostComputer) {
let device = String(exif.Image.HostComputer);
if (exif.Image?.Software) device += `, iOS ${exif.Image.Software}`;
items.push(["Device", device]);
}
if (exif.Photo?.LensModel) items.push(["Lens", String(exif.Photo.LensModel)]);
if (exif.Photo?.ImageUniqueID) items.push(["Unique ID", String(exif.Photo.ImageUniqueID)]);
// Camera settings
const settings = [];
if (exif.Photo?.FNumber) settings.push(`f/${Math.round(exif.Photo.FNumber * 100) / 100}`);
if (exif.Photo?.ExposureTime) settings.push(`1/${Math.round(1 / exif.Photo.ExposureTime)}s`);
if (exif.Photo?.ISOSpeedRatings) settings.push(`ISO ${exif.Photo.ISOSpeedRatings}`);
if (exif.Photo?.FocalLength) settings.push(`${Math.round(exif.Photo.FocalLength * 100) / 100}mm`);
if (settings.length) items.push(["Settings", settings.join(" ")]);
} catch {}
}
// Technical details at the bottom
if (meta.width && meta.height) items.push(["Dimensions", `${meta.width} × ${meta.height}`]);
if (meta.hasProfile && meta.icc) items.push(["ICC Profile", `${(meta.icc.length / 1024).toFixed(1)} KB`]);
if (meta.exif) items.push(["EXIF Data", `${(meta.exif.length / 1024).toFixed(1)} KB`]);
if (meta.xmp) items.push(["XMP Data", `${(meta.xmp.length / 1024).toFixed(1)} KB`]);
if (meta.iptc) items.push(["IPTC Data", `${(meta.iptc.length / 1024).toFixed(1)} KB`]);
} catch {}
// Convert to ordered object
const removed = {};
for (const [k, v] of items) removed[k] = v;
return removed;
}
app.post("/strip", async (req, res) => {
res.setHeader("Connection", "close");
return withConvertSingleFlight(req, res, async () => {
try {
if (!requireAuth(req, res)) return;
if (!req.body?.length) {
return sendError(res, 400, "empty_body", "Empty body", req.requestId);
}
const opts = {
quality: clampInt(req.headers["x-jpeg-quality"], 40, 100, 92),
maxDim: 0,
withoutEnlargement: true,
};
// Extract metadata before stripping
const removed = await extractMetadata(req.body);
let jpeg;
if (looksLikeHeic(req.body)) {
if (isAborted(req, res)) return;
jpeg = await heicToJpeg(req.body, opts);
} else {
await assertSupportedRaster(req.body);
if (isAborted(req, res)) return;
jpeg = await normalizeForVision(req.body, opts);
}
if (isAborted(req, res)) return;
// If client wants JSON, return image + metadata together
const wantsJson = String(req.headers["accept"] || "").includes("application/json");
if (wantsJson) {
res.setHeader("Content-Type", "application/json");
return res.json({
image: jpeg.toString("base64"),
metadata: removed,
});
}
res.setHeader("Content-Type", "image/jpeg");
return res.send(jpeg);
} catch (e) {
const status = e?.statusCode || 500;
const code = e?.code || "strip_failed";
console.error(JSON.stringify({ requestId: req.requestId, err: String(e?.stack || e) }));
return sendError(
res,
status,
code,
status === 415 ? "Unsupported media type" : "Metadata strip failed",
req.requestId
);
}
});
});
app.post("/convert", async (req, res) => { app.post("/convert", async (req, res) => {
res.setHeader("Connection", "close"); try {
const token = process.env.CONVERTER_TOKEN;
const auth = req.headers.authorization || "";
return withConvertSingleFlight(req, res, async () => { if (!token || auth !== `Bearer ${token}`) {
try { return res.status(401).send("Unauthorized");
if (!requireAuth(req, res)) return;
if (!req.body?.length) {
return sendError(res, 400, "empty_body", "Empty body", req.requestId);
}
const opts = parseOptions(req);
if (isPdfRequest(req)) {
if (isAborted(req, res)) return;
const jpeg = await pdfFirstPageToJpeg(req.body, opts);
if (isAborted(req, res)) return;
res.setHeader("Content-Type", "image/jpeg");
return res.send(jpeg);
}
if (looksLikeHeic(req.body)) {
if (isAborted(req, res)) return;
const jpeg = await heicToJpeg(req.body, opts);
if (isAborted(req, res)) return;
res.setHeader("Content-Type", "image/jpeg");
return res.send(jpeg);
}
await assertSupportedRaster(req.body);
if (isAborted(req, res)) return;
const jpeg = await normalizeForVision(req.body, opts);
if (isAborted(req, res)) return;
res.setHeader("Content-Type", "image/jpeg");
return res.send(jpeg);
} catch (e) {
const status = e?.statusCode || 500;
const code = e?.code || "conversion_failed";
console.error(JSON.stringify({ requestId: req.requestId, err: String(e?.stack || e) }));
return sendError(
res,
status,
code,
status === 415 ? "Unsupported media type" : "Conversion failed",
req.requestId
);
} }
});
const input = req.body; // Buffer
if (!input || input.length === 0) {
return res.status(400).send("Empty body");
}
const jpeg = await sharp(input, { failOnError: false })
.rotate() // respect EXIF orientation
.jpeg({ quality: 85 })
.toBuffer();
res.setHeader("Content-Type", "image/jpeg");
return res.status(200).send(jpeg);
} catch (e) {
return res.status(500).send(String(e?.stack || e));
}
}); });
app.post("/convert/pdf", async (req, res) => { // Express will throw for oversized bodies; return a clean 413
res.setHeader("Connection", "close"); app.use((err, _req, res, next) => {
req.setTimeout(DEFAULT_REQ_TIMEOUT_PDF_MS); if (err?.type === "entity.too.large") {
res.setTimeout(DEFAULT_REQ_TIMEOUT_PDF_MS); return res.status(413).send("Payload too large (max 30mb)");
}
return withConvertSingleFlight(req, res, async () => { return next(err);
try {
if (!requireAuth(req, res)) return;
if (!req.body?.length) {
return sendError(res, 400, "empty_body", "Empty body", req.requestId);
}
if (!isPdfRequest(req)) {
return sendError(res, 415, "unsupported_media_type", "Expected a PDF", req.requestId);
}
const opts = parseOptions(req);
if (isAborted(req, res)) return;
const jpegs = await pdfAllPagesToJpegs(req.body, opts);
if (isAborted(req, res)) return;
res.setHeader("Content-Type", "application/json");
return res.json({
pages: jpegs.map((buf) => buf.toString("base64")),
});
} catch (e) {
const status = e?.statusCode || 500;
const code = e?.code || "conversion_failed";
console.error(JSON.stringify({ requestId: req.requestId, err: String(e?.stack || e) }));
return sendError(
res,
status,
code,
status === 415 ? "Unsupported media type" : "PDF conversion failed",
req.requestId
);
}
});
}); });
/* ------------------------------------------------------------------ */
/* Helpers */
/* ------------------------------------------------------------------ */
function execFilePromise(cmd, args, timeoutMs) {
return new Promise((resolve, reject) => {
execFile(cmd, args, { timeout: timeoutMs }, (err, _stdout, stderr) => {
if (err) {
if (err.code === "ENOENT") return reject(new Error(`Missing dependency: ${cmd}`));
if (err.killed || err.signal === "SIGTERM") {
return reject(new Error(`${cmd} timed out after ${timeoutMs}ms`));
}
return reject(new Error(stderr || String(err)));
}
resolve();
});
});
}
function clampInt(v, min, max, fallback) {
const n = Number(v);
if (!Number.isFinite(n)) return fallback;
return Math.max(min, Math.min(max, Math.floor(n)));
}
async function safeUnlink(p) {
try { await fs.unlink(p); } catch {}
}
/* ------------------------------------------------------------------ */
const port = Number(process.env.PORT) || 8080; const port = Number(process.env.PORT) || 8080;
const server = app.listen(port, "0.0.0.0", () =>
console.log(`converter listening on :${port}`)
);
server.keepAliveTimeout = 5_000; // IMPORTANT for Fly.io: bind to 0.0.0.0 (not localhost)
server.headersTimeout = 10_000; app.listen(port, "0.0.0.0", () => {
console.log(`converter listening on ${port}`);
});