Skip to content

t1k:marketing:analytics:crawldata

FieldValue
Moduleanalytics
Version1.21.13
Effortmedium
Tools—
/t1k:marketing:analytics:crawldata

Pull ad-spend from Mintegral, Google Ads, and Unity Ads, pull ad-revenue from Adjust (primary), AppLovin MAX, CloudX, and AdMob, and merge everything per app package name into one Profit/ROAS report: a Title (human-readable project name), a column per spend network, a spend Total, Ad Revenue (Adjust drives Profit/ROAS), a column per other revenue source, Profit (revenue − spend), and ROAS (revenue ÷ spend). Output is sorted by package; Title / Profit / ROAS are derived at output time, never stored, and new apps map automatically (Title from Adjust’s app dimension). For the operator’s daily deliverable — a Net-Revenue report that uses AppLovin (MAX) for chosen apps, MAX + Adjust non-MAX for maxPlusAdjustNonMax titles, and Adjust for the rest — see Net-Revenue daily report.

The pull-and-reconcile work delegates to cheap-tier agents; only the interpretation stays on a reasoning tier (routing SSOT: rules/orchestration-rules.md § “Task-Type → Agent Routing”):

StepAgent
Pull one named vendor report for stated dates and report the figurest1k-fact-fetcher
Reconcile spend/revenue across two or more sources against a stated tolerancet1k-data-verifier
Aggregate a single source into the per-package tablet1k-metrics-reporter
Explain WHY a package’s spend or revenue looks wrongt1k-debugger — reasoning tier, not cheap

Profit and ROAS are derived at output time from columns already returned, so this is cheap-tier work. Once the question turns from do these sources agree? to why don’t they?, it changes tier and goes to a different agent.

  • Pulling a day’s or a date range’s ad spend, combined per package
  • Producing a merged spend-vs-revenue report with Profit / ROAS columns
  • Standing up the daily scheduled Excel export
  • Producing the Net-Revenue daily report (AppLovin chosen side)
  • Adding a new ad network / revenue source, or per-app revenue overrides
  • Computing CloudX First Look P90 CPM bid floors from MAX report data
  • Investigating why a package’s spend or revenue looks wrong

Ten live connectors. China is excluded from every revenue source; spend sources are all-countries.

SourceKindOne-liner
mintegralspendAdvertiser Performance Reporting API; joins by package_name. Multi-account: extra advertiser accounts via MINTEGRAL_<n>_ACCESS_KEY/_API_KEY (n = 2–9), all summed into the one Mintegral column (same shape as Unity’s multi-org). A half-configured extra logs a skip warning instead of silently under-reporting.
googleadsspendGAQL via the Google Ads API; joins each App campaign’s app_id → package.
unityspendAdvertising Statistics API v2 acquisitions report (Service Account Basic auth); Management API maps Unity app id → store package; UTC day buckets, 429 backoff. Multi-org: extra accounts via UNITY_ADS_<n>_* (n = 2–9), all merged into the one unity column. Unified-Platform-migrated org (403): the app list falls back to the Unified /applications endpoint; with no rows from the legacy stats feed, spend is unknown, never $0; see Gotchas.
applovin-uaspendAppLovin advertiser (AppDiscovery UA) report, sharing APPLOVIN_REPORT_KEY. Buying is largely stopped (cents/day); if it restarts, do not trust a day’s ROAS until its spend is accounted for. Window, figures and endpoint notes: references/applovin-ua-retirement.md.
adjustrevenue (primary)Datascape Report Service; drives Profit/ROAS; supplies the Title. Aggregates every network, CloudX included — nothing is added on top.
adjust-s2srevenue (S2S source)Same Report Service, scoped to the S2S ad-revenue source via the ad_revenue_sources=S2S Ad Revenue filter (a filter, not a dimension); emits an adjust-s2s column. Reconciliation against cloudx only — the Net-Revenue report no longer consumes it (the S2S swap is retired). Reuses ADJUST_API_TOKEN.
adjust-nonmaxrevenue (non-MAX sources)Same Report Service as adjust, every ad_revenue_source except the MAX ones (keep-list derived at runtime; fails loudly if a MAX id is missing). Feeds Net Revenue for maxPlusAdjustNonMax titles only. Reuses ADJUST_API_TOKEN.
applovinrevenue (reference)MAX Ad Revenue Report; estimated revenue re-bucketed to UTC+8; own column, does not change Profit/ROAS. Chosen side for appLovinPreferred apps in the Net-Revenue report.
cloudxrevenue (reference)CloudX per-app report; joins by app_id (= package), one UTC+8-day window. Already inside Adjust — crawled for reconciliation against adjust-s2s, never added on top.
admobrevenue (reference)AdMob Reporting API (OAuth2 admob.readonly); publisher ESTIMATED_EARNINGS by app, apps.list maps app id → store package; own column, does not change Profit/ROAS.

Per-app overrides and exclusions (tool/report-overrides.json: appLovinPreferred, maxPlusAdjustNonMax, excludedPackages). Every generator that reads per-package rows drops excluded packages through tool/lib/report-overrides.mjs; a generator that re-implements the rule is a defect. Gate: bun tool/test-report-exclusions.mjs. Details: references/report-overrides.md.

Clone the companion repo (below); the crawler lives at .claude/skills/t1k-marketing-crawldata/tool/ (the repo-root tool/ holds only ad-hoc scripts, not the crawler). From that directory:

Terminal window
cd .claude/skills/t1k-marketing-crawldata/tool
bun install # first run only
bun run pull -- --yesterday --xlsx ../data/spend.xlsx # merged daily export
bun run pull -- --date 2026-06-10 --source cloudx # a single source

A scheduler (launchd / cron) can run the merged daily export automatically. All flags, output formats, the daily-export setup, and per-app revenue overrides (--html + tool/report-overrides.json) are documented in the companion repo.

Net-Revenue daily report (AppLovin chosen side)

Section titled “Net-Revenue daily report (AppLovin chosen side)”

The operator’s daily deliverable, built by tool/gen-daily-report.mjs (in the companion repo) on top of a --pivot crawl CSV — interactive HTML + Excel. It sets Net Revenue per title, three classes (rule history and the why: references/net-revenue-history.md):

MarkerClass (report-overrides.json)Net Revenue
♦appLovinPreferredAppLovin (MAX), Adjust replaced
◆+maxPlusAdjustNonMaxMAX + adjust-nonmax; Rev AppLovin shows MAX
(none)everyone else, and any title with MAX = 0Adjust (CloudX is already inside it)

Reconcile: Net = Adjust − Σ♦ Adjust + Σ♦ MAX − Σ◆+ Adjust + Σ◆+ (MAX + adjust-nonmax), the ◆+ term only where MAX > 0. A spend-<date>.csv without the adjust-nonmax column counts as MAX + 0 for ◆+ titles, with one [net-basis] warning per run; do not re-crawl saved days to fill it. The rule is netRevenue() in the companion repo’s tool/lib/report-overrides.mjs; its TypeScript and Apps Script mirrors change with it (tool/test-report-net-basis.mjs).

The S2S swap is retired; adjust-s2s is kept for reconciliation only. History: references/net-revenue-history.md.

Build the daily report from a --pivot crawl: bun gen-daily-report.mjs ../data <date>. Commands and the 10-column layout: references/report-commands.md.

14-day sparklines — REUSE saved data, never re-crawl

Section titled “14-day sparklines — REUSE saved data, never re-crawl”

The sparklines reuse saved spend-<date>.csv files; each day is crawled once and old days are never re-crawled. Detail: references/report-commands.md.

Per-title trend report — gen-trend-report.mjs

Section titled “Per-title trend report — gen-trend-report.mjs”

gen-trend-report.mjs <dir> <endDate> [days=7] builds the trend HTML from saved files (0 crawls). Detail: references/report-commands.md.

Discord delivery — notify-discord-daily.mjs

Section titled “Discord delivery — notify-discord-daily.mjs”

notify-discord-daily.mjs posts the headline and attaches both files; a missing webhook exits 0. Detail: references/report-commands.md.

  • Adjust under-records MAX revenue on some builds — ◆+ titles use MAX + adjust-nonmax. Measured ratios and the window: references/adjust-applovin-max-api-vs-sdk.md.

  • Adjust’s “AppLovin Max” API source is empty and never backfills, so Adjust never self-corrects. Status: references/adjust-applovin-max-api-vs-sdk.md.

  • MAX’s Meta figure lags 1–2 days on the newest day. Do not read the newest 1–2 days of a MAX pull as final. Evidence: references/net-revenue-history.md.

  • AdMob networkReport ESTIMATED_EARNINGS is AdMob-network-only and does not over-report. The double count runs the other way: AdMob bidding into MAX is already on MAX’s ADMOB_BIDDING line. Split with mediationReport:generate + MEDIATION_GROUP: "AdMob (default)" is never summed on top of MAX; a named group is safe to sum. Pass localizationSettings: { currencyCode: 'USD' }, never timeZone='America/Los_Angeles'. Details: references/hard-rule-details.md.

  • A monetization partner can also sell traffic; that cost side is easy to miss. Crawl the partner’s spend as well as its revenue. A $0 outside its buying window is real, not a gap. references/applovin-ua-retirement.md.

  • A missing spend COLUMN does not mean missing spend. total is accumulated from crawled rows in pivot.ts, not summed from displayed columns. Check where a total comes from before predicting what removing a column does. references/applovin-ua-retirement.md.

  • Adjust carries AppLovin MAX twice: exclude the API side (AppLovin Max), never the SDK side (Applovin MAX SDK). The API side is missing most days and never backfills. Measured split: references/adjust-applovin-max-api-vs-sdk.md.

  • ad_revenue_sources__not_in is silently ignored — the reports-service has no exclusion filter, so an exclusion must be expressed as an INCLUDE of every other source on the bare ad_revenue_sources key. Derive that keep-list at runtime from filters_data so a newly-added source is kept rather than silently dropped.

  • Mintegral silently drops out of pulls longer than 8 days. A --start/--end range >8 days returns rows with no Mintegral spend (its per-request cap), so the total is silently understated. Pull in ≤8-day chunks, or rely on the per-day reuse model (single-day files always include Mintegral).

  • MAX (AppLovin) ≈ 92–94% of total ad revenue; CloudX ≈ 4–5% and growing. MAX mediation is the dominant monetization; CloudX is a small S2S contributor.

  • A Unity Service Account missing Advertise API Viewer fails silently: the apps call returns 200 with zero results, every app lands under unknown, and the day total still looks right. Assert total > 0 on a new org. Details: references/hard-rule-details.md.

  • A Unity org migrated to the Unified Platform returns 403, not an empty list - a different failure from the missing role above; the connector falls back to the Unified /applications list, and unity: FAILED is reserved for every configured account failing. Triage and fix: references/unity-unified-platform-403.md.

  • Unresolved rows reconcile in three passes: exact title, then normalized title, then package slug. Every pass needs a UNIQUE match, so Android/iOS siblings are skipped rather than merged onto the wrong platform. An app whose bundle id and title state no name still strands: fix the role, not the title. Details: references/hard-rule-details.md.

  • Unity’s daily quota dies after a few full crawls, and a failed source is non-fatal. Two or three full crawls in one day exhaust it; later days show Unity at 0. Read per-source row counts in crawl.log before trusting a re-crawl. Details: references/hard-rule-details.md.

  • An unconfigured buying account looks like an app with no spend. When a title’s ROAS looks implausibly good, check for a missing buying account before believing it. Details: references/hard-rule-details.md.

Full setup, per-connector API reference & the runnable crawler

Section titled “Full setup, per-connector API reference & the runnable crawler”

This skill is a discoverable playbook — the code, the per-connector setup docs, and the credential reference are owned by the companion repo (single source of truth):

The1Studio/marketing-crawl-ads-spend https://github.com/The1Studio/marketing-crawl-ads-spend

Clone that repo to run the crawl — it is self-sufficient (crawler code + setup docs + .env.example). Everything lives under .claude/skills/t1k-marketing-crawldata/:

The companion repo’s setup docs, credential reference, crawler and .env.example are the reference set; file-by-file list: references/report-commands.md.

Credentials live ONLY in a gitignored credentials/.env in that repo — never in this kit, never on the command line.

CloudX First Look bid-floor calculation (P90)

Section titled “CloudX First Look bid-floor calculation (P90)”

Distinct from the spend/revenue crawl: compute CloudX first-look price floors as the impression-weighted P90 eCPM per app × format × geo from AppLovin MAX report data. The MAX Reporting API is Mediation Partner Reporting, so the weighted empirical-CDF method is mandatory (a naive percentile(ecpm,90) inflates floors). Output is a results table + multi-tab .xlsx. Read-only — nothing is pushed to MAX; floors are recommendations.

Unlike the crawler (which lives in the companion repo), the First Look floor scripts are bundled with this skill — self-contained, zero-dependency (Node 26+ built-ins only) scripts under scripts/cloudx-first-look/ that read a local .env (MAX_REPORT_KEY=, same value as APPLOVIN_REPORT_KEY):

Commands: references/report-commands.md.

Full methodology, per-app geo buckets, uplift tiers, and gotchas: references/cloudx-first-look-floors.md.

  • This skill is read-only (reporting). Any mutating ad-network call moves real money — see rules/ad-network-mcp-write-safety.md before any write.
  • Never print or commit API keys, tokens, or account ids. Document env-var names only; real values live solely in the companion repo’s gitignored credentials/.env (or, for the bundled First Look scripts, a local .env).
  • iOS apps come back as store ids (e.g. id6751266232) in some networks’ package fields — that is expected, not a bug.

Operator playbook + connector overview. The crawler, per-connector setup docs, and credential reference are maintained in The1Studio/marketing-crawl-ads-spend (single source of truth); deepen this playbook iteratively via /t1k:sync-back.