t1k:marketing:analytics:crawldata
| Field | Value |
|---|---|
| Module | analytics |
| Version | 1.21.13 |
| Effort | medium |
| Tools | — |
How to invoke
Section titled “How to invoke”/t1k:marketing:analytics:crawldataAd Spend vs Revenue Crawl (Profit / ROAS)
Section titled “Ad Spend vs Revenue Crawl (Profit / ROAS)”Pull ad-spend from Mintegral, Google Ads, and Unity Ads, pull
ad-revenue from Adjust (primary), AppLovin MAX, CloudX, and
AdMob, and merge everything per app package name into one Profit/ROAS
report: a Title (human-readable project name), a column per spend network,
a spend Total, Ad Revenue (Adjust drives Profit/ROAS), a column per other
revenue source, Profit (revenue − spend), and ROAS (revenue ÷ spend).
Output is sorted by package; Title / Profit / ROAS are derived at output time,
never stored, and new apps map automatically (Title from Adjust’s app
dimension). For the operator’s daily deliverable — a Net-Revenue report that uses
AppLovin (MAX) for chosen apps, MAX + Adjust non-MAX for maxPlusAdjustNonMax titles,
and Adjust for the rest — see
Net-Revenue daily report.
Executed by
Section titled “Executed by”The pull-and-reconcile work delegates to cheap-tier agents; only the interpretation
stays on a reasoning tier (routing SSOT: rules/orchestration-rules.md
§ “Task-Type → Agent Routing”):
| Step | Agent |
|---|---|
| Pull one named vendor report for stated dates and report the figures | t1k-fact-fetcher |
| Reconcile spend/revenue across two or more sources against a stated tolerance | t1k-data-verifier |
| Aggregate a single source into the per-package table | t1k-metrics-reporter |
| Explain WHY a package’s spend or revenue looks wrong | t1k-debugger — reasoning tier, not cheap |
Profit and ROAS are derived at output time from columns already returned, so this is cheap-tier work. Once the question turns from do these sources agree? to why don’t they?, it changes tier and goes to a different agent.
When to Use
Section titled “When to Use”- Pulling a day’s or a date range’s ad spend, combined per package
- Producing a merged spend-vs-revenue report with Profit / ROAS columns
- Standing up the daily scheduled Excel export
- Producing the Net-Revenue daily report (AppLovin chosen side)
- Adding a new ad network / revenue source, or per-app revenue overrides
- Computing CloudX First Look P90 CPM bid floors from MAX report data
- Investigating why a package’s spend or revenue looks wrong
Connectors at a glance
Section titled “Connectors at a glance”Ten live connectors. China is excluded from every revenue source; spend sources are all-countries.
| Source | Kind | One-liner |
|---|---|---|
mintegral | spend | Advertiser Performance Reporting API; joins by package_name. Multi-account: extra advertiser accounts via MINTEGRAL_<n>_ACCESS_KEY/_API_KEY (n = 2–9), all summed into the one Mintegral column (same shape as Unity’s multi-org). A half-configured extra logs a skip warning instead of silently under-reporting. |
googleads | spend | GAQL via the Google Ads API; joins each App campaign’s app_id → package. |
unity | spend | Advertising Statistics API v2 acquisitions report (Service Account Basic auth); Management API maps Unity app id → store package; UTC day buckets, 429 backoff. Multi-org: extra accounts via UNITY_ADS_<n>_* (n = 2–9), all merged into the one unity column. Unified-Platform-migrated org (403): the app list falls back to the Unified /applications endpoint; with no rows from the legacy stats feed, spend is unknown, never $0; see Gotchas. |
applovin-ua | spend | AppLovin advertiser (AppDiscovery UA) report, sharing APPLOVIN_REPORT_KEY. Buying is largely stopped (cents/day); if it restarts, do not trust a day’s ROAS until its spend is accounted for. Window, figures and endpoint notes: references/applovin-ua-retirement.md. |
adjust | revenue (primary) | Datascape Report Service; drives Profit/ROAS; supplies the Title. Aggregates every network, CloudX included — nothing is added on top. |
adjust-s2s | revenue (S2S source) | Same Report Service, scoped to the S2S ad-revenue source via the ad_revenue_sources=S2S Ad Revenue filter (a filter, not a dimension); emits an adjust-s2s column. Reconciliation against cloudx only — the Net-Revenue report no longer consumes it (the S2S swap is retired). Reuses ADJUST_API_TOKEN. |
adjust-nonmax | revenue (non-MAX sources) | Same Report Service as adjust, every ad_revenue_source except the MAX ones (keep-list derived at runtime; fails loudly if a MAX id is missing). Feeds Net Revenue for maxPlusAdjustNonMax titles only. Reuses ADJUST_API_TOKEN. |
applovin | revenue (reference) | MAX Ad Revenue Report; estimated revenue re-bucketed to UTC+8; own column, does not change Profit/ROAS. Chosen side for appLovinPreferred apps in the Net-Revenue report. |
cloudx | revenue (reference) | CloudX per-app report; joins by app_id (= package), one UTC+8-day window. Already inside Adjust — crawled for reconciliation against adjust-s2s, never added on top. |
admob | revenue (reference) | AdMob Reporting API (OAuth2 admob.readonly); publisher ESTIMATED_EARNINGS by app, apps.list maps app id → store package; own column, does not change Profit/ROAS. |
Per-app overrides and exclusions (tool/report-overrides.json: appLovinPreferred, maxPlusAdjustNonMax, excludedPackages). Every generator that reads per-package rows drops excluded packages through tool/lib/report-overrides.mjs; a generator that re-implements the rule is a defect. Gate: bun tool/test-report-exclusions.mjs. Details: references/report-overrides.md.
Running a crawl
Section titled “Running a crawl”Clone the companion repo (below); the crawler lives at
.claude/skills/t1k-marketing-crawldata/tool/ (the repo-root tool/ holds only
ad-hoc scripts, not the crawler). From that directory:
cd .claude/skills/t1k-marketing-crawldata/toolbun install # first run onlybun run pull -- --yesterday --xlsx ../data/spend.xlsx # merged daily exportbun run pull -- --date 2026-06-10 --source cloudx # a single sourceA scheduler (launchd / cron) can run the merged daily export automatically. All
flags, output formats, the daily-export setup, and per-app revenue overrides
(--html + tool/report-overrides.json) are documented in the companion repo.
Net-Revenue daily report (AppLovin chosen side)
Section titled “Net-Revenue daily report (AppLovin chosen side)”The operator’s daily deliverable, built by tool/gen-daily-report.mjs (in the
companion repo) on top of a --pivot crawl CSV — interactive HTML + Excel. It sets
Net Revenue per title, three classes (rule history and the why:
references/net-revenue-history.md):
| Marker | Class (report-overrides.json) | Net Revenue |
|---|---|---|
| ♦ | appLovinPreferred | AppLovin (MAX), Adjust replaced |
| ◆+ | maxPlusAdjustNonMax | MAX + adjust-nonmax; Rev AppLovin shows MAX |
| (none) | everyone else, and any title with MAX = 0 | Adjust (CloudX is already inside it) |
Reconcile: Net = Adjust − Σ♦ Adjust + Σ♦ MAX − Σ◆+ Adjust + Σ◆+ (MAX + adjust-nonmax), the ◆+ term only where MAX > 0. A spend-<date>.csv without the adjust-nonmax column counts as MAX + 0 for ◆+ titles, with one [net-basis] warning per run; do not re-crawl saved days to fill it. The rule is netRevenue() in the companion repo’s tool/lib/report-overrides.mjs; its TypeScript and Apps Script mirrors change with it (tool/test-report-net-basis.mjs).
The S2S swap is retired; adjust-s2s is kept for reconciliation only. History: references/net-revenue-history.md.
Build the daily report from a --pivot crawl: bun gen-daily-report.mjs ../data <date>. Commands and the 10-column layout: references/report-commands.md.
14-day sparklines — REUSE saved data, never re-crawl
Section titled “14-day sparklines — REUSE saved data, never re-crawl”The sparklines reuse saved spend-<date>.csv files; each day is crawled once and old days are never re-crawled. Detail: references/report-commands.md.
Per-title trend report — gen-trend-report.mjs
Section titled “Per-title trend report — gen-trend-report.mjs”gen-trend-report.mjs <dir> <endDate> [days=7] builds the trend HTML from saved files (0 crawls). Detail: references/report-commands.md.
Discord delivery — notify-discord-daily.mjs
Section titled “Discord delivery — notify-discord-daily.mjs”notify-discord-daily.mjs posts the headline and attaches both files; a missing webhook exits 0. Detail: references/report-commands.md.
Hard rules
Section titled “Hard rules”-
Adjust under-records MAX revenue on some builds — ◆+ titles use MAX +
adjust-nonmax. Measured ratios and the window:references/adjust-applovin-max-api-vs-sdk.md. -
Adjust’s “AppLovin Max” API source is empty and never backfills, so Adjust never self-corrects. Status:
references/adjust-applovin-max-api-vs-sdk.md. -
MAX’s Meta figure lags 1–2 days on the newest day. Do not read the newest 1–2 days of a MAX pull as final. Evidence:
references/net-revenue-history.md. -
AdMob
networkReportESTIMATED_EARNINGSis AdMob-network-only and does not over-report. The double count runs the other way: AdMob bidding into MAX is already on MAX’sADMOB_BIDDINGline. Split withmediationReport:generate+MEDIATION_GROUP:"AdMob (default)"is never summed on top of MAX; a named group is safe to sum. PasslocalizationSettings: { currencyCode: 'USD' }, nevertimeZone='America/Los_Angeles'. Details:references/hard-rule-details.md. -
A monetization partner can also sell traffic; that cost side is easy to miss. Crawl the partner’s spend as well as its revenue. A
$0outside its buying window is real, not a gap.references/applovin-ua-retirement.md. -
A missing spend COLUMN does not mean missing spend.
totalis accumulated from crawled rows inpivot.ts, not summed from displayed columns. Check where a total comes from before predicting what removing a column does.references/applovin-ua-retirement.md. -
Adjust carries AppLovin MAX twice: exclude the API side (
AppLovin Max), never the SDK side (Applovin MAX SDK). The API side is missing most days and never backfills. Measured split:references/adjust-applovin-max-api-vs-sdk.md. -
ad_revenue_sources__not_inis silently ignored — the reports-service has no exclusion filter, so an exclusion must be expressed as an INCLUDE of every other source on the baread_revenue_sourceskey. Derive that keep-list at runtime fromfilters_dataso a newly-added source is kept rather than silently dropped. -
Mintegral silently drops out of pulls longer than 8 days. A
--start/--endrange >8 days returns rows with no Mintegral spend (its per-request cap), so the total is silently understated. Pull in ≤8-day chunks, or rely on the per-day reuse model (single-day files always include Mintegral). -
MAX (AppLovin) ≈ 92–94% of total ad revenue; CloudX ≈ 4–5% and growing. MAX mediation is the dominant monetization; CloudX is a small S2S contributor.
-
A Unity Service Account missing
Advertise API Viewerfails silently: theappscall returns200with zero results, every app lands underunknown, and the day total still looks right. Asserttotal > 0on a new org. Details:references/hard-rule-details.md. -
A Unity org migrated to the Unified Platform returns
403, not an empty list - a different failure from the missing role above; the connector falls back to the Unified/applicationslist, andunity: FAILEDis reserved for every configured account failing. Triage and fix:references/unity-unified-platform-403.md. -
Unresolved rows reconcile in three passes: exact title, then normalized title, then package slug. Every pass needs a UNIQUE match, so Android/iOS siblings are skipped rather than merged onto the wrong platform. An app whose bundle id and title state no name still strands: fix the role, not the title. Details:
references/hard-rule-details.md. -
Unity’s daily quota dies after a few full crawls, and a failed source is non-fatal. Two or three full crawls in one day exhaust it; later days show Unity at 0. Read per-source row counts in
crawl.logbefore trusting a re-crawl. Details:references/hard-rule-details.md. -
An unconfigured buying account looks like an app with no spend. When a title’s ROAS looks implausibly good, check for a missing buying account before believing it. Details:
references/hard-rule-details.md.
Full setup, per-connector API reference & the runnable crawler
Section titled “Full setup, per-connector API reference & the runnable crawler”This skill is a discoverable playbook — the code, the per-connector setup docs, and the credential reference are owned by the companion repo (single source of truth):
The1Studio/marketing-crawl-ads-spendhttps://github.com/The1Studio/marketing-crawl-ads-spend
Clone that repo to run the crawl — it is self-sufficient (crawler code +
setup docs + .env.example). Everything lives under
.claude/skills/t1k-marketing-crawldata/:
The companion repo’s setup docs, credential reference, crawler and .env.example are the reference set; file-by-file list: references/report-commands.md.
Credentials live ONLY in a gitignored credentials/.env in that repo — never in
this kit, never on the command line.
CloudX First Look bid-floor calculation (P90)
Section titled “CloudX First Look bid-floor calculation (P90)”Distinct from the spend/revenue crawl: compute CloudX first-look price floors
as the impression-weighted P90 eCPM per app × format × geo from AppLovin
MAX report data. The MAX Reporting API is Mediation Partner Reporting, so the
weighted empirical-CDF method is mandatory (a naive percentile(ecpm,90)
inflates floors). Output is a results table + multi-tab .xlsx. Read-only —
nothing is pushed to MAX; floors are recommendations.
Unlike the crawler (which lives in the companion repo), the First Look floor
scripts are bundled with this skill — self-contained, zero-dependency (Node
26+ built-ins only) scripts under scripts/cloudx-first-look/
that read a local .env (MAX_REPORT_KEY=, same value as APPLOVIN_REPORT_KEY):
Commands: references/report-commands.md.
Full methodology, per-app geo buckets, uplift tiers, and gotchas:
references/cloudx-first-look-floors.md.
Security
Section titled “Security”- This skill is read-only (reporting). Any mutating ad-network call moves
real money — see
rules/ad-network-mcp-write-safety.mdbefore any write. - Never print or commit API keys, tokens, or account ids. Document env-var
names only; real values live solely in the companion repo’s gitignored
credentials/.env(or, for the bundled First Look scripts, a local.env). - iOS apps come back as store ids (e.g.
id6751266232) in some networks’ package fields — that is expected, not a bug.
Status
Section titled “Status”Operator playbook + connector overview. The crawler, per-connector setup docs,
and credential reference are maintained in The1Studio/marketing-crawl-ads-spend
(single source of truth); deepen this playbook iteratively via /t1k:sync-back.