Data Quality Gate - deterministic post-scrape cleaner + verdict
Post-scrape data cleaner, no LLM: repairs mojibake, HTML, invisible chars.
Community: Submitted by a user or imported; check the owner before granting accessOnlineNo sign-inGlobalFreeCan modify data
What it can do
What data it sees
Do you need an account
No: the server works without sign-in
Post-scrape data cleaner, no LLM: repairs mojibake, HTML, invisible chars. Plus a verdict.
Server tool list (3)
Raw names from tools/list. Only developers need these.
| check_dataset_quality | Call this before using any dataset. Returns a deterministic quality verdict (RELIABLE / USABLE_WITH_CLEANING / UNRELIABLE) with exact facts: completeness, nulls, type consistency, impossible values, duplicates, outliers, and (on financial/trading data) cross-source price divergence. 100% deterministic, no LLM. Free -- this MCP endpoint runs the engine directly; POST /api (plain REST, same engine) is x402-gated at $0.01/call instead. Input: rawJson (a JSON array of row objects, or a single object); datasetId is accepted but not resolvable on this deployment -- pass rawJson instead. |
| clean_scraped_data | PAID TIER, CALLED FROM A FREE ENDPOINT -- READ THIS FIRST: this tool does NOT return your cleaned data, and its response has NO `data` key. It returns {"status": "payment_required", "what_we_found_in_your_data": {...}, "get_the_repaired_data": {...}}. Do not treat its output as cleaned data and do not write it back into your dataset. WHAT YOU GET HERE, FREE: an exact inventory of the defects in YOUR data -- which repair rules would fire, how many values each would change, and which cases cannot be repaired safely and need your decision. HOW TO GET THE REPAIRED DATA: make the one call named in `get_the_repaired_data` -- POST https://www.aidatatools.dev/api/clean, $0.04 via x402, no account, no API key, no signup. That response body IS the cleaned dataset, in the shape you posted it. WHY THE SPLIT: detection is free on this endpoint and always has been (check_dataset_quality reports the same defects). The repaired artifact is the paid product, because it is re-bought on every extraction run rather than cached like a verdict. WHAT THE PAID CALL DOES: removes leftover HTML tags and entities, decodes mojibake ('Café' -> 'Café'), strips invisible characters (zero-width, BOM, soft hyphen), normalises non-breaking spaces and trims values -- across nested objects and arrays too. 100% deterministic, no LLM: the same input always yields byte-identical output, and cleaning twice equals cleaning once. It repairs how data was ENCODED, never what it SAYS: masked placeholders ('N/A', 'None'), near-duplicate rows and failed extractions ('access denied', 'captcha', which mean that record must be re-scraped) are reported with a proposal, never silently deleted or rewritten. The full boundary -- 7 rules applied automatically, 5 needing an explicit opt-in, 8 only ever reported -- is at GET https://www.aidatatools.dev/api/clean. |
| clean_scraped_data_audited | PAID TIER, CALLED FROM A FREE ENDPOINT -- READ THIS FIRST: this tool does NOT return your cleaned data, and its response has NO `data` key. It returns {"status": "payment_required", "what_we_found_in_your_data": {...}, "get_the_repaired_data": {...}}. Do not treat its output as cleaned data and do not write it back into your dataset. WHAT YOU GET HERE, FREE: an exact inventory of the defects in YOUR data -- which repair rules would fire, how many values each would change, and which cases cannot be repaired safely and need your decision. HOW TO GET THE REPAIRED DATA: make the one call named in `get_the_repaired_data` -- POST https://www.aidatatools.dev/api/clean/audit, $0.12 via x402, no account, no API key, no signup. That response body IS the cleaned dataset, in the shape you posted it. WHY THE SPLIT: detection is free on this endpoint and always has been (check_dataset_quality reports the same defects). The repaired artifact is the paid product, because it is re-bought on every extraction run rather than cached like a verdict. WHAT THE PAID CALL DOES: the same repair as clean_scraped_data, plus a complete audit trail: every transformation with its path, rule, before and after value, a replay_id, and input/output SHA-256. The ledger is a full inverse patch -- applying it in reverse reconstructs your original input byte for byte. Use it when you must be able to PROVE later what changed and why. |