TL;DR: The mistake is treating anger as demand. Across 1,044,461 scored discussions we find that complaint intensity and willingness to pay move almost independently: of the 245,319 discussions carrying a high-intensity pain point, only 14.5% also showed high willingness to pay. Three things predict money better than volume or tone — a commercial action taken in the same breath, the platform the complaint was made on, and whether the person is already paying for something in the category.
Why "lots of people complain about this" is a weak filter
Complaint volume tells you a category is populated. It does not tell you a budget exists. These are different claims, and the gap between them is where most idea validation goes wrong.
Our pipeline scores every discussion on two separate axes. Intensity rates how badly the described problem hurts, per pain point, from 1 to 10. Willingness to pay rates whether the discussion contains evidence that someone would spend money to fix it. Both are scored by the same model on the same text.
If intensity were a good proxy for demand, the two would track each other. They do not:
| Share | |
|---|---|
| Pain points rated intensity ≥8 | 25.0% of 2,042,978 |
| Discussions rated willingness to pay ≥8 | 3.6% of 1,044,461 |
| Discussions rated willingness to pay 1–2 | 44.2% |
And directly on the overlap: among the 245,319 discussions that contain at least one intensity-≥8 pain point, 14.5% also score ≥8 on willingness to pay. The other 85.5% describe something that genuinely hurts and that nobody has shown any sign of paying to fix.
That is the whole problem with reading complaint threads for ideas. The loudest material is selected for emotion, and emotion is free.
The three signals that do predict money
1. A commercial action in the same breath
The strongest single marker is not what the person says about the problem — it is what they are doing about it. Hiring someone. Putting it out to contract. Naming the tool they currently pay for and what it costs.
We tag discussions where a hiring or purchasing action is detectable. The difference is large and consistent:
| Willingness to pay ≥7 | Mean score | |
|---|---|---|
| Commercial action detected (54,503 discussions) | 26.2% | 4.7 |
| No commercial action (the rest) | 12.3% | 3.4 |
Twice the hit rate. This is why job postings and contractor requests are underrated as research material: a job posting is a complaint with a budget line attached. Someone has already lost the internal argument about whether this is worth money.
The word-level signals that co-occur with high scores follow the same logic. Across a 30,000-discussion sample, the terms that show up most often in scored pain points are cost (15.1%), invest (10.2%), team (8.0%), buy (6.9%), right now (6.2%), subscription (5.8%), and budget (4.5%). Notice what they have in common: they are all about resources already committed or about to be, not about how bad something feels.
2. Where the complaint was made
This one surprised us enough that we re-ran it. Platforms differ enormously in how often their complaints carry payment evidence — not by a few percent, but by more than an order of magnitude.
The table below compares each source's share of high willingness-to-pay discussions (≥8) against its share of low ones (≤3). A ratio above 1 means the platform is over-represented among discussions that show money behind them.
| Source | Share of high WTP | Share of low WTP | Ratio |
|---|---|---|---|
| Discourse product forums | 10.4% | 0.5% | 22.6x |
| HackerNews | 3.4% | 0.2% | 14.8x |
| GitHub issues | 6.1% | 0.5% | 11.6x |
| App store reviews (iOS) | 22.4% | 3.3% | 6.8x |
| Google Play reviews | 4.7% | 2.4% | 1.9x |
| Dev.to | 1.8% | 3.1% | 0.57x |
| Medium | 9.0% | 16.2% | 0.56x |
| Bluesky | 10.7% | 23.5% | 0.46x |
| Mastodon | 16.0% | 45.6% | 0.35x |
| YouTube | 0.2% | 0.9% | 0.26x |
Remote job boards score higher still — WeWorkRemotely at 215x, RemoteOK at 15.3x — but their low-WTP counts are small enough (73 and 675 discussions) that we would not lean on the exact multiple. Directionally they agree with the hiring finding above.
The pattern underneath is about who is in the room. A Discourse forum for a specific product is populated by people who already installed it, usually at work. A broadcast social feed is populated by everyone. Mastodon contributes 45.6% of all low-willingness-to-pay discussions in our dataset and 16.0% of the high ones — it is not a bad platform, it is a general-audience one, and general audiences complain about things they were never going to buy.
Practical version: narrow, product-specific venues beat wide ones, and venues where the complaint costs the author something — a support ticket, a GitHub issue, a job posting — beat venues where it costs nothing.
3. Whether they are already paying for something in the category
App store reviews sit at 6.8x on the table above despite being consumer-facing and full of noise. The reason is a structural filter: to leave a review you generally installed the product, and often paid for it. The category is pre-validated by the reviewer's own behaviour before they type a word.
This generalises. A complaint about a tool someone pays for is worth several complaints about a problem in the abstract, because the first one has already answered the hardest question — does anyone in this category open their wallet at all.
A working checklist
Before committing to an idea, try to answer yes to at least two:
- Is someone spending money adjacent to this already? A tool they pay for, a contractor they hired, a spreadsheet someone maintains on salary.
- Did the complaint cost the author anything to make? Filing an issue, writing a detailed forum post, posting a job. Effort is a weak proxy for budget, but it is a real one.
- Does the same complaint appear against multiple products in the category? One product being bad is an execution gap. Every product having the same hole is a market gap.
- Is there a workaround in the description? Someone doing manual work weekly has already priced the problem in hours.
- Does the complaint survive without the emotion? Strip the adjectives. If nothing specific is left, there was no information in it.
What is deliberately not on this list: how many people complained, and how angry they were. Both are easy to measure and neither survived contact with our own data.
Honest caveats
- These scores come from a language model, not from purchase records. They measure evidence of willingness to pay present in the text. Someone can be a buyer and never signal it; someone can talk about budget and never spend.
- The platform ratios reflect our crawl mix, not the whole internet. We collect more heavily from some sources than others. The ratios control for that by comparing each source against itself across score bands, but a source we barely touch could still surprise us.
- A high score is a reason to investigate, not a reason to build. Nothing in a public complaint tells you whether the buyer is reachable, or what they would actually pay.
- Low willingness to pay is not the same as no opportunity. Consumer products often monetise around the complaint rather than through it. The signal is calibrated for tools people buy.
Related reading
- Where to find SaaS ideas — the sourcing side of the same question
- How to find SaaS ideas in app store reviews — working the 6.8x source specifically
- What is a pain point database? — how the scoring above is produced
- Run the filters against live data in the PainHunt dashboard, or check a specific idea in validate