Doing general purpose extractors is hard, a logarithmic curve of value for exponential cost.
- Doing general purpose extractors is hard, a logarithmic curve of value for exponential cost.
- Anyone who's written a web scraper knows this in their bones.
- No individual case is hard, and easy for someone to say "the fix for this case is easy."
- The problem is that all of the individual cases are below the Coasian Floor[so] of being worth tackling individually.
- Also often all of them are quite different, so solving one doesn't necessarily help with the others.
- To tackle general purpose extractors in a sustainable way requires a social sifting process, where the savviest users, in solving their own problems, produce solutions that everyone else can draft off of, too.