What are WebPipes?

WebPipes is a work-in-progress proposal for linking together web services. Essentially, you can think of WebPipes as UNIX programs. They are small utilities available over HTTP that do one thing, and do it well. WebPipes follow the Unix philosophy:

  1. Write programs that do one thing and do it well.
  2. Write programs to work together.
  3. Write programs to handle text streams, because that is a universal interface.

The magic comes when you start chaining pipes together. You can create really amazing workflows by combining a few small pieces together.


How do they work?

Since a WebPipe is essentially just a script that receives input via HTTP POST/GET, there's not much else to know other than how to structure the output. Their precise behavior is covered in the WebPipe Specification.


Where can I find a full list WebPipes?

You can view a list of some example WebPipes, or on the WebPipes.org GitHub.


Is there a WebPipe specification?

To participate in the wider WebPipe ecosystem, a few conventions have been adopted which enables WebPipes to "speak" to one another. Check out the WebPipe Spec.


What's the difference between WebHooks and WebPipes?

Although, WebHooks share much in common with WebPipes, each solve different problems. WebHooks were designed, in part, to solve the push/poll. Essentially, a WebHook is a URL callback since it responds to events. For example, GitHub implements WebHooks. On post-receive, GitHub calls a user-defined URL like a callback. It also feeds along a few arguments that tells you about that commit.

Likewise, WebPipes are essentially simple scripts publicly accessible via a URL. You POST/GET some input, and it'll provide you some output.


Why should I care?

Although not yet widely implemented, WebPipes offer developers more leverage. By connecting small and independent units, the whole becomes greater than the sum of its parts.

But the real beauty is the potential for WebPipes to enable the "programmable web" and an entirely manner for creating applications. Some folks have even experimented with wrapping Unix programs around HTTP.


How do I implement WebPipes?

Since WebPipes are basically just scriptlets, they're easy to implement. Read the WebPipes Specification to learn the specifics.


Example WebPipes

  • Minify CSS - Takes CSS markup and compresses it.
  • Parse Markdown - Takes Markdown data and converts it to HTML.
  • Parse RSS - Takes an RSS feed and splits it into an array of records.
  • Send Email - Send an email.
  • Formatter - Input a mustache template and some arguments, outputs a formatted string.

See more in the WebPipe Registry →


Resources

  • Specification: WebPipe Specification - A definitive compilation of standards and best practices.
  • GitHub: WebPipes GitHub - Repository of code, discussion, and issues.
  • Group: WebPipe Group - A Google Group for community discussion.
  • Testing: PostBin - Debug webhooks/pipes by capturing and logging requests.
  • Testing: WebHookApp - Debug your HTTP requests. Auto-generates a cURL command.

Specification Authors