YQL has become a great way to extend Pipes. You can harvest the power of YQL by using YQL statements in the YQL module. Another way of using even more of YQL's power is to use the Pipes Webservice module. This enables you to use YQL's Execute functionality to create javascript functions that will work on your data in any Pipe.

In this blog post we'll go through the steps on how to use YQL Execute to power the Pipes Webservice Module - for the impatient, here's the example Pipe that we'll be creating.

In order to use YQL Execute, you'll need a website to host your XML file. The XML file is where YQL Execute allows you to manipulate data by using server side javascript. We can then post data to YQL via the Pipes Webservice module to allow further processing of our Pipes Data.

Some users on the Pipes message boards asked if we could add a "yindex" element that numbers each item in the output. This becomes trivial when using YQL Execute!

The first thing we will create is our XML file for YQL Execute. You can copy and paste the structure of the XML below, editing only whats inside the execute node.
The execute content is wrapped within a CDATA statement. Since we're dealing with json, we'll import Crockfords json parser and stringifier by using the y.include() method. First, we parse the json data using JSON.parse(), then loop through the json structure and add new elements by using pdata.items[i].new_element_here within the loop. "yindex" is the new item we want added to our Pipes data structure. We then assign a number to yindex within the loop and then use JSON.stringify() to bring it back together. Finally, we assign our "output" variable as our response.object. Then we want to host this XML file on a publicly accessible web host.

Now we'll set up our Pipe. We need to use the URL builder to setup the URL that the Webservice module will access.

In the URL Builder module, we'll use

• http://query.yahooapis.com as our base
• v1/public/yql as our Path element
• These name/values as our query params
  • q: use "http://paul.donnelly.org/yql/yindex.xml" as yindex; select * from yindex where data = @data
  • format: json
  • diagnostics: false

And simply wire the URL builder module into the url terminal in the Web Service module. Be sure to use query.results.result.items as the Path to item list.

Here is the example Pipe that takes the Pipes blog feed, uses YQL execute as a Webservice module to create an additional element in the Pipes data output: "yindex" which adds a number to each item in the Pipes results. Another example YQL Execute statement is: use "http://paul.donnelly.org/yql/random.xml" as ran; select * from ran where data = @data that randomizes a given feed.

Did you ever want to do something more with Pipes data? By using a Pipes webservice module you can!

There are several ways you can achieve this. You can host your own - or even host it on Google App Engine. Sam Pullara has a great post on how to extend Pipes with GAE here.

One host that we've been playing around with is AppJet. Using AppJet as your host for your Pipes webservice module is easy to make, fast and effective. AppJet uses JavaScript as the server side language making it even easier to use for web developers.

For this example, I want to randomize my feed. First you'll need to create an appjet login, after that you can clone my example AppJet app. Appjet like Pipes lets you view source, which for developers is one of the best ways to learn and see how things work.

Some things to take note. You'll want to name your AppJet function name "post_<somename here>". This tells AppJet that you're posting data to this function. You'll also want to add page.setMode("plain"); and var data = JSON.parse(request.params.data); as your first and second line.

var data, gives you access to the pipes output data array. You can do whatever you want with it. For my example I just do a randomized sort on it. If you want to manipulate each item in the data array, its easy to loop through it and change the feed as necessary (see my commented code in the AppJet example). You can also add items to the data array by declaring something like: data.items[i].newitem = <whatever>;

AppJet makes it so easy and is a great way to extend Pipes functionality. Here is my example Pipe that will bring it all together.

You'll want to make sure that "Path to items list:" has a value of "items" or similar for this example to work.

For you web2.0 savvy people that customized your Pipes URL to http://pipes.yahoo.com/coolname/coolpipe, you might be wondering how to get your Pipe ID.

This is especially useful to Flash developers who use our yahooapis.com crossdomain policy file as the web2.0 URLs aren't supported in that domain.

To get the ID of your Pipe go to "More Options" on the Pipe info page and click "Get as RSS" (or as JSON etc). This will open a new window and you'll see the Pipe ID in the URL.

If you are a Flash developer, just replace pipes.yahoo.com with pipes.yahooapis.com as described in this previous post: http://blog.pipes.yahoo.net/2007/03/10/pipes-adds-flash-developer-support/

We've added a new Web Service module to Pipes that allows developers to power their own modules and extend the functionality Pipes offers.

In the Pipes Editor, the new Web Service module is listed under Operators:

• The Web Service module POSTs the items in a pipe in JSON format to an external web service. This allows developers to extend the functionality Pipes offers to do whatever they need. The original items are replaced by the web services JSON or RSS response. Here's an example.
  

Note: We've documented the source code for the web service that we call in the example pipe.

We worked with the Yahoo! Maps team to put together two examples that demonstrate how you can take the output from a Pipe containing GeoData and overlay its content on a Y! Map. Using these examples, you can learn how you can start putting Pipes powered Maps on your website.

The Pipe being called in the below examples is called Where2 with flickr. This Pipe aggregates event listings in the San Francisco Bay Area from upcoming.org. Then, uses the location information for each event to search Y! Local for nearby restaurants and flickr for nearby pictures. The output of this Pipe contains all the data you need to decide where to head out to!

Both examples demonstrate how to take the output from a Pipe and overly its data on a Y! Map:

• The first example uses the JSON output from a Pipe.
• The second example uses the RSS output from a Pipe.

To learn more about how each example was made you can view there respective sources. There's more information about the Yahoo! Maps AJAX API on the Yahoo! Developer Network website. If you have specific questions about the Y! Maps AJAX API we suggest you visit the yws-maps-ajax mailing list.

We're eager to answer any questions you might have about these examples so start a thread anytime on the Pipes Message Boards.

Many Pipes developers have asked how they can get Pipes content onto their web site or blog.

We've put together a small piece of sample Javascript to show how you can get a Pipe's output into your page - without using any other server!

Of course, how you present it to the user is up to you! Here's a simple example that uses the sample code to do just that.

The IBM developerWorks website has posted a feature article on Pipes that includes both an in-depth description of the project and a great tutorial. Here's how they describe the article:

This tutorial provides a basic understanding of Yahoo Pipes, a service that filters, transforms, and aggregates content feeds. A complete run-through of the toolset available is included, as are three demonstrations that illustrate the capabilities and features of the service. Finally, step-by-step instructions for one of the demonstrations complete the tutorial, along with a recorded movie showing the steps in detail.

The intent of this tutorial is simply to describe the Yahoo Pipes service for managing available content feeds, focusing on the features, tools, and capabilities available.