Pipes has had a problem handling certain character sets in various feeds and XML documents. The good news is that we believe we've finally isolated the issue and the release today should solve the mangling problem for a majority of feeds in our system. If you are using the http://william.cswiz.org/tool/xmliconv/ hack, you can still continue to use it, but you should be able to fetch the feeds directly now.

We also fixed the pubDate on our output. We will display the 4 digits of the year (2008) vs. the 2 digits (08).

This week we have two updates to our site: A new module to fetch a pages source and a small fix for easy web paths when running a Pipe.

This weeks release incorporates a new module named "Fetch Page". The new module will fetch the source of a given web site as a string. This data can then be converted into a RSS feed or merged with other data in your Pipe using the Regex module. The module is located under the "Sources" menu in the editor.

The module can fetch an entire page or you can specify a "starting" and "ending" point. We also offer support for specifying a delimiter to allow you to easily separate the text into strings.

Our team as also created a few examples:
1) Fetch Page Example #1 is our first example. The goal of this Pipe is to extract train schedules from a local train scheduled provided in the San Francisco Bay Area.
2) Fetch Page Example #2 is a simple example that utilizes the first example as a sub-Pipe to obtain addition train stops.

Additional information about this new module along with its limitations can be found in the documentation modules/sources section.

Our second change to the site this week is a minor fix/enhancement to allow for easy web paths of all Pipe.runs page. This will allow you to use the easy paths you created for your Pipes. For example, the Pipe above "Fetch Page Example #1" would look like the following below (please note the "?" instead of the "&"):

Original name:
"http://pipes.yahoo.com/pipes/pipe.run?_id=1qoLq6ai3BGv06982R2EvQ&_render=rss&station=Palo+Alto"

Easy web address name:
"http://pipes.yahoo.com/pipes_team/fetch_page_example?_render=rss&station=Palo+Alto"

While every release of Pipes fixes many bugs, we don't often talk about them. However the latest Pipes update addresses one of the biggest problems reported on the message boards and now Pipes supports a wider diversity language encodings and character sets found in many feeds. We still have a few remaining types to nail down but many more feeds should now work well.

Another frequently requested enhancement finally arrives on the Regex module, now our power users (you know who you are) can make use of the "m" "i" "g" and "s" flags.

We've also enabled run counting - yes, finally they're back! We've re-counted all those hanging chads, errr, runs, and updated the statistics on each Pipe. Take a look at your Pipes and see how popular they have been.

Fed up with that avatar? You can now use any picture or yourself so other developers can see how pretty you really are. You can change your picture without even leaving your Pipes page:

Thanks to everyone who has continued to provide the Pipes team with valuable suggestions. Today, we pushed a number of improvements that we want to tell you more about.

1) You can now use subfields in the Regex and Rename Modules. It's also possible to refer to an item using dot syntax: ${published.timezone}.

If something in the preview pane is nested three levels deep, foo->bar->baz, you can refer to it as ${foo.bar.baz} in Regex replacements. Here's an example to get you started.

2) Titles in a Pipe that contain quotes and/or angle brackets no longer get mangled

3) You can now use URLs that have urlencoded characters (like %22) in the Fetch Module.

4) Modules can now contain arbitrary text. In the past, fields that contained values that looked like tags (for example: <something>) were getting unexpectedly escaped or stripped.

5) Certain BBC feeds that were broken now work.