Marmite: Re-purposing Web Content through End-User Programming

Marmite is an end-user programming system for the web that enables non-programmers to recreate some of the functionality of mashups without programming. Users construct a data-flow from a set of operators. Operators are GUI representations of web services or represent operations the local machine such as filtering or scraping information off of web pages.

Marmite is different from other mashup systems such as Yahoo Pipes!, Microsoft Popfly, and Intel MashMaker because:

• Marmite deals with both Web Services and screen-scraping methods of creating mashups.
• The state of the data is shown at each individual step.
• Fine-grain control and constant feedback between steps makes incremental programming possible (which shrinks the design/test/debug cycle).

Marmite is inspired by UNIX pipes and Apple Automator.

Downloads

Marmite is available here for you to play with it and try it out. I don't plan to update this or support it much further beyond fixing any heinous bugs you may find. If you like it and want to actually use it for something, I'd be happy to show you how to make any necessary changes. However, this software is really unsupported and you may find many bugs in it. When in doubt, hit "Reload." DISCLAIMER: Some things work and some things don't.

I am actually redesigning Marmite from scratch and taking a different approach with respect to the UI design. Also, the new system will support construction of operators from within the system. I hope to initially release it in Feb 2008.

Marmite 0.2 (click here to install into Firefox) Access it from the Tools Menu.

Marmite screencast 60MB (right-click to download)

Known Issues and Limitations

(that were major enough for me to remember to place here)

• Not all operators in the listing have been constructed. Incomplete operators are listed to give an idea of what operators might be possible.
• The Yahoo! Maps operator doesn't always work. I haven't been able to reproduce it reliably. (It either works well or fails to work at all for periods of time. I think this is something on their side.)
• You can't save a workflow. Sorry.
• The suggested operators list is the same regardless of what actual operators are already in the workflow. I decided at some point this was sufficient for the user study since there weren't that many operators to begin with. Users never noticed that it was the same.
• There is a race condition when you open up one of the scraping operators where the wizard panel may be rendered behind the page. Simply close the window and hit play on the operator again.
• The address scraping operator looks for URLs that point to Yahoo Maps. A more sophisticated, semantically clever scraper was beyond the scope of this project. Maybe next time.

Contact Info

Feedback, questions, and other comments are much appreciated.