Interconnecting Web Services


The only way to untangle a complex software service, such as Zemanta's, is to break it down into smaller components so that developers can work mostly independent without stepping on each other toes all the time. For the past 10-15 years the dominant way for implementing software services is to use web standards and interconnect them by http(s). At first we all used XML as a serialization format, but nowadays JSON is the usual choice, since JSON is much simpler to understand and handle than the XML bloat. For exposing and consuming data JSON-over-HTML works like charm. It's based on reliable, transparent, and widely used technology that's easy to understand and handle by programmers. But web services are not just about exposing data, but also about exposing functionality. Unfortunately JSON-over-HTML is not very suitable as a RPC (Remote Procedure Call) mechanism since it lacks validation and control semantics. Consequently, programmers have developed some alternatives, such as Apache Thrift, that is also used at Zemanta. But as put forward in a verbose Finnish manner by our new colleague Kalle "using thrift is not very entertaining".

Especially in the initial phase of product/feature development service interfaces are changing a lot and robustness of thrift comes at (unacceptable) price of slower development. Therefore in the prototyping phase programmers prefer JSON-over-HTML for RPC even if that means implementing validation and control code for each service from scratch. Once a web service is defined and put in production, the flexibility of JSON-over-HTML is not so important any more, while lack of robustness comes to the fore, stabbing the programmer in the back with increasing frequency due to inadequate validation and control mechanisms and copy/paste code.

My ideal way of interconnecting web services would be something that provides flexibility of JSON-over-HTTP in the prototyping phase, and robustness of thrift once the service becomes a critical part of the system. Unfortunately, I don't know of any good way how to achieve this. If you have some ideas yourself, let me know in the comments.

Enhanced by Zemanta