Category: Technology

SCAM – Story of Data

Read many articles on SMAC (Social, Mobile, Analytics and Cloud) but none could connect all the dots. So, I tried mine by reordering SMAC as SCAM. What do you think of it?SMAC

From Data Scientist Viewpoint:

“Social Networking brings customer together to share data about products/services usage. To address the scale of this shared social data, Cloud platforms provides scalable & elastic storage to facilitate this large and complex social interactions between customers. To make sense of this complex social interaction in cloud, Analytic collects and correlates this Big Data to bring insights into customer’s behavior about product/services usage. Closing this loop, using these analyzed insights, Mobile delivers personalized products/services into customer’s context.”

From Mobile Developer viewpoint:

“Cloud is the platform which makes the lives of developers a lot more easier and helping developers churn out Apps on the fly (services like BaaS and PaaS is making the lives of developers a lot easier in terms of cost and time). Social engagement is the means through which you spread word about your Apps and to help expand your user base. Analytics is where you understand the user behavior, the market behavior and gain insights that help you make your App even better. And you start with the cycle again.”



Platform providers has unique opportunity to adopt SOA as business model by opening up their platform as web service and allowing developers and partners to build applications. Platform owners can leverage new avenue to monetize the API usage, share revenue for each transaction and sell advertisement as well.SOA Conf

It is win-win situation for all, it is hay days not just for platform builders but also for stakeholders like entrepreneurs, developers, aggregator, end users and marketplace. Their way of cutting gold is by affiliations and partnerships, revenue sharing, advertisements and subscription.

Platform owner needs to formulates their SOA strategy to open-up platform using Web API analogous to Web Services.  Also needs to create environment for platform adaptation and creation of ecosystem around platform for easy entry.  In all this most challenging aspect of this offering is to address heterogeneous users and developers.

Marketing strategy is highly important to give you edge in this PAAS game.

  • Expand footprint of platform
  • Drive traffic
  • Innovation (Feature development)
  • Support new business model
  • Content acquisition (user-generated)
  • Co-branding

Designing the Web API(Web service) is an art and critical to success.  These API needs to support heterogeneous technologies and best way to adopt is is going for open standards like XML/SOAP.  Platform API should be architected with homogeneous and conceptual coherence like WS/REST. Also these must follow underlying domain semantics and should be easy to learn.

API need to have right granularity to offer comprehensive tools for app building. Mesh-up apps wants to control noun object and composite apps want access for verb actions. Fine grained API address coverage for noun object (Domain Objects) like Order/Customer etc as well as verb objects through coarse grained APIs.

After hammering API next come technology choices.

  • Transport Protocol – SOAP vs REST
  • Service Standards – WS *
  • Data Structure Exchange – XML/JSON
  • Standard – OASIS/W3C

How to solve evolving standard problems? Answer lies in Model Driven Development approach.

Cloud is for REST of the World

IT industry is always been hyped with ever-growing buzzwords and its excessive usage. New entrant, Cloud Computing has been buzzing in Enterprise world and every IT company iscloud img try to get on to this bandwagon by positioning themselves with feature differentiators.

But I have tried to find the mean of cloud computing for myself. I went to find out from an Architect view-point for 5-W questions – What/Why/How/When/Where.

Lets find out what is the problem with current model?


A business owners can’t anticipate the growth of their business and to choose an infrastructure scheme/architecture, they just have to rely on historical data or gut feeling. Later point, they meet with reality to realize that initial calculations were well below/above the reality mark  and forced to choose another scheme/architecture.  And it goes on.

But with advent of Cloud, above problem has been address to certain extend for time being.  Now their focus is just application not infrastructure or application/middleware server capabilities. Cloud will take care of these.  Enterprise can focus on application and can choose to go either conservatively or aggressively. This  allows them the flexibility of change depending upon response and requirements.

Cloud Computing is an IT Infrastructure utility model of  “Pay As You Go”. Till today, cloud has comes in many shapes. E.g

  • Service Oriented Architecture
  • Software As A Service
  • Software + Service
  • Rich Internet Applications

All these are certainly applications to use cloud as a base. Lets go to fundamental components of Cloud architecture.

Roles: In two-tier system architecture,  Web Layer and Business Layer are mapped to corresponding roles in Cloud architecture.

  • Web Roles – Web Interface to system
  • Worker Role – Business Logic implemented using Cloud APIs

Storage: All the data system generates and uses needs to be stored and keep it available all the time.

  • Table/Blob/Relational – System data in many forms

Messaging: Asynchronous communication is the glue between Web and Worker roles to keep system connected as well as decoupled.

  • Queue – Queue is preferred choice of communication between Web and Worker.

Technology: There are many competing technologies trying to provide “Right” solution, but my choice goes to RESTful Architecture.

Patterns: Recurring problems brings patterns of solutions. According to architect of cloud, there are five problem space in cloud.

Now, lets discuss about application designing in cloud.

Cloud Computation:

A system which require heavy data crunching on continuous basis. E.g. search engine indexing. MapReduce has been the solution to do this in grids but throws a challenge of managing grid’s growth.


Cloud Solution: Cloud provider will take the responsibility to managing Grids.

Cloud Storage:

A system where data keeps growing, storage longevity becomes critical factor and data type can range from Blobs, Tables to Relational.  It also throws another issue of tight affinity between logical data and actual hardware storage.  To add to more complication, architect assess distance of Data Access Layer(DAL) between client and server.

  • Code Near – Data is controlled by Client’s DAL and generally has large data upload. E.g. offline data
  • Code Far – Data is controlled by server DAL and generally smaller chunk of contiguous data. E.g. Browser, Chat.


Cloud Solution: Data in cloud is logical entity defined in URL style. Cloud uses REST data modeling to address any type of data. This releives Cloud Infrastructure Provider to manage affinity of storage and hardware independently. REST lets designer define the a logical business data expressed in terms of URL. Client Applications can perform CRUD operations directly through these URL.

Cloud Communication:

A system which requires to talk to multiple external systems to perform their operations in coördination. But this poses the of negotiation on infrastructure and security and protocol. Preferred choice is HTTP but custom solution needs to be implemented.


Cloud Solution: Cloud wraps REST service over messaging solution to provide uniform communication between multiple parties without any overhead or responsibility on either parties.

Cloud Scalability:

A system where user base is very large and traffic load is not uniform. To bear the burden of heavy loads, Network Load Balancers are called in service but still either does not fit the required bill and keep both architect and Sys Admin on toe all across.


Cloud Solution: Cloud provides will manage it with elastic computing and storage capacity depending on load dynamically.

Cloud Multi-Tenancy:

A system which provides services to multiple business partners. E.g B2B model. In this each partner demands its own data schema and UI customization on their choice. Providing this service from one setup is not advisable but it can be done with proper design consideration.


Cloud Solution: Cloud provider can provide multiple entry point to this system, but design and identity consideration needs to be looked carefully  by system architect.

>Rational has a superb collection of tools to manage complete software product development lifecycle. It is amazing vision to automate whole process. Yes, it is a learning from other matured engineering branches like automobile.

Rational Composer:

  • Role: Business Analyst 
  • Purpose: Defining and composing requirement

Rational RequisitePro:

  • Role: Project Manager
  • Purpose: Managing requirement with project lifecycle.

Rational Software Architect: 

  • Role: Architect and Designer  
  • Purpose: Once requirements are defined, that is where Architect jumps in with

Rational Software Analyzer:

  • Role: Developer/QA Engineer
  • Purpose: Static Code Analysis and Code review 

Rational Purify: 

  • Role: System Analyst
  • Purpose: System Performance Matrices  

Rational Tester: 

  • Role: QA Engineer
  • Purpose: This tools help QA engineers to do regression testing.

Rational Team Concert:

  • Role: Team
  • Purpose: Team collaboration tool for Agile development

Rational Quality Manager: 

  • Role: Quality Manager
  • Purpose: Test planning, workflow control, tracking and metrics reporting

Jazz is the Rational program to bring collaboration to this diverse and comprehensive tool kit. It is an attempt to bring all above in one umberalla. If that happenes, IT development will be just like GM Car plant where car gets manufactured without human hand touch. In IT, it will be without bothering keyboard of developers.

With this vision, I think developer’s days are numbered. 😦

>Few years back I used OS Commerce to build a NGO website to sell handicraft works. Recently, I came across a presentation about other open source Content Management Systems which are widely used across industry as well as small business. Here I share my learning from it.

Content Management Systems
Content Management Systems (CMS) are designed to allow non-tech users to create, maintain and publish their website without knowledge of underlying technology. It provides main service of publishing document/information about dept, corp, product, services etc without any admistrative help. It also provides  document management system (DMS) capability to manage versioning of documents. Its usage has reached to Blog and Web 2.0 applications as well as to very large corporations and gov organization. There are three major CMS system with open source flavor.

All of these are based on LAMP(Linux, Apache, Mysql, PHP) technology stack. Three years back, I, myself, used WordPress platform to build and manage self-hosted Blog service for myself. It was fantastic experience but decided to give away my blog management to The Boss “WordPress”. 🙂

When we talk about CMS we can’t forget to talk about Reporting tool. Here are few of them.


A XML based great Business Intelligence tool. With Java interface, it has become almost de-fecto standard of report implementation for Java developers.
A visual tool interface to JasperReport technology base. Its WYSIWYS editor brings fantastic edge in report development for JasperReport. This allows user to do everything from dragging a image to change a font, or bring anything they wish.
JFree Report: 
It can be used to create chart of variety of kind from simple pie chart to Gantt and parato charts. Java based.
This CMS has been the most famous amongst all. It is for enterprise reporting, analysis, dashboard, data mining, workflow and ETL capabilities for Business Intelligence (BI) needs. Pentaho uses a subscription-based customer model. The developer community involved in the project comprises around 8,000 members. Pentaho generates revenue via technical support and management services for enterprise customers. Good part is that it can access data from a variety of Data source like OLAP, XML, RDBMS etc.
It is a professional open-source company that provides free documentation and paid professional service to users of its software.

Most of above makes money by support activities or extension development.  But who makes money even in this. Microsoft!!! They also have their foot in this space with a closed paid product called Microsoft Sharepoint System (MOSS).

>In SOA, governance plays out to be most critical in implementing solution. It brings standard, rules, best practices and regulations on which all enterprise stands on. They also bring historical background for successful transitioning of technology stacks. Here are major phases on which SOA Governance stands:

  • Planning
  • Monitoring
  • Execution
  • Assurance
  • Publication

Five major governance considerations for an architect to design an SOA.
Information Management: In enterprise, information definition and its flow are key components. Metadata defines the structure of information and its relationship with other. This will be influenced by Business, Technology and Governance with in organization.

Policy Management: Policy is about a standard rules and regulation learned over a period time by organizations. It is required to keep maintain security, integrity and consistency with existing enterprise systems. Enterprise architect has to ensure that any new implementation complies with these policy. Policy management is applied both at design and run-time, but new policies and services needs to be interlinked to provide improvement. This is a four steps process.

  1. Definition of policies
  2. Association with organization behavior
  3. Enforcement of these new policies
  4. Reporting and monitoring these policies effect

Service Contract Management:  Contract is between two parties, Provider and consumer, about a service. SOA architect has to understand consumers demands and connect with providers service offering and design a contract to satisfy both parties. This process need consideration to many different aspects like:

  1. Owner service offering
  2. Cost Model
  3. Targeted service level
  4. Maintenance Schedule for service
  5. Process of enhancement requests
  6. Customer support – call frequency
  7. Responsibilities of both provider and consumer

Life cycle Management: SOA services also have life just like any product. In this, provider and consumer play major role in service design. Following is the phase-wise transition of provider and consumer.
Provider Life-cycle for a change:

  • Plan > Build > Assure > Deploy/Publish > Monitor

Consumer Life-cycle for a change:

  • Plan > Discover > Contract > Build > Monitor

Interoperability: SOA is design to work with any technology implemented services. But throws other inter departmental and organizational issues like:

  • Governance
  • Quality Standard
  • Management


Wot d hell is SOA!!!

It is very nature for any technology to readjust after certain time. Enterprise applications are no exception.   Tight integration between enterprise applications lead to the need of loosely coupled module. It is analogous to transition from static libs based applications to loosely coupled DLL and DCOM technologies. Web Service has brought relief to enterprise applications from its complex and tightly coupled framework.

Why SOA? 
Due to itrative adoptation of technology by oragnizations, Enterprise applications needs to co-exist in heterogeneous environments. For any new or old application, one new particulate functionality either had to be redeveloped or reintegrated which throws a situation of multiple version of similar application specification exist within one organization. This puts other operational issues maintenance, version control and patch updates. SOA solved this problem holistically by bring these different application for similar functionality as a service at one central location and provide interface to all.

What is SOA? 
I used to think it is more about Web Service communicating with each other over ESB. But when I started to look close, found web service with lots of customized standard from J2EE world.

Here is my startup take on it. More will follow after some dipping into river.
There are four major pillors to SOA.

  1. Web Service Description Language (WSDL):  It is self explanatory description of service in a platform independent XML file.
  2. SOAP Messages: It is communication transport layer where business documents are exchanged in heterogeneous environment where providers have very little info about each other.
  3. Universal Definition, Discovery and Integration (UDDI): An enterprise level service directory/registory to look-up and invoke services.
  4. Quality of Service (QoS): Service level agreement about secuity consideration, authentication and authorization standard and reliability of messages in the system

SOA Infrastructure: 
There are three Major layers in SOA infra.

  • WS-I Profile: Interopearbility standard
  • Protocols- SOAP, UDDI, WSDL

Existingplatform like J2EE, .Net etc are natural adopters to SOA to provide proven infrastructure parameters like scalability, reliability, availability, and performance.

Just web service implementation for SOA will not be sufficient to satisfy numerous other quality of service standards. Organization for the Advancement of Structured Information Standards (OASIS) plays very important role in these evolving standard. Here are majors ones:

  • Security: Web Service Security
    •  Required for credential exchange, message integrity and message confidentiality. SOA advocates existing Security Assertion Markup Language (SAML) standard.
  • Reliability: WS-Reliability and WS-ReliableMessaging
    • Relieable message delivery is critical in SOA system to adopt messaging characteristic like once-and-only-once delivery, at-most-once delivery, duplicate message elimination, guaranteed message delivery, and acknowledgment.
  • Policy: WS-Policy
    • provides a standard way to defined provider Policy assertions.
  • Orchestration: BPEL4WS or WSBPEL (Web Services Business Process Execution Language)
    • Provides a standard to integrate enterprise data, applications and components. It involves process requirements, such as asynchronous communication, parallel processing, data transformation, and compensation.
  • Management: Web Services for Distributed Management (WSDM)
    • Provides a standard for System Administrators to manage enterprise infrastructure to run expending services in a heterogeneous environments.

Major SOA Platform Vendors:

  • JBoss – personal favorite
  • IBM
  • HP
  • Oracle


SOA – Evolution into today

When I understood the meaning of SOA, first feeling I got, that RMI taken from Object level communication to Component level.
This is how I look at whole evolution of SOA:

Monolithic Applications:
First came, procedural language to build very large scale application systems.

Structured Object Oriented:
With monolithic approach, larger systems were enormously complex and hard to maintain. Solution was to divide a system into smaller logically divided units based on functionality.

Client and Server:
Logical progression of OO – Grouping objects/functions on one computer and calling these from others.

3 Tier App:
Now, server wanted to control some of its resources by itself and provide services. To access caged functionality, one extra layer added to Client/Server model.

N Tier App:
For simplify more complicated systems, layered request/response agreement installed between application/layers.

Distributed Objects (DO):
Back to freedom to interact with object on N-Tier systems, an ubiquitous and heterogeneous system evolved to interact with orthogonal objects (not one-to-one interactions).

DO created many objects interacting with many object on many computers. Solution emerged by aggregation of logically/functionally connected objects into components and design uniform interfacefor communications.

Service Oriented Architecture:
Now these orthogonal interacting components needs simplified, publishable and accessibility through a service provider proxies which provides widely accepted standard interface.

What next? May be….
Centralized Service Model, Contract Oriented Architecture, IMakeNUBreakArchitecture etc.. 😉

%d bloggers like this: