Big Data Sandbox Options for a Sustainable Future

Roadmap

In the case of options 2, 3 or 4 being deemed the desirable target, there are two ways to implement the final infrastructure:

• 
  Implementing the chosen option directly.
  
• 
  Implementing the option 1 infrastructure, then progressively expanding it until the final design is reached.
  

The progressive development is possible because, compared to option 1:

• 
  Option 2 is a direct expansion of the infrastructure and governance (multiple systems, governed by multiple boards).
  
• 
  Option 3 is a direct expansion of the infrastructure (multiple systems).
  

Then compared to options 2 and 3, option 4 is a direct expansion in infrastructure and/or governance complexity (adding production systems to option 2, or 'networking' option 3). Therefore, there is a natural progression path from option 1 to option 2 (through 'networking' the option 1 model) or to option 3 (with the addition of a production node). From there, option 4 is achievable as a latter expansion.

While it could be tempting to implement the chosen option directly, therefore achieving the final chosen design faster, this would also constitute a risk in terms of user adoption and community development. This is because the option 1 infrastructure is the simplest model for users, with a single platform to access and work with. This, in turn, allows an easier buildup of knowledge and experience through a tight user community. This is in contrast with a more complex system with distributed user communities where new users would have to learn from a more complex system with a reduced local user community.

In the context of driving adoption of a new platform / methodology, peer learning (from work colleagues or co-users of a system) is a critical factor. It may therefore be desirable to ensure that a 'knowledgeable' community is first developed in ideal quick-learning conditions before the infrastructure is made more complex or the user base split.

The current expectations are that the potential need for option 2 or 3 will become clearer in 2016 based on the experiences with option 1.